<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<br class="">
<div><br class="">
<blockquote type="cite" class="">
<div class="">On Sep 11, 2019, at 10:06, Michael Di Domenico <<a href="mailto:mdidomenico4@gmail.com" class="">mdidomenico4@gmail.com</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div class="">On Tue, Sep 10, 2019 at 5:48 PM Andreas Dilger <<a href="mailto:adilger@whamcloud.com" class="">adilger@whamcloud.com</a>> wrote:<br class="">
<blockquote type="cite" class=""><br class="">
I don't think "lfs find -xdev" has never been a priority for Lustre, since it is rare for Lustre filesystems to be<br class="">
mounted in a nested manner. Since people already run multiple "lfs find" tasks in parallel on different<br class="">
clients to get better performance, it isn't hard to run separate tasks from the top-level mountpoint of<br class="">
different filesystems. What is the use case for this?<br class="">
</blockquote>
<br class="">
doesn't xdev keep find from crossing mount points, not necessarily<br class="">
only in a nested manner but also if there's a link to a directory in a<br class="">
different filesystem. i believe 'find' without -xdev will follow and<br class="">
descend. but this predicates that my understanding is sound (which it<br class="">
probably isn't). i generally add -xdev to my finds as a habit to keep<br class="">
from scanning nfs volumes.<br class="">
</div>
</div>
</blockquote>
<div><br class="">
</div>
Yes, -xdev is to avoid crossing mountpoints, but like I wrote it is rare to have nested Lustre</div>
<div>mountpoints, so this wouldn't really be useful in most cases. The find tree walking does</div>
<div>*not* follow symlinks into the target directory, only mountpoints.</div>
<div><br class="">
<blockquote type="cite" class="">
<div class="">
<div class=""><br class="">
<blockquote type="cite" class="">along the same vein, can anyone state whether there's any actual<br class="">
performance gain walking the filesystem using find vs lfs find?<br class="">
<br class="">
For "find" vs. "lfs find" performance, this depends heavily on what the search parameters are. If just<br class="">
the filename, they will be the same. If it includes some MDT-specific attributes (e.g. uid, gid) then<br class="">
"lfs find" can be significantly faster (e.g 3-5x). If it is uses file size, then they will be about the same<br class="">
unless there are other MDT-only parameters, or once LSOM support is landed (hopefully 2.13).<br class="">
</blockquote>
<br class="">
okay, that's what i thought or recalled correctly from hearing<br class="">
somewhere else. in my particular instance i was just using 'find<br class="">
-type f' and didn't see any appreciable difference in scanning speed<br class="">
between the two<br class="">
</div>
</div>
</blockquote>
<br class="">
</div>
<div>For Lustre, ext4, and most other filesystems, the file type is also stored in the directory entry, so that</div>
<div>"find" can determine the type without a "stat". That is safe since the file type cannot be changed after</div>
<div>the file is created.</div>
<div><br class="">
</div>
<div>In a scan like "(lfs) find -name '*foo*' -type f" it only needs to read the directory entries (including the</div>
<div>file type) and process each entry. There is nothing that "lfs find" can optimize. With mode, uid, gid, and</div>
<div>*some* timestamp queries, "lfs find" can fetch only the MDS attributes and skip any OST RPCs for</div>
<div>that file if the (non)match can be decided without the OST attributes.</div>
<div><br class="">
</div>
<div>Once the Lazy Size-on-MDT (LSOM) integration is finished (<a href="https://review.whamcloud.com/35167" class="">https://review.whamcloud.com/35167</a>) it</div>
<div>will be possible for "lfs find --lazy" to use *only* attributes from the MDS (size, timestamps) to speed</div>
<div>up scanning and avoid OST RPC overhead.</div>
<br class="">
<div class="">
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div>Cheers, Andreas</div>
<div>--</div>
<div>Andreas Dilger</div>
<div>Principal Lustre Architect</div>
<div>Whamcloud</div>
<div><br class="">
</div>
<div><br class="">
</div>
<div><br class="">
</div>
</div>
</div>
</div>
</div>
</div>
<br class="Apple-interchange-newline">
<br class="Apple-interchange-newline">
</div>
<br class="">
</body>
</html>