[Lustre-discuss] Client directory entry caching

Daire Byrne daire.byrne at gmail.com
Thu Jul 29 03:47:25 PDT 2010


Hi,

I was wondering if it is possible to have the client completely cache
a recursive listing of a lustre filesystem such that on a second run
it doesn't have to talk to the MDT again? Taking the simplest case
where I only have one client that is browsing a million file tree
(say), I would expect that once the ldlm has cached the locks
(lru_size) then a second recursive scan (find, ls -R) shouldn't need
to talk to the MDT/OST again. But this is not the case probably
because a recursive scan needs to do a open() and getdents() on each
directory it finds. If I just stat all the files without doing a
recursive scan then it gets everything from the client cache as
expected without the MDS chatter - e.g.

  find /mnt/lustre -type f > /tmp/files.txt
  cat /tmp/files.txt | xargs ls -l

Is there any way to improve the browsing speed and cache directory
contents - especially for the case where I only have a single client
accessing an entire tree? As an aside I also noticed that a "ls -l"
does a getxattr - does that get cached by the client too? I can
imagine it might cause quite a bit of MDS chatter.

Cheers,

Daire



More information about the lustre-discuss mailing list