[lustre-discuss] DNE v3 and directory inode changing

[Andreas Dilger]

On Mar 24, 2023, at 13:20, Bertschinger, Thomas Andrew Hjorth <bertschinger at lanl.gov<mailto:bertschinger at lanl.gov>> wrote:

Thanks, this is helpful. We certainly don't need the auto-split feature and were just experimenting with it, so this should be fine for us. And we have been satisfied with the round robin directory creation so far. Just out of curiosity, is the auto-split feature still being actively worked on and expected to be complete/production-ready within some defined period of time?

Nobody is currently working on directory split.  There are a number of other DNE optimizations that are underway (mostly internal code, locking, and recovery improvements without much "visible" to the outside world).

The main feature in progress on the metadata side is the metadata writeback cache (WBC) that will greatly improve single-client workloads in a single directory tree (e.g. untar and then build/process files in that tree).   That should help significantly with genomics and machine learning workloads that have this kind of usage pattern.

Cheers, Andreas
--
Andreas Dilger
Lustre Principal Architect
Whamcloud







-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20230324/ffeba19f/attachment.htm>