[lustre-discuss] DNE2 settings are not propagated?

Mannthey, Keith keith.mannthey at intel.com
Fri Mar 20 13:57:43 PDT 2020

Thanks Andreas and Cory very helpful.   -D will help for what I am using it for.


From: Spitz, Cory James <cory.spitz at hpe.com>
Sent: Friday, March 20, 2020 12:57 PM
To: Andreas Dilger <adilger at whamcloud.com>; Mannthey, Keith <keith.mannthey at intel.com>
Cc: lustre-discuss at lists.lustre.org; Koutoupis, Petros <petros.koutoupis at hpe.com>
Subject: Re: [lustre-discuss] DNE2 settings are not propagated?

As Andreas says, it isn’t really recommended, but you can get sub-directories to inherit the striping by using the -D, --default option to `lfs mkdir` or `lfs setdirstripe`.   From the man page:

       -D, --default
              Set  the  default striping pattern of subdirectories. Newly cre-
              ated sub-directories will use the new default striping  pattern,
              but  existing  sub-directories  will not be affected.  The newly
              created sub-directories will also inherit the specified  default
              striping  pattern.  Only  default  stripe count is supported for

              Note that striping all directories across all MDTs by default is
              not  recommended  at  this  time, as the clients will have to do
              more RPCs to create and access each directory,  hurting  perfor-
              mance rather than improving it.  Default striped directories are
              preferred for cases where large subdirectories will  be  created
              (e.g. file-per-process job output directories).


On 3/20/20, 2:33 PM, "lustre-discuss on behalf of Andreas Dilger" <lustre-discuss-bounces at lists.lustre.org<mailto:lustre-discuss-bounces at lists.lustre.org> on behalf of adilger at whamcloud.com<mailto:adilger at whamcloud.com>> wrote:

On Mar 20, 2020, at 13:02, Mannthey, Keith <keith.mannthey at intel.com<mailto:keith.mannthey at intel.com>> wrote:

I am doing some work with Lustre 2.13 (Downloaded from Whamcloud) and DNE 2 there are 6 MDTs.

[root at colts52-clx01 for-test]# lfs getdirstripe TEST
lmv_stripe_count: 6 lmv_stripe_offset: 1 lmv_hash_type: fnv_1a_64
mdtidx           FID[seq:oid:ver]
    1           [0x2c0000400:0x5:0x0]
    5           [0x300000403:0x5:0x0]
    3           [0x340000403:0x5:0x0]
    0           [0x200000403:0x5:0x0]
    4           [0x240000402:0x5:0x0]
    2           [0x280000401:0x5:0x0]
[root at colts52-clx01 for-test]# mkdir TEST/now
[root at colts52-clx01 for-test]# lfs getdirstripe TEST/now/
lmv_stripe_count: 0 lmv_stripe_offset: 2 lmv_hash_type: none

Directory stripping settings are not being propagated.  Is this expected?

Yes, this working as designed.  It isn't necessarily good for all directories to be striped, as that adds overhead without necessary improving performance.

The current recommendation is that DNE should be used for e.g. a top-level directory to distribute files and subdirectories across MDTs, or in the case of large directories with millions of files.

Cheers, Andreas
Andreas Dilger
Principal Lustre Architect

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20200320/ba2d860c/attachment.html>

More information about the lustre-discuss mailing list