[lustre-discuss] DoM question

Dilger, Andreas andreas.dilger at intel.com
Fri Apr 6 03:29:23 PDT 2018

On Apr 5, 2018, at 03:01, Martin BALVERS <martin.balvers at nutricia.com> wrote:
> I have a question about the new Data on MDT feature.
> The default dom_stripesize is 1M, does this mean that smaller files will
> also consume 1M on the MDT ?
> I was thinking of using this for my home dirs, but there are a lot of
> smaller files there, so maybe dom_stripesize=64k would be better.

The dom_stripesize setting is the MAXIMUM size that can be stored on the MDT.
It does not affect the amount of data allocated to the file, which will be in
units of 4096-byte blocks.

The intent is that, depending on how much space is available on the MDT and
your file size distribution, you set a PFL layout for the filesystem that puts
the first component on the MDT, and the rest of the file on the OSTs, like:

  lfs setstripe -E1M -L mdt -E64M -c1 -S1M -E8G -c4 -E-1 -c-1 -S4M /mnt/testfs

or whatever is appropriate for your file size mix.  This alleviates the need
for most users to set a layout for their files, while having good performance
for a wide variety of use cases.

Since most sites have file size distribution like 90% of files are below 1MB
and use only 5% of space, while 90% of space is used by only 5% of very large
files, it isn't a big deal that every file has the first 1MB on the MDT.  You
would only save 10% of space on the MDT by optimizing the remaining files to
not store the first 1MB there.

Note that if you are using DoM and FLR, you probably want to format your MDT
with non-default parameters (e.g. 256KB/inode, "-i 262144", if using ldiskfs),
or it will normally 50% filled with inodes by default and only has a limited
amount of space for data (< 1.5KB/inode), directories, logs, etc.

For ZFS this is less of a concern, since it dynamically allocates blocks and
inodes, but it is still more likely to fill with data depending on your workload.

Cheers, Andreas
Andreas Dilger
Lustre Principal Architect
Intel Corporation

More information about the lustre-discuss mailing list