[Lustre-discuss] 1.8.1 test setup achieved, what about maximum mdt size

[Andreas Dilger]

On 18-Oct-09, at 16:04, Piotr Wadas wrote:
> Now, I did a simple count of MDT size as described in lustre 1.8.1  
> manual,
> and setup mdt as recommended. The question is, no matter I did right  
> count
> or not, what actually will happen, if MDT partition runs out of space?
> Any chances to dump the whole MGS+MDT combined fs, supply a bigger  
> block
> device, or extend partition size with some e2fsprogs/tune2fs trick ?
> This assumes, that no matter how big MDT is, it will be exhausted  
> someday.

It is true that the MDT device can become full at some point, but this
happens fairly rarely given that most Lustre HPC users have very large
files, and the size of the MDT is MUCH smaller than the space needed for
the file data.  The maximum size of MDT is 8TB, and if you format the
filesystem with "-i 2048" you can get 4B inodes therein, which is the
maximum.  Even the largest filesystem we have seen doesn't use that many
inodes.

Once ZFS backing filesystems are available, this fixed inode limit will
be gone (for all practical purposes).  It will allow up to 2^48 files
per fileset, and it with CMD (when it finally arrives) will allow  
multiple
MDTs in a single Lustre filesystem.

> One possible solution is simply to add/create another FS, with another
> MGS/MDT. But the question persists :)

If you are using LVM you can increase the size of the MDT device and
resize the filesystem to add more inodes to the filesystem.

> And one more thing - I use combined MGS/MDT. What's actually about MGS
> size? I mean, if I use separate MGS and MDT, what size it should have,
> and how management service works, regarding to its block-device  
> storage ?


The MGS needs only some MB of space, maybe 128MB is the most it would
ever need.

Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.