[lustre-discuss] 1 MDS and 1 OSS

[Dilger, Andreas]

On Oct 31, 2017, at 05:46, Mohr Jr, Richard Frank (Rick Mohr) <rmohr at utk.edu> wrote:
> 
>> On Oct 30, 2017, at 4:46 PM, Brian Andrus <toomuchit at gmail.com> wrote:
>> 
>> Someone please correct me if I am wrong, but that seems a bit large of an MDT. Of course drives these days are pretty good sized, so the extra is probably very inexpensive.
> 
> That probably depends on what the primary usage will be.  If the applications create lots of small files (like some biomed programs), then a larger MDT would result in more inodes allowing more Lustre files to be created.

With mirroring the MDT ends up as ~2.4TB (about 1.2B files for ldiskfs, 600M files for ZFS), which gives an minimum average file size of 120TB/1.2B = 100KB on the OSTs (200KB for ZFS).  That said, by default you won't be able to create so many files on the OSTs unless you reduce the inode ratio for ldiskfs at format time, or use ZFS (which doesn't have a fixed inode count, but uses twice as much space per inode ob the MDT). 

Having a larger MDT isn't bad if you plan future expansion.  That said, you would get better performance over FDR if you used SSDs for the MDT rather than HDDs (if you aren't already planning this), and for a single OSS you probably don't need the extra MDT capacity.  With both ldiskfs+LVM and ZFS you can also expand the MDT size in the future if you need more capacity.

Cheers, Andreas
--
Andreas Dilger
Lustre Principal Architect
Intel Corporation