[lustre-discuss] Lustre newbie problems: formatting disk or partition with Lustre filesystem fails

[Dilger, Andreas]

The size of the MDT depends only on how many files you will have. If you have an average file size of 1MB then this will mean about 30TB/1MB=30M files at 2KB/file, or about 60GB for the MDT (assuming you are using ldiskfs).

However, it should be noted that sharing space on the same RAID-6 array between the MDT and OST(s) will not give the best performance due to lots of seeking and the MDT IO pattern not being well suited for RAID-6, but rather RAID-1 mirroring. 

You would be far better off to get a couple of small SSD drives for the MDT (even 100GB is small for SSDs these days) and mirror them for the MDT+MGT storage and use all 30TB for the OSTs. 

Cheers, Andreas

> On Nov 28, 2016, at 10:54, Stefano Turolla <turolla at genzentrum.lmu.de> wrote:
> 
> Thanks Rick, that was great, I started having a look to the tutorial,
> could you just give a suggestion on how big the MDT partition should be
> in my case (6 nodes + 1 master and 30TB data)?
> 
> regards
> Stefano
> 
> On 11/28/2016 05:10 PM, Mohr Jr, Richard Frank (Rick Mohr) wrote:
>>> On Nov 28, 2016, at 9:58 AM, Stefano Turolla <turolla at genzentrum.lmu.de> wrote:
>>> 
>>> thanks for the quick reply, I am maybe doing the wrong thing. What I am
>>> trying to achieve is to have a Lustre volume to be shared among the
>>> nodes, and the 30TB is the size of existing storage.
>>> 
>>> Should I create a separate (and much smaller) partition for MDT? How I
>>> then set up the data partition to be shared? For the time being I would
>>> prefer having MGS and MDT on the same system.
>> Yes, you will need a separate partition or block device for the combined MGT/MDT.  That storage will be used to store the metadata info for the files in your Lustre file system.  The actual file data will be stored on one or more OST devices.  In most Lustre setups, there is a server that mounts the MDT/MGT device (which is referred to as the MDS/MGS node).   One or more additional servers then mount the OSTs (these are referred to as the OSS nodes).  However, it is possible to have a single server that mounts the MDT/MGT as well as the OSTs.
>> 
>> If you are interested in some entry-level Lustre tutorials, check out http://lustre.ornl.gov/lustre101-courses.
>> 
>> --
>> Rick Mohr
>> Senior HPC System Administrator
>> National Institute for Computational Sciences
>> http://www.nics.tennessee.edu
> 
> -- 
> Stefano Turolla
> IT Sysadmin Gene Center
> University of Munich
> Feodor-Lynen-Str. 25 81377 Munich, Germany
> tel. +49 (0)89 2180 71055
> email. turolla at genzentrum.lmu.de
> 
> _______________________________________________
> lustre-discuss mailing list
> lustre-discuss at lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org