[Lustre-discuss] mkfs options/tuning for RAID based OSTs

Dennis Nelson dnelson at sgi.com
Tue Oct 19 14:56:33 PDT 2010


Segment size should be 128.

128 KB * 8 data drives = 1 MB.


On 10/19/10 3:42 PM, "Edward Walter" <ewalter at cs.cmu.edu> wrote:

> Hello All,
> 
> We're doing a fresh Lustre 1.8.4 install using Sun StorageTek 2540
> arrays for our OST targets.  We've configured these as RAID6 with no
> spares which means we have the equivalent of 10 data disks and 2 parity
> disks in play on each OST.
> 
> We configured the "Segment Size" on these arrays at 512 KB.  I believe
> this is equivalent to the "chunk size" in the Lustre operations manual
> (section 10.1.1).  Based on the formulae in the manual: in order to have
> my stripe width fall below 1MB; I need to reconfigure my "Segment Size"
> like this:
> 
> Segment Size <= 1024KB/(12-2) = 102.4 KB
> so 16KB, 32KB or 64KB are optimal values
> Does this seem right?
> 
> Do I really need to do this (reinitialize the arrays/volumes) to get my
> Segment Size below 1MB?  What impact will/won't this have on performance?
> 
> When I format the OST filesystem; I need to provide options for both
> stripe and stride.  The manual indicates that the units for these values
> are 4096-byte (4KB) blocks.  Given that, I should use something like:
> 
> -E stride= (one of)
>     16KB/4KB = 4
>     32KB/4KB = 8
>     64KB/4KB = 16
> 
> stripe= (one of)
>     16KB*10/4KB = 40
>     32KB*10/4KB = 80
>     64KB*10/4KB = 160
> 
> so for example I would issue the following:
> mkfs.lustre --mountfsoptions="stripe=160" --mkfsoptions="-E stride=16 -m
> 1" ...
> 
> Is it better for to opt for the higher values or lower values here?
> 
> Also, does anyone have recommendations for "aligning" the filesystem so
> that the fs blocks align with the RAID chunks?  We've done things like
> this for SSD drives.  We'd normally give Lustre the entire RAID device
> (without partitions) so this hasn't been an issue in the past.  For this
> installation though; we're creating multiple volumes (for size/space
> reasons) so partitioning is a necessary evil now.
> 
> Thanks for any feedback!
> 
> -Ed Walter
> Carnegie Mellon University
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-discuss




More information about the lustre-discuss mailing list