[Lustre-discuss] clarification on mkfs.lustre options

[Andreas Dilger]

On 2010-07-30, at 13:14, Sebastian Gutierrez <gutseb at cs.stanford.edu> wrote:
>> If you are planning on expanding this at the RAID6 level to be an 8+2 configuration, you should specify "-E stripe=256,stride=64".  
> 
> Are there any potential negatives here?   I initially used a 6 disk raid 10 but I ended up with wasted space on the filesystem since I could not fit the 1M lustre I/O into the number of active disks cleanly.  Would there be a way to minimize the amount of wasted space if I wanted to stick to raid 1/0?  
> I assume that aligned I/O is always preferred. 

For RAID-1+0 the alignment is much less important. While there is still some negative effect if the 1MB read of write is not aligned (because it will make an extra pair of disks active to fill the RPC) this is not nearly so bad as RAID-5/6 where it will cause also the parity chunk to be rewritten. 

If you are using a 6-disk RAID-1+0 then it would be OK for example to configure the RAID chunksize to be 128kB. While this means that a 1MB IO would handle 3*128kB from two pairs of disks and 4*128kB from the third pair of disks (each IO would be sequential though).

 It means a given pair of disks would do a bit more work than the others for a given RPC, but since the IO is sequential (assuming the request itself is sequential) it will not need an extra seek for the last disk and the extra IO is a minimal effort. 

>> 
>> Also, 16TB LUN support is only available with ext4, so if you have 2TB drives you need to make sure to download the right ldiskfs package.
> 
> I am creating these filesystems with lustre 1.8.3 from the prebuilt RPMs.  Do you mean that there is a different ldiskfs package I should use?  

There should be an ldiskfs-ext4 RPM available for download with 1.8.3 and later (for specific vendor kernels). The ext3 code has had a  lot more testing than ext4 so we recommend using ext3 unless there is a reason to use ext4 (e.g. > 8TB LUN size).  

> Should i also use the flexible block group options on the OSTs?
> -O flex_bg and -G nr_merged_groups

The flex_bg feature is only available with ext4. We haven't done any testing with this feature yet, but in theory it can help. 

Cheers,  Andreas