[Lustre-discuss] clarification on mkfs.lustre options

[Sebastian Gutierrez]

Hello Andreas 
Thank you very much for you help.  I do have a few more questions.

>> 
>> Therefore 
>> My mkfs options should be 
>> --mkfsoptions="-E stripe=128 -E stride=32" /dev/sdb
> 
> If you are planning on expanding this at the RAID6 level to be an 8+2 configuration, you should specify "-E stripe=256,stride=64".  

Are there any potential negatives here?   I initially used a 6 disk raid 10 but I ended up with wasted space on the filesystem since I could not fit the 1M lustre I/O into the number of active disks cleanly.  Would there be a way to minimize the amount of wasted space if I wanted to stick to raid 1/0?  
I assume that aligned I/O is always preferred. 
 

> Note that you cannot specify mulitple separate "-E" options to mke2fs, it would only use the last one specified.

I want to say I tried the mkfs.lustre "-E stripe=xx,stride=xx" but I received some error.  I will see if this happens again. 

> 
>> purchase 6 more disks
>> This will allow for a new raid 6 that is aligned the same way with 3 hotspares.
> 
>> or (I need clarification if this understanding is correct)
>> 
>> purchase 10 more disks
>> Expand the current raid 6 to a larger 10 disk/raid 6 with a 4 disk raid 1/0 for a external journal plus a hot spare.  
> 
> Using a 4-disk RAID-10 external journal is unlikely to give you any extra performance, since journal IO is nearly sequential (though sometimes small block writes if there are few clients and you are not using async journal).

I see.  

> 
> Also, 16TB LUN support is only available with ext4, so if you have 2TB drives you need to make sure to download the right ldiskfs package.
> 

I am creating these filesystems with lustre 1.8.3 from the prebuilt RPMs.  Do you mean that there is a different ldiskfs package I should use?  

I guess this brings up another question.  
Should i also use the flexible block group options on the OSTs?
-O flex_bg and -G nr_merged_groups

> Depending on the hardware options on your RAID, it may be that you need a separate hot spare for each LUN, in which case having 2 hot spares makes sense.  Otherwise, you can probably use only 13 or 14 drives.
> 
>> Is my understanding of the documentation accurate? 
>> Do both of these options seem like potential upgrade options?
> 
> Either of them seem reasonable.
> 
> If the hardware allows in-place RAID reshaping then it is possible.  I'd always recommend to make a backup before doing this, because one never knows what might happen if this operation is interrupted for some reason.

Understood.  

Thanks Again
Sebastian