[Lustre-discuss] RAID Stripe alignment

Andreas Dilger andreas.dilger at oracle.com
Mon Dec 6 09:55:40 PST 2010


Is your write cache persistent?  One major factor in having Lustre read and write alignment is that any misaligned write  will cause read-modify-write, and misaligned reads will cause 2x reads if the RAID layer is doing parity verification.

If your RAID layer is hiding this overhead via cache, you need to be absolutely sure that it is safe in case of crashes and failover of either or both the OSS and RAID controller. 

Cheers, Andreas

On 2010-12-06, at 9:44, Kaizaad Bilimorya <kaizaad at sharcnet.ca> wrote:

> Hello,
> 
> I read this thread
> http://www.mail-archive.com/lustre-discuss@lists.lustre.org/msg07791.html
> 
> and the Lustre Operations manual "10.1 Considerations for Backend Storage"
> 
> in order to determine the best performing setup for our OSS hardware.
> 
> HP DL180 G6
> - CentOS 5.5 and Lustre 1.8.4
> - HP Smart Array P410 controller (512 MB cache, 25% Read / 75% Write)
> - 600 GB SAS drives
> 
> The stripesizes (in HP P410 terminology is the amount of data read/written 
> to each disk) available on the P410 array controllers are 8,16,32,64,128,256
> 
> Two of the scenarios that we tested are:
> 
> 1) 1 x 12 disk RAID6 lun
> chunksize = 1024 / 10 disks = 102.4 => use stripesize=64
> - not optimally aligned but maximum space usage
> - setup on oss[2-4]
> - sgpdd_survey results:
>   http://www.sharcnet.ca/~kaizaad/orcafs/unaligned.html
> 
> 2) 1 x 10 disk RAID6 lun
> chunksize = 1024 / 8 = 128 => use stripesize=128
> - optimally aligned but at the sacrifice of 2 disks of space
> - setup on oss[8-10]
> - sgpdd_survey results:
>   http://www.sharcnet.ca/~kaizaad/orcafs/aligned.html
> 
> 
> In our cases, the graphs seem to indicate that the the underlying RAID 
> alignment setup doesn't matter much, which is totally counter intuitive to 
> the recommendations by the Lustre list and manual.
> 
> Is there something we are missing here? Maybe I misunderstood the 
> recommendations? Or are we just bottlenecking on a component in the setup 
> so a proper RAID alignment doesn't show as being beneficial? Any insight 
> is appreciated.
> 
> thanks
> -k
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-discuss



More information about the lustre-discuss mailing list