[Lustre-discuss] OST RAID sets

[Mohr Jr, Richard Frank (Rick Mohr)]

On Oct 20, 2014, at 9:26 AM, Jon Tegner <tegner at foi.se> wrote:

> "For better performance, we recommend that you create RAID sets with 4 or 8 data disks plus one or two parity disks."
> 
> We were experimenting with OSTs with 6 disks, and tried raid set of both 4 and 5 disks (raid5). While testing for performance we didn't notice any difference (4 was not noticeable faster than 5), so just wondering if there are any references where the performance advantage of 4 or 8 data disks are verified.


Most likely the recommendation is based on the idea of creating a RAID set where a single stripe is exactly 1 MB.  Lustre's maximum request size was 1 MB prior to Lustre 2.4.  (In v2.4, there is support for 4 MB requests, but I think the default might still be 1 MB requests.)  Using 4 or 8 disks, you can have a bulk 1MB I/O request that uses an entire RAID stripe (if aligned properly) which can potentially avoid read-modify-write penalties.  However, the effect of this penalty might vary considerably depending upon the kind of software/hardware RAID setup you are using.  Also, if your workload is not a bunch of 1MB streaming writes, then using 4 or 8 disks may not provide an advantage.

-- 
Rick Mohr
Senior HPC System Administrator
National Institute for Computational Sciences
http://www.nics.tennessee.edu