[Lustre-discuss] OST RAID sets

Mohr Jr, Richard Frank (Rick Mohr) rmohr at utk.edu
Mon Oct 20 10:51:20 PDT 2014

On Oct 20, 2014, at 9:26 AM, Jon Tegner <tegner at foi.se> wrote:

> "For better performance, we recommend that you create RAID sets with 4 or 8 data disks plus one or two parity disks."
> We were experimenting with OSTs with 6 disks, and tried raid set of both 4 and 5 disks (raid5). While testing for performance we didn't notice any difference (4 was not noticeable faster than 5), so just wondering if there are any references where the performance advantage of 4 or 8 data disks are verified.

Most likely the recommendation is based on the idea of creating a RAID set where a single stripe is exactly 1 MB.  Lustre's maximum request size was 1 MB prior to Lustre 2.4.  (In v2.4, there is support for 4 MB requests, but I think the default might still be 1 MB requests.)  Using 4 or 8 disks, you can have a bulk 1MB I/O request that uses an entire RAID stripe (if aligned properly) which can potentially avoid read-modify-write penalties.  However, the effect of this penalty might vary considerably depending upon the kind of software/hardware RAID setup you are using.  Also, if your workload is not a bunch of 1MB streaming writes, then using 4 or 8 disks may not provide an advantage.

Rick Mohr
Senior HPC System Administrator
National Institute for Computational Sciences

More information about the lustre-discuss mailing list