[Lustre-discuss] Hardware or software RAID?

[Andreas Dilger]

On Nov 20, 2008  12:26 +0000, Mark Dixon wrote:
> Which method of RAID is preferred with Lustre: hardware or software?
> 
> This might seem like a daft question, but I'm a newbie. The list, 
> operations guide and various "best practice" papers do not appear to 
> express a preference.

The great thing about Lustre is that you can make this decision entirely
based on what kind of price/performance/reliability you need, and not
because of a specific hardware requirement.

> I'm thinking about a system with:
> 
> * 2x failover MDS with a RAID1 or RAID10 volume
> * 2x failover OSS with RAID5 or RAID6 volumes.
> 
> I'm trying to gauge whether it's worth having shared storage arrays for 
> each failover set with hardware RAID, or just leave them as dual-attached 
> JBODs.
> 
> My instinct says that hardware RAID from a reputable vendor is best - 
> particularly because there's a battery-backed cache - but I see from the 
> lists that Lustre has put a lot of effort in improving the Linux MD RAID 
> layer.

There are a number of large clusters (TACC Ranger in particular) that use
software RAID on JBODs, but the majority of systems use hardware RAID in order to
maximize performance (at an increased cost of course).

These days I would tend to recommend using RAID-6 over RAID-5 just because
the large disks available take a long time to rebuild, and there is a
non-zero risk of a second disk failing during that time.


Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.