[Lustre-discuss] software raid

[Stuart Midgley]

Hi Brian

Long time no speak.

Anyway, we use to use software raid exclusively but have slowly stopped.  Using 3ware cards in Rackable nodes now.  All going well so far.  Though, for our MDS we are running a 3 way mirror on sas disks.

md has a few issues... all of them tend to end at the same place... losing data.  We have had situations where md returns crap data cause its getting it from a disk, but doesn't actually verify it against other disks (the disk hasn't actually thrown hardware errors)... you manually fail the disk and all of a sudden the file is no longer corrupt.

We have also had situations where md says the write occurred successfully, but really it has just hit the cache on the disk and hasn't been committed to platter... and a short time later, the disk reports the error to md but for a much earlier read/write.  The data is now corrupt on disk and flushed form all of lustre's caches.

With all our software raid we now do /sbin/hdparm -W 0 "$dev"  to disable write caching on the disk.  This has helped, but obviously hurts performance.





--
Dr Stuart Midgley
sdm900 at gmail.com



On 24/03/2011, at 10:54 AM, Brian O'Connor wrote:

> 
> This has probably been asked and answered.
> 
> Is software raid(md) still considered bad practice?
> 
> I would like to use ssd drives for an mdt, but using fast ssd drives
> behind a raid controller seems to defeat the purpose.
> 
> There was some thought that the decision not to support
> software raid was mostly about Sun/Oracle trying to sell hardware
> raid.
> 
> thoughts?
> 
> -- 
> Brian O'Connor
> -----------------------------------------------------------------------
> SGI Consulting
> Email: briano at sgi.com, Mobile +61 417 746 452
> Phone: +61 3 9963 1900, Fax:  +61 3 9963 1902
> 357 Camberwell Road, Camberwell, Victoria, 3124
> AUSTRALIA
> http://www.sgi.com/support/services
> -----------------------------------------------------------------------
> 
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-discuss