[Lustre-discuss] Optimal OSS OST drives for boxed deployment

[Peter Grandi]

>>> I have two OSS each have six 1TB drives.  sda contains the
>>> kernel and the operating system.  sdb,sdc,sdd,sde,sdf are the
>>> targets and make only a raid 5.
>>> Is it advisable to add another drive to each of these OSS's
>>> to facilitate raid 6 for the targets?

Why RAID6? Do you realize that it has a very different
performance profile from RAID5? And that all parity RAID have
some unpleasant combination of performance/availability.

>>> sda has only / partion and occupies the entire 1TB drive.

Bit of a waste, and no redundancy.

[ ... ]

> Purchased 4 more 1TB spinpoint drives for the OSS's.

Using the same drive type in all the drives or many drives in an
array is not a wonderful idea, never mind having them purchased
at the same time and delivered together.

Still with 8 drive per OSS you can get pretty impressive
performance if you use the right RAID setup.

> This should allow for proper raid 6 if the boards, power
> supplies and backup power can handle the load. 

There is nothing "proper" about RAID6/5: http://WWW.BAARF.com/

Unless there are very good reasons (and there are AFAICS only
two cases in which there are), one should nearly always use
RAID10 (in some cases RAID1 and in very few cases RAID0).

In your case the sort of layout I would use is:

* Each drive should have two partitions (or more if more than
  one OST is desired), one say 10-30GB, and the other covering
  the rest of the disk minus the last say 10G or so.

* The first partition of the first 2 disks should be a RAID1
  with the root filesystem. One might want to put the MDT in
  there too. The other first partitions could be used for swap
  (no RAID) or for '/var' for example (RAID10).

* The second (and further) partitions of all disks in one RAID10
  for the OSTs.

There is a case for leaving the 8th drive on each OSS as a hot
spare, and keeping a 9th drive on a shelf as a cold spare.

If you were really into maximum availability, you could do
RAID10 across the two OSSes, using DRBD, there are a few HOWTOs
on how to do that (but performance will suffer of course).