[Lustre-discuss] Redundancy with object storage?

[Brian J. Murrell]

On Mon, 2007-12-03 at 20:32 -0600, D. Dante Lorenso wrote:
> 
> The problem that I see is that if any one 
> piece of the 4-node system fails, the whole system will fail.

Not quite.  If an OST fails, only the objects on that OST become failed.
The filesystem will continue to run and service requests from the
available OSTs.  That means, depending on striping policies, that some
or all of a file's contents might not be available.

> Is it possible to configure Lustre to write Objects to more than 1 node 
> simultaneously such that I am guaranteed that if one node goes down that 
> all files are still accessible?

That's called RAID, and right now, no.  It's on the roadmap though.

> This would effectively mean that I 
> would use 2 times the storage space for each object written and would 
> require that every cluster have a minimum of 2 nodes.

This is a description of mirroring.

> I understand the concepts of using DRBD and replicating block devices as 
> well as creating a full separate cluster for fail-over, but I'm hoping 
> to build redundancy into a single cluster without having to duplicate my 
> network with a bunch of active/passive machine combinations.

Using a reliable (some form of RAID -- which drbd qualifies as) shared
storage device is the only way to mitigate the SPOF scenario with Lustre
currently.

As far as duplication, etc. there is no reason why you cannot mirror
your drbd devices amongst the hardware you have currently (i.e pair your
machines up and create drbd based, mirrored devices) and along with some
form of failover (i.e. HA heartbeat) to get some redundancy.

If you were already resigned to halving your storage by mirroring OSTs
at the Lustre layer, dropping that mirroring down to drbd should not
impose any more significant costs.  Or maybe I'm misunderstanding your
concerns with using drbd.

b.