[Lustre-discuss] Geographic cluster

[Brian J. Murrell]

On Fri, 2009-08-07 at 11:18 -0700, Evan Rempel wrote:
> Is it possible to use Lustre to make a single filesystem
> that has a mirror between two locations such that if
> the power were to be lost in one location that the other
> location could continue to run uninterrupted?

Mirror or replication?  There is a difference.  Mirror implies that the
two sides are coherent, meaning that a write on one side cannot be
considered complete until the other side is written too.

Replication is lazy mirroring with some limitations.  The major
limitation is that because it is lazy, you do not have a guarantee that
just because a file is on one side, it is available on the other side.
It will only become available to the other side when resource
constraints (such as bandwidth and latency) allow it to be available.
If you have lots of bandwidth and low latency, the delay could be quite
short.

The other thing is that I don't know that our replication feature is
going to be bi-directional.  That is, I don't know that you can have to
active filesystems replicating to each other.

> I would really like to have an active/active cluster so
> that I could run an apache cluster on top
> of the filesystem, and load balance all Web activity
> across any of the apache nodes which read/write to
> storage which is synchronously mirrored between
> two storage systems that are a couple of kilometeres
> apart.

This is a difficult problem to solve.  As long as each side is writing
to different files or parts of the filesystem, it's easy, but what
happens when both sides open the same file for a write operation?  Last
write wins?  That's not usually acceptable.  

Perhaps one of the engineers working on our replication feature will
chime in on whether replication can be bi-directional or not.  Just
given the complexity of the conflict problem, I tend to doubt it.

It might be that the replication system calls out to a user defined
function to resolve conflicts.  But that is just a total WAG on my part.

b.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: This is a digitally signed message part
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20090807/530f5bce/attachment.pgp>