[Lustre-discuss] NFS vs Lustre

Kevin Van Maren Kevin.Vanmaren at Sun.COM
Mon Aug 31 13:59:41 PDT 2009


Yes, the semantics are similar to a local filesystem: data can be lost 
after close() under several cases, including:

1) the client crashes before the server has written the data to disk 
(data that made it to the server should be written, but that is asynch),

2) the server returns an error to the client (EIO, ie due to errors on 
the OST),

3) the client is evicted by the server (ie, due to communication issues) 
before writing data to disk, or

4) server reboots and recovery fails (ie, in 1.6.x a _different_ client 
does not reconnect to replay transactions).  With version-based recovery 
in 1.8, clients might be able to still replay some transactions even if 
another client crashed/rebooted while the server was down.

fsync() is the best way to ensure data is on disk, for both Lustre and a 
local filesystem.

Kevin


Brian J. Murrell wrote:
> On Mon, 2009-08-31 at 21:56 +0200, Daniel Kobras wrote:
>   
>> Hi!
>>     
>
> Hi,
>
>   
>> Lustre's
>> standard config follows Posix and allows dirty client-side caches after
>> close(). Performance improves as a result, of course, but in case something
>> goes wrong on the net or the server, users potentially lose data just like on
>> any local Posix filesystem.
>>     
>
> I don't think this is true.  This is something that I am only
> peripherally knowledgeable about and I am sure somebody like Andreas or
> Johann can correct me if/where I go wrong...
>
> You are right that there is an opportunity for a client to write to an
> OST and get it's write(2) call returned before data goes to physical
> disk.  But Lustre clients know that, and therefore they keep the state
> needed to replay that write(2) to the server until the server sends back
> a commit callback.  The commit callback is what tells the client that
> the data actually went to physical media and that it can now purge any
> state required to replay that transaction.
>
> Until that write callback is received, the client holds on to whatever
> state it would need to do that write(2) all over again, for exactly the
> case you cite which is the server goes down before the data goes to
> physical media.
>
> It is this data that the client is caching until the commit callback is
> received that is used by the recovery mechanisms that start when a
> target comes back on-line.
>
> Hope that clarifies things, and further, I hope my understanding is
> correct as is my explanation.
>
> b.
>
>   
> ------------------------------------------------------------------------
>
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-discuss
>   




More information about the lustre-discuss mailing list