[Lustre-discuss] NFS vs Lustre

[Nicolas Williams]

On Sun, Aug 30, 2009 at 10:51:41PM +0200, Daniel Kobras wrote:
> On Sat, Aug 29, 2009 at 11:56:40AM -0600, Lee Ward wrote:
> > NFS4 addresses those by:
> > 
> > 1) Introducing state. Can do full POSIX now without the lock servers.
> > Lots of resiliency mechanisms introduced to offset the downside of this,
> > too.
> 
> NFS4 implementations are able to handle Posix advisory locks, but unlike
> Lustre, they don't support full Posix filesystem semantics. For example, NFS4
> still follows the traditional NFS close-to-open cache consistency model whereas
> with Lustre, individual write()s are atomic and become immediately visible to
> all clients.

NFSv4 can't handle O_APPEND, and has those close-to-open semantics.
Those are the two large departures from POSIX in NFSv4.

NFSv4.1 also adds metadata/data separation and data distribution, much
like Lustre, but with the same POSIX semantics departures mentioned
above.  Also, NFSv4.1's "pNFS" concept doesn't have room for
"capabilities" (in the distributed filesystem sense, not in the Linux
capabilities sense), which means that OSSes and MDSes have to
communicate to get permissions to be enforced.  There are also
differences with respect to recovery, etcetera.

One thing about NFS is that it's meant to be neutral w.r.t. the type of
filesystem it shares.  So NFSv4, for example, has features for dealing
with filesystems that don't have a notion of persistent inode number.
Whereas Lustre has its own on-disk format and therefore can't be used to
share just any type of filesystem.

Nico
--