[Lustre-discuss] NFS vs Lustre

Peter Grandi pg_lus at lus.for.sabi.co.UK
Mon Aug 31 09:09:21 PDT 2009


Interesting discussion of NFS vs. Lustre even if they are so
different in aims...

[ ... ]

lee> 3) It must support all of the transports we are interested in.

Except for some corner cases (that an HEP site might well have)
that today tends to reduce to the classic Ethernet/IP pair...

lee> 4) It must be scalable, in that we can cheaply attach
lee>    storage and both performance (reading *and* writing) and
lee>    capacity within a single mounted file system increase in
lee>    direct proportion.

I suspect that scalability is more of a dream, as to me it
involves more requirements including scalable backup (not so
easy) and scalable 'fsck' (not so easy).

These are easier with Lustre because it does not provide "a
single mounted file system" but a single mounted *namespace*
which is a very different thing, even if largely equivalent for
most users.

[ ... ]

lee> NFS4 does most everything Lustre can with one very
lee> important exception, IO bandwidth. [ ... ] Lustre IO
lee> performance *does* scale. It uses a 3rd-party transfer.

That can summarized by saying that Lustre is a parallel
distributed metafilesystem, while NFS is a protocol used to
access what usually is something not distributed and an actual
filesystem. The limitations of the NFS protocol can be overcome,
and as you say, pNFS turns it into a parallel distributed
metafilesystem too:

lee> NFS4 has a proposed extension, called pNFS, to address this
lee> problem. It just introduces the 3rd-party data transfers
lee> that Lustre enjoys. If and when that is a standard, and is
lee> well supported by clients and vendors, the really big
lee> technical difference will virtually disappear. It's been a
lee> long time coming, though. It's still not there. Will it
lee> ever be, really?

My impression is that it is a lot more real than it was only a
couple years ago, and here is an amusing mashup:

  http://FT.ORNL.gov/pubs-archive/ipdps2009-wyu-final.pdf

   «Parallel NFS (pNFS) is an emergent open standard for
    parallelizing data transfer over a variety of I/O
    protocols. Prototypes of pNFS are actively being developed
    by industry and academia to examine its viability and
    possible enhancements. In this paper, we present the design,
    implementation, and evaluation of lpNFS, a Lustre-based
    parallel NFS. [ ... ] Our initial performance evaluation
    shows that the performance of lpNFS is comparable to that of
    original Lustre.»

lee> Done. That was useful for me. I think five years ago I
lee> might have opted for Lustre in the "create many small
lee> files" case, where I would consider NFS today,

Looks optimistic to me -- I don't see any good solution to the
"create many small files case, at least as to shared storage.

For smaller situations I am looking out of interest to some
other distributed filesystems, which are a bit more researchy,
but seem fairly reliable already.



More information about the lustre-discuss mailing list