[Lustre-discuss] GlusterFS and Lustre

[laytonjb at charter.net]

---- Craig Tierney <Craig.Tierney at noaa.gov> wrote: 
> rishi pathak wrote:
> > I came across this www.gluster.org <http://www.gluster.org>
> > Has any one tried it .
> > Is it a true parallel file system allowing concurrent read and write to 
> > a file by many  processes.
> > Will it be suitable for HPC applications.
> > 
> > 
> 
> I wouldn't call GlusterFS a parallel filesystem in the same way I would
> refer to Lustre or PVFS. GlusterFS is a distributed filesystem,
> where complete files are contained on one of multiple servers. 

This isn't quite accurate. Depending upon the translators you use, the files
can be stripped across servers. For clusters it is almost always the case that
the files will be stripped.

> striping, even they say striping for their implementation is bad 
> (http://www.gluster.org/docs/index.php/GlusterFS_FAQ#Why_is_striping_bad.3F).
> Because of GlusterFS's modular architecture it was easy for them to implement.
> They do have MPI-IO support on their roadmap, so maybe they are planning to work around
> the issues described in the link above in user space.
> 
> GlusterFS is much more like Ibrix or Netapp/GX than Lustre.  It seems best as a distributed NFS
> replacement.  In my minimal testing, performance scales linearly as you add data servers.
> Metadata performance is reasonable (by feel, not by actual measurements).

One of the design ideas behind GlusterFS is that it doesn't have a metadata
server. So i'm not sure what you were measuring. It may have been the
metadata performance for the underlying file system rather than GlusterFS.

I haven't tested it yet, but it has some interesting ideas (all in user-space so
there are no kernel mods to worry about, no metadata server, stackable
translators for tuning performance). 

Jeff