[Lustre-discuss] Lustre SNS and Hadoop

[Andreas Dilger]

On Jul 11, 2009  00:52 -0400, Mag Gam wrote:
> Does anyone know the status of SNS and Lustre? I know it was postponed
> until further notice but has anyone word any other news?

Well, ORNL presented a paper at ISC a few weeks ago with an implementation
of data mirroring that might be similar to SNS.  I haven't seen more than
the presentation about it yet, but I've asked them to post the code to
lustre-devel so we can all look at it.

> Also, since SNS isn't going to be ready for primetime, is anyone using
> Hadoop and Lustre? Since Hadoop has a redundancy failover won't that
> make more sense with Lustre?

We've tested Hadoop running on Lustre, and in some workloads Lustre is
much faster than the HDFS, but not in all of them.  It was just a test,
and we haven't done much to optimize it.

Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.