[Lustre-discuss] Applications of Lustre - streaming?

[Jason Brooks]

Hello,

The question of hdfs storage via lustre has been in the foreground of my thinking.  the hadoop hdfs processes are not aware of block devices: they only know of a filesystem mount point to begin storing data in hdfs.  THUS…

If we provide a filesystem interface (say a lustre mount point) whose latencies and throughput approach that of local disk storage (say, via infiniband), could we not have the various hadoop nodes store their data in the lustre filesystem?  would hadoop even care?

I realize that this may not be a good place to bring it up.  But there you go…

One of these days, (with all of my ample spare time), I will benchmark it.  and report of course…

--jason

From: Jon Yeargers <yeargers at ohsu.edu<mailto:yeargers at ohsu.edu>>
Date: Friday, December 7, 2012 9:26 AM
To: "lustre-discuss at lists.lustre.org<mailto:lustre-discuss at lists.lustre.org>" <lustre-discuss at lists.lustre.org<mailto:lustre-discuss at lists.lustre.org>>
Subject: [Lustre-discuss] Applications of Lustre - streaming?

Can Lustre be used to store data like streaming audio / video? I’ve been scolded about considering it for DB storage but I’m looking at the relative merits of Lustre vs HDFS.

I’m moving to a clustered DB setup and wondering about Cassandra / Lustre vs Hadoop (IE HBase / HDFS). One offers flexibility in terms of mixing hardware components while the other is a ‘one stop shop’.

Not trying to elicit a religious war – and yes, I’ve been reading as much as I can find about this. Just hoping for the opinion(s) of this side of the table.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20121207/31acfb8b/attachment.htm>