[lustre-discuss] Lustre with Hadoop (Hortonworks Data Platform)

[Mohr Jr, Richard Frank (Rick Mohr)]

> On Jan 9, 2017, at 4:21 AM, Markham Benjamin <ben.markham at xiilab.com> wrote:
> 
> I was wondering the use cases of using Lustre on Hadoop. One key things about HDFS is that it runs on commodity hardware. Unless I’m being misinformed, Lustre doesn’t exactly run on commodity hardware.

I don’t think you want to use Lustre on Hadoop. You would want to use Hadoop on Lustre (i.e. - Lustre replaces HDFS).

And Lustre does run on commodity hardware.  Sites usually run Lustre on generic x86_64 servers.  I know some that use Dells, and I am using Appro/Cray blades.  Sometimes sites will use more specialized storage hardware (like DDN or NetApp), but that is not required for Lustre.

--
Rick Mohr
Senior HPC System Administrator
National Institute for Computational Sciences
http://www.nics.tennessee.edu