[Lustre-discuss] Understanding lustre setup ..

[Colin Faber]

Hi,

In general the clients are isolated nodes which only act as single 
client nodes accessing the greater file system (MGS, MDS, and OSS (OSTS))

You can run a client mount on any of these nodes however it's not 
recommended as it can lead to deadlock and memory contention problems.

Ideally your hadoop data source from lustre would be a lustre client. 
This can consist of a network boot machine which then mounts Lustre FS 
via it's interconnect.


So think of it like this:

(LUSTRE: [MGS <-> MDS] <-> [OSS { OST OST OST }]) <-> Client A (mount 
/mnt/lustre)
                |______________________________________^

Then Client A acts as data store exporting /mnt/lustre to your hadoop 
cluster.

I hope this makes sense =)

-cf



On 03/11/2013 11:33 AM, linux freaker wrote:
> Hello,
>
> I have been reading
> http://wiki.lustre.org/images/1/1b/Hadoop_wp_v0.4.2.pdf for setting up
> Hadoop over lustre.
> Generally in hadoop setup, we have 1 Namenode and various number of datanodes.
> If I want to setup the same keeping Lustre as backend, in the document
> it is mentioned that:
>
> ".............Our experiments run on cluster with 8 nodes in total,
> one is mds/namenode, the rest are
> OSS/DataNode".
>
> I wonder where does the Lustre Client fit here?
>
> For Hadoop to work , we mention filesystem parameter esp /lustre here.
> We dont have /lustre on OSS.
> How is it possible?
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-discuss