[Lustre-discuss] small/inexpensive cluster design

Andrew Uselton acuselton at lbl.gov
Thu Apr 21 10:52:03 PDT 2011


Howdy all,
  I had a question from a colleague and did not have a ready answer.
What is the community's experience with putting together a small and
inexpensive cluster
that serves Lustre from (some of) the compute nodes's local disks?

They have run some simple tests with using a) just local disc, b) simple NFS
service
mounted to compute nodes, and c) Lustre with OSS and MDS on the same node.
A typical workload for them is to compile the "Visit" visualization package.
On a local disk
this takes 2 to 3 hours. On NFS it was closer to 24 hours, and on the small
Lustre example
it was about 5 hours. Now they'd like to go a little further and try to find
a Lustre solution that
would improve performance as compared to local disk. Their workload will be
mostly metadata intensive
rather than bulk I/O intensive. Is there any experience like that out there?
Cheers,
Andrew

Notes from the one asking the question:
----------------------------------------------------------
 What I would like to do now is to develop the cheapest
 small cluster possible that still has good I/O
 performance. NetAps raise the cost significantly. Also, I

 think the whole system must come out of the box with the
 application and all dependencies built and good I/O.


 So one possible way would be a system with a head node and
 N compute nodes, each with multiple CPUs and cores, of
 course. I can then imagine a Lustre file system with the
 MDS on the head node and perhaps M OSSs on the compute
 nodes, which then serve up their local disks. Of course,
 now the compute nodes are running both the computational
 application (on all cores likely) and 0 or 1 OSS.

 It sounds like from what you are saying that at a minimum
 I would need two interfaces per node: one over which the
 MPI communication goes for the apps, and one for serving
 the Lustre file system on those nodes which are serving
 that. Is this right?

 Is this a reasonable direction to go?  (Having both OSS and
 computation on some nodes.)

 Are there examples of good systems designs out there?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20110421/20ab476e/attachment.htm>


More information about the lustre-discuss mailing list