[Lustre-discuss] Rule of thumb for setting up lustre resources...

[Klaus Steden]

Hi Mark,

See my comments inline below.

cheers,
Klaus

On 6/14/08 11:22 AM, "Mark True" <darfoo at gmail.com>did etch on stone
tablets:

> 
> Hello!
> 
> I am new to the list, but I have been researching Lustre for quite some time
> and finally have an occasion to use it.  I am trying to do some capacity
> planning and I am wondering if there are some general rules of thumb for
> configuring a Lustre environment.
> 
> Specifically:
> 
> A> If increasing the number of OSTs increases throughput, is there a
> relationship that can be used to determine how many OSTs we're likely to need
> at the outset to establish a baseline minimum throughput.  For examples, if I
> want to get 3GB sustained throughput how many OSTs will facilitate this.
> 
> B> Does the MGS and MDS have to be separate for best performance, or can they
> be consolidated into one server without causing too much hardship
> 
>
> C>  Right now I am looking at a model where I am connecting all the OSTs, and
> the MDS/MGS together using infiniband, and connecting the storage via
> fibrechannel.   Is this the ideal solution or am I going in the wrong
> direction.  
>
This is a good solution, and will give you good performance overall,
although you can mix different storage technologies and network technologies
within the same storage environment and it should remain relatively
transparent. I've got a cluster that handles both FC storage and iSCSI
storage, but I know there are people out there using DRBD, and I'm dying to
try Infiniband-based storage as well. Anything that presents a block device
to an OSS should be suitable for use with Lustre, but some will perform
better than others.

Bottom line, I think, is pick the best technology for your price range and
performance needs. Infiniband + FC is pretty much the top of the mountain,
though.
> 
> D> Just wondering what clustering software people use on the front end with
> Lustre typically, if they are going to be using this as a filesystem for some
> kind of HPC environment, what is the most popular clustering technology for
> this.
> 
Our CFS clusters are all organized as part of ROCKS clusters. I know a
number of people on this list are on the ROCKS list, so there's good
cross-pollination between technologies. It's a mature cluster architecture
designed for HPC, and bundles a number of useful solutions and tools onboard
(MPI, SGE, Torque, distributed compilers, visualization, etc.). It's also
relatively easy to integrate with Lustre, as you can simply drop in the
pre-built Lustre RPMs into the cluster installer and be ready to go in a few
minutes.
>
> E> Does Heartbeat install next to whatever HPC clustering technology you have?
>
I'm using Linux-HA, and it wasn't built into my cluster software distro, but
it was easy enough to drop into the mix, and as of late last year had native
disk support for Lustre file systems.
> 
> Thanks, and I hope that I can soon be someone who contributes rather than just
> asking questions :)
> 
> --Mark T.
> 
> 
> 
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-discuss