[Lustre-discuss] Lnet configuration: 1 ost per gige interface.

Steden Klaus Klaus.Steden at thomson.net
Tue Oct 21 14:17:31 PDT 2008

Hi Rob,

I'm not sure on this point (but Andreas no doubt will know) but you might be able to get more overall bandwidth with a single logical network interface if you use a different aggregation mode, such as adaptive load balancing, instead of LACP. ALB and TLB modes will still operate on a dumb switch (at least, they operate without a configuration change on intelligent switches).

As long as Lustre doesn't care about the internal details of 'bond0', you should be able to set it up with ALB on a dumb switch and use 'bond0' as your interface.

Just a thought.


-----Original Message-----
From: lustre-discuss-bounces at lists.lustre.org on behalf of Andreas Dilger
Sent: Tue 10/21/2008 11:47 AM
To: Hendelman, Rob
Cc: lustre-discuss at lists.lustre.org
Subject: Re: [Lustre-discuss] Lnet configuration: 1 ost per gige interface.
On Oct 21, 2008  11:41 -0500, Hendelman, Rob wrote:
> I have assigned 3 ip's (on different /28's) in linux to the gig-e
> interface on the mds/mgs box, and 1 ip (also on different /28's) to each
> interface on the oss server.  The client (only 1 to start) is also going
> to have 3 gige nics, with 1 ip on each subnet.
> First off, what does modules.conf look like on the mgs/mds?  Should it
> be something like:
> Options lnet networks=tcp0(eth0),tcp1(eth0:0),tcp2(eth0:1)
> Should the modules.conf on the oss/clients be something like:
> Options lnet networks=tcp0(eth0),tcp1(eth1),tcp2(eth2)?
> Can I then format each ost with the mgs/mds parameter pointing to tcp0,
> tcp1, or tcp2 to force it to use a specific gige interface?

You can use the "ip2nets" option to keep the configuration the same
on all of the nodes, instead of explicitly specifying the mapping.
This allows mapping IP subnets to Lustre networks regardless of what
the interface names are.  This is in section 7.1.1 of the manual.

Cheers, Andreas
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.

Lustre-discuss mailing list
Lustre-discuss at lists.lustre.org

More information about the lustre-discuss mailing list