[Lustre-discuss] Lnet configuration: 1 ost per gige interface.

[Hendelman, Rob]

Hello everyone,

I'd like to dedicate 1 gigabit Ethernet interface per OST.  What is the
way to do this?  My OST's are fairly slow (raid 10, 4 sata drives ea) by
enterprise standards but still can sustain 80-90MByte/second on large
sequential reads.  I imagine if I striped a file over 3 OST's I would
hit speed limitations using GigE.

What I'd like to do is have multiple gige nics connected to a (dumb)
switch to start with.  I'd put each gige nic in a separate /28.  My
first OSS box has 3 OST's (12 drives) and 3 gig-e nics.  My MDS/MGS has
a single gige nic.

I have assigned 3 ip's (on different /28's) in linux to the gig-e
interface on the mds/mgs box, and 1 ip (also on different /28's) to each
interface on the oss server.  The client (only 1 to start) is also going
to have 3 gige nics, with 1 ip on each subnet.

First off, what does modules.conf look like on the mgs/mds?  Should it
be something like:

Options lnet networks=tcp0(eth0),tcp1(eth0:0),tcp2(eth0:1)

Should the modules.conf on the oss/clients be something like:

Options lnet networks=tcp0(eth0),tcp1(eth1),tcp2(eth2)?

Can I then format each ost with the mgs/mds parameter pointing to tcp0,
tcp1, or tcp2 to force it to use a specific gige interface?

I have been reading through the Lustre 1.6 operations manual but have
not gained a clear understanding of exactly how to set this up so it
works properly.

I am running Lustre 1.6.5.1 on centos 5.1 with the rpm's provided by the
Sun website.

Thanks,

Robert

Robert Hendelman Jr
Magnetar Capital LLC
Rob.Hendelman at magnetar.com
1-847-905-4557



The information contained in this message and its attachments 
is intended only for the private and confidential use of the 
intended recipient(s).  If you are not the intended recipient 
(or have received this e-mail in error) please notify the 
sender immediately and destroy this e-mail. Any unauthorized 
copying, disclosure or distribution of the material in this e-
mail is strictly prohibited.