[Lustre-discuss] multihomed OST's configuration

D. Marc Stearman marc at llnl.gov
Tue Jul 8 08:00:51 PDT 2008



On Jul 7, 2008, at 3:13 AM, mdavid wrote:
> [root at se003 ~]# cat /etc/modprobe.conf
> alias eth0 bnx2
> alias eth1 bnx2
> alias eth2 e1000
> alias eth3 e1000
> alias scsi_hostadapter megaraid_sas
> options lnet networks=tcp(eth0,eth1,eth2,eth3)
>
> and mount some of them again without problem, but
> lctl > list_nids
> 10.100.1.51 at tcp
>
> 10.100.1.51 is the ip of the eth1, the others do not show up, I
> suppose they should
>
> or should I use the lctl network commands on the MDS/MDT (both on one
> machine)
> to get all the routes somehow?

You will only have one nid per logical LNET network (tcp0, tcp1,  
o2ib0, o2ib1, etc).  So in your setup, you will only have one nid,  
for tcp0, regardless of how many eth devices you include.  If you did  
something like this:

tcp0(eth0),tcp1(eth1),tcp2(eth2),tcp3(eth3), you would end up with 4  
nids, but that isn't what you want.  It sounds like you have  it  
configured correctly.

If you were to do bonding, you would use the standard linux bonding  
methods to bond eth0-eth3 -> bond0, so your lnet options line would  
look like this:

options lnet networks=tcp(bond0)

and you would still have one nid.

Bonding is great if all your ethernet devices are plugged into a  
single switch that supports bonding.  If your network interfaces are  
on different ethernet switches for redundancy (or because you are  
lacking in ports), you cannot use bonding, and must go mutlihomed.

All of our servers (500+) are multihomed.

-Marc

----
D. Marc Stearman
LC Lustre Administration Lead
marc at llnl.gov
925.423.9670
Pager: 1.888.203.0641




More information about the lustre-discuss mailing list