[Lustre-discuss] How do you make an MGS/OSS listen on 2 NICs?

Lundgren, Andrew Andrew.Lundgren at Level3.com
Tue Jan 15 12:35:52 PST 2008


We actually set lustre up to run on some non-routable aliased IP addresses, so we wanted to limit it two two specific NIC cards.  I modified my line to look like yours, where I specify tcp1 for the second interface and now it is pingable there, but I am not positive it is correct.  When I didn't have any lnet options, it just grabbed the 1st NIC it saw and ignored the rest.

--
Andrew

________________________________
From: Klaus Steden [mailto:klaus.steden at thomson.net]
Sent: Tuesday, January 15, 2008 11:50 AM
To: Lundgren, Andrew; 'Lustre-discuss at clusterfs.com'
Subject: Re: [Lustre-discuss] How do you make an MGS/OSS listen on 2 NICs?


Try putting quotes around the argument to the 'networks=' statement. If you've got only eth0 and eth1 in your system, you don't need to specify them explicitly, either, Lustre will automatically bind all available interfaces.

Here's what my config looks like:

-- cut --
[root at mds-0-0 ~]# lctl list_nids
172.16.129.252 at tcp
172.16.128.252 at tcp1
root at mds-0-0 ~]# lctl ping 172.16.128.250 at tcp1
12345-0 at lo
12345-172.16.129.250 at tcp
12345-172.16.128.250 at tcp1
[root at mds-0-0 ~]# lctl ping 172.16.128.250 at tcp0
^C
[root at mds-0-0 ~]# lctl ping 172.16.129.250 at tcp0
12345-0 at lo
12345-172.16.129.250 at tcp
12345-172.16.128.250 at tcp1
[root at mds-0-0 ~]# lctl ping 172.16.129.250 at tcp1
^C
[root at mds-0-0 ~]# grep lnet /etc/modprobe.conf
options lnet networks="tcp0(eth0),tcp1(bond0)"
-- cut --

hth,
Klaus

On 1/15/08 9:28 AM, "Lundgren, Andrew" <Andrew.Lundgren at Level3.com>did etch on stone tablets:

I am running on CentOS 5 distribution without adding any updates from CentOS. I am using the lustre 1.6.4.1 kernel and software.

I have two NICs that run though different switches.

I have the lustre options in my modprobe.conf to look like this:

options lnet networks=tcp0(eth1,eth0)

My MGS seems to be only listening on the first interface however.

When I try and ping the 1st interface (eth1) , it works when I go for the second (eth0) it does not.

# lctl ping 192.168.135.81 at tcp
12345-0 at lo
12345-192.168.135.81 at tcp
# lctl ping 192.168.135.80 at tcp
failed to ping 192.168.135.80 at tcp: Input/output error

The following is in /var/log/messages

Jan 15 17:18:15 dint0001 kernel: LustreError: 120-3: Refusing connection from 192.168.135.80 for 192.168.135.80 at tcp <mailto:192.168.135.80 at tcp><mailto:192.168.135.80 at tcp> : No matching NI
Jan 15 17:18:15 dint0001 kernel: LustreError: 3251:0:(socklnd_cb.c:2167:ksocknal_recv_hello()) Error -104 reading HELLO from 192.168.135.80
Jan 15 17:18:15 dint0001 kernel: LustreError: 11b-b: Connection to 192.168.135.80 at tcp <mailto:192.168.135.80 at tcp><mailto:192.168.135.80 at tcp> at host 192.168.135.80 on port 988 was reset: is it running a compatible version of Lustre and is 192.168.135.80 at tcp <mailto:192.168.135.80 at tcp><mailto:192.168.135.80 at tcp> one of its NIDs?

Where/How do I configure it to listen on both devices?

Thank you!

--
Andrew Lundgren




________________________________
_______________________________________________
Lustre-discuss mailing list
Lustre-discuss at clusterfs.com
https://mail.clusterfs.com/mailman/listinfo/lustre-discuss

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20080115/02a7ce57/attachment.htm>


More information about the lustre-discuss mailing list