[Lustre-discuss] How do you make an MGS/OSS listen on 2 NICs?
Lundgren, Andrew
Andrew.Lundgren at Level3.com
Tue Jan 15 12:35:52 PST 2008
We actually set lustre up to run on some non-routable aliased IP addresses, so we wanted to limit it two two specific NIC cards. I modified my line to look like yours, where I specify tcp1 for the second interface and now it is pingable there, but I am not positive it is correct. When I didn't have any lnet options, it just grabbed the 1st NIC it saw and ignored the rest.
--
Andrew
________________________________
From: Klaus Steden [mailto:klaus.steden at thomson.net]
Sent: Tuesday, January 15, 2008 11:50 AM
To: Lundgren, Andrew; 'Lustre-discuss at clusterfs.com'
Subject: Re: [Lustre-discuss] How do you make an MGS/OSS listen on 2 NICs?
Try putting quotes around the argument to the 'networks=' statement. If you've got only eth0 and eth1 in your system, you don't need to specify them explicitly, either, Lustre will automatically bind all available interfaces.
Here's what my config looks like:
-- cut --
[root at mds-0-0 ~]# lctl list_nids
172.16.129.252 at tcp
172.16.128.252 at tcp1
root at mds-0-0 ~]# lctl ping 172.16.128.250 at tcp1
12345-0 at lo
12345-172.16.129.250 at tcp
12345-172.16.128.250 at tcp1
[root at mds-0-0 ~]# lctl ping 172.16.128.250 at tcp0
^C
[root at mds-0-0 ~]# lctl ping 172.16.129.250 at tcp0
12345-0 at lo
12345-172.16.129.250 at tcp
12345-172.16.128.250 at tcp1
[root at mds-0-0 ~]# lctl ping 172.16.129.250 at tcp1
^C
[root at mds-0-0 ~]# grep lnet /etc/modprobe.conf
options lnet networks="tcp0(eth0),tcp1(bond0)"
-- cut --
hth,
Klaus
On 1/15/08 9:28 AM, "Lundgren, Andrew" <Andrew.Lundgren at Level3.com>did etch on stone tablets:
I am running on CentOS 5 distribution without adding any updates from CentOS. I am using the lustre 1.6.4.1 kernel and software.
I have two NICs that run though different switches.
I have the lustre options in my modprobe.conf to look like this:
options lnet networks=tcp0(eth1,eth0)
My MGS seems to be only listening on the first interface however.
When I try and ping the 1st interface (eth1) , it works when I go for the second (eth0) it does not.
# lctl ping 192.168.135.81 at tcp
12345-0 at lo
12345-192.168.135.81 at tcp
# lctl ping 192.168.135.80 at tcp
failed to ping 192.168.135.80 at tcp: Input/output error
The following is in /var/log/messages
Jan 15 17:18:15 dint0001 kernel: LustreError: 120-3: Refusing connection from 192.168.135.80 for 192.168.135.80 at tcp <mailto:192.168.135.80 at tcp><mailto:192.168.135.80 at tcp> : No matching NI
Jan 15 17:18:15 dint0001 kernel: LustreError: 3251:0:(socklnd_cb.c:2167:ksocknal_recv_hello()) Error -104 reading HELLO from 192.168.135.80
Jan 15 17:18:15 dint0001 kernel: LustreError: 11b-b: Connection to 192.168.135.80 at tcp <mailto:192.168.135.80 at tcp><mailto:192.168.135.80 at tcp> at host 192.168.135.80 on port 988 was reset: is it running a compatible version of Lustre and is 192.168.135.80 at tcp <mailto:192.168.135.80 at tcp><mailto:192.168.135.80 at tcp> one of its NIDs?
Where/How do I configure it to listen on both devices?
Thank you!
--
Andrew Lundgren
________________________________
_______________________________________________
Lustre-discuss mailing list
Lustre-discuss at clusterfs.com
https://mail.clusterfs.com/mailman/listinfo/lustre-discuss
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20080115/02a7ce57/attachment.htm>
More information about the lustre-discuss
mailing list