<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD><TITLE>Re: [Lustre-discuss] How do you make an MGS/OSS listen on 2 NICs?</TITLE>
<META http-equiv=Content-Type content="text/html; charset=us-ascii">
<META content="MSHTML 6.00.6000.16587" name=GENERATOR></HEAD>
<BODY>
<DIV dir=ltr align=left><SPAN class=769003420-15012008><FONT face=Arial
color=#0000ff size=2>We actually set lustre up to run on some non-routable
aliased IP addresses, so we wanted to limit it two two specific NIC cards.
I modified my line to look like yours, where I specify tcp1 for the second
interface and now it is pingable there, but I am not positive it is
correct. When I didn't have any lnet options, it just grabbed the 1st NIC
it saw and ignored the rest.</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=769003420-15012008><FONT face=Arial
color=#0000ff size=2></FONT></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN class=769003420-15012008><FONT face=Arial
color=#0000ff size=2>--</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=769003420-15012008><FONT face=Arial
color=#0000ff size=2>Andrew</FONT></SPAN></DIV><BR>
<BLOCKQUOTE
style="PADDING-LEFT: 5px; MARGIN-LEFT: 5px; BORDER-LEFT: #0000ff 2px solid; MARGIN-RIGHT: 0px">
<DIV class=OutlookMessageHeader lang=en-us dir=ltr align=left>
<HR tabIndex=-1>
<FONT face=Tahoma size=2><B>From:</B> Klaus Steden
[mailto:klaus.steden@thomson.net] <BR><B>Sent:</B> Tuesday, January 15, 2008
11:50 AM<BR><B>To:</B> Lundgren, Andrew;
'Lustre-discuss@clusterfs.com'<BR><B>Subject:</B> Re: [Lustre-discuss] How do
you make an MGS/OSS listen on 2 NICs?<BR></FONT><BR></DIV>
<DIV></DIV><FONT face="Verdana, Helvetica, Arial"><SPAN
style="FONT-SIZE: 12px"><BR>Try putting quotes around the argument to the
‘networks=’ statement. If you’ve got only eth0 and eth1 in your system, you
don’t need to specify them explicitly, either, Lustre will automatically bind
all available interfaces.<BR><BR>Here’s what my config looks like:<BR><BR>--
cut --<BR>[root@mds-0-0 ~]# lctl
list_nids<BR>172.16.129.252@tcp<BR>172.16.128.252@tcp1<BR>root@mds-0-0 ~]#
lctl ping
172.16.128.250@tcp1<BR>12345-0@lo<BR>12345-172.16.129.250@tcp<BR>12345-172.16.128.250@tcp1<BR>[root@mds-0-0
~]# lctl ping 172.16.128.250@tcp0<BR>^C<BR>[root@mds-0-0 ~]# lctl ping
172.16.129.250@tcp0<BR>12345-0@lo<BR>12345-172.16.129.250@tcp<BR>12345-172.16.128.250@tcp1<BR>[root@mds-0-0
~]# lctl ping 172.16.129.250@tcp1<BR>^C<BR>[root@mds-0-0 ~]# grep lnet
/etc/modprobe.conf<BR>options lnet networks="tcp0(eth0),tcp1(bond0)"<BR>-- cut
--<BR><BR>hth,<BR>Klaus<BR><BR>On 1/15/08 9:28 AM, "Lundgren, Andrew"
<Andrew.Lundgren@Level3.com>did etch on stone
tablets:<BR><BR></SPAN></FONT>
<BLOCKQUOTE><SPAN style="FONT-SIZE: 12px"><FONT face=Arial>I am running on
CentOS 5 distribution without adding any updates from CentOS. I am using the
lustre 1.6.4.1 kernel and software.<BR></FONT><FONT
face="Verdana, Helvetica, Arial"><BR></FONT><FONT face=Arial>I have two NICs
that run though different switches. <BR></FONT><FONT
face="Verdana, Helvetica, Arial"><BR></FONT><FONT face=Arial>I have the
lustre options in my modprobe.conf to look like this:<BR></FONT><FONT
face="Verdana, Helvetica, Arial"><BR></FONT><FONT face=Arial>options lnet
networks=tcp0(eth1,eth0)<BR></FONT><FONT
face="Verdana, Helvetica, Arial"><BR></FONT><FONT face=Arial>My MGS seems to
be only listening on the first interface however.<BR></FONT><FONT
face="Verdana, Helvetica, Arial"><BR></FONT><FONT face=Arial>When I try and
ping the 1st interface (eth1) , it works when I go for the second (eth0) it
does not.<BR></FONT><FONT face="Verdana, Helvetica, Arial"><BR></FONT><FONT
face=Arial># lctl ping
192.168.135.81@tcp<BR>12345-0@lo<BR>12345-192.168.135.81@tcp<BR># lctl ping
192.168.135.80@tcp<BR>failed to ping 192.168.135.80@tcp: Input/output
error<BR></FONT><FONT face="Verdana, Helvetica, Arial"><BR></FONT><FONT
face=Arial>The following is in /var/log/messages<BR></FONT><FONT
face="Verdana, Helvetica, Arial"><BR></FONT><FONT face=Arial>Jan 15 17:18:15
dint0001 kernel: LustreError: 120-3: Refusing connection from 192.168.135.80
for 192.168.135.80@tcp</FONT><FONT face="Verdana, Helvetica, Arial"> <A
href="mailto:192.168.135.80@tcp"><mailto:192.168.135.80@tcp></A>
</FONT><FONT face=Arial>: No matching NI<BR>Jan 15 17:18:15 dint0001 kernel:
LustreError: 3251:0:(socklnd_cb.c:2167:ksocknal_recv_hello()) Error -104
reading HELLO from 192.168.135.80<BR>Jan 15 17:18:15 dint0001 kernel:
LustreError: 11b-b: Connection to 192.168.135.80@tcp</FONT><FONT
face="Verdana, Helvetica, Arial"> <A
href="mailto:192.168.135.80@tcp"><mailto:192.168.135.80@tcp></A>
</FONT><FONT face=Arial>at host 192.168.135.80 on port 988 was reset: is it
running a compatible version of Lustre and is 192.168.135.80@tcp</FONT><FONT
face="Verdana, Helvetica, Arial"> <A
href="mailto:192.168.135.80@tcp"><mailto:192.168.135.80@tcp></A>
</FONT><FONT face=Arial>one of its NIDs?<BR></FONT><FONT
face="Verdana, Helvetica, Arial"><BR></FONT><FONT face=Arial>Where/How do I
configure it to listen on both devices?<BR></FONT><FONT
face="Verdana, Helvetica, Arial"><BR></FONT><FONT face=Arial>Thank
you!<BR></FONT><FONT face="Verdana, Helvetica, Arial"><BR></FONT><FONT
face=Arial>--<BR>Andrew Lundgren<BR> <BR></FONT><FONT
face="Verdana, Helvetica, Arial"><BR></FONT><FONT
face=Arial><BR></FONT><FONT face="Verdana, Helvetica, Arial"><BR>
<HR align=center width="95%" SIZE=3>
</FONT></SPAN><FONT size=2><FONT face="Monaco, Courier New"><SPAN
style="FONT-SIZE: 10px">_______________________________________________<BR>Lustre-discuss
mailing
list<BR>Lustre-discuss@clusterfs.com<BR>https://mail.clusterfs.com/mailman/listinfo/lustre-discuss<BR></SPAN></FONT></FONT></BLOCKQUOTE><FONT
size=2><FONT face="Monaco, Courier New"><SPAN
style="FONT-SIZE: 10px"><BR></BLOCKQUOTE></SPAN></FONT></FONT></BODY></HTML>