<HTML xmlns="http://www.w3.org/TR/REC-html40" xmlns:v =
"urn:schemas-microsoft-com:vml" xmlns:o =
"urn:schemas-microsoft-com:office:office" xmlns:w =
"urn:schemas-microsoft-com:office:word" xmlns:m =
"http://schemas.microsoft.com/office/2004/12/omml" xmlns:mv =
"http://macVmlSchemaUri"><HEAD>
<META content="text/html; charset=utf-8" http-equiv=Content-Type>
<META name=Title content="">
<META name=Keywords content="">
<META name=Generator content="Microsoft Word 15 (filtered medium)">
<STYLE>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</STYLE>
<STYLE><!--
/* Font Definitions */
@font-face
{font-family:Verdana;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p
{mso-style-priority:99;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.EmailStyle17
{mso-style-type:personal;
font-family:"Calibri",sans-serif;
color:#365F91;
font-weight:normal;
font-style:normal;}
span.EmailStyle20
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
span.msoIns
{mso-style-type:export-only;
mso-style-name:"";
text-decoration:underline;
color:teal;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></STYLE>
</HEAD>
<BODY lang=EN-US dir=ltr vLink=purple link=blue bgColor=white>
<DIV dir=ltr>
<DIV style="FONT-SIZE: 12pt; FONT-FAMILY: 'Calibri'; COLOR: #000000">
<DIV>Hi Chris</DIV>
<DIV> </DIV>
<DIV>I installed RHEL 6.7, MLNX_OFED_LINUX-3.4-1.0.0.0-rhel6.7-x86_64 and then
Lustre 2.8 in my Lustre MDS/MGT/OSS servers.</DIV>
<DIV> </DIV>
<DIV>My ib0 is working fine and I can ping other nodes.</DIV>
<DIV>my lustre.conf file has</DIV>
<DIV>options lnet networks=o2ib(ib0)</DIV>
<DIV> </DIV>
<DIV>With this, If I run “service lnet start” it fails with error </DIV>
<DIV>LNET configure error 22: Invalid argument</DIV>
<DIV> </DIV>
<DIV>dmesg give me output as below (I just captured last line but there are many
lines with symbol error or so)</DIV>
<DIV> </DIV>
<DIV>LNetError: 16770:0:(api-ni.c:1276:lnet_startup_lndni()) Can't load LND
o2ib, module ko2iblnd, rc=256</DIV>
<DIV> </DIV>
<DIV style="FONT-FAMILY: ; COLOR: ">
<DIV dir=ltr>
<DIV style="FONT-FAMILY: ; COLOR: ">
<DIV>If I specify tcp in lustre.conf,, it works fine.</DIV>
<DIV> </DIV>
<DIV>I have reinstalled Lustre and then Mellanox OFED driver but still the
problem is same, not able to make infiniband up with Lustre
Lnet</DIV></DIV></DIV></DIV>
<DIV> </DIV>
<DIV style="FONT-SIZE: 12pt; FONT-FAMILY: 'Calibri'; COLOR: #000000">
<DIV dir=ltr>
<DIV style="FONT-SIZE: 12pt; FONT-FAMILY: 'Calibri'; COLOR: #000000">
<DIV><FONT size=2 face=Verdana>Regards</FONT></DIV>
<DIV style="FONT-SIZE: 12pt; FONT-FAMILY: 'Calibri'; COLOR: #000000">
<DIV dir=ltr>
<DIV style="FONT-SIZE: 12pt; FONT-FAMILY: 'Calibri'; COLOR: #000000">
<DIV style="FONT-SIZE: 12pt; FONT-FAMILY: 'Calibri'; COLOR: #000000">
<DIV dir=ltr>
<DIV style="FONT-SIZE: 12pt; FONT-FAMILY: 'Calibri'; COLOR: #000000">
<DIV style="FONT-SIZE: 12pt; FONT-FAMILY: 'Calibri'; COLOR: #000000">
<DIV dir=ltr>
<DIV style="FONT-SIZE: 12pt; FONT-FAMILY: 'Calibri'; COLOR: #000000">
<DIV style="FONT-SIZE: 12pt; FONT-FAMILY: 'Calibri'; COLOR: #000000">
<DIV dir=ltr>
<DIV style="FONT-SIZE: 12pt; FONT-FAMILY: 'Calibri'; COLOR: #000000">
<DIV> </DIV>
<DIV><FONT size=2 face=Verdana><STRONG>Ravi Konila</STRONG></FONT></DIV>
<DIV><FONT size=2 face=Verdana>Sr. Technical Consultant</FONT></DIV>
<DIV> </DIV>
<DIV> </DIV></DIV></DIV></DIV></DIV></DIV></DIV></DIV></DIV></DIV></DIV></DIV></DIV></DIV></DIV></DIV>
<DIV
style='FONT-SIZE: small; TEXT-DECORATION: none; FONT-FAMILY: "Calibri"; FONT-WEIGHT: normal; COLOR: #000000; FONT-STYLE: normal; DISPLAY: inline'>
<DIV style="FONT: 10pt tahoma">
<DIV><FONT size=3 face=Calibri></FONT> </DIV>
<DIV style="BACKGROUND: #f5f5f5">
<DIV style="font-color: black"><B>From:</B> <A title=hornc@cray.com>Chris
Horn</A> </DIV>
<DIV><B>Sent:</B> Thursday, October 12, 2017 9:02 PM</DIV>
<DIV><B>To:</B> <A title=ravibhatk@gmail.com>Ravi Konila</A> ; <A
title=parag_k@citilindia.com>Parag Khuraswar</A> ; <A
title=lustre-discuss@lists.lustre.org>'Lustre User Discussion Mailing List'</A>
</DIV>
<DIV><B>Subject:</B> Re: [lustre-discuss] Lnet not starting</DIV></DIV></DIV>
<DIV> </DIV></DIV>
<DIV
style='FONT-SIZE: small; TEXT-DECORATION: none; FONT-FAMILY: "Calibri"; FONT-WEIGHT: normal; COLOR: #000000; FONT-STYLE: normal; DISPLAY: inline'>
<DIV class=WordSection1>
<P class=MsoNormal>dmesg output should provide more information about the
“Invalid argument” error that you are seeing, but my guess would be that Lustre
was compiled against a different IB stack than what you have
installed.<o:p></o:p></P>
<P class=MsoNormal><o:p></o:p> </P>
<P class=MsoNormal>Chris Horn<o:p></o:p></P>
<P class=MsoNormal><o:p></o:p> </P>
<DIV
style="BORDER-TOP: #b5c4df 1pt solid; BORDER-RIGHT: medium none; BORDER-BOTTOM: medium none; PADDING-BOTTOM: 0in; PADDING-TOP: 3pt; PADDING-LEFT: 0in; BORDER-LEFT: medium none; PADDING-RIGHT: 0in">
<P class=MsoNormal><B><SPAN style="FONT-SIZE: 12pt; COLOR: black">From:
</SPAN></B><SPAN style="FONT-SIZE: 12pt; COLOR: black">lustre-discuss
<lustre-discuss-bounces@lists.lustre.org> on behalf of Ravi Konila
<ravibhatk@gmail.com><BR><B>Reply-To: </B>Ravi Konila
<ravibhatk@gmail.com><BR><B>Date: </B>Thursday, October 12, 2017 at 7:58
AM<BR><B>To: </B>Parag Khuraswar <parag_k@citilindia.com>, 'Lustre User
Discussion Mailing List' <lustre-discuss@lists.lustre.org><BR><B>Subject:
</B>Re: [lustre-discuss] Lnet not starting<o:p></o:p></SPAN></P></DIV>
<DIV>
<P class=MsoNormal><o:p></o:p> </P></DIV>
<DIV>
<DIV>
<DIV>
<P class=MsoNormal><SPAN style="FONT-SIZE: 12pt; COLOR: black">Hi
Parag<o:p></o:p></SPAN></P></DIV>
<DIV>
<P class=MsoNormal><SPAN
style="FONT-SIZE: 12pt; COLOR: black"> <o:p></o:p></SPAN></P></DIV>
<DIV>
<P class=MsoNormal><SPAN style="FONT-SIZE: 12pt; COLOR: black">Even I am facing
the same issue with RHEL 6.7 and Lustre 2.8. Now I am also trying with RHEL 7.3
and Lustre 2.10.0.<o:p></o:p></SPAN></P></DIV>
<DIV>
<P class=MsoNormal><SPAN style="FONT-SIZE: 12pt; COLOR: black">I am planning to
install Mellanox OFED driver with 3.4 stack. Looks like there is some problem
with OFED 4.x stack with Lustre 2.10.0.<o:p></o:p></SPAN></P></DIV>
<DIV>
<P class=MsoNormal><SPAN style="FONT-SIZE: 12pt; COLOR: black">Let me try the
same and update. <o:p></o:p></SPAN></P></DIV>
<DIV>
<DIV>
<DIV>
<DIV>
<DIV>
<DIV>
<DIV>
<DIV>
<DIV>
<DIV>
<DIV>
<DIV>
<DIV>
<DIV>
<DIV>
<DIV>
<P class=MsoNormal><SPAN style="FONT-SIZE: 12pt; COLOR: black">When I start
“service lnet start” it gives LNET configure error 22: Invalid
argument<o:p></o:p></SPAN></P></DIV>
<DIV>
<P class=MsoNormal><SPAN style="FONT-SIZE: 12pt; COLOR: black">but it works fine
with tcp.<o:p></o:p></SPAN></P></DIV>
<DIV>
<P class=MsoNormal><SPAN
style="FONT-SIZE: 12pt; COLOR: black"> <o:p></o:p></SPAN></P></DIV>
<DIV>
<P class=MsoNormal><SPAN
style='FONT-SIZE: 10pt; FONT-FAMILY: "Verdana",sans-serif; COLOR: black'>Regards</SPAN><SPAN
style="FONT-SIZE: 12pt; COLOR: black"><o:p></o:p></SPAN></P></DIV>
<DIV>
<P class=MsoNormal><SPAN
style="FONT-SIZE: 12pt; COLOR: black"> <o:p></o:p></SPAN></P></DIV>
<DIV>
<P class=MsoNormal><STRONG><SPAN
style='FONT-SIZE: 10pt; FONT-FAMILY: "Verdana",sans-serif; COLOR: black'>Ravi
Konila</SPAN></STRONG><SPAN
style="FONT-SIZE: 12pt; COLOR: black"><o:p></o:p></SPAN></P></DIV>
<DIV>
<P class=MsoNormal><SPAN
style='FONT-SIZE: 10pt; FONT-FAMILY: "Verdana",sans-serif; COLOR: black'>Sr.
Technical Consultant</SPAN><SPAN
style="FONT-SIZE: 12pt; COLOR: black"><o:p></o:p></SPAN></P></DIV>
<DIV>
<P class=MsoNormal><SPAN style="FONT-SIZE: 12pt; COLOR: black">Maruti Suzuki
India Ltd<o:p></o:p></SPAN></P></DIV>
<DIV>
<P class=MsoNormal><SPAN
style="FONT-SIZE: 12pt; COLOR: black"> <o:p></o:p></SPAN></P></DIV></DIV></DIV></DIV></DIV></DIV></DIV></DIV></DIV></DIV></DIV></DIV></DIV></DIV></DIV></DIV>
<DIV>
<DIV>
<DIV>
<P class=MsoNormal><SPAN
style='FONT-SIZE: 10pt; FONT-FAMILY: "tahoma",serif; COLOR: black'> <o:p></o:p></SPAN></P></DIV>
<DIV>
<DIV>
<P class=MsoNormal style="BACKGROUND: whitesmoke"><B><SPAN
style='FONT-SIZE: 10pt; FONT-FAMILY: "tahoma",serif; COLOR: black'>From:</SPAN></B><SPAN
style='FONT-SIZE: 10pt; FONT-FAMILY: "tahoma",serif; COLOR: black'> Parag
Khuraswar <o:p></o:p></SPAN></P></DIV>
<DIV>
<P class=MsoNormal style="BACKGROUND: whitesmoke"><B><SPAN
style='FONT-SIZE: 10pt; FONT-FAMILY: "tahoma",serif; COLOR: black'>Sent:</SPAN></B><SPAN
style='FONT-SIZE: 10pt; FONT-FAMILY: "tahoma",serif; COLOR: black'> Thursday,
October 12, 2017 6:11 PM<o:p></o:p></SPAN></P></DIV>
<DIV>
<P class=MsoNormal style="BACKGROUND: whitesmoke"><B><SPAN
style='FONT-SIZE: 10pt; FONT-FAMILY: "tahoma",serif; COLOR: black'>To:</SPAN></B><SPAN
style='FONT-SIZE: 10pt; FONT-FAMILY: "tahoma",serif; COLOR: black'> 'Lustre User
Discussion Mailing List' <o:p></o:p></SPAN></P></DIV>
<DIV>
<P class=MsoNormal style="BACKGROUND: whitesmoke"><B><SPAN
style='FONT-SIZE: 10pt; FONT-FAMILY: "tahoma",serif; COLOR: black'>Subject:</SPAN></B><SPAN
style='FONT-SIZE: 10pt; FONT-FAMILY: "tahoma",serif; COLOR: black'>
[lustre-discuss] Lnet not starting<o:p></o:p></SPAN></P></DIV></DIV></DIV>
<DIV>
<P class=MsoNormal><SPAN
style="FONT-SIZE: 12pt; COLOR: black"> <o:p></o:p></SPAN></P></DIV></DIV>
<DIV>
<P class=MsoNormal><SPAN style="FONT-SIZE: 12pt; COLOR: #365f91">Hi,</SPAN><SPAN
style="COLOR: black"><o:p></o:p></SPAN></P>
<P class=MsoNormal><SPAN style="COLOR: black"> <o:p></o:p></SPAN></P>
<P class=MsoNormal><SPAN style="FONT-SIZE: 12pt; COLOR: #365f91">I am installing
Lustre 2.10.0 on RHEL 7.3. </SPAN><SPAN
style="COLOR: black"><o:p></o:p></SPAN></P>
<P class=MsoNormal><SPAN style="FONT-SIZE: 12pt; COLOR: #365f91">IB is working
fine but lnet is not coming up. Lustre service is running.</SPAN><SPAN
style="COLOR: black"><o:p></o:p></SPAN></P>
<P class=MsoNormal><SPAN style="FONT-SIZE: 12pt; COLOR: #365f91">Ibstat also
show link up and active.</SPAN><SPAN style="COLOR: black"><o:p></o:p></SPAN></P>
<P class=MsoNormal><SPAN style="FONT-SIZE: 12pt; COLOR: #365f91">Lustre and lnet
modules are also loaded.</SPAN><SPAN style="COLOR: black"><o:p></o:p></SPAN></P>
<P class=MsoNormal><SPAN style="COLOR: black"> <o:p></o:p></SPAN></P>
<P class=MsoNormal><SPAN
style="FONT-SIZE: 12pt; COLOR: #365f91">Regards,</SPAN><SPAN
style="COLOR: black"><o:p></o:p></SPAN></P>
<P class=MsoNormal><SPAN
style="FONT-SIZE: 12pt; COLOR: #365f91">Parag</SPAN><SPAN
style="COLOR: black"><o:p></o:p></SPAN></P>
<P class=MsoNormal><SPAN style="COLOR: black"> <o:p></o:p></SPAN></P>
<P class=MsoNormal><SPAN style="COLOR: black"> <o:p></o:p></SPAN></P>
<DIV class=MsoNormal style="TEXT-ALIGN: center" align=center><SPAN
style="FONT-SIZE: 12pt; COLOR: black">
<HR align=center SIZE=2 width="100%">
</SPAN></DIV>
<P class=MsoNormal><SPAN
style="FONT-SIZE: 12pt; COLOR: black">_______________________________________________<BR>lustre-discuss
mailing
list<BR>lustre-discuss@lists.lustre.org<BR>http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org<o:p></o:p></SPAN></P></DIV></DIV></DIV></DIV></DIV></DIV></DIV></BODY></HTML>