<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv=Content-Type content="text/html; charset=us-ascii">
<meta name=Generator content="Microsoft Word 12 (filtered medium)">
<style>
<!--
/* Font Definitions */
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0in;
margin-right:0in;
margin-bottom:0in;
margin-left:.5in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;}
@page Section1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.Section1
{page:Section1;}
/* List Definitions */
@list l0
{mso-list-id:672683653;
mso-list-type:hybrid;
mso-list-template-ids:-795966976 67698689 67698691 67698693 67698689 67698691 67698693 67698689 67698691 67698693;}
@list l0:level1
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Symbol;}
ol
{margin-bottom:0in;}
ul
{margin-bottom:0in;}
-->
</style>
<!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang=EN-US link=blue vlink=purple>
<div class=Section1>
<p class=MsoNormal><span style='font-size:14.0pt;color:#1F497D'>Hi all,<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:14.0pt;color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:14.0pt;color:#1F497D'>I was
experimenting with 1.8.2 patchless client and bump into a problem with using
Infiniband.<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:14.0pt;color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoListParagraph style='text-indent:-.25in;mso-list:l0 level1 lfo1'><![if !supportLists]><span
style='font-size:14.0pt;font-family:Symbol;color:#1F497D'><span
style='mso-list:Ignore'>·<span style='font:7.0pt "Times New Roman"'>
</span></span></span><![endif]><span style='font-size:14.0pt;color:#1F497D'>Server
kickstarted with RHEL 5.3<o:p></o:p></span></p>
<p class=MsoListParagraph style='text-indent:-.25in;mso-list:l0 level1 lfo1'><![if !supportLists]><span
style='font-size:14.0pt;font-family:Symbol;color:#1F497D'><span
style='mso-list:Ignore'>·<span style='font:7.0pt "Times New Roman"'>
</span></span></span><![endif]><span style='font-size:14.0pt;color:#1F497D'>Installed
lustre-client-1.8.2-2.6.18_164.11.1.el5_lustre.1.8.2.x86_64.rpm<o:p></o:p></span></p>
<p class=MsoListParagraph style='text-indent:-.25in;mso-list:l0 level1 lfo1'><![if !supportLists]><span
style='font-size:14.0pt;font-family:Symbol;color:#1F497D'><span
style='mso-list:Ignore'>·<span style='font:7.0pt "Times New Roman"'>
</span></span></span><![endif]><span style='font-size:14.0pt;color:#1F497D'>Installed
lustre-client-modules-1.8.2-2.6.18_164.11.1.el5_lustre.1.8.2.x86_64.rpm<o:p></o:p></span></p>
<p class=MsoListParagraph style='text-indent:-.25in;mso-list:l0 level1 lfo1'><![if !supportLists]><span
style='font-size:14.0pt;font-family:Symbol;color:#1F497D'><span
style='mso-list:Ignore'>·<span style='font:7.0pt "Times New Roman"'>
</span></span></span><![endif]><span style='font-size:14.0pt;color:#1F497D'>Performed
“chkconfig –level 2345 openibd on” so ib0 is working<o:p></o:p></span></p>
<p class=MsoListParagraph style='text-indent:-.25in;mso-list:l0 level1 lfo1'><![if !supportLists]><span
style='font-size:14.0pt;font-family:Symbol;color:#1F497D'><span
style='mso-list:Ignore'>·<span style='font:7.0pt "Times New Roman"'>
</span></span></span><![endif]><span style='font-size:14.0pt;color:#1F497D'>Performed
“rsync –alP /lib/modules/2.6.18-164.11.1.el5 /lib/modules/2.6.18-128.el5<o:p></o:p></span></p>
<p class=MsoListParagraph style='text-indent:-.25in;mso-list:l0 level1 lfo1'><![if !supportLists]><span
style='font-size:14.0pt;font-family:Symbol;color:#1F497D'><span
style='mso-list:Ignore'>·<span style='font:7.0pt "Times New Roman"'>
</span></span></span><![endif]><span style='font-size:14.0pt;color:#1F497D'>Performed
“depmod –a”<o:p></o:p></span></p>
<p class=MsoListParagraph style='text-indent:-.25in;mso-list:l0 level1 lfo1'><![if !supportLists]><span
style='font-size:14.0pt;font-family:Symbol;color:#1F497D'><span
style='mso-list:Ignore'>·<span style='font:7.0pt "Times New Roman"'>
</span></span></span><![endif]><span style='font-size:14.0pt;color:#1F497D'>/etc/modprobe.conf
has “options lnet networks=o2ib0(ib0),tcp1(eth2),tcp2(eth3)”<o:p></o:p></span></p>
<p class=MsoListParagraph style='text-indent:-.25in;mso-list:l0 level1 lfo1'><![if !supportLists]><span
style='font-size:14.0pt;font-family:Symbol;color:#1F497D'><span
style='mso-list:Ignore'>·<span style='font:7.0pt "Times New Roman"'>
</span></span></span><![endif]><span style='font-size:14.0pt;color:#1F497D'>/etc/fstab
has 10.103.34.42@o2ib0:/spfs
/lustre1_fifo lustre
rw,noauto,_netdev 0 0<o:p></o:p></span></p>
<p class=MsoListParagraph style='text-indent:-.25in;mso-list:l0 level1 lfo1'><![if !supportLists]><span
style='font-size:14.0pt;font-family:Symbol;color:#1F497D'><span
style='mso-list:Ignore'>·<span style='font:7.0pt "Times New Roman"'>
</span></span></span><![endif]><span style='font-size:14.0pt;color:#1F497D'>Performed
“modprobe lnet”<o:p></o:p></span></p>
<p class=MsoListParagraph style='text-indent:-.25in;mso-list:l0 level1 lfo1'><![if !supportLists]><span
style='font-size:14.0pt;font-family:Symbol;color:#1F497D'><span
style='mso-list:Ignore'>·<span style='font:7.0pt "Times New Roman"'>
</span></span></span><![endif]><span style='font-size:14.0pt;color:#1F497D'>Performed
“lctl net up” with error - LNET configure error 100: Network is
down<o:p></o:p></span></p>
<p class=MsoListParagraph style='text-indent:-.25in;mso-list:l0 level1 lfo1'><![if !supportLists]><span
style='font-size:14.0pt;font-family:Symbol;color:#1F497D'><span
style='mso-list:Ignore'>·<span style='font:7.0pt "Times New Roman"'>
</span></span></span><![endif]><span style='font-size:14.0pt;color:#1F497D'>bundle
of errors on /var/log/messages, portion of the tail message:<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:14.0pt;color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:14.0pt;color:#1F497D'>Feb 16 16:20:43
bg8mo33sn kernel: ko2iblnd: disagrees about version of symbol
ib_destroy_fmr_pool<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:14.0pt;color:#1F497D'>Feb 16 16:20:43
bg8mo33sn kernel: ko2iblnd: Unknown symbol ib_destroy_fmr_pool<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:14.0pt;color:#1F497D'>Feb 16 16:20:43
bg8mo33sn modprobe: FATAL: Error inserting ko2iblnd (/lib/modules/2.6.18-128.el5/2.6.18-164.11.1.el5/kernel/net/lustre/ko2iblnd.ko):
Unknown symbol in module, or unknown parameter (see dmesg)<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:14.0pt;color:#1F497D'>Feb 16 16:20:43
bg8mo33sn kernel: ko2iblnd: disagrees about version of symbol rdma_destroy_id<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:14.0pt;color:#1F497D'>Feb 16 16:20:44
bg8mo33sn kernel: ko2iblnd: Unknown symbol rdma_destroy_id<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:14.0pt;color:#1F497D'>Feb 16 16:20:44
bg8mo33sn kernel: ko2iblnd: disagrees about version of symbol rdma_accept<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:14.0pt;color:#1F497D'>Feb 16 16:20:44
bg8mo33sn kernel: ko2iblnd: Unknown symbol rdma_accept<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:14.0pt;color:#1F497D'>Feb 16 16:20:44
bg8mo33sn kernel: ko2iblnd: disagrees about version of symbol ib_dealloc_pd<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:14.0pt;color:#1F497D'>Feb 16 16:20:44
bg8mo33sn kernel: ko2iblnd: Unknown symbol ib_dealloc_pd<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:14.0pt;color:#1F497D'>Feb 16 16:20:44
bg8mo33sn kernel: ko2iblnd: disagrees about version of symbol
ib_fmr_pool_map_phys<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:14.0pt;color:#1F497D'>Feb 16 16:20:44
bg8mo33sn kernel: ko2iblnd: Unknown symbol ib_fmr_pool_map_phys<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:14.0pt;color:#1F497D'>Feb 16 16:20:44
bg8mo33sn kernel: LustreError: 4768:0:(api-ni.c:1043:lnet_startup_lndnis())
Can't load LND o2ib, module ko2iblnd, rc=256<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:14.0pt;color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoListParagraph style='text-indent:-.25in;mso-list:l0 level1 lfo1'><![if !supportLists]><span
style='font-size:14.0pt;font-family:Symbol;color:#1F497D'><span
style='mso-list:Ignore'>·<span style='font:7.0pt "Times New Roman"'>
</span></span></span><![endif]><span style='font-size:14.0pt;color:#1F497D'>In
1.8.1.1 there was kernel-ib-1.4.2-2.6.18_128.7.1.el5.x86_64.rpm. No problem
with 1.8.1.1.<o:p></o:p></span></p>
<p class=MsoListParagraph style='text-indent:-.25in;mso-list:l0 level1 lfo1'><![if !supportLists]><span
style='font-size:14.0pt;font-family:Symbol;color:#1F497D'><span
style='mso-list:Ignore'>·<span style='font:7.0pt "Times New Roman"'>
</span></span></span><![endif]><span style='font-size:14.0pt;color:#1F497D'>In
1.8.2 there is no specific RPM to handle IB? Using stocked openib from RHEL
worked for patched kernel installation. <o:p></o:p></span></p>
<p class=MsoListParagraph style='text-indent:-.25in;mso-list:l0 level1 lfo1'><![if !supportLists]><span
style='font-size:14.0pt;font-family:Symbol;color:#1F497D'><span
style='mso-list:Ignore'>·<span style='font:7.0pt "Times New Roman"'>
</span></span></span><![endif]><span style='font-size:14.0pt;color:#1F497D'>Are
there any other tricks to make this work with patchless client?<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:14.0pt;color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:14.0pt;color:#1F497D'>Thanks in
advance…<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:14.0pt;color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoNormal><b><i><span style='font-size:20.0pt;color:#1F497D'>Steve<o:p></o:p></span></i></b></p>
<p class=MsoNormal><span style='color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoNormal><b><i><span style='font-size:16.0pt;color:#1F497D'>Stephen
Chu<o:p></o:p></span></i></b></p>
<p class=MsoNormal><b><span style='font-size:12.0pt;color:#1F497D'>AT&T
Labs CSO<o:p></o:p></span></b></p>
<p class=MsoNormal><span style='color:#1F497D'>C5-3C03<o:p></o:p></span></p>
<p class=MsoNormal><span style='color:#1F497D'>200 Laurel Ave<o:p></o:p></span></p>
<p class=MsoNormal><span style='color:#1F497D'>Middletown, NJ<o:p></o:p></span></p>
<p class=MsoNormal><span style='color:#1F497D'>stephenchu@att.com<o:p></o:p></span></p>
<p class=MsoNormal><o:p> </o:p></p>
</div>
</body>
</html>