<div dir="ltr">Hi Parag - I've been away from this field for awhile, but this message may be the most helpful at this point:<div><div><br></div><div><span style="color:rgb(31,73,125);font-family:Calibri,sans-serif;font-size:18.66666603088379px">modprobe: ERROR: could not insert 'ko2iblnd': Invalid argument</span><br></div><div><br></div><div>Last I knew, there were two IB releases - Mellanox/MOFED and Intel (formerly QLogic) - and the Lustre kernel you use needs to be compiled against the IB release that is installed on the node.<br></div><div><br></div><div>So, I see a couple possibilities of why the Lustre IB module will not load:</div><div>1. Since you are using MOFED, the kernel you have may instead support Intel IB (thus the invalid argument).</div><div>2. One of the options provided may not be valid for the module, or may conflict with another provided option:</div><div><br></div><div><span style="color:rgb(31,73,125);font-family:Calibri,sans-serif;font-size:18.66666603088379px">require_privileged_port=0 use_privileged_port=0 timeout=150 retry_count=7 map_on_demand=32 peer_credits=63 concurrent_sends=63 ntx=32768 credits=32768 fmr_pool_size=8193</span><br><div class="gmail_extra"><br clear="all"><div><div class="gmail-m_-7955658491389144023gmail_signature"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div>As item 1 is probably a binary, and item 2 a matrix, confirming that the kernel that you have supports the module that you are trying to load seems to be a better starting point. :)</div><div><br></div><div>As Intel HPDD seems to provide a kernel that matches the version are using, item 1 seems likely.</div><div><a href="https://downloads.hpdd.intel.com/public/lustre/lustre-2.9.0/el7.3.1611/server/RPMS/x86_64/">https://downloads.hpdd.intel.com/public/lustre/lustre-2.9.0/el7.3.1611/server/RPMS/x86_64/</a><br></div><div>If that is the case, a couple options include using the Intel IB, or compiling a Lustre kernel that supports the Mellanox OFED.<br></div><div><br></div><div>Hope this helps.</div><div>Brett</div><div dir="ltr"><div><div></div></div></div></div></div></div></div></div></div></div></div></div></div></div>
<br><div class="gmail_quote">On Sun, Nov 12, 2017 at 1:16 AM, Parag Khuraswar <span dir="ltr"><<a href="mailto:parag_k@citilindia.com" target="_blank">parag_k@citilindia.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div lang="EN-US"><div class="gmail-m_-7955658491389144023gmail-m_-3866864905143472575WordSection1"><p class="MsoNormal"><span style="font-size:14pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)">Hi Brett,<u></u><u></u></span></p><p class="MsoNormal"><span style="font-size:14pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)"><u></u> <u></u></span></p><p class="MsoNormal"><span style="font-size:14pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)">I am using MOFED “MLNX_OFED_LINUX-4.1-1.0.2.0” with kernel “3.10.0-514.el7.x86_64”<u></u><u></u></span></p><p class="MsoNormal"><span style="font-size:14pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)"><u></u> <u></u></span></p><p class="MsoNormal"><span style="font-size:14pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)">o/p of modprobe –v ko2iblnd<u></u><u></u></span></p><p class="MsoNormal"><span style="font-size:14pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)"><u></u> <u></u></span></p><p class="MsoNormal"><span style="font-size:14pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)">[root@mds1 ~]# modprobe -v ko2iblnd<u></u><u></u></span></p><p class="MsoNormal"><span style="font-size:14pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)">install /usr/sbin/ko2iblnd-probe require_privileged_port=0 use_privileged_port=0 timeout=150 retry_count=7 map_on_demand=32 peer_credits=63 concurrent_sends=63 ntx=32768 credits=32768 fmr_pool_size=8193<u></u><u></u></span></p><p class="MsoNormal"><span style="font-size:14pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)">insmod /lib/modules/3.10.0-514.el7.x8<wbr>6_64/extra/lustre/net/ko2iblnd<wbr>.ko require_privileged_port=0 use_privileged_port=0 timeout=150 retry_count=7 map_on_demand=32 peer_credits=63 concurrent_sends=63 ntx=32768 credits=32768 fmr_pool_size=8193<u></u><u></u></span></p><p class="MsoNormal"><span style="font-size:14pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)">modprobe: ERROR: could not insert 'ko2iblnd': Invalid argument<u></u><u></u></span></p><p class="MsoNormal"><span style="font-size:14pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)">modprobe: ERROR: Error running install command for ko2iblnd<u></u><u></u></span></p><p class="MsoNormal"><span style="font-size:14pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)">modprobe: ERROR: could not insert 'ko2iblnd': Operation not permitted<u></u><u></u></span></p><p class="MsoNormal"><span style="font-size:14pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)">[root@mds1 ~]#<u></u><u></u></span></p><p class="MsoNormal"><span style="font-size:14pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)"><u></u> <u></u></span></p><p class="MsoNormal"><span style="font-size:14pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)"><u></u> <u></u></span></p><p class="MsoNormal"><span style="font-family:Calibri,sans-serif;color:rgb(31,73,125)">Regards,<u></u><u></u></span></p><p class="MsoNormal"><span style="font-family:Calibri,sans-serif;color:rgb(31,73,125)">Parag<u></u><u></u></span></p><p class="MsoNormal"><span style="font-family:Calibri,sans-serif;color:rgb(31,73,125)"><u></u> <u></u></span></p><p class="MsoNormal"><span style="font-size:14pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)"><u></u> <u></u></span></p><p class="MsoNormal"><b><span style="font-size:10pt;font-family:Tahoma,sans-serif">From:</span></b><span style="font-size:10pt;font-family:Tahoma,sans-serif"> Brett Lee [mailto:<a href="mailto:brettlee.lustre@gmail.com" target="_blank">brettlee.lustre@gmail.<wbr>com</a>] <br><b>Sent:</b> Saturday, November , 2017 8:08 PM<br><b>To:</b> Parag Khuraswar<br><b>Cc:</b> Mannthey, Keith; <a href="mailto:lustre-discuss@lists.lustre.org" target="_blank">lustre-discuss@lists.lustre.or<wbr>g</a><br><b>Subject:</b> Re: [lustre-discuss] error while configuring lnet<u></u><u></u></span></p><div><div class="gmail-m_-7955658491389144023gmail-h5"><p class="MsoNormal"><u></u> <u></u></p><div><p class="MsoNormal">Hi Parag,<u></u><u></u></p><div><p class="MsoNormal"><u></u> <u></u></p></div><div><p class="MsoNormal">You may need to confirm that the in-kernel IB and the IB in the kernel module "match" (are compatible). I think that loading the module (`sudo modprobe -v ko2iblnd`) may be sufficient to verify the match (it's been a while, others may correct me).<u></u><u></u></p></div><div><p class="MsoNormal"><u></u> <u></u></p></div><div><p class="MsoNormal">Please indicate which kernel and which IB you are using.<u></u><u></u></p></div><div><p class="MsoNormal"><br clear="all"><u></u><u></u></p><div><div><div><div><div><div><div><div><div><div><div><div><div><div><div><p class="MsoNormal">Brett<u></u><u></u></p></div><div><p class="MsoNormal">--<u></u><u></u></p></div><div><p class="MsoNormal">Protect yourself against cybercrime<u></u><u></u></p><div><div><p class="MsoNormal">PDS Software Solutions<u></u><u></u></p></div><div><p class="MsoNormal"><a href="https://www.trustpds.com/" target="_blank">https://www.TrustPDS.com</a><u></u><u></u></p></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div><p class="MsoNormal"><u></u> <u></u></p><div><p class="MsoNormal">On Fri, Nov 10, 2017 at 8:30 PM, Parag Khuraswar <<a href="mailto:parag_k@citilindia.com" target="_blank">parag_k@citilindia.com</a>> wrote:<u></u><u></u></p><div><div><p class="MsoNormal"><span style="font-size:14pt;color:rgb(31,73,125)">Hi Keith,</span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:14pt;color:rgb(31,73,125)"> </span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:14pt;color:rgb(31,73,125)">Below errors I am getting while adding lnet and mounting mdt.</span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:14pt;color:rgb(31,73,125)"> </span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:14pt;color:rgb(31,73,125)">dmesg logs while adding lnet </span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:14pt;color:rgb(31,73,125)">==============================<wbr>===========</span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:14pt;color:rgb(31,73,125)">[317831.432182] LNetError: 28362:0:(api-ni.c:1861:lnet_st<wbr>artup_lndnet()) Can't load LND o2ib, module ko2iblnd, rc=256</span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:14pt;color:rgb(31,73,125)">==============================<wbr>===========</span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:14pt;color:rgb(31,73,125)"> </span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:14pt;color:rgb(31,73,125)"> </span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:14pt;color:rgb(31,73,125)"> </span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:14pt;color:rgb(31,73,125)">dmesg logs while mounting mdt</span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:14pt;color:rgb(31,73,125)">==============================<wbr>============</span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:14pt;color:rgb(31,73,125)">[290476.172602] LNetError: 23040:0:(api-ni.c:1861:lnet_st<wbr>artup_lndnet()) Can't load LND o2ib, module ko2iblnd, rc=256</span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:14pt;color:rgb(31,73,125)">[317478.730515] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro</span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:14pt;color:rgb(31,73,125)">[317480.166277] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,n<wbr>o_mbcache,nodelalloc</span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:14pt;color:rgb(31,73,125)">[317480.313296] LustreError: 28268:0:(ldlm_lib.c:483:client<wbr>_obd_setup()) can't add initial connection</span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:14pt;color:rgb(31,73,125)">[317480.313600] LustreError: 28268:0:(obd_config.c:608:clas<wbr>s_setup()) setup <a href="mailto:MGC10.2.1.204@o2ib" target="_blank">MGC10.2.1.204@o2ib</a> failed (-2)</span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:14pt;color:rgb(31,73,125)">[317480.313603] LustreError: 28268:0:(obd_mount.c:202:lustr<wbr>e_start_simple()) <a href="mailto:MGC10.2.1.204@o2ib" target="_blank">MGC10.2.1.204@o2ib</a> setup error -2</span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:14pt;color:rgb(31,73,125)">[317480.313632] LustreError: 28268:0:(obd_mount_server.c:15<wbr>73:server_put_super()) no obd home-MDT0000</span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:14pt;color:rgb(31,73,125)">[317480.313635] LustreError: 28268:0:(obd_mount_server.c:13<wbr>2:server_deregister_mount()) home-MDT0000 not registered</span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:14pt;color:rgb(31,73,125)">[317480.433934] Lustre: server umount home-MDT0000 complete</span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:14pt;color:rgb(31,73,125)">[317480.433940] LustreError: 28268:0:(obd_mount.c:1504:lust<wbr>re_fill_super()) Unable to mount (-2)</span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:14pt;color:rgb(31,73,125)">==============================<wbr>============</span><u></u><u></u></p><div><p class="MsoNormal"><span style="color:rgb(31,73,125)"> </span><u></u><u></u></p><p class="MsoNormal"><span style="color:rgb(31,73,125)"> </span><u></u><u></u></p><p class="MsoNormal"><span style="color:rgb(31,73,125)">Regards,</span><u></u><u></u></p><p class="MsoNormal"><span style="color:rgb(31,73,125)">Parag</span><u></u><u></u></p><p class="MsoNormal"><span style="color:rgb(31,73,125)"> </span><u></u><u></u></p></div><p class="MsoNormal"><span style="font-size:14pt;color:rgb(31,73,125)"> </span><u></u><u></u></p><div><div style="border-style:solid none none;border-top-width:1pt;border-top-color:rgb(181,196,223);padding:3pt 0in 0in"><p class="MsoNormal"><b><span style="font-size:10pt;font-family:Tahoma,sans-serif">From:</span></b><span style="font-size:10pt;font-family:Tahoma,sans-serif"> Mannthey, Keith [mailto:<a href="mailto:keith.mannthey@intel.com" target="_blank">keith.mannthey@intel.c<wbr>om</a>] <br><b>Sent:</b> Saturday, November , 2017 2:06 AM</span><u></u><u></u></p><div><div><p class="MsoNormal"><br><b>To:</b> Parag Khuraswar; <a href="mailto:lustre-discuss@lists.lustre.org" target="_blank">lustre-discuss@lists.lustre.or<wbr>g</a><br><b>Subject:</b> RE: [lustre-discuss] error while configuring lnet<u></u><u></u></p></div></div></div></div><div><div><p class="MsoNormal"> <u></u><u></u></p><p class="MsoNormal"><a name="m_-7955658491389144023_m_-3866864905143472575_m_8884962771233506054__MailEndCompose"><span style="color:rgb(31,73,125)">If you have ib0 device check dmesg for more hints on what is going wrong. </span></a><u></u><u></u></p><p class="MsoNormal"><span style="color:rgb(31,73,125)"> </span><u></u><u></u></p><p class="MsoNormal"><span style="color:rgb(31,73,125)">Thanks,</span><u></u><u></u></p><p class="MsoNormal"><span style="color:rgb(31,73,125)">Keith </span><u></u><u></u></p><div style="border-style:none none none solid;border-left-width:1.5pt;border-left-color:blue;padding:0in 0in 0in 4pt"><div><div style="border-style:solid none none;border-top-width:1pt;border-top-color:rgb(225,225,225);padding:3pt 0in 0in"><p class="MsoNormal"><a name="m_-7955658491389144023_m_-3866864905143472575_m_8884962771233506054______replyseparato"></a><b>From:</b> Parag Khuraswar [mailto:<a href="mailto:parag_k@citilindia.com" target="_blank">parag_k@citilindia.com</a><wbr>] <br><b>Sent:</b> Friday, November 10, 2017 10:59 AM<br><b>To:</b> Mannthey, Keith <<a href="mailto:keith.mannthey@intel.com" target="_blank">keith.mannthey@intel.com</a>>; <a href="mailto:lustre-discuss@lists.lustre.org" target="_blank">lustre-discuss@lists.lustre.or<wbr>g</a><br><b>Subject:</b> RE: [lustre-discuss] error while configuring lnet<u></u><u></u></p></div></div><p class="MsoNormal"> <u></u><u></u></p><p class="MsoNormal"><span style="font-size:14pt;color:rgb(31,73,125)">Hi,</span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:14pt;color:rgb(31,73,125)"> </span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:14pt;color:rgb(31,73,125)">Basically I am trying to add lnet. Deleting is just try whether it is happing or not.</span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:14pt;color:rgb(31,73,125)">Main is I want to add o2ib network. Which is giving error “invalid argument ”</span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:14pt;color:rgb(54,95,145)">==================</span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:14pt;color:rgb(54,95,145)">[root@mds2 ~]# lnetctl net add --net o2ib --if ib0</span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:14pt;color:rgb(54,95,145)">add:</span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:14pt;color:rgb(54,95,145)"> - net:</span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:14pt;color:rgb(54,95,145)"> errno: -22</span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:14pt;color:rgb(54,95,145)"> descr: "cannot add network: Invalid argument"</span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:14pt;color:rgb(31,73,125)">================</span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:14pt;color:rgb(31,73,125)">I am really not able to understand what argument is invalid in my command.</span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:14pt;color:rgb(31,73,125)">I am able to ping ib0 network</span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:14pt;color:rgb(31,73,125)"> </span><u></u><u></u></p><div><p class="MsoNormal"><span style="color:rgb(31,73,125)">Regards,</span><u></u><u></u></p><p class="MsoNormal"><span style="color:rgb(31,73,125)">Parag</span><u></u><u></u></p><p class="MsoNormal"><span style="color:rgb(31,73,125)"> </span><u></u><u></u></p></div><p class="MsoNormal"><span style="font-size:14pt;color:rgb(31,73,125)"> </span><u></u><u></u></p><div><div style="border-style:solid none none;border-top-width:1pt;border-top-color:rgb(181,196,223);padding:3pt 0in 0in"><p class="MsoNormal"><b><span style="font-size:10pt;font-family:Tahoma,sans-serif">From:</span></b><span style="font-size:10pt;font-family:Tahoma,sans-serif"> Mannthey, Keith [<a href="mailto:keith.mannthey@intel.com" target="_blank">mailto:keith.mannthey@intel.c<wbr>om</a>] <br><b>Sent:</b> Friday, November , 2017 10:51 PM<br><b>To:</b> Parag Khuraswar; <a href="mailto:lustre-discuss@lists.lustre.org" target="_blank">lustre-discuss@lists.lustre.or<wbr>g</a><br><b>Subject:</b> RE: [lustre-discuss] error while configuring lnet</span><u></u><u></u></p></div></div><p class="MsoNormal"> <u></u><u></u></p><p class="MsoNormal"><span style="color:rgb(31,73,125)">What are you trying to accomplish? </span><u></u><u></u></p><p class="MsoNormal"><span style="color:rgb(31,73,125)"> </span><u></u><u></u></p><p class="MsoNormal"><span style="color:rgb(31,73,125)">From below:</span><u></u><u></u></p><p class="MsoNormal"><span style="color:rgb(31,73,125)"> </span><u></u><u></u></p><p class="MsoNormal"><a href="mailto:10.1.1.205@tcp" target="_blank">10.1.1.205@tcp</a><span style="color:rgb(31,73,125)"> is on 0@lo not eno1 and in general you should not need the “—if” option to delete a fabric. </span><u></u><u></u></p><p class="MsoNormal"><span style="color:rgb(31,73,125)"> </span><u></u><u></u></p><p class="MsoNormal"><span style="color:rgb(31,73,125)">Try: # lnetctl net del --net tcp</span><u></u><u></u></p><p class="MsoNormal"><span style="color:rgb(31,73,125)"> </span><u></u><u></u></p><p class="MsoNormal"><span style="color:rgb(31,73,125)">Can you do a normal ping over ib0? </span><u></u><u></u></p><p class="MsoNormal"><span style="color:rgb(31,73,125)"> </span><u></u><u></u></p><p class="MsoNormal"><span style="color:rgb(31,73,125)">“dmesg” can sometime provide greater details about errors like this. </span><u></u><u></u></p><p class="MsoNormal"><span style="color:rgb(31,73,125)"> </span><u></u><u></u></p><p class="MsoNormal"><span style="color:rgb(31,73,125)">Thanks,</span><u></u><u></u></p><p class="MsoNormal"><span style="color:rgb(31,73,125)">Keith </span><u></u><u></u></p><p class="MsoNormal"><span style="color:rgb(31,73,125)"> </span><u></u><u></u></p><p class="MsoNormal"><span style="color:rgb(31,73,125)"> </span><u></u><u></u></p><div style="border-style:none none none solid;border-left-width:1.5pt;border-left-color:blue;padding:0in 0in 0in 4pt"><div><div style="border-style:solid none none;border-top-width:1pt;border-top-color:rgb(225,225,225);padding:3pt 0in 0in"><p class="MsoNormal"><b>From:</b> lustre-discuss [<a href="mailto:lustre-discuss-bounces@lists.lustre.org" target="_blank">mailto:lustre-discuss-bounces<wbr>@lists.lustre.org</a>] <b>On Behalf Of </b>Parag Khuraswar<br><b>Sent:</b> Friday, November 10, 2017 9:10 AM<br><b>To:</b> <a href="mailto:lustre-discuss@lists.lustre.org" target="_blank">lustre-discuss@lists.lustre.or<wbr>g</a><br><b>Subject:</b> [lustre-discuss] error while configuring lnet<u></u><u></u></p></div></div><p class="MsoNormal"> <u></u><u></u></p><p class="MsoNormal"><span style="font-size:14pt;color:rgb(54,95,145)">Hi,</span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:14pt;color:rgb(54,95,145)"> </span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:14pt;color:rgb(54,95,145)">I am trying to add lnet but getting below error. </span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:14pt;color:rgb(54,95,145)">======================</span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:14pt;color:rgb(54,95,145)">[root@mds2 ~]# lnetctl net show</span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:14pt;color:rgb(54,95,145)">net:</span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:14pt;color:rgb(54,95,145)"> - net type: lo</span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:14pt;color:rgb(54,95,145)"> local NI(s):</span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:14pt;color:rgb(54,95,145)"> - nid: 0@lo</span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:14pt;color:rgb(54,95,145)"> status: up</span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:14pt;color:rgb(54,95,145)"> - net type: tcp</span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:14pt;color:rgb(54,95,145)"> local NI(s):</span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:14pt;color:rgb(54,95,145)"> - nid: <a href="mailto:10.1.1.205@tcp" target="_blank">10.1.1.205@tcp</a></span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:14pt;color:rgb(54,95,145)"> status: up</span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:14pt;color:rgb(54,95,145)">[root@mds2 ~]# lnetctl net add --net o2ib --if ib0</span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:14pt;color:rgb(54,95,145)">add:</span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:14pt;color:rgb(54,95,145)"> - net:</span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:14pt;color:rgb(54,95,145)"> errno: -22</span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:14pt;color:rgb(54,95,145)"> descr: "cannot add network: Invalid argument"</span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:14pt;color:rgb(54,95,145)">[root@mds2 ~]# lnetctl net del --net tcp --if eno1</span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:14pt;color:rgb(54,95,145)">del:</span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:14pt;color:rgb(54,95,145)"> - net:</span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:14pt;color:rgb(54,95,145)"> errno: -22</span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:14pt;color:rgb(54,95,145)"> descr: "cannot del network: Invalid argument"</span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:14pt;color:rgb(54,95,145)">[root@mds2 ~]# lctl list_nids</span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:14pt;color:rgb(54,95,145)"><a href="mailto:10.1.1.205@tcp" target="_blank">10.1.1.205@tcp</a></span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:14pt;color:rgb(54,95,145)">[root@mds2 ~]#</span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:14pt;color:rgb(54,95,145)">==============================<wbr>======</span><u></u><u></u></p><p class="MsoNormal"><span style="color:rgb(54,95,145)"> </span><u></u><u></u></p><p class="MsoNormal"><span style="color:rgb(54,95,145)"> </span><u></u><u></u></p><p class="MsoNormal"><span style="color:rgb(54,95,145)">Regards,</span><u></u><u></u></p><p class="MsoNormal"><span style="color:rgb(54,95,145)">Parag</span><u></u><u></u></p><p class="MsoNormal"><span style="color:rgb(54,95,145)"> </span><u></u><u></u></p><p class="MsoNormal"> <u></u><u></u></p></div></div></div></div></div></div><p class="MsoNormal" style="margin-bottom:12pt"><br>______________________________<wbr>_________________<br>lustre-discuss mailing list<br><a href="mailto:lustre-discuss@lists.lustre.org" target="_blank">lustre-discuss@lists.lustre.or<wbr>g</a><br><a href="http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org" target="_blank">http://lists.lustre.org/listin<wbr>fo.cgi/lustre-discuss-lustre.<wbr>org</a><u></u><u></u></p></div><p class="MsoNormal"><u></u> <u></u></p></div></div></div></div></div></div></blockquote></div><br></div></div></div></div>