<html><head><style type="text/css"><!-- DIV {margin:0px;} --></style></head><body><div style="font-family:tahoma,new york,times,serif;font-size:12pt"><div style="font-family: tahoma,new york,times,serif; font-size: 12pt;">Thank you. After the self compiled 'lustre-modules-1.8.3-2.6.18_164.11.1.el5_lustre.1.8.3_201011012110' installed on the client side, the problem resolved.<br><br>Yufei<br><br><br><br><div style="font-family: times new roman,new york,times,serif; font-size: 12pt;"><font face="Tahoma" size="2"><hr size="1"><b><span style="font-weight: bold;">From:</span></b> Wang Yibin <wang.yibin@oracle.com><br><b><span style="font-weight: bold;">To:</span></b> ren yufei <renyufei83@yahoo.com.cn><br><b><span style="font-weight: bold;">Cc:</span></b> lustre-discuss@lists.lustre.org<br><b><span style="font-weight: bold;">Sent:</span></b> Wed, November 3, 2010 10:03:31 PM<br><b><span style="font-weight: bold;">Subject:</span></b> Re:
[Lustre-discuss] Lustre FS in Infiniband - client mount problem<br></font><br>
<base>As the error message said, the lustre modules probably were not loaded when you were trying to mount lustre client.<div><br></div><div>Please provide information - specifically, the loaded modules and more dmesg log.</div><div><br><div><div>在 2010-11-4，上午2:30， ren yufei 写道：</div><br class="Apple-interchange-newline"><blockquote type="cite"><span class="Apple-style-span" style="border-collapse: separate; font-family: Courier; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; font-size: medium;"><div><div style="margin: 0px; font-family: tahoma,times,serif; font-size: 12pt;"><span style="font-family: tahoma,times,serif;">Dear all,</span><br style="font-family: tahoma,times,serif;"><br style="font-family: tahoma,times,serif;"><span style="font-family: tahoma,times,serif;">I
have setup some nodes (include MDT/MGS/OSS/Client) with Mellanox 40G RNIC connected via Mellanox MTS3600 switch, and deployed lustre FS in this cluster</span><font style="font-family: tahoma,times,serif;" size="4">. The MDT/MGS node and OSS nodes works but client could not mount on this FS. The error information is as follows.<br><br></font><font style="font-family: tahoma,times,serif;" size="4">Client: 192.168.1.23<br>MDS: 192.168.1.11<br><br>-- error information<br>Client # mount -t lustre 192.168.1.11@o2ib0:/lustre /mnt/lustre<br>mount.lustre: mount 192.168.1.11@o2ib0:/lustre at /mnt/lustre failed: No such device<br>Are the lustre modules loaded?<br>Check /etc/modprobe.conf and /proc/filesystems<br>Note 'alias lustre llite' should be removed from modprobe.conf<br><br>Client # ls -l /mnt/lustre<br>total 0<br></font><font style="font-family: tahoma,times,serif;" size="4"><br>Client # dmesg | tail<br>LustreError: 165-2: Nothing registered for client
mount! Is the 'lustre' module loaded?<br>LustreError: 5116:0:(obd_mount.c:2045:lustre_fill_super()) Unable to mount (-19)<br><br>-- environment information.<br><br>Client:<br># lctl list_nids<br>192.168.1.23@o2ib<br># lctl ping 192.168.1.11@o2ib0<br>12345-0@lo<br>12345-192.168.1.11@o2ib<br><br>MDS:<br></font><font style="font-family: tahoma,times,serif;" size="4"># lctl list_nids<br>192.168.1.11@o2ib</font><font style="font-family: tahoma,times,serif;" size="4"><br><br>lctl > device_list<br> 0 UP mgs MGS MGS 13<br> 1 UP mgc MGC192.168.1.11@o2ib bb9cf87d-fd14-b679-85ce-f0fa1a866aff 5<br> 2 UP mdt MDS MDS_uuid 3<br> 3 UP lov lustre-mdtlov lustre-mdtlov_UUID 4<br> 4 UP mds lustre-MDT0000 lustre-MDT0000_UUID 3<br> 5 UP osc lustre-OST0000-osc lustre-mdtlov_UUID 5<br> 6 UP osc lustre-OST0001-osc lustre-mdtlov_UUID 5<br>...<br><br>By the way, All these nodes could connect or access to each other via
ping/iperf(TCP)/ibv_rc_ pingpong/ibv_ud_pingpong/ib_ write_lat.</font><span style="font-family: tahoma,times,serif;"><span class="Apple-converted-space"> </span></span><font style="font-family: tahoma,times,serif;" size="4">However,<span class="Apple-converted-space"> </span><b>the 'rping' client, based on librdmacm, could not connect to the server side.<span class="Apple-converted-space"> </span></b>error is:<br><br># rping -c 192.168.1.11<br>cq completion failed status 5<br>wait for CONNECTED state 10<br>connect error -1<br>cma event RDMA_CM_EVENT_REJECTED, error 8<br></font><br style="font-family: tahoma,times,serif;"><font style="font-family: tahoma,times,serif;" size="4"><br>Thank you very much.<br><br>Yufei</font><br style="font-family: tahoma,times,serif;"><div style="margin: 0px; font-family: tahoma,times,serif;"><br></div></div><br>_______________________________________________<br>Lustre-discuss mailing list<br><a rel="nofollow"
ymailto="mailto:Lustre-discuss@lists.lustre.org" target="_blank" href="mailto:Lustre-discuss@lists.lustre.org">Lustre-discuss@lists.lustre.org</a><br><span><a target="_blank" href="http://lists.lustre.org/mailman/listinfo/lustre-discuss">http://lists.lustre.org/mailman/listinfo/lustre-discuss</a></span><br></div></span></blockquote></div><br></div></div></div>
</div><br>
</body></html>