<html><head><style type="text/css"><!-- DIV {margin:0px;} --></style></head><body><div style="font-family:tahoma,new york,times,serif;font-size:12pt"><span style="font-family: tahoma,new york,times,serif;">Dear all,</span><br style="font-family: tahoma,new york,times,serif;"><br style="font-family: tahoma,new york,times,serif;"><span style="font-family: tahoma,new york,times,serif;">I have setup some nodes (include MDT/MGS/OSS/Client) with Mellanox 40G RNIC connected via Mellanox MTS3600 switch, and deployed lustre FS in this cluster</span><font style="font-family: tahoma,new york,times,serif;" size="4">. The MDT/MGS node and OSS nodes works but client could not mount on this FS. The error information is as follows.<br><br></font><font style="font-family: tahoma,new york,times,serif;" size="4">Client: 192.168.1.23<br>
MDS: 192.168.1.11<br><br>-- error information<br>Client # mount -t lustre 192.168.1.11@o2ib0:/lustre /mnt/lustre<br>mount.lustre: mount 192.168.1.11@o2ib0:/lustre at /mnt/lustre failed: No such device<br>Are the lustre modules loaded?<br>Check /etc/modprobe.conf and /proc/filesystems<br>Note 'alias lustre llite' should be removed from modprobe.conf<br><br>Client # ls -l /mnt/lustre<br>total 0<br></font><font style="font-family: tahoma,new york,times,serif;" size="4"><br>Client # dmesg | tail<br>LustreError: 165-2: Nothing registered for client mount! Is the 'lustre' module loaded?<br>LustreError: 5116:0:(obd_mount.c:2045:lustre_fill_super()) Unable to mount (-19)<br><br>-- environment information.<br><br>Client:<br># lctl list_nids<br>192.168.1.23@o2ib<br># lctl ping 192.168.1.11@o2ib0<br>12345-0@lo<br>12345-192.168.1.11@o2ib<br><br>MDS:<br></font><font style="font-family: tahoma,new york,times,serif;" size="4"># lctl list_nids<br>
192.168.1.11@o2ib</font><font style="font-family: tahoma,new york,times,serif;" size="4"><br><br>lctl > device_list<br> 0 UP mgs MGS MGS 13<br> 1 UP mgc MGC192.168.1.11@o2ib bb9cf87d-fd14-b679-85ce-f0fa1a866aff 5<br> 2 UP mdt MDS MDS_uuid 3<br> 3 UP lov lustre-mdtlov lustre-mdtlov_UUID 4<br> 4 UP mds lustre-MDT0000 lustre-MDT0000_UUID 3<br> 5 UP osc lustre-OST0000-osc lustre-mdtlov_UUID 5<br> 6 UP osc lustre-OST0001-osc lustre-mdtlov_UUID 5<br>...<br><br>By the way, All these nodes could connect or access
to each other via ping/iperf(TCP)/ibv_rc_<wbr>pingpong/ibv_ud_pingpong/ib_<wbr>write_lat.</font><span style="font-family: tahoma,new york,times,serif;"> </span><font style="font-family: tahoma,new york,times,serif;" size="4">However, <b>the 'rping' client, based on librdmacm, could not connect to the server side. </b>error is:<br><br># rping -c 192.168.1.11<br>cq completion failed status 5<br>wait for CONNECTED state 10<br>connect error -1<br>cma event RDMA_CM_EVENT_REJECTED, error 8<br></font><br style="font-family: tahoma,new york,times,serif;"><font style="font-family: tahoma,new york,times,serif;" size="4"><br>Thank you very much.<br><br>Yufei</font><br style="font-family: tahoma,new york,times,serif;"><div style="font-family: tahoma,new york,times,serif;"><br></div>
</div><br>
</body></html>