Dear Lustre users,<br> I am using lustre file system(<span class="gI">lustre-1.8.1</span>) over ib.Now i required lustre over ib and over Ethernet also.<br> I modified the modprobe.conf on client , mds(mdt),oss(ost).<br>
I add below line in modprobe.conf.<br> options lnet networks=o2ib(ib0),tcp1(eth1)<br> I am able to mount lustre over ib on client but not able to mount over Ethernet.<br> I am getting following error on stdout .<br>
<br> mount.lustre: mount 172.29.2.25@tcp1:/home at /mnt failed: No such file or directory<br> Is the MGS specification correct?<br> Is the file system name correct?<br> If upgrading, is the copied client log valid? (see upgrade docs)<br>
<br> and /var/log/message is<br><br> kernel: LustreError: 6943:0:(ldlm_lib.c:329:client_obd_setup()) can't add initial connection<br> kernel: LustreError: 6943:0:(obd_config.c:370:class_setup()) setup home-MDT0000-mdc-ffff810f39c8ac00 failed (-2)<br>
kernel: LustreError: 6943:0:(obd_config.c:1197:class_config_llog_handler()) Err -2 on cfg command:<br> kernel: Lustre: cmd=cf003 0:home-MDT0000-mdc 1:home-MDT0000_UUID 2:172.31.65.49@o2ib<br> kernel: LustreError: 15c-8: MGC172.29.2.25@tcp1: The configuration from log 'home-client' failed (-2). This may be the result of communication errors between this node and the MGS, a bad configuration, or other errors. See the syslog for more information.<br>
kernel: LustreError: 6933:0:(llite_lib.c:1171:ll_fill_super()) Unable to process log: -2<br> kernel: LustreError: 6933:0:(obd_config.c:441:class_cleanup()) Device 2 not setup<br> kernel: LustreError: 6933:0:(ldlm_request.c:1030:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway<br>
kernel: LustreError: 6933:0:(ldlm_request.c:1533:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108<br> kernel: Lustre: client ffff810f39c8ac00 umount complete<br> kernel: LustreError: 6933:0:(obd_mount.c:1997:lustre_fill_super()) Unable to mount (-2)<br>
<br> I am able to ping mgs from client .<br><br> lctl list_nids command output on mgs give following output.<br><br> 172.31.65.49@o2ib<br> 172.29.2.25@tcp1<br><br> also i able to mount lustre client on mgs itself over Ethernet <br>
<br> Following is tunefs.lustre output<br><br><span class="gI"></span><br> 1) mgs tunefs.lustre output<br><br> checking for existing Lustre data: found CONFIGS/mountdata<br> Reading CONFIGS/mountdata<br>
<br> Read previous values:<br> Target: MGS<br> Index: unassigned<br> Lustre FS: scratch<br> Mount type: ldiskfs<br> Flags: 0x174<br> (MGS needs_index first_time update writeconf )<br>
Persistent mount opts: errors=remount-ro,iopen_nopriv,user_xattr<br> Parameters:<br><br><br> Permanent disk data:<br> Target: MGS<br> Index: unassigned<br> Lustre FS: scratch<br>
Mount type: ldiskfs<br> Flags: 0x174<br> (MGS needs_index first_time update writeconf )<br> Persistent mount opts: errors=remount-ro,iopen_nopriv,user_xattr<br><br> 2) mds(mdt) tunefs.lustre output<br>
Read previous values:<br> Target: home-MDT0000<br> Index: 0<br> Lustre FS: home<br> Mount type: ldiskfs<br> Flags: 0x1<br> (MDT )<br> Persistent mount opts: errors=remount-ro,iopen_nopriv,user_xattr<br>
Parameters: mgsnode=172.31.65.49@o2ib mdt.group_upcall=/usr/sbin/l_getgroups<br><br><br> Permanent disk data:<br> Target: home-MDT0000<br> Index: 0<br> Lustre FS: home<br> Mount type: ldiskfs<br>
Flags: 0x1<br> (MDT )<br> Persistent mount opts: errors=remount-ro,iopen_nopriv,user_xattr<br> Parameters: mgsnode=172.31.65.49@o2ib mdt.group_upcall=/usr/sbin/l_getgroups<br><br> 3) oss(ost) tunefs.lustre output<span class="gI"><br>
</span> Read previous values:<br> Target: home-OST0003<br> Index: 3<br> Lustre FS: home<br> Mount type: ldiskfs<br> Flags: 0x2<br> (OST )<br>
Persistent mount opts: errors=remount-ro,extents,mballoc<br> Parameters: mgsnode=172.31.65.49@o2ib mdt.quota_type=ug<br><br><br> Permanent disk data:<br> Target: home-OST0003<br> Index: 3<br>
Lustre FS: home<br> Mount type: ldiskfs<br> Flags: 0x2<br> (OST )<br> Persistent mount opts: errors=remount-ro,extents,mballoc<br> Parameters: mgsnode=172.31.65.49@o2ib mdt.quota_type=ug<br>
<br> exiting before disk write.<br><br>I am not able to figure out what is the exact problem.<br><br><br>Thanks and regards<br>Vaibhi<br>