Hi Rick<div>Client is configured for IB interface. </div><div>in my understanding i can specific the network of choice in the mount command. tried both tcp and ib. I am still checking on the configurations as suggested in the forumn. will get back with my findings. </div><div><br></div><div>Thanks<span></span><br><br>On Thursday, July 28, 2016, Mohr Jr, Richard Frank (Rick Mohr) <<a href="mailto:rmohr@utk.edu">rmohr@utk.edu</a>> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Is the client supposed to have an IB interface configured, or is it just supposed to mount over ethernet?<br>
<br>
--<br>
Rick Mohr<br>
Senior HPC System Administrator<br>
National Institute for Computational Sciences<br>
<a href="http://www.nics.tennessee.edu" target="_blank">http://www.nics.tennessee.edu</a><br>
<br>
<br>
> On Jul 20, 2016, at 2:09 PM, sohamm <<a href="javascript:;" onclick="_e(event, 'cvml', 'sohamm@gmail.com')">sohamm@gmail.com</a>> wrote:<br>
><br>
> Hi<br>
><br>
> Any guidance/help on this is greatly appreciated.<br>
><br>
> Thanks<br>
><br>
> On Mon, Jul 18, 2016 at 7:25 PM, sohamm <<a href="javascript:;" onclick="_e(event, 'cvml', 'sohamm@gmail.com')">sohamm@gmail.com</a>> wrote:<br>
> Hi Ben<br>
> Both the networks have netmasks of value 255.255.255.0<br>
><br>
> Thanks<br>
><br>
> On Mon, Jul 18, 2016 at 10:08 AM, Ben Evans <<a href="javascript:;" onclick="_e(event, 'cvml', 'bevans@cray.com')">bevans@cray.com</a>> wrote:<br>
> What do your netmasks look like on each network?<br>
><br>
> From: lustre-discuss <<a href="javascript:;" onclick="_e(event, 'cvml', 'lustre-discuss-bounces@lists.lustre.org')">lustre-discuss-bounces@lists.lustre.org</a>> on behalf of sohamm <<a href="javascript:;" onclick="_e(event, 'cvml', 'sohamm@gmail.com')">sohamm@gmail.com</a>><br>
> Date: Monday, July 18, 2016 at 1:56 AM<br>
> To: "<a href="javascript:;" onclick="_e(event, 'cvml', 'lustre-discuss@lists.lustre.org')">lustre-discuss@lists.lustre.org</a>" <<a href="javascript:;" onclick="_e(event, 'cvml', 'lustre-discuss@lists.lustre.org')">lustre-discuss@lists.lustre.org</a>><br>
> Subject: Re: [lustre-discuss] lustre-discuss Digest, Vol 124, Issue 17<br>
><br>
> Hi Thomas<br>
> Below are the results of the commands you suggested.<br>
><br>
> From Client<br>
> [root@dev1 ~]# lctl ping 192.168.200.52@o2ib<br>
> failed to ping 192.168.200.52@o2ib: Input/output error<br>
> [root@dev1 ~]# lctl ping 192.168.111.52@tcp<br>
> 12345-0@lo<br>
> 12345-192.168.200.52@o2ib<br>
> 12345-192.168.111.52@tcp<br>
> [root@dev1 ~]# mount -t lustre 192.168.111.52@tcp:/mylustre /lustre<br>
> mount.lustre: mount 192.168.111.52@tcp:/mylustre at /lustre failed: Input/output error<br>
> Is the MGS running?<br>
> mount: mounting 192.168.111.52@tcp:/mylustre on /lustre failed: Invalid argument<br>
><br>
> cat /var/log/messages | tail<br>
> Jul 18 01:37:04 dev1 user.warn kernel: [2250504.401397] ib1: multicast join failed for ff12:401b:ffff:0000:0000:0000:ffff:ffff, status -22<br>
> Jul 18 01:37:26 dev1 user.warn kernel: [2250526.257309] LNet: No route to 12345-192.168.200.52@o2ib via <?> (all routers down)<br>
> Jul 18 01:37:36 dev1 user.warn kernel: [2250536.481862] ib1: multicast join failed for ff12:401b:ffff:0000:0000:0000:ffff:ffff, status -22<br>
> Jul 18 01:41:53 dev1 user.warn kernel: [2250792.947299] LNet: No route to 12345-192.168.200.52@o2ib via <?> (all routers down)<br>
><br>
><br>
> From MGS<br>
> [root@lustre_mgs01_vm03 ~]# lctl ping 192.168.111.102@tcp<br>
> 12345-0@lo<br>
> 12345-192.168.111.102@tcp<br>
><br>
> Please let me know what else i can try. Looks like i am missing something with the ib config? Do i need router setup as part of lnet ?<br>
> if i am able to ping mgs from client on the tcp network, it should still work ?<br>
><br>
> Thanks<br>
><br>
<br>
<br>
</blockquote></div>