[lustre-discuss] ​luster client mount issues

[Andrus, Brian Contractor]

Are you running IPoIB?
Can you do “lsmod |grep lnet”? Also, ensure you have the right network settings in your /etc/modprobe.d/lnet.conf file (or wherever you may have defined the networks)


Brian Andrus
ITACS/Research Computing
Naval Postgraduate School
Monterey, California
voice: 831-656-6238





From: lustre-discuss [mailto:lustre-discuss-bounces at lists.lustre.org] On Behalf Of sohamm
Sent: Thursday, July 28, 2016 6:55 PM
To: Mohr Jr, Richard Frank (Rick Mohr)
Cc: lustre-discuss at lists.lustre.org
Subject: Re: [lustre-discuss] ​luster client mount issues

Hi Rick
Client is configured for IB interface.
in my understanding i can specific the network of choice in the mount command. tried both tcp and ib. I am still checking on the configurations as suggested in the forumn. will get back with my findings.

Thanks

On Thursday, July 28, 2016, Mohr Jr, Richard Frank (Rick Mohr) <rmohr at utk.edu<mailto:rmohr at utk.edu>> wrote:
Is the client supposed to have an IB interface configured, or is it just supposed to mount over ethernet?

--
Rick Mohr
Senior HPC System Administrator
National Institute for Computational Sciences
http://www.nics.tennessee.edu


> On Jul 20, 2016, at 2:09 PM, sohamm <sohamm at gmail.com<javascript:;>> wrote:
>
> Hi
>
> Any guidance/help on this is greatly appreciated.
>
> Thanks
>
> On Mon, Jul 18, 2016 at 7:25 PM, sohamm <sohamm at gmail.com<javascript:;>> wrote:
> Hi Ben
> Both the networks have netmasks of value 255.255.255.0
>
> Thanks
>
> On Mon, Jul 18, 2016 at 10:08 AM, Ben Evans <bevans at cray.com<javascript:;>> wrote:
> What do your netmasks look like on each network?
>
> From: lustre-discuss <lustre-discuss-bounces at lists.lustre.org<javascript:;>> on behalf of sohamm <sohamm at gmail.com<javascript:;>>
> Date: Monday, July 18, 2016 at 1:56 AM
> To: "lustre-discuss at lists.lustre.org<javascript:;>" <lustre-discuss at lists.lustre.org<javascript:;>>
> Subject: Re: [lustre-discuss] lustre-discuss Digest, Vol 124, Issue 17
>
> Hi Thomas
> Below are the results of the commands you suggested.
>
> From Client
> [root at dev1 ~]# lctl ping 192.168.200.52 at o2ib<mailto:192.168.200.52 at o2ib>
> failed to ping 192.168.200.52 at o2ib<mailto:192.168.200.52 at o2ib>: Input/output error
> [root at dev1 ~]# lctl ping 192.168.111.52 at tcp<mailto:192.168.111.52 at tcp>
> 12345-0 at lo
> 12345-192.168.200.52 at o2ib<mailto:12345-192.168.200.52 at o2ib>
> 12345-192.168.111.52 at tcp<mailto:12345-192.168.111.52 at tcp>
> [root at dev1 ~]# mount -t lustre 192.168.111.52 at tcp:/mylustre<mailto:192.168.111.52 at tcp:/mylustre> /lustre
> mount.lustre: mount 192.168.111.52 at tcp:/mylustre<mailto:192.168.111.52 at tcp:/mylustre> at /lustre failed: Input/output error
> Is the MGS running?
> mount: mounting 192.168.111.52 at tcp:/mylustre<mailto:192.168.111.52 at tcp:/mylustre> on /lustre failed: Invalid argument
>
> cat /var/log/messages | tail
> Jul 18 01:37:04 dev1 user.warn kernel: [2250504.401397] ib1: multicast join failed for ff12:401b:ffff:0000:0000:0000:ffff:ffff, status -22
> Jul 18 01:37:26 dev1 user.warn kernel: [2250526.257309] LNet: No route to 12345-192.168.200.52 at o2ib<mailto:12345-192.168.200.52 at o2ib> via <?> (all routers down)
> Jul 18 01:37:36 dev1 user.warn kernel: [2250536.481862] ib1: multicast join failed for ff12:401b:ffff:0000:0000:0000:ffff:ffff, status -22
> Jul 18 01:41:53 dev1 user.warn kernel: [2250792.947299] LNet: No route to 12345-192.168.200.52 at o2ib<mailto:12345-192.168.200.52 at o2ib> via <?> (all routers down)
>
>
> From MGS
> [root at lustre_mgs01_vm03 ~]# lctl ping 192.168.111.102 at tcp<mailto:192.168.111.102 at tcp>
> 12345-0 at lo
> 12345-192.168.111.102 at tcp<mailto:12345-192.168.111.102 at tcp>
>
> Please let me know what else i can try. Looks like i am missing something with the ib config? Do i need router setup as part of lnet ?
> if i am able to ping mgs from client on the tcp network, it should still work ?
>
> Thanks
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20160729/f2d4a59c/attachment-0001.htm>