[lustre-discuss] ​luster client mount issues

[sohamm]

Hi All,
I was able to get the lustre client mounted successfully.

I was getting this error whenever i tried to mount client via tcp/o2ib
network.

[867363.885584] LustreError: 162-5: Missing mount data: check that
/sbin/mount.lustre isinstalled.
[867363.885637] LustreError: 13214:0:(obd_mount.c:1325:lustre_fill_super())
Unable to mount  (-22)
[867363.885659] LustreError: 13214:0:(obd_mount.c:1325:lustre_fill_super())
Skipped 1 previous similar message
[867364.107157] LustreError: 15c-8: MGC192.168.111.52 at tcp: The
configuration from log 'mylustre-client' failed (-2). This may be the
result of communication errors between this node and the MGS, a bad
configuration, or other errors. See the syslog for more information.
[867364.107209] LustreError: 13215:0:(llite_lib.c:1046:ll_fill_super())
Unable to process log: -2
[867364.107729] Lustre: Unmounted mylustre-client
MGC192.168.111.52 at tcp

but when checked on the mgs with cat /proc/fs/lustre/device, i could not
see any entry for MGC192.168.111.52 at tcp . Only MGC192.168.200.52 at o2ib was
present. Also some of the disks were missing. Which led me to look into my
disks health. To my surprise the iscsi disks in zpool were "Degraded". Not
sure what caused that. So i reconfigured the entire setup and was able to
mount the client via o2ib without any issues. Couple of other tweaks i did
this time is

1. Disabled the firewalld and ufw ( earlier i added an exception to allow
the iscsi disks )
2. used the following command to mount MGS and mdt ( both in single
command).
mkfs.lustre --fsname=lustre --mgs --mdt --backfstype=zfs mgs01/data --index
0

earlier i used below two commands
mkfs.lustre --mgs --backfstype=zfs mds1_1/mgs
mkfs.lustre --mdt --backfstype=zfs --fsname=mylustre --index=1
--mgsnode=192.168.200.52 at o2ib mds1_1/mdt1

3. modified lustre.conf to have the preferred network as first parameter.
Not sure if thats how its supposed to work.
options lnet networks=o2ib(ib0),tcp0(eth0) if i want to connect over o2ib
options lnet networks=tcp0(eth0),o2ib(ib0) if i want to connect over tcp

Thanks all for your help. Hopefully my learning's will help others.


On Mon, Aug 1, 2016 at 7:42 AM, Mohr Jr, Richard Frank (Rick Mohr) <
rmohr at utk.edu> wrote:

>
> > On Jul 28, 2016, at 9:54 PM, sohamm <sohamm at gmail.com> wrote:
> >
> > Client is configured for IB interface.
>
> So it looks like there might be something wrong with the LNet config on
> the client then.  Based on the output from “lctl ping” that you ran from
> the server, the client only reported a NID on the tcp network.
>
> > in my understanding i can specific the network of choice in the mount
> command. tried both tcp and ib.
>
> That is true, but sometimes if the client and server both have interfaces
> on two different networks (like ethernet and IB) there can be some subtle
> issues.  When you specify the NID for the MGS to mount the file system, the
> client will retrieve information about the MDS/OSS servers from the MGS you
> specified.  This information includes the NIDS that the MDS/OSS servers
> will listen for requests.  If a client sees that a server has a NID on tcp0
> and a NID on o2ib0, and the client also has NIDs on tcp0 and o2ib0, then
> the client sees that there are two paths to the same server and it will
> just pick one of the paths (which might not be the one you want).  And if
> the path it chooses happens to be down, it won’t matter if the other path
> is up.
>
> (Now, I should make a disclaimer about the above statements.  I believe
> that is how it worked on Lustre versions like 1.8 and 2.4.  I have not
> tried this with newer Lustre versions, so the behavior could be different.
> I also have not experimented with anything like specifying weights for LNet
> routes, so I don’t know if that could be used to prefer one interface over
> another.)
>
> --
> Rick Mohr
> Senior HPC System Administrator
> National Institute for Computational Sciences
> http://www.nics.tennessee.edu
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20160816/b1d57d58/attachment.htm>