[lustre-discuss] lustre client 2.9 cannot mount 2.10.0 OSTs
Riccardo Veraldi
Riccardo.Veraldi at cnaf.infn.it
Mon Aug 7 19:34:27 PDT 2017
thanks, yes I noticed that Issued and I changed hte name. I Also rebuild
the FS but now it does not work for another unknown reason:
Aug 7 19:05:38 psana1510 kernel: [289134.511260] Lustre:
3333:0:(client.c:2114:ptlrpc_expire_one_request()) @@@ Request sent has
failed due to network error: [sent 1502157938/real 1502157938]
req at ffff88203e699800 x1574823717862480/t0(0)
o250->MGC192.168.48.254 at tcp@192.168.48.254 at tcp:26/25 lens 520/544 e 0 to
1 dl 1502157943 ref 1 fl Rpc:eXN/0/ffffffff rc 0/-1
Aug 7 19:05:38 psana1510 kernel: [289134.515771] Lustre:
3333:0:(client.c:2114:ptlrpc_expire_one_request()) Skipped 1 previous
similar message
Aug 7 19:05:44 psana1510 kernel: [289140.510450] LustreError:
5566:0:(mgc_request.c:251:do_config_log_add()) MGC192.168.48.254 at tcp:
failed processing log, type 1: rc = -5
Aug 7 19:06:14 psana1510 kernel: [289170.512640] LustreError: 15c-8:
MGC192.168.48.254 at tcp: The configuration from log 'scrtch12-client'
failed (-5). This may be the result of communication errors between this
node and the MGS, a bad configuration, or other errors. See the syslog
for more information.
Aug 7 19:06:14 psana1510 kernel: [289170.517479] Lustre: Unmounted
scrtch12-client
Aug 7 19:06:14 psana1510 kernel: [289170.519294] LustreError:
5566:0:(obd_mount.c:1505:lustre_fill_super()) Unable to mount (-5)
the MDS has always TCP port 9888 closed.
I have seen that lnetctl is missing in my rpm. I rebuilt the rpm from
SPEC file. but I noticed lnetctl is missing.
could that be the reason of LNET problem ?
I noticed I Was missing libyaml on my build server and I Remember there
was a Lustre bug related to building and packaging the Lustre rpms
if libyaml is missing.
my question is if lnetctl is needed to have lustre lnet configured at
startup.
this is my lustre.conf in modprobe.d
options lnet network=tcp2(eth0)
also on my previous lustre 2.9 and 2.8 rpm builds lnetctl is missing but
LNET gets configured without troubles when lnet module is loaded
Rick
On 8/7/17 7:05 PM, Cowe, Malcolm J wrote:
> Lustre file system names cannot exceed 8 characters in length, but “scratch12” is 9 characters. Try changing the fsname to a smaller string. You can do this with tunefs.lustre on all the storage targets, but I can’t remember if you need to use --erase-params and recreate all the options. Alternatively, reformat.
>
> Malcolm.
>
> On 8/8/17, 11:30 am, "lustre-discuss on behalf of Riccardo Veraldi" <lustre-discuss-bounces at lists.lustre.org on behalf of Riccardo.Veraldi at cnaf.infn.it> wrote:
>
> trying to debug more this problem looks like tcp port 9888 is closed on
> the MDS.
> this is weird. lnet module is running. There is no firewall and OSSs and
> MDS are on the same subnet.
> but I Cannot connect to port 9888.
> There is anything which changed in Lustre 2.10.0 related to lnet and TCP
> ports that I need to take care of in the configuration ?
>
> On 8/7/17 6:13 PM, Riccardo Veraldi wrote:
> > Hello,
> >
> > I have a new Lustre cluster based on Lustre 2.10.0/ZFS 0.7.0 on Centos 7.3
> > Lustre FS creation went smooth.
> > When I tryed then to mount from the clients, Lustre is not able to mount
> > any of the OSTs.
> > It stops at MGS/MDT level.
> >
> > this is from the client side:
> >
> > mount.lustre: mount 192.168..48.254 at tcp2:/scratch12 at
> > /reg/data/scratch12 failed: Invalid argument
> > This may have multiple causes.
> > Is 'scratch12' the correct filesystem name?
> > Are the mount options correct?
> > Check the syslog for more info.
> >
> > Aug 7 17:58:53 psana1510 kernel: [285130.463377] LustreError:
> > 29240:0:(mgc_request.c:335:config_log_add()) logname scratch12-client is
> > too long
> > Aug 7 17:58:53 psana1510 kernel: [285130.463772] Lustre:
> > 3333:0:(client.c:2114:ptlrpc_expire_one_request()) @@@ Request sent has
> > failed due to network error: [sent 1502153933/real 1502153933]
> > req at ffff88203d75ec00 x1574823717093632/t0(0)
> > o250->MGC192.168.48.254 at tcp2@192.168.48.254 at tcp2:26/25 lens 520/544 e 0
> > to 1 dl 1502153938 ref 1 fl Rpc:eXN/0/ffffffff rc 0/-1
> > Aug 7 17:58:53 psana1510 kernel: [285130.469156] LustreError: 15b-f:
> > MGC192.168.48.254 at tcp2: The configuration from log
> > 'scratch12-client'failed from the MGS (-22). Make sure this client and
> > the MGS are running compatible versions of Lustre.
> > Aug 7 17:58:53 psana1510 kernel: [285130.472072] Lustre: Unmounted
> > scratch12-client
> > Aug 7 17:58:53 psana1510 kernel: [285130.473827] LustreError:
> > 29240:0:(obd_mount.c:1505:lustre_fill_super()) Unable to mount (-22)
> >
> > from the MDS side there is nothing in syslog. So I tried to engage tcpdump:
> >
> > 17:58:53.745610 IP psana1510.pcdsn.1023 >
> > psanamds12.pcdsn.cyborg-systems: Flags [S], seq 1356843681, win 29200,
> > options [mss 1460,sackOK,TS val 284847388 ecr 0,nop,wscale 7], length 0
> > 17:58:53.745644 IP psanamds12.pcdsn.cyborg-systems >
> > psana1510.pcdsn.1023: Flags [R.], seq 0, ack 1356843682, win 0, length 0
> > 17:58:58.757421 ARP, Request who-has psanamds12.pcdsn tell
> > psana1510.pcdsn, length 46
> > 17:58:58.757441 ARP, Reply psanamds12.pcdsn is-at 00:1a:4a:16:01:56 (oui
> > Unknown), length 28
> >
> > OSS, nothing in the log file or in tcpdump
> >
> > lustre client is 2.9 and the server 2.10.0
> >
> > I have no firewall running and no SElinux
> >
> > this never happened to me before. I am usually running older lustre
> > versions on clients but I never had this problem before.
> > Any hint ?
> >
> > thank you very much
> >
> > Rick
> >
> >
> >
> > _______________________________________________
> > lustre-discuss mailing list
> > lustre-discuss at lists.lustre.org
> > http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>
>
> _______________________________________________
> lustre-discuss mailing list
> lustre-discuss at lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>
>
> _______________________________________________
> lustre-discuss mailing list
> lustre-discuss at lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>
More information about the lustre-discuss
mailing list