[Lustre-discuss] MDT fails to mount after lustre upgrade

[Anjana Kar]

After upgrading a CentOS MGS/MDS server running zfs lustre

from
lustre: 2.3.64
kernel: patchless_client
build: v2_3_64-1dkms-1-PRISTINE-2.6.32-358.2.1.el6.netboot

to
lustre: 2.5.57
kernel: patchless_client
build: 2.5.57-g3423b5b-CHANGED-2.6.32-431.11.2.el6.netboot

the MDT fails to mount with the messages below.

"/sbin/service lustre start mgsmdt" brings the zpool online:

NAME                     USED  AVAIL  REFER  MOUNTPOINT
lustre-mgs-mdt          47.2G  26.2G    30K /lustre-mgs-mdt
lustre-mgs-mdt/mgsmdt0  46.5G  26.2G  27.8G /lustre-mgs-mdt/mgsmdt0

"lctl list_nids" shows
10.10.101.160 at o2ib
128.182.75.160 at tcp10

Any suggestions on how to get MDT to mount?

Thanks,
-Anjana Kar
  Pittsburgh Supercomputing Center
  kar at psc.edu

------------------------------------------------------------------------------------------------------------------- 
2014-04-21T13:05:03.641223-04:00 icon0.psc.edu kernel: SPL: using hostid 
0xb680a04b
LustreError: 11-0: iconfs-MDT0000-lwp-MDT0000: Communicating with 0 at lo, 
operation mds_connect failed with -11.
2014-04-21T13:05:06.820694-04:00 icon0.psc.edu kernel: LustreError: 
11-0: iconfs-MDT0000-lwp-MDT0000: Communicating with 0 at lo, operation 
mds_connect failed with -11.
LustreError: 3537:0:(mdd_device.c:1050:mdd_prepare()) iconfs-MDD0000: 
failed to initialize lfsck: rc = -17
LustreError: 3537:0:(obd_mount_server.c:1776:server_fill_super()) Unable 
to start targets: -17 2014-04-21T13:05Lustre: Failing over 
iconfs-MDT0000 :09.821666-04:00 icon0.psc.edu kernel: LustreError: 
3537:0:(mdd_device.c:1050:mdd_prepare()) iconfs-MDD0000: failed to 
initialize lfsck: rc = -17
2014-04-21T13:05:09.821688-04:00 icon0.psc.edu kernel: LustreError: 
3537:0:(obd_mount_server.c:1776:server_fill_super()) Unable to start 
targets: -17 2014-04-21T13:05:09.834743-04:00 icon0.psc.edu kernel: 
Lustre: Failing over iconfs-MDT0000