[Lustre-discuss] 1.6.7.2 Adding an OST issues

[Scott Barber]

All MGS/MDT/OSTs running CentOS 5.4 x86_64 2.6.18-92.1.26.el5_lustre.1.6.7.2smp

A few weeks ago I had added an OST (OST000c) incorrectly. I set it as
inactive by:
lctl --device 17 deactivate
and
lctl conf_param webcache-OST000c.osc.active=0

No data was ever written to it, so no harm done. I then reformatted
the OST properly and added it (OST000d) to the lustre successfully.
The volume ran great.

I have a few more OSTs on a new OSS to add to the volume. The came
from another newer lustre array (v1.8.1.1) that we were "downsizing"
so I removed the 1.8.1.1 lustre RPMs and installed the v1.6.7.2 RPMs
and rebooted. I then reformatted the OSTs like this:
mkfs.lustre -v --reformat --fsname webcache --ost
--mgsnode=webcache-mds01 at tcp /dev/sdb
and
mkfs.lustre -v --reformat --fsname webcache --ost
--mgsnode=webcache-mds01 at tcp /dev/sdc

I mounted them up, but the volume size did not increase. After
checking the MDS I found something weird.
Here's a 'lctl dl' on the MGS:
[root at webcache-mds01 ~]# lctl dl
  0 UP mgs MGS MGS 124
  1 UP mgc MGC10.4.20.1 at tcp 0dbf20ab-e2af-2170-ee62-3c92fc63b2a7 5
  2 UP mdt MDS MDS_uuid 3
  3 UP lov webcache-mdtlov webcache-mdtlov_UUID 4
  4 UP mds webcache-MDT0000 webcache-MDT0000_UUID 245
  5 UP osc webcache-OST0000-osc webcache-mdtlov_UUID 5
  6 UP osc webcache-OST0001-osc webcache-mdtlov_UUID 5
  7 UP osc webcache-OST0002-osc webcache-mdtlov_UUID 5
  8 UP osc webcache-OST0003-osc webcache-mdtlov_UUID 5
  9 UP osc webcache-OST0004-osc webcache-mdtlov_UUID 5
 10 UP osc webcache-OST0005-osc webcache-mdtlov_UUID 5
 11 UP osc webcache-OST0006-osc webcache-mdtlov_UUID 5
 12 UP osc webcache-OST0007-osc webcache-mdtlov_UUID 5
 13 UP osc webcache-OST0008-osc webcache-mdtlov_UUID 5
 14 UP osc webcache-OST0009-osc webcache-mdtlov_UUID 5
 15 UP osc webcache-OST000a-osc webcache-mdtlov_UUID 5
 16 UP osc webcache-OST000b-osc webcache-mdtlov_UUID 5
 17 UP osc webcache-OST000c-osc webcache-mdtlov_UUID 5
 18 UP osc webcache-OST000d-osc webcache-mdtlov_UUID 5
 19 UP osc webcache-OST000e-osc webcache-OST000c-osc-mdtlov_UUID 4
 20 UP osc webcache-OST000f-osc webcache-OST000c-osc-mdtlov_UUID 4

Why do the 2 new OST have "webcache-OST000c-osc-mdtlov_UUID 4" instead
of "webcache-mdtlov_UUID 5" after the OSC name?
When I mounted the 2 new OSTs the syslog on the MDS shows:

Mar 29 22:35:07 webcache-mds01 kernel: LustreError:
1377:0:(obd_config.c:836:class_process_config()) no device for:
webcache-OST000c-osc-mdtlov
Mar 29 22:35:07 webcache-mds01 kernel: LustreError:
1377:0:(obd_config.c:1102:class_config_llog_handler()) Err -22 on cfg
command:
Mar 29 22:35:07 webcache-mds01 kernel: Lustre:    cmd=cf00d
0:webcache-OST000c-osc-mdtlov  1:webcache-OST000e_UUID  2:14  3:1

Mar 29 22:36:49 webcache-mds01 kernel: LustreError:
1384:0:(obd_config.c:836:class_process_config()) no device for:
webcache-OST000c-osc-mdtlov
Mar 29 22:36:49 webcache-mds01 kernel: LustreError:
1384:0:(obd_config.c:1102:class_config_llog_handler()) Err -22 on cfg
command:
Mar 29 22:36:49 webcache-mds01 kernel: Lustre:    cmd=cf00d
0:webcache-OST000c-osc-mdtlov  1:webcache-OST000f_UUID  2:15  3:1

Something is right and I'm not sure where to start digging. Thoughts?

Thanks,
Scott Barber
Senior Systems Administrator
iMemories.com