[Lustre-discuss] Adding OST's not increasing storage?

[Josephine Palencia]

Test Setup: (combining them first while ENOMEM error fixed for mgs hardware)
------
mds00w.psc.teragrid.org: combined mdt/mgs and client
oss00w.psc.teragrid.org: ost1
oss00w.psc.teragrid.org: ost2
operon22.psc.edu       : ost3 and client


Combined mgs/mdt on mds00w:
-----------------------------
root at mds00w ~]# df -h
/dev/sda8             9.9G  489M  8.9G   6% /mnt/test/mdt


Added 1 ost from oss00w with 1.4TB
------------------
[root at oss00w ~]# df -h
/dev/sda7             1.3T  1.1G  1.3T   1% /mnt/test/ost0

Shows 1.3T storage correctly on mds00w as client.
root at mds00w ~]# df -h
mds00w at tcp0:/testfs   1.3T  1.1G  1.3T   1% /mnt/testfs

Added 2nd ost from oss01w  with 1.4TB:
------------------
[root at oss01w ~]# df -h
/dev/sda7             1.3T  1.1G  1.3T   1% /mnt/test/ost1

mds00w shows both osts active
-------------------
root at mds00w ~]# cat 
/proc/fs/lustre/lov/testfs-clilov-ffff81007c659000/target_obd
0: testfs-OST0000_UUID ACTIVE
1: testfs-OST0001_UUID ACTIVE
[root at mds00w ~]# cat /proc/fs/lustre/lov/testfs-MDT0000-mdtlov/target_obd
0: testfs-OST0000_UUID ACTIVE
1: testfs-OST0001_UUID ACTIVE

but df from mds00w shows  only 1.3T storage available  (other 1.3T in used?)
Should be around 2.6T


Adding a third ost with only 150GB storage gives 142GB storage showing on
both clients. Total storage should be around 2.7TB.
-------------

[root at operon22 ~]# df -h
/dev/hdb1             151G  1.1G  142G   1% /mnt/test/ost3
mds00w at tcp0:/testfs   2.8T  2.6T  142G  95% /mnt/testfs
[root at mds00w ~]# df -h
/dev/sda8             9.9G  489M  8.9G   6% /mnt/test/mdt
mds00w at tcp0:/testfs   2.8T  2.6T  142G  95% /mnt/testfs


[root at mds00w ~]# cat /proc/fs/lustre/devices
   0 UP mgs MGS MGS 11
   1 UP mgc MGC128.182.112.60 at tcp e6af805d-1e32-b002-d315-54fb78e7e558 5
   2 UP lov testfs-MDT0000-mdtlov testfs-MDT0000-mdtlov_UUID 4
   3 UP mdt testfs-MDT0000 testfs-MDT0000_UUID 7
   4 UP mds mdd_obd-testfs-MDT0000-0 mdd_obd_uuid-testfs-MDT0000-0 3
   5 UP osc testfs-OST0000-osc-MDT0000 testfs-MDT0000-mdtlov_UUID 5
   6 UP lov testfs-clilov-ffff8101278aec00 f327440b-f5e9-c9cc-66fc-7a6001402368 4
   7 UP lmv testfs-clilmv-ffff8101278aec00 f327440b-f5e9-c9cc-66fc-7a6001402368 4
   8 UP mdc testfs-MDT0000-mdc-ffff8101278aec00 f327440b-f5e9-c9cc-66fc-7a6001402368 5
   9 UP osc testfs-OST0000-osc-ffff8101278aec00 f327440b-f5e9-c9cc-66fc-7a6001402368 5
  10 UP osc testfs-OST0001-osc-ffff8101278aec00 f327440b-f5e9-c9cc-66fc-7a6001402368 5
  11 UP osc testfs-OST0001-osc-MDT0000 testfs-MDT0000-mdtlov_UUID 5
  12 UP osc testfs-OST0002-osc-ffff8101278aec00 f327440b-f5e9-c9cc-66fc-7a6001402368 5
  13 UP osc testfs-OST0002-osc-MDT0000 testfs-MDT0000-mdtlov_UUID 5

Show all OST's active.

[root at mds00w ~]# cat /proc/fs/lustre/lov/testfs-clilov-ffff8101278aec00/target_obd
0: testfs-OST0000_UUID ACTIVE
1: testfs-OST0001_UUID ACTIVE
2: testfs-OST0002_UUID ACTIVE
[root at mds00w ~]# cat /proc/fs/lustre/lov/testfs-MDT0000-mdtlov/target_obd
0: testfs-OST0000_UUID ACTIVE
1: testfs-OST0001_UUID ACTIVE
2: testfs-OST0002_UUID ACTIVE

Encountering issues deactivating/activating OST's as well but that's in 
another email.

Thanks,
josephin


PS.
Version:  lustre: 1.9.50
Kernel-2.6.18-92.1.6-lustre-1.9.50 #2 SMP x86_64