[lustre-discuss] quota problem on lustre 2.5.3 when doing lfs_migrate
Philippe Weill
Philippe.Weill at latmos.ipsl.fr
Wed Apr 15 22:38:31 PDT 2015
hello
we have problem with quota on a 2.5.3 filesystem (ext4 based on
scientific linux 6 )when using lfs_migrate after desactivating an ost on
mds
file seem to be well migrated but allocated space stay in quota on the
desactivated device
see sample in attached text document
--
Weill Philippe - Administrateur Systeme et Reseaux
CNRS/UPMC/IPSL LATMOS (UMR 8190)
-------------- next part --------------
[admtest at cluster ~]$ lfs df -h /data
UUID bytes Used Available Use% Mounted on
datafs-MDT0000_UUID 299.9G 5.7G 274.3G 2% /data[MDT:0]
datafs-OST0000_UUID 51.5T 43.7T 7.3T 86% /data[OST:0]
datafs-OST0001_UUID 51.5T 42.7T 8.3T 84% /data[OST:1]
datafs-OST0002_UUID 51.5T 42.9T 8.0T 84% /data[OST:2]
datafs-OST0003_UUID 51.5T 43.4T 7.6T 85% /data[OST:3]
datafs-OST0004_UUID 51.5T 43.0T 8.0T 84% /data[OST:4]
datafs-OST0005_UUID 51.5T 42.8T 8.2T 84% /data[OST:5]
datafs-OST0006_UUID 57.7T 610.3G 56.5T 1% /data[OST:6]
datafs-OST0007_UUID 57.7T 774.6G 56.3T 1% /data[OST:7]
datafs-OST0008_UUID 57.7T 775.7G 56.3T 1% /data[OST:8]
datafs-OST0009_UUID 57.7T 729.1G 56.4T 1% /data[OST:9]
filesystem summary: 539.6T 261.2T 273.0T 49% /data
admtest at cluster ~]$ lfs quota -uv admtest /data/admtest
Disk quotas for user admtest (uid 1310):
Filesystem kbytes quota limit grace files quota limit grace
/data/admtest 4 838860800 1048576000 - 2 300000 310000 -
datafs-MDT0000_UUID
4 - 0 - 2 - 65536 -
datafs-OST0000_UUID
0 - 1177916 - - - - -
datafs-OST0001_UUID
0 - 16777216 - - - - -
datafs-OST0002_UUID
0 - 2416584 - - - - -
datafs-OST0003_UUID
0 - 116088 - - - - -
datafs-OST0004_UUID
0 - 2174436 - - - - -
datafs-OST0005_UUID
0 - 1263524 - - - - -
datafs-OST0006_UUID
0 - 7650664 - - - - -
datafs-OST0007_UUID
0 - 4194304 - - - - -
datafs-OST0008_UUID
0 - 16777216 - - - - -
datafs-OST0009_UUID
0 - 4194304 - - - - -
[admtest at cluster ~]$ dd if=/dev/zero of=/data/admtest/10G bs=1M count=10240
10240+0 enregistrements lus
10240+0 enregistrements écrits
10737418240 octets (11 GB) copiés, 21,4806 s, 500 MB/s
[admtest at cluster ~]$ lfs getstripe /data/admtest/10G
/data/admtest/10G
lmm_stripe_count: 1
lmm_stripe_size: 1048576
lmm_stripe_offset: 1
obdidx objid objid group
1 4322572 0x41f50c 0
[admtest at cluster ~]# lfs quota -uv admtest /data
Disk quotas for user admtest (uid 1310):
Filesystem kbytes quota limit grace files quota limit grace
/data 10485796 838860800 1048576000 - 3 300000 310000 -
datafs-MDT0000_UUID
4 - 0 - 3 - 65536 -
datafs-OST0000_UUID
0 - 1177916 - - - - -
datafs-OST0001_UUID
10485792 - 16777216 - - - - -
datafs-OST0002_UUID
0 - 2416584 - - - - -
datafs-OST0003_UUID
0 - 116088 - - - - -
datafs-OST0004_UUID
0 - 2174436 - - - - -
datafs-OST0005_UUID
0 - 1263524 - - - - -
datafs-OST0006_UUID
0 - 7650664 - - - - -
datafs-OST0007_UUID
0 - 4194304 - - - - -
datafs-OST0008_UUID
0 - 16777216 - - - - -
datafs-OST0009_UUID
0 - 4194304 - - - - -
Total allocated inode limit: 65536, total allocated block limit: 56742252
[root at mds2-ipsl ~]# lctl dl |grep data
3 UP osd-ldiskfs datafs-MDT0000-osd datafs-MDT0000-osd_UUID 17
5 UP lod datafs-MDT0000-mdtlov datafs-MDT0000-mdtlov_UUID 4
6 UP mdt datafs-MDT0000 datafs-MDT0000_UUID 79
7 UP mdd datafs-MDD0000 datafs-MDD0000_UUID 4
8 UP qmt datafs-QMT0000 datafs-QMT0000_UUID 4
9 UP osp datafs-OST0000-osc-MDT0000 datafs-MDT0000-mdtlov_UUID 5
10 UP osp datafs-OST0001-osc-MDT0000 datafs-MDT0000-mdtlov_UUID 5
11 UP osp datafs-OST0002-osc-MDT0000 datafs-MDT0000-mdtlov_UUID 5
12 UP osp datafs-OST0003-osc-MDT0000 datafs-MDT0000-mdtlov_UUID 5
13 UP osp datafs-OST0004-osc-MDT0000 datafs-MDT0000-mdtlov_UUID 5
14 UP osp datafs-OST0005-osc-MDT0000 datafs-MDT0000-mdtlov_UUID 5
15 UP lwp datafs-MDT0000-lwp-MDT0000 datafs-MDT0000-lwp-MDT0000_UUID 5
28 UP osp datafs-OST0006-osc-MDT0000 datafs-MDT0000-mdtlov_UUID 5
29 UP osp datafs-OST0007-osc-MDT0000 datafs-MDT0000-mdtlov_UUID 5
30 UP osp datafs-OST0008-osc-MDT0000 datafs-MDT0000-mdtlov_UUID 5
31 UP osp datafs-OST0009-osc-MDT0000 datafs-MDT0000-mdtlov_UUID 5
[root at mds2-ipsl ~]# lctl --device 10 deactivate
Lustre: setting import datafs-OST0001_UUID INACTIVE by administrator request
[root at nfs-lustre ~]# lfs_migrate -y /data/admtest/10G
/data/admtest/10G: done
[root at nfs-lustre ~]# lfs getstripe /data/admtest/10G
/data/admtest/10G
lmm_stripe_count: 1
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 1
lmm_stripe_offset: 8
obdidx objid objid group
8 139595 0x2214b 0
[admtest at cluster ~]$ lfs quota -uv admtest /data/admtest
Disk quotas for user admtest (uid 1310):
Filesystem kbytes quota limit grace files quota limit grace
/data/admtest 20971560 838860800 1048576000 - 4 300000 310000 -
datafs-MDT0000_UUID
4 - 0 - 4 - 65536 -
datafs-OST0000_UUID
0 - 1177916 - - - - -
datafs-OST0001_UUID
10485792 - 16777216 - - - - -
datafs-OST0002_UUID
0 - 2416584 - - - - -
datafs-OST0003_UUID
0 - 116088 - - - - -
datafs-OST0004_UUID
0 - 2174436 - - - - -
datafs-OST0005_UUID
0 - 1263524 - - - - -
datafs-OST0006_UUID
0 - 7650664 - - - - -
datafs-OST0007_UUID
0 - 4194304 - - - - -
datafs-OST0008_UUID
10485764 - 16777216 - - - - -
datafs-OST0009_UUID
0 - 4194304 - - - - -
More information about the lustre-discuss
mailing list