[Lustre-discuss] question about size on MDS (MDT) for lustre-1.8

Nathan.Dauchy at noaa.gov Nathan.Dauchy at noaa.gov
Thu Jan 6 16:42:24 PST 2011


Greetings,

I am looking for more information regarding the "size on MDS" feature as
it exists for lustre-1.8.x.  Testing on our system (which started out as
1.6.6 and is now 1.8.x) indicates that there are many files which do not
have the size information stored on the MDT.  So, my basic question:
under what conditions will the "size hint" attribute be updated?  Is
there any way to force the MDT to query the OSTs and update it's
information?


More details and background information...

We are currently running:
  Client: CentOS-5.5, kernel-2.6.18-194.11.4.el5, lustre-1.8.4
  Server: CentOS-5.3, kernel-2.6.18-164.15.1.el5, lustre-1.8.3.ddn4.1
with no immediate plans to upgrade to lustre-2.x.

I found the high level design here:
  http://wiki.lustre.org/images/9/94/Size_on_mds-hld.pdf
but that seems to be targeted at 2.0.x for full implementation.

I also discovered this old thread on the topic:
  http://lists.lustre.org/pipermail/lustre-discuss/2009-July/011149.html
in which Andreas wrote "In Lustre 1.6.7 the approximate file size
started to be stored on the MDT inodes..." and "This size is not
actively updated for pre-existing files...".  (It seems as though this
is still true for 1.8.x.)

The end goal of this is to facilitate efficient checks of disk usage on
a per-directory basis (essentially we want "volume based quotas").  I'm
hoping to run something once a day on the MDS like the following:
    lvcreate  -s -p r -n mdt_snap /dev/mdt
    mount -t ldiskfs -o ro /dev/mdt_snap /mnt/snap
    cd /mnt/snap/ROOT
    du --apparent-size ./* > volume_usage.log
    cd /
    umount /mnt/snap
    lvremove /dev/mdt_snap
Since the data is going to be up to one day old anyway, I don't really
mind that the file size is "approximate", but it does have to be
reasonably close.

With the MDT LVM snapshot method I can check the whole 300TB file system
in about 3 hours, whereas checking from a client takes weeks.

Here is why I am relatively certain that the size-on-MDS attributes are
not updated (lightly edited):

[root at mds0 ~]# ls -l /mnt/snap/ROOT/test/rollover/user_acct_file
-rw-r--r-- 1 9999 9000 0 Mar 23  2010
/mnt/snap/ROOT/test/rollover/user_acct_file
[root at mds0 ~]# du /mnt/snap/ROOT/test/rollover/user_acct_file
0       /mnt/snap/ROOT/test/rollover/user_acct_file
[root at mds0 ~]# du --apparent-size
/mnt/snap/ROOT/test/rollover/user_acct_file
0       /mnt/snap/ROOT/test/rollover/user_acct_file

[root at c448 ~]# ls -l /mnt/lfs0/test/rollover/user_acct_file
-rw-r--r-- 1 user group 184435207 Mar 23  2010
/mnt/lfs0/test/rollover/user_acct_file
[root at c448 ~]# du /mnt/lfs0/test/rollover/user_acct_file
180120  /mnt/lfs0/test/rollover/user_acct_file
[root at c448 ~]# du --apparent-size /mnt/lfs0/test/rollover/user_acct_file
180113  /mnt/lfs0/test/rollover/user_acct_file


Thanks very much for any answers or suggestions you can provide!

-Nathan




More information about the lustre-discuss mailing list