[Lustre-devel] Research on filesystem metadata operation distribution

Andreas Dilger adilger at whamcloud.com
Thu May 5 11:05:42 PDT 2011

On May 5, 2011, at 11:54, Thomas Roth wrote:
> At GSI, we have
> lctl get_param mds.*.stats | egrep "open|close|rename|link|attr|sync"
> open                      10302752480 samples [reqs]
> close                     528292519 samples [reqs]
> unlink                    22292174 samples [reqs]
> rename                    542512 samples [reqs]
> getxattr                  408511496 samples [reqs]
> setxattr                  838368 samples [reqs]
> setattr                   27097846 samples [reqs]
> getattr                   738233548 samples [reqs]
> Output of 'lfs df' is attached.

While it wasn't my original goal in asking for this data, a question I've been asking all of the sites that have filesystems with OSTs of different sizes is whether they use (and/or continue to use) an external process for balancing the space used on the OSTs (e.g. migrating of files, or marking OSTs inactive on the MDS once they have reached some threshold of space usage), or if the existing space balancing mechanism in the MDS was able to get this relatively uniform space utilization?

> The filesystem is used for storing HEP data from theory calculations, simulations and HEP experimental data for use in analysis. People also use it as a software repository, to compile their programs (ouch) and as a general purpose distributed file system (a certain sysadmin is known to store his music files there).
> Regards,
> Thomas
> On 04/21/2011 08:40 PM, Andreas Dilger wrote:
>> I'm trying to get some data about the relative distribution of MDS operations in the wild, and I'd be grateful if some people with production filesystems that have been running for at least a week could collect some simple stats and email them to me.  They can be collected by any regular user on the MDS node:
>>     lctl get_param mds.*.stats | egrep "open|close|rename|link|attr|sync"
>> It would be useful to also include "lfs df" and "lfs df -i" information, as well as a brief description of what the filesystem is used for (scratch, home, project, archive, etc).
>> As a reminder, I'm also interested if some Lustre admins could run the "fsstats" tool from http://www.pdsi-scidac.org/fsstats/ and send me the output.  Sending the output to PDSI via their submission form may also produce some positive results.
>>     http://www.pdsi-scidac.org/fsstats/files/fsstats-1.4.5.tar.gz
>> Thanks in advance for any data.  I've set replies to go only to lustre-devel, to avoid clogging the larger readership of lustre-discuss, but it may be useful for others to have this in a list archive and/or searchable via Google in the future so I don't necessarily want to keep it all to myself.
>> Cheers, Andreas
>> --
>> Andreas Dilger
>> Principal Engineer
>> Whamcloud, Inc.
>> _______________________________________________
>> Lustre-discuss mailing list
>> Lustre-discuss at lists.lustre.org
>> http://lists.lustre.org/mailman/listinfo/lustre-discuss
> -- 
> --------------------------------------------------------------------
> Thomas Roth
> Department: Informationstechnologie
> GSI Helmholtzzentrum für Schwerionenforschung GmbH
> Planckstraße 1
> 64291 Darmstadt
> www.gsi.de
> Gesellschaft mit beschränkter Haftung
> Sitz der Gesellschaft: Darmstadt
> Handelsregister: Amtsgericht Darmstadt, HRB 1528
> Geschäftsführung: Professor Dr. Dr. h.c. Horst Stöcker,
> Dr. Hartmut Eickhoff
> Vorsitzende des Aufsichtsrates: Dr. Beatrix Vierkorn-Rudolph
> Stellvertreter: Ministerialdirigent Dr. Rolf Bernhardt
> <lfs_df.txt><lfs_df_i.txt>_______________________________________________
> Lustre-devel mailing list
> Lustre-devel at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-devel

Cheers, Andreas
Andreas Dilger 
Principal Engineer
Whamcloud, Inc.

More information about the lustre-devel mailing list