[Lustre-discuss] Lustre MDS Queries

Andreas Dilger adilger at sun.com
Wed May 14 16:35:40 PDT 2008


On May 14, 2008  14:36 -0500, Daniel Ferber wrote:
> I need to find out what MDS data I can access directly from an application,
> via IO system calls or Lustre commands ­ assuming I am the owner of the
> file. 

You can access file attributes (e.g. owner, group, timestamps, and soon
file size) from the MDS.

> Or is any access to MDS data indirect, e.g. stat() and does that go just
> to the MDS, or to the file as well?

All file data is currently resident only on the OSTs, so data access is
always "indirect" as you term it.

> I have a customer who has a large number of 1K files, that I suspect are
> simply pointers to something ­ a crude MDS in other words. So on one hand
> this is a perfect app for Lustre, just load the real files into Lustre and
> the Lustre MDS points to the actual files!
> 
> But if the small files point to other objects that are not files, like a
> physical file cabinet, are there system calls or commands that can be used
> to load and read data into the Lustre MDS itself?

Having 1kB files in Lustre is probably not a very good use model.  There
will be 512+ bytes of metadata used in the MDT filesystem, and then another
512+ bytes of metadata in the OST filesystem, and a minimum of 4kB for the
data block.  This is less efficient than just having a single file server
that stores the file data directly.

There has always been discussion about more efficient small-file data
storage in Lustre, putting the file data in the MDT filesystem in that
case, but there hasn't been enough demand for it.  It is kind of circular
though, because lack of this capability reduces demand...


Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.




More information about the lustre-discuss mailing list