[lustre-discuss] Tuning for metadata performance

[Vicker, Darby J. (JSC-EG111)[Jacobs Technology, Inc.]]

Sure. Thanks for the ideas.  

Yes, I think that is correct.  If I understand correctly, that's one reason DoM is faster for small files.  But my understanding is also limited.  

By a "single OSS", do you mean the same OSS for all files?  Or just 1 OSS for each individual file (but not necessarily the same OSS for all files).  I think you mean the latter.  All the lustre results I've sent so far are effectively using a single OSS (but not the same OSS) for all/almost all the files.  Our default PFL uses a single OST up to 32 MB, 4 OST's up to 1GB and 8 OST's beyond that.  In the git repo I've been using for this test, there are only 6 files bigger than 32 MB.  And there was the test where I explicitly set the stripe count to 1 for all files (ephemeral1s).  

-----Original Message-----
From: lustre-discuss <lustre-discuss-bounces at lists.lustre.org> on behalf of Michael Di Domenico <mdidomenico4 at gmail.com>
Date: Thursday, January 14, 2021 at 7:06 AM
Cc: "lustre-discuss at lists.lustre.org" <lustre-discuss at lists.lustre.org>
Subject: Re: [lustre-discuss] [EXTERNAL] Re: Tuning for metadata performance

    interesting.  thanks for the detail.

    completely off the cuff, i could be totally wrong here, and i maybe
    misremembering something, but i believe i recall reading somewhere,
    when getxattr and getattr are called the MDT contacts the OSS's as
    well to get some info.  if that's true, this double hop is probably
    the extra wait time. the nfs server wouldn't have to do this, it just
    checks the local disk.

    if i'm not crazy and what i said has some truth, the one thing that
    might show a differential is if you lock all the files in the repo to
    a single OSS (you may have done this, i'm not sure)

    Again I'm no lustre genius, just trying to help.  the lustre dev's
    likely can explain better than i can.  i might be leading you down a
    rabbit hole.