[Lustre-discuss] question about size on MDS (MDT) for lustre-1.8

[Nathan Dauchy]

On 01/12/2011 05:45 PM, Robin Humble wrote:
> On Thu, Jan 06, 2011 at 05:42:24PM -0700, Nathan.Dauchy at noaa.gov wrote:
>> I am looking for more information regarding the "size on MDS" feature as
>> it exists for lustre-1.8.x.  Testing on our system (which started out as
>> 1.6.6 and is now 1.8.x) indicates that there are many files which do not
>> have the size information stored on the MDT.  So, my basic question:
>> under what conditions will the "size hint" attribute be updated?  Is
>> there any way to force the MDT to query the OSTs and update it's
>> information?
> 
> atime (and the MDT size hint) wasn't being updated for most of the 1.8
> series due to this bug:
>   https://bugzilla.lustre.org/show_bug.cgi?id=23766
> the atime fix is now in 1.8.5, but I'm not sure if anyone has verified
> whether or not the MDT size hint is now behaving as originally intended.

Thanks for the pointer, Robin!

> your approach of mounting and searching a MDT snapshot should be
> possible, but it would seem neater just to have a tool on a client send
> the right rpc's to the MDS and get the information that way.

It would be great to have both options available.  I was assuming the
MDT snapshot would be easier, and potentially faster than waiting for
network transactions too.

> like you, we are finding that the timescales for our filesystem
> trawling scripts are getting out of hand, mostly (we think) due to
> retrieving size information from very busy OSTs. a tool that only hit
> the MDT and found (filename, uid, gid, approx size) should help a lot.
> so +1 on this topic.

On a somewhat related note... we have recently discovered that the
object caching added in 1.8 consumes all the memory on the OSS nodes,
leaving insufficient block device cache for the inodes.  This was making
'ls -l' and 'du' run 10-20x longer than when we were running
lustre-1.6.7.  If you are running 1.8 and want to try turning off the
object caching, these are the settings you should look at:

lctl conf_param fsname-OST00XX.ost.read_cache_enable=0
lctl conf_param fsname-OST00XX.ost.writethrough_cache_enable=0

> BTW, once you have 1.8.5 on the MDS, then a hack to populate the MDT
> size hints might be to read 4k from every file in the system. that
> should update atime and the size hint. please let us know if this works.

Unfortunately, we aren't in a position to upgrade to 1.8.5 any time
soon.  If anyone can test this and see if it is possible to update the
size hint AFTER initial object creation it would be very much appreciated!

Regards,
Nathan