[lustre-discuss] Size difference between du and quota

Martin Hecht hecht at hlrs.de
Thu May 21 09:36:38 PDT 2015


Hi,

a few more things which may play a role:

- as you are suspecting, the difference of used blocks vs. used bytes
might be the reason, especially if there are many very small files, but
there are more possible causes:

- some tools use 2^10 bytes and some others use 1000 bytes as kb which
might explain small discrpancies. du and ls are examples for different
output. However, this can not explain the whole difference.

- your find looks for regular files only (type f), but directories and
symbolic links consume a few kb as well. If there are many symbolic
links, this would be the explanation, I think.

- there are also cases in which quota can get out of sync (I don't
remember the cause, but I have already seen warnings about this in the
syslog of lustre servers). e2fsck on the ldiskfs level is supposed to
fix this issue, but I also had cases in which I had to turn off quota by
means of tune2fs on the ldisk level and turn it on again in order to
trigger something like a background quotacheck in lustre 2. In lustre
1.8 there used to be a tool "quotacheck"

- preallocated stripes on many ost's might be an issue as well, although
I don't see the discrepancy described on our file systems.

- there might also be orphane objects on the disk, i.e. stripes which
are not referenced anymore on the lustre level, but which still consume
disk space (not sure if these may affect quota). An online lfsck is
supposed to clean them up in lustre 2. In lustre 1.8 one had to run
several e2fsck on the ldisk level and build databases to run an lfsck,
but that's not supported anymore in lustre 2.

best regards,
Martin

On 05/20/2015 10:50 AM, Phill Harvey-Smith wrote:
> Hi all,
>
> One of my users is reporting a massive size difference between the
> figures reported by du and quota.
>
> doing a du -hs on his directory reports :
>  du -hs .
> 529G    .
>
> doing a lfs quota -u username /storage reports
> Filesystem  kbytes   quota   limit   grace   files   quota   limit  
> grace
> /storage 621775192  640000000 640010000       -  601284  1000000
> 1100000       -
>
> Though this user does have a lot of files :
>
> find . -type f | wc -l
> 581519
>
> So I suspect that it is the typical thing that quota is reporting used
> blocks whilst du is reporting used bytes, which can of course be
> wildly different due to filesystem overhead and wasted unused space at
> the end of files where a block is allocated but only partially used.
>
> Is this likely to be the case ?
>
> I'm also not entirely sure what versions of lustre the client machines
> and MDS / OSS servers are running, as I didn't initially set the
> system up.
>
> Cheers.
>
> Phill.
>
> _______________________________________________
> lustre-discuss mailing list
> lustre-discuss at lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org





More information about the lustre-discuss mailing list