[lustre-discuss] question regarding du vs df on lustre

[Einar Næss Jensen]

Thank you for answering Andreas.

Lustre version is 2.12.8

It is indeed when we delete io500 files when we discovered this, but we also see it when deleting other files, that the "df" lags 1-2 hours behind.
We see it both on nvme and ssd drives. Haven't checkd hdd drives/osts yet.

This is a new lustre setup, and benchmarks are good (in our opinion). For now it is just this annoyance bugging us. 
We didn't notice on previous lustre setup but will check if we see it there also.


Einar




________________________________________
From: Andreas Dilger <adilger at whamcloud.com>
Sent: Monday, April 11, 2022 18:01
To: Einar Næss Jensen
Cc: lustre-discuss at lists.lustre.org
Subject: Re: [lustre-discuss] question regarding du vs df on lustre

Lustre is returning the file unlink from the MDS immediately, but deleting
the objects from the OSTs asynchronously in the background.

How many files are being deleted in this case?  If you are running tests
like IO500, where there are many millions of small files plus some huge
files, then it may be that huge object deletion is behind small objects?

That said, it probably shouldn't take hours to finish if the OST storage is
NVMe based.

Cheers, Andreas

> On Apr 4, 2022, at 05:05, Einar Næss Jensen <einar.nass.jensen at ntnu.no> wrote:
>
> Hello lustre people.
>
> We are experimenting with lustre on nvme, and observe the following issue:
> After running benchmarks and deleting benchmark files, we see that df and du reports different sizes:
>
> [root at idun-02-27 ~]#  du -hs /nvme/
> 38M     /nvme/
> [root at idun-02-27 ~]# df -h|grep nvme
> 10.3.1.2 at o2ib:/nvme                   5.5T  3.9T  1.3T  76% /nvme
>
>
> It takes several hours before du and df agrees.
>
> What is causing this?
> How can we get updated records for df immediately when deleting files?
>
>
> Best REegards
> Einar
> _______________________________________________
> lustre-discuss mailing list
> lustre-discuss at lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org