[lustre-discuss] Full OST

[Andreas Dilger]

On Sep 3, 2021, at 01:49, Alastair Basden <a.g.basden at durham.ac.uk<mailto:a.g.basden at durham.ac.uk>> wrote:

Hi,

We have a file system where each OST is a single SSD.

One of those is reporting as 100% full (lfs df -h /snap8):
snap8-OST004d_UUID          5.8T        2.0T        3.5T  37% /snap8[OST:77]
snap8-OST004e_UUID          5.8T        5.5T        7.5G 100% /snap8[OST:78]
snap8-OST004f_UUID          5.8T        2.0T        3.4T  38% /snap8[OST:79]

However, I can't find any files on it:
lfs find --ost snap8-OST004e /snap8/
returns nothing.

I guess that it has filled up, and that there is some bug or other that is now preventing proper behaviour - but I could be wrong.

Does anyone have any suggestions?

Essentially, I'd like to find some of the files and delete or migrate some, and thus return it to useful production.

Next step would be to unmount OST004e, run a full e2fsck, and then check lost+found and/or a regular "find /mnt/ost -type f -size +1M" or similar to find where the files are.

Cheers, Andreas
--
Andreas Dilger
Lustre Principal Architect
Whamcloud







-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20210908/941b198d/attachment.html>