[lustre-discuss] OST still has inodes and size after deleting all files
Pavlo Khmel
pavlo.khmel at gmail.com
Sat Jan 20 09:01:01 PST 2024
Thank you. I checked all those files with "ll_decode_filter_fid" and "lfs fid2path":
On OSS servers:
# umount /lustre/ost01
# mount -t ldiskfs /dev/mapper/ost01 /mnt/
# find /mnt/O/0/d*/ -type f
/mnt/O/0/d11/450605003
/mnt/O/0/d12/450605484
. . .
# ll_decode_filter_fid /mnt/O/0/d11/450605003
/mnt/O/0/d11/450605003: parent=[0x200019425:0x733f:0x0] stripe=0 stripe_size=1048576 stripe_count=1 layout_version=0 range=0
# umount /mnt
# mount -t lustre /dev/mapper/ost01 /lustre/ost01
On client:
# lfs fid2path /cluster [0x200019425:0x733f:0x0]
lfs fid2path: cannot find /cluster [0x200019425:0x733f:0x0]: No such file or directory
I got a reply "No such file or directory" for all those unknown files.
So I can assume those are "stray objects from deleted files".
Best regards,
Pavlo Khmel
> On 20 Jan 2024, at 04:45, Andreas Dilger <adilger at whamcloud.com> wrote:
>
>
>
>> On Jan 19, 2024, at 13:48, Pavlo Khmel via lustre-discuss <lustre-discuss at lists.lustre.org <mailto:lustre-discuss at lists.lustre.org>> wrote:
>>
>> Hi,
>>
>> I'm trying to remove 4 OSTs.
>>
>> # lfs osts
>> OBDS:
>> 0: cluster-OST0000_UUID ACTIVE
>> 1: cluster-OST0001_UUID ACTIVE
>> 2: cluster-OST0002_UUID ACTIVE
>> 3: cluster-OST0003_UUID ACTIVE
>> . . .
>>
>> I moved all files to other OSTs. "lfs find" cannot find any files on these 4 OSTs.
>>
>> # time lfs find --ost 0 --ost 1 --ost 2 --ost 3 /cluster
>>
>> real 936m8.528s
>> user 13m48.298s
>> sys 210m1.245s
>>
>> But still: 2624 inods are in use and 14.5G total size.
>>
>> # lfs df -i | grep -e OST0000 -e OST0001 -e OST0002 -e OST0003
>> cluster-OST0000_UUID 4293438576 644 4293437932 1% /cluster[OST:0]
>> cluster-OST0001_UUID 4293438576 640 4293437936 1% /cluster[OST:1]
>> cluster-OST0002_UUID 4293438576 671 4293437905 1% /cluster[OST:2]
>> cluster-OST0003_UUID 4293438576 669 4293437907 1% /cluster[OST:3]
>>
>> # lfs df -h | grep -e OST0000 -e OST0001 -e OST0002 -e OST0003
>> cluster-OST0000_UUID 29.2T 3.8G 27.6T 1% /cluster[OST:0]
>> cluster-OST0001_UUID 29.2T 3.7G 27.6T 1% /cluster[OST:1]
>> cluster-OST0002_UUID 29.2T 3.3G 27.6T 1% /cluster[OST:2]
>> cluster-OST0003_UUID 29.2T 3.7G 27.6T 1% /cluster[OST:3]
>>
>> I tried to check the file-system for errors:
>>
>> # umount /lustre/ost01
>> # e2fsck -fy /dev/mapper/ost01
>>
>> and
>>
>> # lctl lfsck_start --device cluster-OST0001
>> # lctl get_param -n osd-ldiskfs.cluster-OST0001.oi_scrub
>> . . .
>> status: completed
>>
>> I tried to mount OST as ldiskfs and there are several files in /O/0/d*/
>>
>> # umount /lustre/ost01
>> # mount -t ldiskfs /dev/mapper/ost01 /mnt/
>> # ls -Rhl /mnt/O/0/d*/
>> . . .
>> /mnt/O/0/d11/:
>> -rw-rw-rw- 1 user1 group1 603K Nov 8 21:37 450605003
>> /mnt/O/0/d12/:
>> -rw-rw-rw- 1 user1 group1 110K Jun 16 2023 450322028
>> -rw-rw-rw- 1 user1 group1 21M Nov 8 22:17 450605484
>> . . .
>>
>> Is it expected behavior? Is it save to delere OST even with those files?
>
>
> You can run the debugfs "stat" command to print the "fid" xattr and it will print the MDT
> parent FID for use with "lfs fid2path" on the client to see if there are any files related
> to these objects. You could also run "ll_decode_filter_fid" to do the same thing on the
> mounted ldiskfs filesystem.
>
> It is likely that there are a few stray objects from deleted files, but hard to say for sure.
>
> Cheers, Andreas
> --
> Andreas Dilger
> Lustre Principal Architect
> Whamcloud
>
>
>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20240120/4a5d9b6d/attachment-0003.htm>
More information about the lustre-discuss
mailing list