[lustre-discuss] lfsck repair quota

Yong, Fan fan.yong at intel.com
Wed Apr 17 07:42:15 PDT 2019


For ldiskfs based backend, the e2fsck will link ldiskfs orphan inodes into backend /lost+found directory that is invisible to Lustre namespace. After that, you can run namespace LFSCK that will move the orphan inodes (with valid FID) from the backend /lost+found to Lustre lost+found directory that is visible to Lustre namespace (exported as $mount/.lustre/lost+found). Then you can manually check and decide whether keep or remove them.

Anyway, the orphan objects/inodes are counted by Lustre quota even though they were invisible to applications. The LFSCK does not repair quota, instead, it can make some invisible objects to become visible. Then the space used by visible objects (can be counted via client side user space statistics tools, such as "du") may be more close the quota usage.

On the other hand, the layout LFSCK may find out some OST-side orphans, and add them to the Lustre lost+found directory. That is also helpful for quota unmatched trouble.

But if the quota accounting object itself crashed, then LFSCK can help nothing.


--
Cheers,
Nasf

-----Original Message-----
From: lustre-discuss [mailto:lustre-discuss-bounces at lists.lustre.org] On Behalf Of Fernando Perez
Sent: Wednesday, April 17, 2019 4:33 PM
To: lustre-discuss at lists.lustre.org
Subject: Re: [lustre-discuss] lfsck repair quota

Dear Martin.

I tried to run the e2fsck in the mdt three years ago and the logs shows a lot of this kind of messages:

> Unattached inode 26977505
> Connect to /lost+found? yes
> Inode 26977505 ref count is 2, should be 1. Fix? yes

In fact I think that the e2fsc ran so slow due that all the mdt inodes were corrected.

Regards.

=============================================
Fernando Pérez
Institut de Ciències del Mar (CSIC)
Departament Oceanografía Física i Tecnològica Passeig Marítim de la Barceloneta,37-49
08003 Barcelona
Phone:  (+34) 93 230 96 35
=============================================

On 4/16/19 8:43 PM, Martin Hecht wrote:
> Are there a lot of inodes moved to lost+found by the fsck, which contribute to the occupied quota now?
>
> ----- Ursprüngliche Mail -----
> Von: Fernando Pérez <fperez at icm.csic.es>
> An: lustre-discuss at lists.lustre.org
> Gesendet: Tue, 16 Apr 2019 16:24:13 +0200 (CEST)
> Betreff: Re: [lustre-discuss] lfsck repair quota
>
> Thank you Rick.
>
> I followed these steps for the ldiskfs OSTs and MDT, but the quotes for all users is more corrupted than before.
>
> I tried to run e2fsck in ldiskfs OSTs MDT, but the problem was the MDT e2fsck ran very slow ( 10 inodes per second for more than 100 million inodes).
>
> According to the lustre wiki I though that the lfsck could repair corrupted quotes:
>
> http://wiki.lustre.org/Lustre_Quota_Troubleshooting
>
> Regards.
>
> ============================================
> Fernando Pérez
> Institut de Ciències del Mar (CSIC)
> Departament Oceanografía Física i Tecnològica Passeig Marítim de la 
> Barceloneta,37-49
> 08003 Barcelona
> Phone:  (+34) 93 230 96 35
> ============================================
>
>> El 16 abr 2019, a las 15:34, Mohr Jr, Richard Frank (Rick Mohr) <rmohr at utk.edu> escribió:
>>
>>
>>> On Apr 15, 2019, at 10:54 AM, Fernando Perez <fperez at icm.csic.es> wrote:
>>>
>>> Could anyone confirm me that the correct way to repair wrong quotes in a ldiskfs mdt is lctl lfsck_start -t layout -A?
>> As far as I know, lfsck doesn’t repair quota info. It only fixes internal consistency within Lustre.
>>
>> Whenever I have had to repair quotas, I just follow the procedure you 
>> did (unmount everything, run “tune2fs -O ^quota <dev>”, run “tune2fs 
>> -O quota <dev>”, and then remount).  But all my systems used ldiskfs, 
>> so I don’t know if the ZFS OSTs introduce any sort of complication.  
>> (Actually, I am not even sure if/how you can regenerate quota info 
>> for ZFS.)
>>
>> --
>> Rick Mohr
>> Senior HPC System Administrator
>> National Institute for Computational Sciences 
>> http://www.nics.tennessee.edu
>>
_______________________________________________
lustre-discuss mailing list
lustre-discuss at lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org


More information about the lustre-discuss mailing list