[lustre-discuss] Errors large directory feature ldiskfs

[Fernando Perez]

Thank you Andreas and Artem.

We have had problems in the filesystem due to this issue, because a user 
had an automatic process (cron) that tried to write a lot of new files 
in this folder each 15 minutes.

After some days there were a lot of hang process in the clients, the 
mds+mdt server had the uptime very high, and the metadata operations ran 
very slow in the rest of the lustre clients.

We solve the problem restarting lustre and deleting the stuck folder, 
because all the commands to try to see or move or copy files from this 
folder hang.

Regards.

=============================================
Fernando Pérez
Institut de Ciències del Mar (CSIC)
Departament Oceanografía Física i Tecnològica
Passeig Marítim de la Barceloneta,37-49
08003 Barcelona
Phone:  (+34) 93 230 96 35
=============================================

On 10/04/2018 02:54 PM, Andreas Dilger wrote:
> On Oct 4, 2018, at 04:20, Fernando Perez <fperez at icm.csic.es> wrote:
>> Thank you Artem.
>>
>> Do you know if the lustre filesystem can work properly with these errors and without enable this feature?
> Yes, this essentially means someone is creating huge directories (10M entries or more), and they will see ENOSPC errors when trying to create files in that directory.
>
> You can see which directory it is by looking up the FID on a client:
>
>      lfs fid2path <mountpoint> [0x200002347:0x1:0x0]
>
> With directories this large, user tools like "ls" and "rm" don't work very well, which is one reason why there hasn't been a priority to allow larger directories.  Enabling the large_dir feature is one way to handle this (though it is still in testing), and creating a DNE striped directory for this directory across multiple MDTs is another way.
>
> Cheers, Andreas
>
>
>> On 10/04/2018 11:47 AM, Artem Blagodarenko wrote:
>>> Hello Fernando,
>>>
>>> The option is enabled by command:
>>>
>>> tune2fs -O large_dir <ldiskfs_partition>
>>>
>>> BUT, this feature is still in testing, so I do not suggest  use it in production.
>>> Also, current Lustre FS e2fsprogs don’t support this feature. Now it available only for EXT4 since E2fsprogs 1.44.0 (March 7, 2018) and linux 4.13.
>>> I hope this feature will be enabled in WC e2fsprogs which will be released with Lustre FS 2.12.
>>>
>>> Best regards,
>>> Artem Blagodarenko.
>>>
>>>> On 4 Oct 2018, at 12:28, Fernando Perez <fperez at icm.csic.es> wrote:
>>>>
>>>> Sorry, we have lustre 2.10.4 release installed in our lustre filesystem, migrated previously from 2.4 to 2.8 and 2.10.4
>>>>
>>>> Regards.
>>>>
>>>> =============================================
>>>> Fernando Pérez
>>>> Institut de Ciències del Mar (CSIC)
>>>> Departament Oceanografía Física i Tecnològica
>>>> Passeig Marítim de la Barceloneta,37-49
>>>> 08003 Barcelona
>>>> Phone:  (+34) 93 230 96 35
>>>> =============================================
>>>>
>>>> On 10/04/2018 11:16 AM, Fernando Perez wrote:
>>>>> Dear all.
>>>>>
>>>>> We have a lustre system with combined mds + mdt ldiskfs and several osts with zfs and ldiskfs.
>>>>>
>>>>> I have these errors in our mds server log:
>>>>>
>>>>> LDISKFS-fs warning (device dm-4): ldiskfs_dx_add_entry:2618: inode 30544460: comm mdt01_047: index 2: reach max htree level 2
>>>>> Oct  4 09:32:24 nemo kernel: LDISKFS-fs warning (device dm-4): ldiskfs_dx_add_entry:2622: Large directory feature is not enabled on this filesystem
>>>>> Oct  4 09:32:24 nemo kernel: Lustre: 25646:0:(osd_handler.c:466:osd_ldiskfs_add_entry()) lustre-MDT0000: directory (inode: 30544460, FID: [0x200002347:0x1:0x0]) has reached maximum entry limit
>>>>>
>>>>> How can I enable the large directory feature in our lustre filesystem to solve these errors?
>>>>>
>>>>> I can't see references to this feature in the lustre documentation. Is it related with the LU-1365?
>>>>>
>>>>> Regards.
>>>>>
>>>> _______________________________________________
>>>> lustre-discuss mailing list
>>>> lustre-discuss at lists.lustre.org
>>>> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>> _______________________________________________
>> lustre-discuss mailing list
>> lustre-discuss at lists.lustre.org
>> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
> Cheers, Andreas
> ---
> Andreas Dilger
> CTO Whamcloud
>
>
>
>