[lustre-discuss] unable to precreate -52/-116

[Marco Grossi]

Hi Amit,

Sounds definitely different from my case.

The only JIRA issue logging a "precreate fid < local used fid" is:
https://jira.whamcloud.com/browse/LU-11536

What puzzle me is the "rc = -52" on the "ofd_create_hdl"; if I mapped it
correctly, is a -EBADE error, i.e. "invalid exchange".

Can you provide:
- HA node list and location of MGS, MDT and OST between nodes

As well as the output of:
- lfs df
- lfs df -i
- lctl get_param osp.*scratch0-OST0029*.prealloc*
- lctl get_param obdfilter.*scratch0-OST0029*.last_id

Regards,
Marco


On 3/9/20 5:23 PM, Kumar, Amit wrote:
> Hi Marco,
> 
> Thank you for the response on this issue.  
> 
> We have an HA setup, I tried to fail over MDT to the secondary pair and then fail it back. This did not help. 
> I also tried restart of the MDS servers, that did not help.
> I have rebooted OSS servers as well, that did not help
> I also tried completely stopping MDS and unmounting MDS for a little while and that did not help either. 
> 
> This error ritually comes back right after MDT is mounted. Additionally I am not able to manually create any files on that particular OST. Any other thoughts.  
> 
> Thank you,
> Amit
> 
> -----Original Message-----
> From: Marco Grossi <marco.grossi at ichec.ie> 
> Sent: Monday, March 9, 2020 11:23 AM
> To: Kumar, Amit <ahkumar at mail.smu.edu>
> Cc: lustre-discuss at lists.lustre.org
> Subject: Re: [lustre-discuss] unable to precreate -52/-116
> 
> Hi Amit,
> 
> We had a similar issue after a set_param of "max_create_count=0"
> 
> In our case re-mounting the MDT (not the OST) fixed the issue.
> 
> Hope it helps.
> 
> Regards,
> Marco
> 
> 
> On 3/3/20 8:25 PM, Kumar, Amit wrote:
>> Dear Lustre,
>>
>>  
>>
>> Recently we had a degraded(Not failed) RAID and had to wait longer to 
>> get compatible disk, as we had received incompatible one and it took 
>> over a week to get the correct one back in place.
>>
>>  
>>
>> During this wait I ended up disabling the OST first and then noticed 
>> continuous IO to the OST and thought of disabling object creation on 
>> it as well. Everything looked normal after that and once the disk was 
>> replaced I reenabled object creation and enabled OST. Since then I 
>> started seeing these messages on OST
>>
>> .(ofd_dev.c:1784:ofd_create_hdl()) scratch0-OST0029: unable to
>> precreate: rc = -52
>>
>> And following messages on MDS
>>
>> .(osp_precreate.c:1282:osp_precreate_thread())
>> scratch0-OST0029-osc-MDT0000: cannot precreate objects: rc = -116
>>
>> .(osp_precreate.c:657:osp_precreate_send())
>> scratch0-OST0029-osc-MDT0000: precreate fid 
>> [0x100290000:0x101b39a:0x0] < local used fid 
>> [0x100290000:0x101b39a:0x0]: rc = -116
>>
>>  
>>
>> These messages don't seem to stop. I am wondering what impact could 
>> these errors have in long run? I have noticed I am not able to create 
>> files on this particular OST using lfs setstripe, when I do so it gets 
>> me an object on another OST by default. Just want to make sure this is 
>> not causing any data loss for files the currently on them and new requests?
>>
>> We plan to upgrade to 2.12 in the summer downtime and assuming that 
>> has a fix based on LU-9442 & LU-11186.  Currently running servers on 
>> lustre
>> 10.4.1 over ZFS-0.7.9-1
>>
>>  
>>
>> Any help is greatly appreciated.
>>
>>  
>>
>> Thank you,
>> Amit
>>
>>
>> _______________________________________________
>> lustre-discuss mailing list
>> lustre-discuss at lists.lustre.org
>> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>>
> 
> --
> Marco Grossi
> ICHEC Systems Team
> 
> 
> ----IF CLASSIFICATION START----
> 
> ----IF CLASSIFICATION END----
> 

-- 
Marco Grossi
ICHEC Systems Team