[lustre-discuss] Slow release of inodes on OST

Åke Sandgren ake.sandgren at hpc2n.umu.se
Wed Feb 5 22:42:00 PST 2020



On 2/6/20 6:58 AM, Andreas Dilger wrote:
> On Feb 4, 2020, at 07:23, Åke Sandgren <ake.sandgren at hpc2n.umu.se
> <mailto:ake.sandgren at hpc2n.umu.se>> wrote:
>>
>> When I create a large number of files on an OST and then remove them,
>> the used inode count on the OST decreases very slowly, it takes several
>> hours for it to go from 3M to the correct ~10k.
>>
>> (I'm running the io500 test suite)
>>
>> Is there something I can do to make it release them faster?
>> Right now it has gone from 3M to 1.5M in 6 hours, (lfs df -i).
> 
> It this the object count or the file count?  Are you possibly using a lot of
> stripes on the files being deleted that is multiplying the work needed?
> 
>> These are SSD based OST's in case it matters.
> 
> The MDS controls the destroy of the OST objects, so there is a rate
> limit, but ~700/s seems low to me, especially for SSD OSTs.
> 
> You could check "lctl get_param osp.*.sync*" on the MDS to see how
> many destroys are pending.  Also, increasing osp.*.max_rpcs_in_flight
> on the MDS might speed this up?  It should default to 32 per OST on
> the MDS vs. default 8 for clients

Should have checked before sending first mail...
The default on the MDS is apparently 8. This is 2.13.0 and I did not
change any params.

lctl get_param osp.*.max_rpcs_in_flight
osp.rsos-OST0000-osc-MDT0000.max_rpcs_in_flight=8

-- 
Ake Sandgren, HPC2N, Umea University, S-90187 Umea, Sweden
Internet: ake at hpc2n.umu.se   Phone: +46 90 7866134 Fax: +46 90-580 14
Mobile: +46 70 7716134 WWW: http://www.hpc2n.umu.se


More information about the lustre-discuss mailing list