[lustre-discuss] Question about max service threads

[Andreas Dilger]

There is actually a parameter for this:

$ lctl get_param ost.OSS.*.thread*
ost.OSS.ost.threads_max=16
ost.OSS.ost.threads_min=3
ost.OSS.ost.threads_started=16
ost.OSS.ost_create.threads_max=10
ost.OSS.ost_create.threads_min=2
ost.OSS.ost_create.threads_started=3
ost.OSS.ost_io.threads_max=16
ost.OSS.ost_io.threads_min=3
ost.OSS.ost_io.threads_started=16
ost.OSS.ost_out.threads_max=10
ost.OSS.ost_out.threads_min=2
ost.OSS.ost_out.threads_started=2
ost.OSS.ost_seq.threads_max=10
ost.OSS.ost_seq.threads_min=2
ost.OSS.ost_seq.threads_started=2

$ lctl get_param mds.MDS.*.thread*
mds.MDS.mdt.threads_max=80
mds.MDS.mdt.threads_min=3
mds.MDS.mdt.threads_started=11
mds.MDS.mdt_fld.threads_max=256
mds.MDS.mdt_fld.threads_min=2
mds.MDS.mdt_fld.threads_started=3
mds.MDS.mdt_io.threads_max=80
mds.MDS.mdt_io.threads_min=3
mds.MDS.mdt_io.threads_started=4
mds.MDS.mdt_out.threads_max=80
mds.MDS.mdt_out.threads_min=2
mds.MDS.mdt_out.threads_started=2
mds.MDS.mdt_readpage.threads_max=56
mds.MDS.mdt_readpage.threads_min=2
mds.MDS.mdt_readpage.threads_started=3
mds.MDS.mdt_seqm.threads_max=256
mds.MDS.mdt_seqm.threads_min=2
mds.MDS.mdt_seqm.threads_started=2
mds.MDS.mdt_seqs.threads_max=256
mds.MDS.mdt_seqs.threads_min=2
mds.MDS.mdt_seqs.threads_started=2
mds.MDS.mdt_setattr.threads_max=56
mds.MDS.mdt_setattr.threads_min=2
mds.MDS.mdt_setattr.threads_started=2


On Sep 21, 2021, at 19:21, Patrick Farrell <pfarrell at ddn.com<mailto:pfarrell at ddn.com>> wrote:

“
Though I can wait for the number threads to automatically decrease, I didn’t find ways which can really indicate the current running threads. I’ve tried thread_started (e.g., lctl get_param mds.MDS.mdt.threads_,started). But this param doesn’t change. ”

I don’t think Lustre exposes a stat which gives *current* count of worker threads.  I’ve always used ps, grep, and wc -l to answer that question :)
________________________________
From: lustre-discuss <lustre-discuss-bounces at lists.lustre.org<mailto:lustre-discuss-bounces at lists.lustre.org>> on behalf of Andreas Dilger via lustre-discuss <lustre-discuss at lists.lustre.org<mailto:lustre-discuss at lists.lustre.org>>
Sent: Tuesday, September 21, 2021 8:03 PM
To: Houkun Zhu <diskun.zhu at gmail.com<mailto:diskun.zhu at gmail.com>>
Cc: lustre-discuss at lists.lustre.org<mailto:lustre-discuss at lists.lustre.org> <lustre-discuss at lists.lustre.org<mailto:lustre-discuss at lists.lustre.org>>
Subject: Re: [lustre-discuss] Question about max service threads

Hello Houkun,
There was patch https://review.whamcloud.com/34400 "LU-947<https://jira.whamcloud.com/browse/LU-947> ptlrpc: allow stopping threads above threads_max" landed for the 2.13 release. You could apply this patch to your 2.12 release, or test with 2.14.0. Note that this patch only lazily stops threads as they become idle, so there is no guarantee that they will all stop immediately when the parameter is changed. It may be some time and processed RPCs before the higher-numbered threads exit.

It might be possible to wake up all of the threads when the threads_max parameter is reduced, to have them check for this condition and exit. However, this is a very unlikely condition under normal usage.

I would recommend to test with increasing the thread count, rather than decreasing it...

Cheers, Andreas

On Sep 20, 2021, at 02:29, Houkun Zhu via lustre-discuss <lustre-discuss at lists.lustre.org<mailto:lustre-discuss at lists.lustre.org>> wrote:


Hi guys,

I’m creating an automatic lustre performance tuning system. But I find it’s hard to tune parameter regarding  max service threads because it seems there is only guarantee of max threads when we increase the parameter. I’ve found a similar discussion<http://lists.lustre.org/htdig.cgi/lustre-discuss-lustre.org/2011-August/010358.html> from 2011, is there any updates?

Though I can wait for the number threads to automatically decrease, I didn’t find ways which can really indicate the current running threads. I’ve tried thread_started (e.g., lctl get_param mds.MDS.mdt.threads_,started). But this param doesn’t change.

Looking forward to your help! Thank you in advance!

Best regards,
Houkun

_______________________________________________
lustre-discuss mailing list
lustre-discuss at lists.lustre.org<mailto:lustre-discuss at lists.lustre.org>
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Cheers, Andreas
--
Andreas Dilger
Lustre Principal Architect
Whamcloud







-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20210922/cef5d9e8/attachment-0001.html>