From andreas.dilger at intel.com Sun Dec 3 00:47:03 2017 From: andreas.dilger at intel.com (Dilger, Andreas) Date: Sun, 3 Dec 2017 00:47:03 +0000 Subject: [lustre-devel] [PATCH] staging: lustre: check result of register_shrinker In-Reply-To: <20171202184046.4338-1-akaraliou.dev@gmail.com> References: <20171202184046.4338-1-akaraliou.dev@gmail.com> Message-ID: <985AF256-3022-45FB-BFCF-1E2B0F622AA8@intel.com> On Dec 2, 2017, at 11:40, Aliaksei Karaliou wrote: > > Lustre code lacks checking the result of register_shrinker() > in several places. register_shrinker() was tagged __must_check > recently so that sparse has started reporting it. Thank you for your patch. Some comments below. > Signed-off-by: Aliaksei Karaliou > --- > drivers/staging/lustre/lustre/ldlm/ldlm_pool.c | 7 +++++-- > drivers/staging/lustre/lustre/obdclass/lu_object.c | 5 +++-- > drivers/staging/lustre/lustre/osc/osc_request.c | 4 +++- > drivers/staging/lustre/lustre/ptlrpc/sec_bulk.c | 13 ++++++++----- > 4 files changed, 19 insertions(+), 10 deletions(-) > > diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c b/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c > index da65d00a7811..7795ececa6d3 100644 > --- a/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c > +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c > @@ -1086,8 +1086,11 @@ int ldlm_pools_init(void) > int rc; > > rc = ldlm_pools_thread_start(); > - if (rc == 0) > - register_shrinker(&ldlm_pools_cli_shrinker); > + if (rc == 0) { > + rc = register_shrinker(&ldlm_pools_cli_shrinker); > + if (rc) > + ldlm_pools_thread_stop(); > + } Rather than nesting conditionals like this, kernel style prefers to use "goto label" to do the cleanup in one place at the end of the function. That keeps the indentation level reasonable, and avoids making the error handling increasingly complex if more conditions are added in the future. The preferred way to handle this would be: rc = ldlm_pools_thread_start(); if (rc) goto out; rc = register_shrinker(&ldlm_pools_cli_shrinker); if (rc) goto out_pools; return 0; out_pools: ldlm_pools_thread_stop(); out: goto out; } Then, if a new error condition is added, it just means an "out_shrinker:" label is added and calling unregister_shrinker() at that point. > diff --git a/drivers/staging/lustre/lustre/obdclass/lu_object.c b/drivers/staging/lustre/lustre/obdclass/lu_object.c > index b938a3f9d50a..9e0256ca2079 100644 > --- a/drivers/staging/lustre/lustre/obdclass/lu_object.c > +++ b/drivers/staging/lustre/lustre/obdclass/lu_object.c > @@ -1951,7 +1951,7 @@ int lu_global_init(void) > * inode, one for ea. Unfortunately setting this high value results in > * lu_object/inode cache consuming all the memory. > */ > - register_shrinker(&lu_site_shrinker); > + result = register_shrinker(&lu_site_shrinker); > > return result; > } > @@ -1961,7 +1961,8 @@ int lu_global_init(void) > */ > void lu_global_fini(void) > { > - unregister_shrinker(&lu_site_shrinker); > + if (lu_site_shrinker.nr_deferred) > + unregister_shrinker(&lu_site_shrinker); > lu_context_key_degister(&lu_global_key); Initially, I didn't think this was needed, but it seems that unregister_shrinker() is not safe to be called on a shrinker that was not initialized properly. While the above check makes this callsite safe, and I'm OK with landing this part of the patch, it might be safer for the kernel as a whole if unregister_shrinker() was made safe against this internally? Something like: void unregister_shrinker(struct shrinker *shrinker) { + if (!shrinker->nr_deferred) + return; + down_write(&shrinker_rwsem); list_del(&shrinker->list); up_write(&shrinker_rwsem); kfree(shrinker->nr_deferred); + shrinker->nr_deferred = NULL; } That avoids the need for the caller to "know" about nr_deferred. > > /* > diff --git a/drivers/staging/lustre/lustre/osc/osc_request.c b/drivers/staging/lustre/lustre/osc/osc_request.c > index 53eda4c99142..45b1ebf33363 100644 > --- a/drivers/staging/lustre/lustre/osc/osc_request.c > +++ b/drivers/staging/lustre/lustre/osc/osc_request.c > @@ -2844,7 +2844,9 @@ static int __init osc_init(void) > if (rc) > goto out_kmem; > > - register_shrinker(&osc_cache_shrinker); > + rc = register_shrinker(&osc_cache_shrinker); > + if (rc) > + goto out_type; > > /* This is obviously too much memory, only prevent overflow here */ > if (osc_reqpool_mem_max >= 1 << 12 || osc_reqpool_mem_max == 0) { > diff --git a/drivers/staging/lustre/lustre/ptlrpc/sec_bulk.c b/drivers/staging/lustre/lustre/ptlrpc/sec_bulk.c > index 77a3721beaee..4eeff832fd4a 100644 > --- a/drivers/staging/lustre/lustre/ptlrpc/sec_bulk.c > +++ b/drivers/staging/lustre/lustre/ptlrpc/sec_bulk.c > @@ -396,6 +396,8 @@ static struct shrinker pools_shrinker = { > > int sptlrpc_enc_pool_init(void) > { > + int rc = -ENOMEM; > + > /* > * maximum capacity is 1/8 of total physical memory. > * is the 1/8 a good number? > @@ -429,12 +431,13 @@ int sptlrpc_enc_pool_init(void) > page_pools.epp_st_outofmem = 0; > > enc_pools_alloc(); > - if (!page_pools.epp_pools) > - return -ENOMEM; > - > - register_shrinker(&pools_shrinker); > + if (page_pools.epp_pools) { > + rc = register_shrinker(&pools_shrinker); > + if (rc) > + enc_pools_free(); > + } Same comment here as above - please use labels to handle error cleanup. > - return 0; > + return rc; > } > > void sptlrpc_enc_pool_fini(void) > -- > 2.11.0 > Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Intel Corporation From andreas.dilger at intel.com Mon Dec 4 08:20:29 2017 From: andreas.dilger at intel.com (Dilger, Andreas) Date: Mon, 4 Dec 2017 08:20:29 +0000 Subject: [lustre-devel] [PATCH] staging: lustre: Fix sparse, using plain integer as NULL pointer in lov_object_fiemap() In-Reply-To: References: Message-ID: <7A48E652-421B-464B-AD2A-C3D5613B48E5@intel.com> > On Nov 30, 2017, at 11:30, Andrii Vladyka wrote: > > Change 0 to NULL in lov_object_fiemap() in order to fix warning produced by sparse > > Signed-off-by: Andrii Vladyka Patches should be inline rather than in an attachment. That said, the patch looks correct, so you can add: Signed-off-by: Andreas Dilger Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Intel Corporation From dan.carpenter at oracle.com Mon Dec 4 08:33:04 2017 From: dan.carpenter at oracle.com (Dan Carpenter) Date: Mon, 4 Dec 2017 11:33:04 +0300 Subject: [lustre-devel] [PATCH] staging: lustre: check result of register_shrinker In-Reply-To: <20171202184046.4338-1-akaraliou.dev@gmail.com> References: <20171202184046.4338-1-akaraliou.dev@gmail.com> Message-ID: <20171204083304.se5gcmgw5iywsskj@mwanda> On Sat, Dec 02, 2017 at 09:40:46PM +0300, Aliaksei Karaliou wrote: > Lustre code lacks checking the result of register_shrinker() > in several places. register_shrinker() was tagged __must_check > recently so that sparse has started reporting it. > > Signed-off-by: Aliaksei Karaliou > --- > drivers/staging/lustre/lustre/ldlm/ldlm_pool.c | 7 +++++-- > drivers/staging/lustre/lustre/obdclass/lu_object.c | 5 +++-- > drivers/staging/lustre/lustre/osc/osc_request.c | 4 +++- > drivers/staging/lustre/lustre/ptlrpc/sec_bulk.c | 13 ++++++++----- > 4 files changed, 19 insertions(+), 10 deletions(-) > > diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c b/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c > index da65d00a7811..7795ececa6d3 100644 > --- a/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c > +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c > @@ -1086,8 +1086,11 @@ int ldlm_pools_init(void) > int rc; > > rc = ldlm_pools_thread_start(); > - if (rc == 0) > - register_shrinker(&ldlm_pools_cli_shrinker); > + if (rc == 0) { > + rc = register_shrinker(&ldlm_pools_cli_shrinker); > + if (rc) > + ldlm_pools_thread_stop(); > + } > > return rc; Originally, the code had two anti-patterns 1) Making the last check in the function different from the others, 2) Success handling instead of failure handling. Success handling leads to spaghetti code. Write it like this instead. rc = ldlm_pools_thread_start(); if (rc) return rc; rc = register_shrinker(&ldlm_pools_cli_shrinker); if (rc) { ldlm_pools_thread_stop(); return rc; } return 0; > } > diff --git a/drivers/staging/lustre/lustre/obdclass/lu_object.c b/drivers/staging/lustre/lustre/obdclass/lu_object.c > index b938a3f9d50a..9e0256ca2079 100644 > --- a/drivers/staging/lustre/lustre/obdclass/lu_object.c > +++ b/drivers/staging/lustre/lustre/obdclass/lu_object.c > @@ -1951,7 +1951,7 @@ int lu_global_init(void) > * inode, one for ea. Unfortunately setting this high value results in > * lu_object/inode cache consuming all the memory. > */ > - register_shrinker(&lu_site_shrinker); > + result = register_shrinker(&lu_site_shrinker); There should be some error handling if the register fails. > > return result; > } > @@ -1961,7 +1961,8 @@ int lu_global_init(void) > */ > void lu_global_fini(void) > { > - unregister_shrinker(&lu_site_shrinker); > + if (lu_site_shrinker.nr_deferred) > + unregister_shrinker(&lu_site_shrinker); > lu_context_key_degister(&lu_global_key); > > /* > diff --git a/drivers/staging/lustre/lustre/osc/osc_request.c b/drivers/staging/lustre/lustre/osc/osc_request.c > index 53eda4c99142..45b1ebf33363 100644 > --- a/drivers/staging/lustre/lustre/osc/osc_request.c > +++ b/drivers/staging/lustre/lustre/osc/osc_request.c > @@ -2844,7 +2844,9 @@ static int __init osc_init(void) > if (rc) > goto out_kmem; > > - register_shrinker(&osc_cache_shrinker); > + rc = register_shrinker(&osc_cache_shrinker); > + if (rc) > + goto out_type; > > /* This is obviously too much memory, only prevent overflow here */ > if (osc_reqpool_mem_max >= 1 << 12 || osc_reqpool_mem_max == 0) { > diff --git a/drivers/staging/lustre/lustre/ptlrpc/sec_bulk.c b/drivers/staging/lustre/lustre/ptlrpc/sec_bulk.c > index 77a3721beaee..4eeff832fd4a 100644 > --- a/drivers/staging/lustre/lustre/ptlrpc/sec_bulk.c > +++ b/drivers/staging/lustre/lustre/ptlrpc/sec_bulk.c > @@ -396,6 +396,8 @@ static struct shrinker pools_shrinker = { > > int sptlrpc_enc_pool_init(void) > { > + int rc = -ENOMEM; > + > /* > * maximum capacity is 1/8 of total physical memory. > * is the 1/8 a good number? > @@ -429,12 +431,13 @@ int sptlrpc_enc_pool_init(void) > page_pools.epp_st_outofmem = 0; > > enc_pools_alloc(); > - if (!page_pools.epp_pools) > - return -ENOMEM; > - > - register_shrinker(&pools_shrinker); > + if (page_pools.epp_pools) { > + rc = register_shrinker(&pools_shrinker); > + if (rc) > + enc_pools_free(); > + } > > - return 0; > + return rc; This new code is *really* hard to read. The original was more readable. Do it like this: if (!page_pools.epp_pools) return -ENOMEM; rc = register_shrinker(&pools_shrinker); if (rc) { enc_pools_free(); return rc; } return 0; regards, dan carpenter From dan.carpenter at oracle.com Mon Dec 4 08:35:00 2017 From: dan.carpenter at oracle.com (Dan Carpenter) Date: Mon, 4 Dec 2017 11:35:00 +0300 Subject: [lustre-devel] [PATCH] staging: lustre: check result of register_shrinker In-Reply-To: <985AF256-3022-45FB-BFCF-1E2B0F622AA8@intel.com> References: <20171202184046.4338-1-akaraliou.dev@gmail.com> <985AF256-3022-45FB-BFCF-1E2B0F622AA8@intel.com> Message-ID: <20171204083500.hqpc3f5arnti46pd@mwanda> On Sun, Dec 03, 2017 at 12:47:03AM +0000, Dilger, Andreas wrote: > The preferred way to handle this would be: > > rc = ldlm_pools_thread_start(); > if (rc) > goto out; > > rc = register_shrinker(&ldlm_pools_cli_shrinker); > if (rc) > goto out_pools; > > return 0; > > out_pools: > ldlm_pools_thread_stop(); > out: > goto out; > } I so badly hate "out" as a label name... But yours made me laugh. regards, dan carpenter From dan.carpenter at oracle.com Mon Dec 4 08:40:18 2017 From: dan.carpenter at oracle.com (Dan Carpenter) Date: Mon, 4 Dec 2017 11:40:18 +0300 Subject: [lustre-devel] [PATCH] staging: lustre: check result of register_shrinker In-Reply-To: References: <20171202184046.4338-1-akaraliou.dev@gmail.com> <985AF256-3022-45FB-BFCF-1E2B0F622AA8@intel.com> Message-ID: <20171204084018.jxfunaapcx36jd6r@mwanda> On Sun, Dec 03, 2017 at 07:59:07PM +0300, ak wrote: > Thank you for your extensive comments. > > I've also thought about adding more protection into unregister_shrinker(), > but not sure how to properly organize the patch set, because there will be > three patches: > * change in mm/vmscan that adds protection and sanitizer. > * fixed change for Lustre driver > * there also two explicit usages of shrinker->nr_deferred in drivers - > good idea to fix too. > All patches have different lists of maintainers, and second and third depend > on first one. And I don't like to send them separately. > So, I'm going to at least prepend this patch with mm/vmscan one. Fix the style for the Lustre patch and resend. Then patch unregister_shrinker(). Then remove the checks. The unregister_shrinker() changes seem like a good idea, but I haven't really looked at it. It might be more involved than it seems. regards, dan carpenter From dan.carpenter at oracle.com Mon Dec 4 19:16:31 2017 From: dan.carpenter at oracle.com (Dan Carpenter) Date: Mon, 4 Dec 2017 22:16:31 +0300 Subject: [lustre-devel] [PATCH] staging: lustre: check result of register_shrinker In-Reply-To: <908ca41a-4fba-1f0f-ddc5-c5e5e3fd1142@gmail.com> References: <20171202184046.4338-1-akaraliou.dev@gmail.com> <985AF256-3022-45FB-BFCF-1E2B0F622AA8@intel.com> <20171204084018.jxfunaapcx36jd6r@mwanda> <908ca41a-4fba-1f0f-ddc5-c5e5e3fd1142@gmail.com> Message-ID: <20171204191631.bkzaruzrlepp3rqy@mwanda> On Mon, Dec 04, 2017 at 09:42:12PM +0300, Aliaksei Karaliou wrote: > On 12/04/2017 11:40 AM, Dan Carpenter wrote: > > On Sun, Dec 03, 2017 at 07:59:07PM +0300, ak wrote: > > > Thank you for your extensive comments. > > > > > > I've also thought about adding more protection into unregister_shrinker(), > > > but not sure how to properly organize the patch set, because there will be > > > three patches: > > > * change in mm/vmscan that adds protection and sanitizer. > > > * fixed change for Lustre driver > > > * there also two explicit usages of shrinker->nr_deferred in drivers - > > > good idea to fix too. > > > All patches have different lists of maintainers, and second and third depend > > > on first one. And I don't like to send them separately. > > > So, I'm going to at least prepend this patch with mm/vmscan one. > > Fix the style for the Lustre patch and resend. Then patch > > unregister_shrinker(). Then remove the checks. > > > > The unregister_shrinker() changes seem like a good idea, but I haven't > > really looked at it. It might be more involved than it seems. > > > > regards, > > dan carpenter > Thanks for the comments too. > I'll send patch with accumulated fixes. > > > } > > > diff --git a/drivers/staging/lustre/lustre/obdclass/lu_object.c b/drivers/staging/lustre/lustre/obdclass/lu_object.c > > > index b938a3f9d50a..9e0256ca2079 100644 > > > --- a/drivers/staging/lustre/lustre/obdclass/lu_object.c > > > +++ b/drivers/staging/lustre/lustre/obdclass/lu_object.c > > > @@ -1951,7 +1951,7 @@ int lu_global_init(void) > > > * inode, one for ea. Unfortunately setting this high value results in > > > * lu_object/inode cache consuming all the memory. > > > */ > > > - register_shrinker(&lu_site_shrinker); > > > + result = register_shrinker(&lu_site_shrinker); > > There should be some error handling if the register fails. > Yeah, I think so, but it seems that it is out of scope of this patch. > The whole negative branch in the mainline kernel looks broken (IMHO). No, it's not. If you introduce new error paths, you're expected to make them clean up instead of leaking resources. regards, dan carpenter From romeusmeister at gmail.com Mon Dec 4 19:45:02 2017 From: romeusmeister at gmail.com (Roman Storozhenko) Date: Mon, 4 Dec 2017 22:45:02 +0300 Subject: [lustre-devel] [PATCH v3] staging: lustre: Replace 'uint32_t' with 'u32' and 'uint64_t' with 'u64' In-Reply-To: <20171130060846.jghmmojo45ropbol@mwanda> References: <20171129164621.GA10881@home> <20171130060846.jghmmojo45ropbol@mwanda> Message-ID: <20171204194502.GA12057@home> On Thu, Nov 30, 2017 at 09:08:46AM +0300, Dan Carpenter wrote: > On Wed, Nov 29, 2017 at 07:46:21PM +0300, Roman Storozhenko wrote: > > There are two reasons for that: > > In my email client the subject line and body are not next to each other. > It looks like this: > > https://marc.info/?l=linux-arm-kernel&m=151187366315885&w=2 > > So it took me a while to realize what you were talking about. Please > assume I'm either reading the subject or the body but not both. > > regards, > dan carpenter > Hello Dan, It's really strange. Moreover the screenshot that you have provided doesn't relate to my patch. Anyway I checked the source file of the message (I use 'mutt -H ) and found that body goes below the subject with one empty separate line (according to patch format recomendations). Could you check is your message related to my patch? If so, please provide the screenshot of my patch. Thanks in advance, Roman Storozhenko From dan.carpenter at oracle.com Mon Dec 4 19:58:23 2017 From: dan.carpenter at oracle.com (Dan Carpenter) Date: Mon, 4 Dec 2017 22:58:23 +0300 Subject: [lustre-devel] [PATCH v3] staging: lustre: Replace 'uint32_t' with 'u32' and 'uint64_t' with 'u64' In-Reply-To: <20171129164621.GA10881@home> References: <20171129164621.GA10881@home> Message-ID: <20171204195823.yiwitsbbhcywwa3m@mwanda> On Wed, Nov 29, 2017 at 07:46:21PM +0300, Roman Storozhenko wrote: > There are two reasons for that: What I'm asking is there are two reasons for what? Where is the first part of that paragraph? regards, dan carpenter From dan.carpenter at oracle.com Mon Dec 4 20:01:34 2017 From: dan.carpenter at oracle.com (Dan Carpenter) Date: Mon, 4 Dec 2017 23:01:34 +0300 Subject: [lustre-devel] [PATCH v3] staging: lustre: Replace 'uint32_t' with 'u32' and 'uint64_t' with 'u64' In-Reply-To: <20171204194502.GA12057@home> References: <20171129164621.GA10881@home> <20171130060846.jghmmojo45ropbol@mwanda> <20171204194502.GA12057@home> Message-ID: <20171204200134.m56ucf4jfnvln2lz@mwanda> On Mon, Dec 04, 2017 at 10:45:02PM +0300, Roman Storozhenko wrote: > On Thu, Nov 30, 2017 at 09:08:46AM +0300, Dan Carpenter wrote: > > On Wed, Nov 29, 2017 at 07:46:21PM +0300, Roman Storozhenko wrote: > > > There are two reasons for that: > > > > In my email client the subject line and body are not next to each other. > > It looks like this: > > > > https://marc.info/?l=linux-arm-kernel&m=151187366315885&w=2 > > > > So it took me a while to realize what you were talking about. Please > > assume I'm either reading the subject or the body but not both. > > > > regards, > > dan carpenter > > > Hello Dan, > > It's really strange. Moreover the screenshot that you have provided doesn't > relate to my patch. Anyway I checked the source file of the message > (I use 'mutt -H ) and found that body goes below the > subject with one empty separate line (according to patch format > recomendations). Mutt is very configurable. Mine looks like the marc.info link. The patch was chosen at random to show how the subject and body are not next to each other. regards, dan carpenter From andreas.dilger at intel.com Tue Dec 5 00:50:25 2017 From: andreas.dilger at intel.com (Dilger, Andreas) Date: Tue, 5 Dec 2017 00:50:25 +0000 Subject: [lustre-devel] [PATCH] staging: lustre: check result of register_shrinker In-Reply-To: <908ca41a-4fba-1f0f-ddc5-c5e5e3fd1142@gmail.com> References: <20171202184046.4338-1-akaraliou.dev@gmail.com> <985AF256-3022-45FB-BFCF-1E2B0F622AA8@intel.com> <20171204084018.jxfunaapcx36jd6r@mwanda> <908ca41a-4fba-1f0f-ddc5-c5e5e3fd1142@gmail.com> Message-ID: On Dec 4, 2017, at 11:42, Aliaksei Karaliou wrote: > > On 12/04/2017 11:40 AM, Dan Carpenter wrote: >> On Sun, Dec 03, 2017 at 07:59:07PM +0300, ak wrote: >>> Thank you for your extensive comments. >>> >>> I've also thought about adding more protection into unregister_shrinker(), >>> but not sure how to properly organize the patch set, because there will be >>> three patches: >>> * change in mm/vmscan that adds protection and sanitizer. >>> * fixed change for Lustre driver >>> * there also two explicit usages of shrinker->nr_deferred in drivers - >>> good idea to fix too. >>> All patches have different lists of maintainers, and second and third depend >>> on first one. And I don't like to send them separately. >>> So, I'm going to at least prepend this patch with mm/vmscan one. >> Fix the style for the Lustre patch and resend. Then patch >> unregister_shrinker(). Then remove the checks. >> >> The unregister_shrinker() changes seem like a good idea, but I haven't >> really looked at it. It might be more involved than it seems. >> >> regards, >> dan carpenter > Thanks for the comments too. > I'll send patch with accumulated fixes. >>> } >>> diff --git a/drivers/staging/lustre/lustre/obdclass/lu_object.c b/drivers/staging/lustre/lustre/obdclass/lu_object.c >>> index b938a3f9d50a..9e0256ca2079 100644 >>> --- a/drivers/staging/lustre/lustre/obdclass/lu_object.c >>> +++ b/drivers/staging/lustre/lustre/obdclass/lu_object.c >>> @@ -1951,7 +1951,7 @@ int lu_global_init(void) >>> * inode, one for ea. Unfortunately setting this high value results in >>> * lu_object/inode cache consuming all the memory. >>> */ >>> - register_shrinker(&lu_site_shrinker); >>> + result = register_shrinker(&lu_site_shrinker); >> There should be some error handling if the register fails. > Yeah, I think so, but it seems that it is out of scope of this patch. > The whole negative branch in the mainline kernel looks broken (IMHO). > In mainline Lustre's git there is a reworked version of upper function obdclass_init(), > which at least calls lu_global_fini() before exiting `module_init` on further failure, > but yeah, still lacks proper cleanup inside lu_global_initcall(). > I'll add one more patch in a patch-set so that maintainers may decide what to do with that. I was looking at the lack of error handling as well, but then I wondered if the module_init() call returns an error, is module_exit() still called, or does the module not load at all in the error case? Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Intel Corporation From dan.carpenter at oracle.com Tue Dec 5 09:33:40 2017 From: dan.carpenter at oracle.com (Dan Carpenter) Date: Tue, 5 Dec 2017 12:33:40 +0300 Subject: [lustre-devel] [PATCH] staging: lustre: check result of register_shrinker In-Reply-To: References: <20171202184046.4338-1-akaraliou.dev@gmail.com> <985AF256-3022-45FB-BFCF-1E2B0F622AA8@intel.com> <20171204084018.jxfunaapcx36jd6r@mwanda> <908ca41a-4fba-1f0f-ddc5-c5e5e3fd1142@gmail.com> Message-ID: <20171205093340.5u2krbdmoaage5b3@mwanda> On Tue, Dec 05, 2017 at 12:50:25AM +0000, Dilger, Andreas wrote: > On Dec 4, 2017, at 11:42, Aliaksei Karaliou wrote: > > > > On 12/04/2017 11:40 AM, Dan Carpenter wrote: > >> On Sun, Dec 03, 2017 at 07:59:07PM +0300, ak wrote: > >>> Thank you for your extensive comments. > >>> > >>> I've also thought about adding more protection into unregister_shrinker(), > >>> but not sure how to properly organize the patch set, because there will be > >>> three patches: > >>> * change in mm/vmscan that adds protection and sanitizer. > >>> * fixed change for Lustre driver > >>> * there also two explicit usages of shrinker->nr_deferred in drivers - > >>> good idea to fix too. > >>> All patches have different lists of maintainers, and second and third depend > >>> on first one. And I don't like to send them separately. > >>> So, I'm going to at least prepend this patch with mm/vmscan one. > >> Fix the style for the Lustre patch and resend. Then patch > >> unregister_shrinker(). Then remove the checks. > >> > >> The unregister_shrinker() changes seem like a good idea, but I haven't > >> really looked at it. It might be more involved than it seems. > >> > >> regards, > >> dan carpenter > > Thanks for the comments too. > > I'll send patch with accumulated fixes. > >>> } > >>> diff --git a/drivers/staging/lustre/lustre/obdclass/lu_object.c b/drivers/staging/lustre/lustre/obdclass/lu_object.c > >>> index b938a3f9d50a..9e0256ca2079 100644 > >>> --- a/drivers/staging/lustre/lustre/obdclass/lu_object.c > >>> +++ b/drivers/staging/lustre/lustre/obdclass/lu_object.c > >>> @@ -1951,7 +1951,7 @@ int lu_global_init(void) > >>> * inode, one for ea. Unfortunately setting this high value results in > >>> * lu_object/inode cache consuming all the memory. > >>> */ > >>> - register_shrinker(&lu_site_shrinker); > >>> + result = register_shrinker(&lu_site_shrinker); > >> There should be some error handling if the register fails. > > Yeah, I think so, but it seems that it is out of scope of this patch. > > The whole negative branch in the mainline kernel looks broken (IMHO). > > In mainline Lustre's git there is a reworked version of upper function obdclass_init(), > > which at least calls lu_global_fini() before exiting `module_init` on further failure, > > but yeah, still lacks proper cleanup inside lu_global_initcall(). > > I'll add one more patch in a patch-set so that maintainers may decide what to do with that. > > I was looking at the lack of error handling as well, but then I wondered if the module_init() call returns an error, is module_exit() still called, or does the module not load at all in the error case? > module_init() is supposed to clean up after itself. module_exit() is only called if init succeeds. regards, dan carpenter From romeusmeister at gmail.com Tue Dec 5 13:48:01 2017 From: romeusmeister at gmail.com (Roman Storozhenko) Date: Tue, 5 Dec 2017 16:48:01 +0300 Subject: [lustre-devel] [PATCH v3] staging: lustre: Replace 'uint32_t' with 'u32' and 'uint64_t' with 'u64' In-Reply-To: <20171204195823.yiwitsbbhcywwa3m@mwanda> References: <20171129164621.GA10881@home> <20171204195823.yiwitsbbhcywwa3m@mwanda> Message-ID: <20171205134801.GA707@home> On Mon, Dec 04, 2017 at 10:58:23PM +0300, Dan Carpenter wrote: > On Wed, Nov 29, 2017 at 07:46:21PM +0300, Roman Storozhenko wrote: > > There are two reasons for that: > > What I'm asking is there are two reasons for what? Where is the first > part of that paragraph? Hello, Dan! Now I understand what did you mean. I shouldn't have written reason in the subject line and reference to it as 'that' in the body. Thanks for you mentioned that. In the following patches I will avoid such types of mistakes. Regards, Romans Storozhenko > > regards, > dan carpenter > From greg at kroah.com Wed Dec 6 08:51:57 2017 From: greg at kroah.com (Greg KH) Date: Wed, 6 Dec 2017 09:51:57 +0100 Subject: [lustre-devel] [PATCH v2 1/2] staging: lustre: check result of register_shrinker In-Reply-To: <20171204192157.4140-1-akaraliou.dev@gmail.com> References: <20171204191631.bkzaruzrlepp3rqy@mwanda> <20171204192157.4140-1-akaraliou.dev@gmail.com> Message-ID: <20171206085157.GA17896@kroah.com> On Mon, Dec 04, 2017 at 10:21:56PM +0300, Aliaksei Karaliou wrote: > Lustre code lacks checking the result of register_shrinker() > in several places. register_shrinker() was tagged __must_check > recently so that sparse has started reporting it. > > Signed-off-by: Aliaksei Karaliou > --- > drivers/staging/lustre/lustre/ldlm/ldlm_pool.c | 12 +++++++++--- > drivers/staging/lustre/lustre/obdclass/lu_object.c | 5 +++-- > drivers/staging/lustre/lustre/osc/osc_request.c | 4 +++- > drivers/staging/lustre/lustre/ptlrpc/sec_bulk.c | 8 +++++++- > 4 files changed, 22 insertions(+), 7 deletions(-) > > v2: Style fixes, as suggested by Cheers, Andreas and Dan Carpenter. > Added one more patch to address resource cleanup, suggested by Dan Carpenter. > > diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c b/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c > index da65d00a7811..9fef2d52d6c2 100644 > --- a/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c > +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c > @@ -1086,10 +1086,16 @@ int ldlm_pools_init(void) > int rc; > > rc = ldlm_pools_thread_start(); > - if (rc == 0) > - register_shrinker(&ldlm_pools_cli_shrinker); > + if (rc) > + return rc; > > - return rc; > + rc = register_shrinker(&ldlm_pools_cli_shrinker); > + if (rc) { > + ldlm_pools_thread_stop(); > + return rc; > + } > + > + return 0; > } > > void ldlm_pools_fini(void) > diff --git a/drivers/staging/lustre/lustre/obdclass/lu_object.c b/drivers/staging/lustre/lustre/obdclass/lu_object.c > index b938a3f9d50a..9e0256ca2079 100644 > --- a/drivers/staging/lustre/lustre/obdclass/lu_object.c > +++ b/drivers/staging/lustre/lustre/obdclass/lu_object.c > @@ -1951,7 +1951,7 @@ int lu_global_init(void) > * inode, one for ea. Unfortunately setting this high value results in > * lu_object/inode cache consuming all the memory. > */ > - register_shrinker(&lu_site_shrinker); > + result = register_shrinker(&lu_site_shrinker); > > return result; return register_shrinker()? > - register_shrinker(&pools_shrinker); > + rc = register_shrinker(&pools_shrinker); > + if (rc) { > + enc_pools_free(); > + return rc; Drop the return rc, and then just do: > + } > > return 0; return rc; there. Makes the patch smaller. thanks, greg k-h From gregkh at linuxfoundation.org Wed Dec 6 08:53:03 2017 From: gregkh at linuxfoundation.org (gregkh at linuxfoundation.org) Date: Wed, 6 Dec 2017 09:53:03 +0100 Subject: [lustre-devel] [PATCH] staging: lustre: Fix sparse, using plain integer as NULL pointer in lov_object_fiemap() In-Reply-To: References: <7A48E652-421B-464B-AD2A-C3D5613B48E5@intel.com> Message-ID: <20171206085303.GA18289@kroah.com> On Mon, Dec 04, 2017 at 12:44:32PM +0200, Andrii Vladyka wrote: > Change 0 to NULL in lov_object_fiemap() in order to fix warning produced by > sparse > > Signed-off-by: Andrii Vladyka > Signed-off-by: Andreas Dilger > --- > > diff --git a/drivers/staging/lustre/lustre/lov/lov_object.c b/drivers/staging/lustre/lustre/lov/lov_object.c > index 105b707..897cf2c 100644 > --- a/drivers/staging/lustre/lustre/lov/lov_object.c > +++ b/drivers/staging/lustre/lustre/lov/lov_object.c > @@ -1335,7 +1335,7 @@ static int lov_object_fiemap(const struct lu_env *env, struct cl_object *obj, > int rc = 0; > int cur_stripe; > int stripe_count; > - struct fiemap_state fs = { 0 }; > + struct fiemap_state fs = { NULL }; > lsm = lov_lsm_addref(cl2lov(obj)); > if (!lsm) Patch is corrupted, and can not apply, please fix up your email client and try it again. greg k-h From greg at kroah.com Wed Dec 6 18:40:14 2017 From: greg at kroah.com (Greg KH) Date: Wed, 6 Dec 2017 19:40:14 +0100 Subject: [lustre-devel] [PATCH v2 1/2] staging: lustre: check result of register_shrinker In-Reply-To: References: <20171204191631.bkzaruzrlepp3rqy@mwanda> <20171204192157.4140-1-akaraliou.dev@gmail.com> <20171206085157.GA17896@kroah.com> Message-ID: <20171206184014.GA10486@kroah.com> On Wed, Dec 06, 2017 at 08:40:43PM +0300, Aliaksei Karaliou wrote: > On 12/06/2017 11:51 AM, Greg KH wrote: > > > On Mon, Dec 04, 2017 at 10:21:56PM +0300, Aliaksei Karaliou wrote: > > > Lustre code lacks checking the result of register_shrinker() > > > in several places. register_shrinker() was tagged __must_check > > > recently so that sparse has started reporting it. > > > > > > Signed-off-by: Aliaksei Karaliou > > > --- > > > drivers/staging/lustre/lustre/ldlm/ldlm_pool.c | 12 +++++++++--- > > > drivers/staging/lustre/lustre/obdclass/lu_object.c | 5 +++-- > > > drivers/staging/lustre/lustre/osc/osc_request.c | 4 +++- > > > drivers/staging/lustre/lustre/ptlrpc/sec_bulk.c | 8 +++++++- > > > 4 files changed, 22 insertions(+), 7 deletions(-) > > > > > > v2: Style fixes, as suggested by Cheers, Andreas and Dan Carpenter. > > > Added one more patch to address resource cleanup, suggested by Dan Carpenter. > > > > > > diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c b/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c > > > index da65d00a7811..9fef2d52d6c2 100644 > > > --- a/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c > > > +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c > > > @@ -1086,10 +1086,16 @@ int ldlm_pools_init(void) > > > int rc; > > > rc = ldlm_pools_thread_start(); > > > - if (rc == 0) > > > - register_shrinker(&ldlm_pools_cli_shrinker); > > > + if (rc) > > > + return rc; > > > - return rc; > > > + rc = register_shrinker(&ldlm_pools_cli_shrinker); > > > + if (rc) { > > > + ldlm_pools_thread_stop(); > > > + return rc; > > > + } > > > + > > > + return 0; > > > } > > > void ldlm_pools_fini(void) > > > diff --git a/drivers/staging/lustre/lustre/obdclass/lu_object.c b/drivers/staging/lustre/lustre/obdclass/lu_object.c > > > index b938a3f9d50a..9e0256ca2079 100644 > > > --- a/drivers/staging/lustre/lustre/obdclass/lu_object.c > > > +++ b/drivers/staging/lustre/lustre/obdclass/lu_object.c > > > @@ -1951,7 +1951,7 @@ int lu_global_init(void) > > > * inode, one for ea. Unfortunately setting this high value results in > > > * lu_object/inode cache consuming all the memory. > > > */ > > > - register_shrinker(&lu_site_shrinker); > > > + result = register_shrinker(&lu_site_shrinker); > > > return result; > > return register_shrinker()? > Yes, It looks easier, thank you for your comments. > But still what to do with the fact that this changed place actually leads to > resources being not freed (what I actually fix in second patch) ? > Should I just merge both commits together ? Don't write a patch that you know is broken, and then fix it up in a follow-on patch. That's not good at all. Just fix one call-site at a time, that should be easier, right? greg k-h From luisbg at kernel.org Wed Dec 6 19:16:58 2017 From: luisbg at kernel.org (Luis de Bethencourt) Date: Wed, 6 Dec 2017 19:16:58 +0000 Subject: [lustre-devel] [PATCH 1/3] staging: lustre: llite: Remove else after goto Message-ID: <20171206191700.26532-1-luisbg@kernel.org> If an "if" branch is terminated by a "goto", there's no need to have an "else" statement and an indented block of code. Remove the "else" statement to simplify the code flow. Signed-off-by: Luis de Bethencourt --- Hi, The following patches remove unneeded 'else' after a 'goto' or 'return'. They are meant to just make the code more readable and aren't functional changes. Thanks, Luis drivers/staging/lustre/lustre/llite/dir.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/staging/lustre/lustre/llite/dir.c b/drivers/staging/lustre/lustre/llite/dir.c index 5b2e47c246f3..f5b67a4923e3 100644 --- a/drivers/staging/lustre/lustre/llite/dir.c +++ b/drivers/staging/lustre/lustre/llite/dir.c @@ -1339,9 +1339,9 @@ static long ll_dir_ioctl(struct file *file, unsigned int cmd, unsigned long arg) cmd == LL_IOC_MDC_GETINFO)) { rc = 0; goto skip_lmm; - } else { - goto out_req; } + + goto out_req; } if (cmd == IOC_MDC_GETFILESTRIPE || -- 2.15.1 From luisbg at kernel.org Wed Dec 6 19:16:59 2017 From: luisbg at kernel.org (Luis de Bethencourt) Date: Wed, 6 Dec 2017 19:16:59 +0000 Subject: [lustre-devel] [PATCH 2/3] staging: lustre: llite: Remove redundant else keyword In-Reply-To: <20171206191700.26532-1-luisbg@kernel.org> References: <20171206191700.26532-1-luisbg@kernel.org> Message-ID: <20171206191700.26532-2-luisbg@kernel.org> There is no need to use 'else' if in main branch 'goto' is present. Signed-off-by: Luis de Bethencourt --- drivers/staging/lustre/lustre/llite/llite_lib.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/drivers/staging/lustre/lustre/llite/llite_lib.c b/drivers/staging/lustre/lustre/llite/llite_lib.c index 8666f1e81ade..e84719662edf 100644 --- a/drivers/staging/lustre/lustre/llite/llite_lib.c +++ b/drivers/staging/lustre/lustre/llite/llite_lib.c @@ -236,7 +236,9 @@ static int client_common_fill_super(struct super_block *sb, char *md, char *dt, "An MDT (md %s) is performing recovery, of which this client is not a part. Please wait for recovery to complete, abort, or time out.\n", md); goto out; - } else if (err) { + } + + if (err) { CERROR("cannot connect to %s: rc = %d\n", md, err); goto out; } -- 2.15.1 From luisbg at kernel.org Wed Dec 6 19:17:00 2017 From: luisbg at kernel.org (Luis de Bethencourt) Date: Wed, 6 Dec 2017 19:17:00 +0000 Subject: [lustre-devel] [PATCH 3/3] staging: lustre: llite: Remove redundant else keyword In-Reply-To: <20171206191700.26532-1-luisbg@kernel.org> References: <20171206191700.26532-1-luisbg@kernel.org> Message-ID: <20171206191700.26532-3-luisbg@kernel.org> There is no need to use 'else' if in main branch 'return' is present. Signed-off-by: Luis de Bethencourt --- drivers/staging/lustre/lustre/llite/vvp_io.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/staging/lustre/lustre/llite/vvp_io.c b/drivers/staging/lustre/lustre/llite/vvp_io.c index bfae98e82d6f..e7a4778e02e4 100644 --- a/drivers/staging/lustre/lustre/llite/vvp_io.c +++ b/drivers/staging/lustre/lustre/llite/vvp_io.c @@ -699,7 +699,7 @@ static int vvp_io_read_start(const struct lu_env *env, result = vvp_prep_size(env, obj, io, pos, tot, &exceed); if (result != 0) return result; - else if (exceed != 0) + if (exceed != 0) goto out; LU_OBJECT_HEADER(D_INODE, env, &obj->co_lu, -- 2.15.1 From anna.fuchs at informatik.uni-hamburg.de Thu Dec 7 10:15:21 2017 From: anna.fuchs at informatik.uni-hamburg.de (Anna Fuchs) Date: Thu, 07 Dec 2017 11:15:21 +0100 Subject: [lustre-devel] LU-10026 Client-side compression Message-ID: <1512641721.3271.6.camel@informatik.uni-hamburg.de> Dear all, the first prototype for client-side compression is now online. https://review.whamcloud.com/#/c/30393/ This is probably a lot of code at once, so I have added a longer description to Jira. https://jira.hpdd.intel.com/browse/LU-10026 The current state is "work in progress", so it contains a lot of TODOs. Those are listed in Jira, even more can be found in the code. Therefore, it is runable under specific conditions and is currently not meant to be autotested. The general project state will be updated at http://wiki.lustre.org/Enh anced_Adaptive_Compression_in_Lustre, further better understandable figures will follow soon. See here the mailing thread for the previous discussion related to this topic. http://lists.onebuilding.org/pipermail/lustre-devel-lustre.org/2 017-January/005208.html It would be great to hear your opinion, though I don't know where to put the main discussion - review comments in Gerrit, comments in Jira or this mailinglist. Best regards, Anna -- Anna Fuchs Universität Hamburg https://wr.informatik.uni-hamburg.de/people/anna_fuchs From joe at perches.com Thu Dec 7 17:24:47 2017 From: joe at perches.com (Joe Perches) Date: Thu, 07 Dec 2017 09:24:47 -0800 Subject: [lustre-devel] [PATCH] staging: lustre: Fix sparse, using plain integer as NULL pointer in lov_object_fiemap() In-Reply-To: <20171206085303.GA18289@kroah.com> References: <7A48E652-421B-464B-AD2A-C3D5613B48E5@intel.com> <20171206085303.GA18289@kroah.com> Message-ID: <1512667487.960.46.camel@perches.com> On Wed, 2017-12-06 at 09:53 +0100, gregkh at linuxfoundation.org wrote: > On Mon, Dec 04, 2017 at 12:44:32PM +0200, Andrii Vladyka wrote: > > Change 0 to NULL in lov_object_fiemap() in order to fix warning produced by > > sparse > > > > Signed-off-by: Andrii Vladyka > > Signed-off-by: Andreas Dilger > > --- > > > > diff --git a/drivers/staging/lustre/lustre/lov/lov_object.c b/drivers/staging/lustre/lustre/lov/lov_object.c > > index 105b707..897cf2c 100644 > > --- a/drivers/staging/lustre/lustre/lov/lov_object.c > > +++ b/drivers/staging/lustre/lustre/lov/lov_object.c > > @@ -1335,7 +1335,7 @@ static int lov_object_fiemap(const struct lu_env *env, struct cl_object *obj, > > int rc = 0; > > int cur_stripe; > > int stripe_count; > > - struct fiemap_state fs = { 0 }; > > + struct fiemap_state fs = { NULL }; > > lsm = lov_lsm_addref(cl2lov(obj)); > > if (!lsm) > > Patch is corrupted, and can not apply, please fix up your email client > and try it again. It would be better to use {} to clear the struct rather than any member initialization. From neilb at suse.com Wed Dec 13 03:15:54 2017 From: neilb at suse.com (NeilBrown) Date: Wed, 13 Dec 2017 14:15:54 +1100 Subject: [lustre-devel] [PATCH 00/13] Assorted lustre clean-ups Message-ID: <151313493380.27582.16490205348451477393.stgit@noble> This series includes a resend of some "list_entry" cleanup patches which now how better commit comments, and also includes some simplifications, removing unnecessary macros and code. Thanks, NeilBrown --- NeilBrown (12): staging: lustre: use list_last_entry to simplify fld_cache_shrink staging: lustre: ldlm: use list_for_each_entry in ldlm_extent_shift_kms() staging: lustre: ldlm: use list_first_entry in ldlm_lockd.c staging: lustre: ldlm: minor list_entry improvements in ldlm_request.c staging: lustre: ldlm: use list_for_each_entry in ldlm_resource.c staging: lustre: lov: use list_for_each_entry in lov_obd.c staging: lustre: libcfs: simplify memory allocation. staging: lustre: libcfs: remove unused rounding functions. staging: lustre: libcfs: discard MKSTR() macro staging: lustre: libcfs: discard MAX_NUMERIC_VALUE staging: lustre: libcfs: discard KLASSERT() staging: lustre: libcfs: discard LASSERT_CHECKED .../lustre/include/linux/libcfs/libcfs_private.h | 91 +++----------------- drivers/staging/lustre/lustre/fld/fld_cache.c | 13 +-- drivers/staging/lustre/lustre/ldlm/ldlm_extent.c | 4 - drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c | 10 +- drivers/staging/lustre/lustre/ldlm/ldlm_request.c | 12 +-- drivers/staging/lustre/lustre/ldlm/ldlm_resource.c | 20 ++-- drivers/staging/lustre/lustre/lov/lov_obd.c | 6 - drivers/staging/lustre/lustre/obdclass/cl_page.c | 4 - .../staging/lustre/lustre/obdclass/obd_config.c | 2 9 files changed, 41 insertions(+), 121 deletions(-) -- Signature From neilb at suse.com Wed Dec 13 03:15:54 2017 From: neilb at suse.com (NeilBrown) Date: Wed, 13 Dec 2017 14:15:54 +1100 Subject: [lustre-devel] [PATCH 02/12] staging: lustre: ldlm: use list_for_each_entry in ldlm_extent_shift_kms() In-Reply-To: <151313493380.27582.16490205348451477393.stgit@noble> References: <151313493380.27582.16490205348451477393.stgit@noble> Message-ID: <151313495483.27582.774362475681930674.stgit@noble> Using list_for_each_entry() means we don't need 'tmp'. Signed-off-by: NeilBrown --- drivers/staging/lustre/lustre/ldlm/ldlm_extent.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_extent.c b/drivers/staging/lustre/lustre/ldlm/ldlm_extent.c index fac9d19d50b6..11b11b5f3216 100644 --- a/drivers/staging/lustre/lustre/ldlm/ldlm_extent.c +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_extent.c @@ -64,7 +64,6 @@ __u64 ldlm_extent_shift_kms(struct ldlm_lock *lock, __u64 old_kms) { struct ldlm_resource *res = lock->l_resource; - struct list_head *tmp; struct ldlm_lock *lck; __u64 kms = 0; @@ -74,8 +73,7 @@ __u64 ldlm_extent_shift_kms(struct ldlm_lock *lock, __u64 old_kms) */ ldlm_set_kms_ignore(lock); - list_for_each(tmp, &res->lr_granted) { - lck = list_entry(tmp, struct ldlm_lock, l_res_link); + list_for_each_entry(lck, &res->lr_granted, l_res_link) { if (ldlm_is_kms_ignore(lck)) continue; From neilb at suse.com Wed Dec 13 03:15:54 2017 From: neilb at suse.com (NeilBrown) Date: Wed, 13 Dec 2017 14:15:54 +1100 Subject: [lustre-devel] [PATCH 03/12] staging: lustre: ldlm: use list_first_entry in ldlm_lockd.c In-Reply-To: <151313493380.27582.16490205348451477393.stgit@noble> References: <151313493380.27582.16490205348451477393.stgit@noble> Message-ID: <151313495486.27582.12193540145417673665.stgit@noble> This is only a small simplification, but it makes the code a little clearer. Signed-off-by: NeilBrown --- drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c b/drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c index ada50b69c5f6..5f6e7c933b81 100644 --- a/drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c @@ -696,13 +696,13 @@ static int ldlm_bl_get_work(struct ldlm_bl_pool *blp, /* process a request from the blp_list at least every blp_num_threads */ if (!list_empty(&blp->blp_list) && (list_empty(&blp->blp_prio_list) || num_bl == 0)) - blwi = list_entry(blp->blp_list.next, - struct ldlm_bl_work_item, blwi_entry); + blwi = list_first_entry(&blp->blp_list, + struct ldlm_bl_work_item, blwi_entry); else if (!list_empty(&blp->blp_prio_list)) - blwi = list_entry(blp->blp_prio_list.next, - struct ldlm_bl_work_item, - blwi_entry); + blwi = list_first_entry(&blp->blp_prio_list, + struct ldlm_bl_work_item, + blwi_entry); if (blwi) { if (++num_bl >= num_th) From neilb at suse.com Wed Dec 13 03:15:54 2017 From: neilb at suse.com (NeilBrown) Date: Wed, 13 Dec 2017 14:15:54 +1100 Subject: [lustre-devel] [PATCH 01/12] staging: lustre: use list_last_entry to simplify fld_cache_shrink In-Reply-To: <151313493380.27582.16490205348451477393.stgit@noble> References: <151313493380.27582.16490205348451477393.stgit@noble> Message-ID: <151313495480.27582.15094470661680270941.stgit@noble> Using list_empty() and list_last_entry() makes the code clearer, and allows a local variable to be discarded. Signed-off-by: NeilBrown --- drivers/staging/lustre/lustre/fld/fld_cache.c | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-) diff --git a/drivers/staging/lustre/lustre/fld/fld_cache.c b/drivers/staging/lustre/lustre/fld/fld_cache.c index 7d6a7106c0a5..ecf8b9e1ed5c 100644 --- a/drivers/staging/lustre/lustre/fld/fld_cache.c +++ b/drivers/staging/lustre/lustre/fld/fld_cache.c @@ -213,19 +213,18 @@ static inline void fld_cache_entry_add(struct fld_cache *cache, */ static int fld_cache_shrink(struct fld_cache *cache) { - struct fld_cache_entry *flde; - struct list_head *curr; int num = 0; if (cache->fci_cache_count < cache->fci_cache_size) return 0; - curr = cache->fci_lru.prev; - while (cache->fci_cache_count + cache->fci_threshold > - cache->fci_cache_size && curr != &cache->fci_lru) { - flde = list_entry(curr, struct fld_cache_entry, fce_lru); - curr = curr->prev; + cache->fci_cache_size && + !list_empty(&cache->fci_lru)) { + struct fld_cache_entry *flde = + list_last_entry(&cache->fci_lru, + struct fld_cache_entry, fce_lru); + fld_cache_entry_delete(cache, flde); num++; } From neilb at suse.com Wed Dec 13 03:15:54 2017 From: neilb at suse.com (NeilBrown) Date: Wed, 13 Dec 2017 14:15:54 +1100 Subject: [lustre-devel] [PATCH 04/12] staging: lustre: ldlm: minor list_entry improvements in ldlm_request.c In-Reply-To: <151313493380.27582.16490205348451477393.stgit@noble> References: <151313493380.27582.16490205348451477393.stgit@noble> Message-ID: <151313495488.27582.6910804861419753279.stgit@noble> Small clarify improvements, and one local variable avoided. Signed-off-by: NeilBrown --- drivers/staging/lustre/lustre/ldlm/ldlm_request.c | 12 ++++-------- 1 file changed, 4 insertions(+), 8 deletions(-) diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_request.c b/drivers/staging/lustre/lustre/ldlm/ldlm_request.c index a9a1551cc9bc..a1b909004de8 100644 --- a/drivers/staging/lustre/lustre/ldlm/ldlm_request.c +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_request.c @@ -1653,7 +1653,7 @@ int ldlm_cli_cancel_list(struct list_head *cancels, int count, */ while (count > 0) { LASSERT(!list_empty(cancels)); - lock = list_entry(cancels->next, struct ldlm_lock, l_bl_ast); + lock = list_first_entry(cancels, struct ldlm_lock, l_bl_ast); LASSERT(lock->l_conn_export); if (exp_connect_cancelset(lock->l_conn_export)) { @@ -1777,7 +1777,7 @@ EXPORT_SYMBOL(ldlm_cli_cancel_unused); static int ldlm_resource_foreach(struct ldlm_resource *res, ldlm_iterator_t iter, void *closure) { - struct list_head *tmp, *next; + struct ldlm_lock *tmp; struct ldlm_lock *lock; int rc = LDLM_ITER_CONTINUE; @@ -1785,18 +1785,14 @@ static int ldlm_resource_foreach(struct ldlm_resource *res, return LDLM_ITER_CONTINUE; lock_res(res); - list_for_each_safe(tmp, next, &res->lr_granted) { - lock = list_entry(tmp, struct ldlm_lock, l_res_link); - + list_for_each_entry_safe(lock, tmp, &res->lr_granted, l_res_link) { if (iter(lock, closure) == LDLM_ITER_STOP) { rc = LDLM_ITER_STOP; goto out; } } - list_for_each_safe(tmp, next, &res->lr_waiting) { - lock = list_entry(tmp, struct ldlm_lock, l_res_link); - + list_for_each_entry_safe(lock, tmp, &res->lr_waiting, l_res_link) { if (iter(lock, closure) == LDLM_ITER_STOP) { rc = LDLM_ITER_STOP; goto out; From neilb at suse.com Wed Dec 13 03:15:54 2017 From: neilb at suse.com (NeilBrown) Date: Wed, 13 Dec 2017 14:15:54 +1100 Subject: [lustre-devel] [PATCH 06/12] staging: lustre: lov: use list_for_each_entry in lov_obd.c In-Reply-To: <151313493380.27582.16490205348451477393.stgit@noble> References: <151313493380.27582.16490205348451477393.stgit@noble> Message-ID: <151313495494.27582.10063352335868964856.stgit@noble> Using the *_entry macro simplifies the code slightly. Signed-off-by: NeilBrown --- drivers/staging/lustre/lustre/lov/lov_obd.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/drivers/staging/lustre/lustre/lov/lov_obd.c b/drivers/staging/lustre/lustre/lov/lov_obd.c index 7ce01026a409..ec70c12e5b40 100644 --- a/drivers/staging/lustre/lustre/lov/lov_obd.c +++ b/drivers/staging/lustre/lustre/lov/lov_obd.c @@ -828,11 +828,9 @@ int lov_setup(struct obd_device *obd, struct lustre_cfg *lcfg) static int lov_cleanup(struct obd_device *obd) { struct lov_obd *lov = &obd->u.lov; - struct list_head *pos, *tmp; - struct pool_desc *pool; + struct pool_desc *pool, *tmp; - list_for_each_safe(pos, tmp, &lov->lov_pool_list) { - pool = list_entry(pos, struct pool_desc, pool_list); + list_for_each_entry_safe(pool, tmp, &lov->lov_pool_list, pool_list) { /* free pool structs */ CDEBUG(D_INFO, "delete pool %p\n", pool); /* In the function below, .hs_keycmp resolves to From neilb at suse.com Wed Dec 13 03:15:54 2017 From: neilb at suse.com (NeilBrown) Date: Wed, 13 Dec 2017 14:15:54 +1100 Subject: [lustre-devel] [PATCH 05/12] staging: lustre: ldlm: use list_for_each_entry in ldlm_resource.c In-Reply-To: <151313493380.27582.16490205348451477393.stgit@noble> References: <151313493380.27582.16490205348451477393.stgit@noble> Message-ID: <151313495491.27582.1409394419999876978.stgit@noble> Having a stand-alone "list_entry()" call is often a sign that something like "list_for_each_entry()" would make the code clearer. Signed-off-by: NeilBrown --- drivers/staging/lustre/lustre/ldlm/ldlm_resource.c | 20 +++++++------------- 1 file changed, 7 insertions(+), 13 deletions(-) diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_resource.c b/drivers/staging/lustre/lustre/ldlm/ldlm_resource.c index 2689ffdf10e3..9958533cc227 100644 --- a/drivers/staging/lustre/lustre/ldlm/ldlm_resource.c +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_resource.c @@ -752,24 +752,22 @@ extern struct ldlm_lock *ldlm_lock_get(struct ldlm_lock *lock); static void cleanup_resource(struct ldlm_resource *res, struct list_head *q, __u64 flags) { - struct list_head *tmp; int rc = 0; bool local_only = !!(flags & LDLM_FL_LOCAL_ONLY); do { - struct ldlm_lock *lock = NULL; + struct ldlm_lock *lock = NULL, *tmp; struct lustre_handle lockh; /* First, we look for non-cleaned-yet lock * all cleaned locks are marked by CLEANED flag. */ lock_res(res); - list_for_each(tmp, q) { - lock = list_entry(tmp, struct ldlm_lock, l_res_link); - if (ldlm_is_cleaned(lock)) { - lock = NULL; + list_for_each_entry(tmp, q, l_res_link) { + if (ldlm_is_cleaned(tmp)) continue; - } + + lock = tmp; LDLM_LOCK_GET(lock); ldlm_set_cleaned(lock); break; @@ -1283,19 +1281,15 @@ void ldlm_res2desc(struct ldlm_resource *res, struct ldlm_resource_desc *desc) */ void ldlm_dump_all_namespaces(enum ldlm_side client, int level) { - struct list_head *tmp; + struct ldlm_namespace *ns; if (!((libcfs_debug | D_ERROR) & level)) return; mutex_lock(ldlm_namespace_lock(client)); - list_for_each(tmp, ldlm_namespace_list(client)) { - struct ldlm_namespace *ns; - - ns = list_entry(tmp, struct ldlm_namespace, ns_list_chain); + list_for_each_entry(ns, ldlm_namespace_list(client), ns_list_chain) ldlm_namespace_dump(level, ns); - } mutex_unlock(ldlm_namespace_lock(client)); } From neilb at suse.com Wed Dec 13 03:15:55 2017 From: neilb at suse.com (NeilBrown) Date: Wed, 13 Dec 2017 14:15:55 +1100 Subject: [lustre-devel] [PATCH 07/12] staging: lustre: libcfs: simplify memory allocation. In-Reply-To: <151313493380.27582.16490205348451477393.stgit@noble> References: <151313493380.27582.16490205348451477393.stgit@noble> Message-ID: <151313495497.27582.5451400666802627086.stgit@noble> 1/ Use kvmalloc() instead of kmalloc or vmalloc 2/ Discard the _GFP() interfaces that are never used. We only ever do GFP_NOFS and GFP_ATOMIC allocations, so support each of those explicitly. Signed-off-by: NeilBrown --- .../lustre/include/linux/libcfs/libcfs_private.h | 42 ++++++-------------- 1 file changed, 12 insertions(+), 30 deletions(-) diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_private.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_private.h index 2f4ff595fac9..c874f9d15c72 100644 --- a/drivers/staging/lustre/include/linux/libcfs/libcfs_private.h +++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_private.h @@ -84,14 +84,6 @@ do { \ lbug_with_loc(&msgdata); \ } while (0) -#ifndef LIBCFS_VMALLOC_SIZE -#define LIBCFS_VMALLOC_SIZE (2 << PAGE_SHIFT) /* 2 pages */ -#endif - -#define LIBCFS_ALLOC_PRE(size, mask) \ - LASSERT(!in_interrupt() || ((size) <= LIBCFS_VMALLOC_SIZE && \ - !gfpflags_allow_blocking(mask))) - #define LIBCFS_ALLOC_POST(ptr, size) \ do { \ if (unlikely(!(ptr))) { \ @@ -103,46 +95,36 @@ do { \ } while (0) /** - * allocate memory with GFP flags @mask + * default allocator */ -#define LIBCFS_ALLOC_GFP(ptr, size, mask) \ +#define LIBCFS_ALLOC(ptr, size) \ do { \ - LIBCFS_ALLOC_PRE((size), (mask)); \ - (ptr) = (size) <= LIBCFS_VMALLOC_SIZE ? \ - kmalloc((size), (mask)) : vmalloc(size); \ + LASSERT(!in_interrupt()); \ + (ptr) = kvmalloc((size), GFP_NOFS); \ LIBCFS_ALLOC_POST((ptr), (size)); \ } while (0) -/** - * default allocator - */ -#define LIBCFS_ALLOC(ptr, size) \ - LIBCFS_ALLOC_GFP(ptr, size, GFP_NOFS) - /** * non-sleeping allocator */ -#define LIBCFS_ALLOC_ATOMIC(ptr, size) \ - LIBCFS_ALLOC_GFP(ptr, size, GFP_ATOMIC) +#define LIBCFS_ALLOC_ATOMIC(ptr, size) \ +do { \ + (ptr) = kmalloc((size), GFP_ATOMIC); \ + LIBCFS_ALLOC_POST(ptr, size); \ +} while (0) /** * allocate memory for specified CPU partition * \a cptab != NULL, \a cpt is CPU partition id of \a cptab * \a cptab == NULL, \a cpt is HW NUMA node id */ -#define LIBCFS_CPT_ALLOC_GFP(ptr, cptab, cpt, size, mask) \ +#define LIBCFS_CPT_ALLOC(ptr, cptab, cpt, size) \ do { \ - LIBCFS_ALLOC_PRE((size), (mask)); \ - (ptr) = (size) <= LIBCFS_VMALLOC_SIZE ? \ - kmalloc_node((size), (mask), cfs_cpt_spread_node(cptab, cpt)) :\ - vmalloc_node(size, cfs_cpt_spread_node(cptab, cpt)); \ + LASSERT(!in_interrupt()); \ + (ptr) = kvmalloc_node((size), GFP_NOFS, cfs_cpt_spread_node(cptab, cpt)); \ LIBCFS_ALLOC_POST((ptr), (size)); \ } while (0) -/** default numa allocator */ -#define LIBCFS_CPT_ALLOC(ptr, cptab, cpt, size) \ - LIBCFS_CPT_ALLOC_GFP(ptr, cptab, cpt, size, GFP_NOFS) - #define LIBCFS_FREE(ptr, size) \ do { \ if (unlikely(!(ptr))) { \ From neilb at suse.com Wed Dec 13 03:15:55 2017 From: neilb at suse.com (NeilBrown) Date: Wed, 13 Dec 2017 14:15:55 +1100 Subject: [lustre-devel] [PATCH 09/12] staging: lustre: libcfs: discard MKSTR() macro In-Reply-To: <151313493380.27582.16490205348451477393.stgit@noble> References: <151313493380.27582.16490205348451477393.stgit@noble> Message-ID: <151313495503.27582.5512761213682609004.stgit@noble> This is only used for tracing when some strings might be NULL. NULL strings are not a problem for tracing, vnsprintf() will report them as "(null)" which is probably better (easier to parse) than an empty string. Also remove a nearby comment that doesn't relate to the (remaining) code at all. Signed-off-by: NeilBrown --- .../lustre/include/linux/libcfs/libcfs_private.h | 8 -------- .../staging/lustre/lustre/obdclass/obd_config.c | 2 +- 2 files changed, 1 insertion(+), 9 deletions(-) diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_private.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_private.h index dee5f650197f..27d40a7589d4 100644 --- a/drivers/staging/lustre/include/linux/libcfs/libcfs_private.h +++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_private.h @@ -235,14 +235,6 @@ do { \ /* logical equivalence */ #define equi(a, b) (!!(a) == !!(b)) -/* -------------------------------------------------------------------- - * Light-weight trace - * Support for temporary event tracing with minimal Heisenberg effect. - * -------------------------------------------------------------------- - */ - -#define MKSTR(ptr) ((ptr)) ? (ptr) : "" - #ifndef HAVE_CFS_SIZE_ROUND static inline size_t cfs_size_round(int val) { diff --git a/drivers/staging/lustre/lustre/obdclass/obd_config.c b/drivers/staging/lustre/lustre/obdclass/obd_config.c index c0e192ae22a9..997c0f9aafb5 100644 --- a/drivers/staging/lustre/lustre/obdclass/obd_config.c +++ b/drivers/staging/lustre/lustre/obdclass/obd_config.c @@ -236,7 +236,7 @@ static int class_attach(struct lustre_cfg *lcfg) uuid = lustre_cfg_string(lcfg, 2); CDEBUG(D_IOCTL, "attach type %s name: %s uuid: %s\n", - MKSTR(typename), MKSTR(name), MKSTR(uuid)); + typename, name, uuid); obd = class_newdev(typename, name); if (IS_ERR(obd)) { From neilb at suse.com Wed Dec 13 03:15:55 2017 From: neilb at suse.com (NeilBrown) Date: Wed, 13 Dec 2017 14:15:55 +1100 Subject: [lustre-devel] [PATCH 11/12] staging: lustre: libcfs: discard KLASSERT() In-Reply-To: <151313493380.27582.16490205348451477393.stgit@noble> References: <151313493380.27582.16490205348451477393.stgit@noble> Message-ID: <151313495508.27582.12134291439268599158.stgit@noble> This appears to be intended for assertions that only make sense in the kernel, but as this code is now kernel-only, it doesn't make sense any more. Signed-off-by: NeilBrown --- .../lustre/include/linux/libcfs/libcfs_private.h | 2 -- drivers/staging/lustre/lustre/obdclass/cl_page.c | 4 ++-- 2 files changed, 2 insertions(+), 4 deletions(-) diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_private.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_private.h index d32ce68e6c58..31403667be6b 100644 --- a/drivers/staging/lustre/include/linux/libcfs/libcfs_private.h +++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_private.h @@ -74,8 +74,6 @@ do { \ # define LINVRNT(exp) ((void)sizeof !!(exp)) #endif -#define KLASSERT(e) LASSERT(e) - void __noreturn lbug_with_loc(struct libcfs_debug_msg_data *msg); #define LBUG() \ diff --git a/drivers/staging/lustre/lustre/obdclass/cl_page.c b/drivers/staging/lustre/lustre/obdclass/cl_page.c index 7f65439f9b95..d3b25667bc3a 100644 --- a/drivers/staging/lustre/lustre/obdclass/cl_page.c +++ b/drivers/staging/lustre/lustre/obdclass/cl_page.c @@ -202,7 +202,7 @@ struct cl_page *cl_page_find(const struct lu_env *env, * vmpage lock is used to protect the child/parent * relationship */ - KLASSERT(PageLocked(vmpage)); + LASSERT(PageLocked(vmpage)); /* * cl_vmpage_page() can be called here without any locks as * @@ -340,7 +340,7 @@ struct cl_page *cl_vmpage_page(struct page *vmpage, struct cl_object *obj) { struct cl_page *page; - KLASSERT(PageLocked(vmpage)); + LASSERT(PageLocked(vmpage)); /* * NOTE: absence of races and liveness of data are guaranteed by page From neilb at suse.com Wed Dec 13 03:15:55 2017 From: neilb at suse.com (NeilBrown) Date: Wed, 13 Dec 2017 14:15:55 +1100 Subject: [lustre-devel] [PATCH 08/12] staging: lustre: libcfs: remove unused rounding functions. In-Reply-To: <151313493380.27582.16490205348451477393.stgit@noble> References: <151313493380.27582.16490205348451477393.stgit@noble> Message-ID: <151313495500.27582.9454376337100991831.stgit@noble> These are all unused except cfs_size_round(). So discard the others, and use the kernel-standard round_up() function to implement cfs_size_round(). Signed-off-by: NeilBrown --- .../lustre/include/linux/libcfs/libcfs_private.h | 29 +------------------- 1 file changed, 1 insertion(+), 28 deletions(-) diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_private.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_private.h index c874f9d15c72..dee5f650197f 100644 --- a/drivers/staging/lustre/include/linux/libcfs/libcfs_private.h +++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_private.h @@ -243,40 +243,13 @@ do { \ #define MKSTR(ptr) ((ptr)) ? (ptr) : "" -static inline size_t cfs_size_round4(int val) -{ - return (val + 3) & (~0x3); -} - #ifndef HAVE_CFS_SIZE_ROUND static inline size_t cfs_size_round(int val) { - return (val + 7) & (~0x7); + return round_up(val, 8); } #define HAVE_CFS_SIZE_ROUND #endif -static inline size_t cfs_size_round16(int val) -{ - return (val + 0xf) & (~0xf); -} - -static inline size_t cfs_size_round32(int val) -{ - return (val + 0x1f) & (~0x1f); -} - -static inline size_t cfs_size_round0(int val) -{ - if (!val) - return 0; - return (val + 1 + 7) & (~0x7); -} - -static inline size_t cfs_round_strlen(char *fset) -{ - return cfs_size_round((int)strlen(fset) + 1); -} - #endif From neilb at suse.com Wed Dec 13 03:15:55 2017 From: neilb at suse.com (NeilBrown) Date: Wed, 13 Dec 2017 14:15:55 +1100 Subject: [lustre-devel] [PATCH 10/12] staging: lustre: libcfs: discard MAX_NUMERIC_VALUE In-Reply-To: <151313493380.27582.16490205348451477393.stgit@noble> References: <151313493380.27582.16490205348451477393.stgit@noble> Message-ID: <151313495506.27582.203930994943994556.stgit@noble> This is unused. drivers/staging/lustre/lnet/lnet/nidstrings.c does use the name, but it includes its own local definition. Signed-off-by: NeilBrown --- .../lustre/include/linux/libcfs/libcfs_private.h | 3 --- 1 file changed, 3 deletions(-) diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_private.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_private.h index 27d40a7589d4..d32ce68e6c58 100644 --- a/drivers/staging/lustre/include/linux/libcfs/libcfs_private.h +++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_private.h @@ -227,9 +227,6 @@ do { \ #define CFS_ALLOC_PTR(ptr) LIBCFS_ALLOC(ptr, sizeof(*(ptr))) #define CFS_FREE_PTR(ptr) LIBCFS_FREE(ptr, sizeof(*(ptr))) -/* max value for numeric network address */ -#define MAX_NUMERIC_VALUE 0xffffffff - /* implication */ #define ergo(a, b) (!(a) || (b)) /* logical equivalence */ From neilb at suse.com Wed Dec 13 03:15:55 2017 From: neilb at suse.com (NeilBrown) Date: Wed, 13 Dec 2017 14:15:55 +1100 Subject: [lustre-devel] [PATCH 12/12] staging: lustre: libcfs: discard LASSERT_CHECKED In-Reply-To: <151313493380.27582.16490205348451477393.stgit@noble> References: <151313493380.27582.16490205348451477393.stgit@noble> Message-ID: <151313495511.27582.5936753382372514332.stgit@noble> This macro isn't used, and comment is about some earlier version of the lustre code that never reached the mainline kernel. Just discard it. Signed-off-by: NeilBrown --- .../lustre/include/linux/libcfs/libcfs_private.h | 7 ------- 1 file changed, 7 deletions(-) diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_private.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_private.h index 31403667be6b..940200ee632e 100644 --- a/drivers/staging/lustre/include/linux/libcfs/libcfs_private.h +++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_private.h @@ -43,13 +43,6 @@ # define DEBUG_SUBSYSTEM S_UNDEFINED #endif -/* - * When this is on, LASSERT macro includes check for assignment used instead - * of equality check, but doesn't have unlikely(). Turn this on from time to - * time to make test-builds. This shouldn't be on for production release. - */ -#define LASSERT_CHECKED (0) - #define LASSERTF(cond, fmt, ...) \ do { \ if (unlikely(!(cond))) { \ From paf at cray.com Wed Dec 13 03:33:26 2017 From: paf at cray.com (Patrick Farrell) Date: Wed, 13 Dec 2017 03:33:26 +0000 Subject: [lustre-devel] [PATCH 12/12] staging: lustre: libcfs: discard LASSERT_CHECKED In-Reply-To: <151313495511.27582.5936753382372514332.stgit@noble> References: <151313493380.27582.16490205348451477393.stgit@noble>, <151313495511.27582.5936753382372514332.stgit@noble> Message-ID: Thanks, Neil - lots of small but nice cleanups. Full series acked. ________________________________ From: lustre-devel on behalf of NeilBrown Sent: Tuesday, December 12, 2017 9:15:55 PM To: Oleg Drokin; James Simmons; Andreas Dilger; Greg Kroah-Hartman Cc: linux-kernel at vger.kernel.org; lustre-devel at lists.lustre.org Subject: [lustre-devel] [PATCH 12/12] staging: lustre: libcfs: discard LASSERT_CHECKED This macro isn't used, and comment is about some earlier version of the lustre code that never reached the mainline kernel. Just discard it. Signed-off-by: NeilBrown --- .../lustre/include/linux/libcfs/libcfs_private.h | 7 ------- 1 file changed, 7 deletions(-) diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_private.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_private.h index 31403667be6b..940200ee632e 100644 --- a/drivers/staging/lustre/include/linux/libcfs/libcfs_private.h +++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_private.h @@ -43,13 +43,6 @@ # define DEBUG_SUBSYSTEM S_UNDEFINED #endif -/* - * When this is on, LASSERT macro includes check for assignment used instead - * of equality check, but doesn't have unlikely(). Turn this on from time to - * time to make test-builds. This shouldn't be on for production release. - */ -#define LASSERT_CHECKED (0) - #define LASSERTF(cond, fmt, ...) \ do { \ if (unlikely(!(cond))) { \ _______________________________________________ lustre-devel mailing list lustre-devel at lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From luisbg at kernel.org Wed Dec 13 10:22:24 2017 From: luisbg at kernel.org (Luis de Bethencourt) Date: Wed, 13 Dec 2017 10:22:24 +0000 Subject: [lustre-devel] [PATCH 12/12] staging: lustre: libcfs: discard LASSERT_CHECKED In-Reply-To: References: <151313493380.27582.16490205348451477393.stgit@noble> <151313495511.27582.5936753382372514332.stgit@noble> Message-ID: <05f81a0a-adda-63d5-b7fe-8e6d2399e643@kernel.org> On 12/13/2017 03:33 AM, Patrick Farrell wrote: > Thanks, Neil - lots of small but nice cleanups. > > Full series acked. > ________________________________ > From: lustre-devel on behalf of NeilBrown > Sent: Tuesday, December 12, 2017 9:15:55 PM > To: Oleg Drokin; James Simmons; Andreas Dilger; Greg Kroah-Hartman > Cc: linux-kernel at vger.kernel.org; lustre-devel at lists.lustre.org > Subject: [lustre-devel] [PATCH 12/12] staging: lustre: libcfs: discard LASSERT_CHECKED > > This macro isn't used, and comment is about some earlier version > of the lustre code that never reached the mainline kernel. > Just discard it. > > Signed-off-by: NeilBrown > --- Thanks Neil! Nice cleanup indeed. Acked-by: Luis de Bethencourt From neilb at suse.com Thu Dec 14 00:00:45 2017 From: neilb at suse.com (NeilBrown) Date: Thu, 14 Dec 2017 11:00:45 +1100 Subject: [lustre-devel] [PATCH 07/12] staging: lustre: libcfs: simplify memory allocation. In-Reply-To: <151313495497.27582.5451400666802627086.stgit@noble> References: <151313493380.27582.16490205348451477393.stgit@noble> <151313495497.27582.5451400666802627086.stgit@noble> Message-ID: <87d13inpoy.fsf@notabene.neil.brown.name> On Wed, Dec 13 2017, NeilBrown wrote: > 1/ Use kvmalloc() instead of kmalloc or vmalloc > 2/ Discard the _GFP() interfaces that are never used. > We only ever do GFP_NOFS and GFP_ATOMIC allocations, > so support each of those explicitly. Hi, I just remembered that posting this patch was, maybe, a little premature. vmalloc() doesn't support GFP_NOFS, so kvmalloc() warns about an attempt to use GFP_NOFS. lustre potentially used vmalloc in GFP_NOFS context, and so now calls kvmalloc() with GFP_NOFS, which triggers a warning. I haven't yet looking into how to remove the warning. Maybe all GFP_NOFS usages should use kmalloc(), not kvmalloc(), and accept the increased chance of failure. I'll dig deeper and hope to have a clearer opinion soon. As it is only a warning, it is probably OK to keep the patch in staging-next, but I wouldn't object if it was dropped. Thanks, NeilBrown > > Signed-off-by: NeilBrown > --- > .../lustre/include/linux/libcfs/libcfs_private.h | 42 ++++++-------------- > 1 file changed, 12 insertions(+), 30 deletions(-) > > diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_private.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_private.h > index 2f4ff595fac9..c874f9d15c72 100644 > --- a/drivers/staging/lustre/include/linux/libcfs/libcfs_private.h > +++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_private.h > @@ -84,14 +84,6 @@ do { \ > lbug_with_loc(&msgdata); \ > } while (0) > > -#ifndef LIBCFS_VMALLOC_SIZE > -#define LIBCFS_VMALLOC_SIZE (2 << PAGE_SHIFT) /* 2 pages */ > -#endif > - > -#define LIBCFS_ALLOC_PRE(size, mask) \ > - LASSERT(!in_interrupt() || ((size) <= LIBCFS_VMALLOC_SIZE && \ > - !gfpflags_allow_blocking(mask))) > - > #define LIBCFS_ALLOC_POST(ptr, size) \ > do { \ > if (unlikely(!(ptr))) { \ > @@ -103,46 +95,36 @@ do { \ > } while (0) > > /** > - * allocate memory with GFP flags @mask > + * default allocator > */ > -#define LIBCFS_ALLOC_GFP(ptr, size, mask) \ > +#define LIBCFS_ALLOC(ptr, size) \ > do { \ > - LIBCFS_ALLOC_PRE((size), (mask)); \ > - (ptr) = (size) <= LIBCFS_VMALLOC_SIZE ? \ > - kmalloc((size), (mask)) : vmalloc(size); \ > + LASSERT(!in_interrupt()); \ > + (ptr) = kvmalloc((size), GFP_NOFS); \ > LIBCFS_ALLOC_POST((ptr), (size)); \ > } while (0) > > -/** > - * default allocator > - */ > -#define LIBCFS_ALLOC(ptr, size) \ > - LIBCFS_ALLOC_GFP(ptr, size, GFP_NOFS) > - > /** > * non-sleeping allocator > */ > -#define LIBCFS_ALLOC_ATOMIC(ptr, size) \ > - LIBCFS_ALLOC_GFP(ptr, size, GFP_ATOMIC) > +#define LIBCFS_ALLOC_ATOMIC(ptr, size) \ > +do { \ > + (ptr) = kmalloc((size), GFP_ATOMIC); \ > + LIBCFS_ALLOC_POST(ptr, size); \ > +} while (0) > > /** > * allocate memory for specified CPU partition > * \a cptab != NULL, \a cpt is CPU partition id of \a cptab > * \a cptab == NULL, \a cpt is HW NUMA node id > */ > -#define LIBCFS_CPT_ALLOC_GFP(ptr, cptab, cpt, size, mask) \ > +#define LIBCFS_CPT_ALLOC(ptr, cptab, cpt, size) \ > do { \ > - LIBCFS_ALLOC_PRE((size), (mask)); \ > - (ptr) = (size) <= LIBCFS_VMALLOC_SIZE ? \ > - kmalloc_node((size), (mask), cfs_cpt_spread_node(cptab, cpt)) :\ > - vmalloc_node(size, cfs_cpt_spread_node(cptab, cpt)); \ > + LASSERT(!in_interrupt()); \ > + (ptr) = kvmalloc_node((size), GFP_NOFS, cfs_cpt_spread_node(cptab, cpt)); \ > LIBCFS_ALLOC_POST((ptr), (size)); \ > } while (0) > > -/** default numa allocator */ > -#define LIBCFS_CPT_ALLOC(ptr, cptab, cpt, size) \ > - LIBCFS_CPT_ALLOC_GFP(ptr, cptab, cpt, size, GFP_NOFS) > - > #define LIBCFS_FREE(ptr, size) \ > do { \ > if (unlikely(!(ptr))) { \ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 832 bytes Desc: not available URL: From paf at cray.com Thu Dec 14 03:34:37 2017 From: paf at cray.com (Patrick Farrell) Date: Thu, 14 Dec 2017 03:34:37 +0000 Subject: [lustre-devel] [PATCH 07/12] staging: lustre: libcfs: simplify memory allocation. In-Reply-To: <151313495497.27582.5451400666802627086.stgit@noble> References: <151313493380.27582.16490205348451477393.stgit@noble>, <151313495497.27582.5451400666802627086.stgit@noble> Message-ID: Acked-by: Patrick Farrell ________________________________ From: lustre-devel on behalf of NeilBrown Sent: Tuesday, December 12, 2017 9:15:55 PM To: Oleg Drokin; James Simmons; Andreas Dilger; Greg Kroah-Hartman Cc: linux-kernel at vger.kernel.org; lustre-devel at lists.lustre.org Subject: [lustre-devel] [PATCH 07/12] staging: lustre: libcfs: simplify memory allocation. 1/ Use kvmalloc() instead of kmalloc or vmalloc 2/ Discard the _GFP() interfaces that are never used. We only ever do GFP_NOFS and GFP_ATOMIC allocations, so support each of those explicitly. Signed-off-by: NeilBrown --- .../lustre/include/linux/libcfs/libcfs_private.h | 42 ++++++-------------- 1 file changed, 12 insertions(+), 30 deletions(-) diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_private.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_private.h index 2f4ff595fac9..c874f9d15c72 100644 --- a/drivers/staging/lustre/include/linux/libcfs/libcfs_private.h +++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_private.h @@ -84,14 +84,6 @@ do { \ lbug_with_loc(&msgdata); \ } while (0) -#ifndef LIBCFS_VMALLOC_SIZE -#define LIBCFS_VMALLOC_SIZE (2 << PAGE_SHIFT) /* 2 pages */ -#endif - -#define LIBCFS_ALLOC_PRE(size, mask) \ - LASSERT(!in_interrupt() || ((size) <= LIBCFS_VMALLOC_SIZE && \ - !gfpflags_allow_blocking(mask))) - #define LIBCFS_ALLOC_POST(ptr, size) \ do { \ if (unlikely(!(ptr))) { \ @@ -103,46 +95,36 @@ do { \ } while (0) /** - * allocate memory with GFP flags @mask + * default allocator */ -#define LIBCFS_ALLOC_GFP(ptr, size, mask) \ +#define LIBCFS_ALLOC(ptr, size) \ do { \ - LIBCFS_ALLOC_PRE((size), (mask)); \ - (ptr) = (size) <= LIBCFS_VMALLOC_SIZE ? \ - kmalloc((size), (mask)) : vmalloc(size); \ + LASSERT(!in_interrupt()); \ + (ptr) = kvmalloc((size), GFP_NOFS); \ LIBCFS_ALLOC_POST((ptr), (size)); \ } while (0) -/** - * default allocator - */ -#define LIBCFS_ALLOC(ptr, size) \ - LIBCFS_ALLOC_GFP(ptr, size, GFP_NOFS) - /** * non-sleeping allocator */ -#define LIBCFS_ALLOC_ATOMIC(ptr, size) \ - LIBCFS_ALLOC_GFP(ptr, size, GFP_ATOMIC) +#define LIBCFS_ALLOC_ATOMIC(ptr, size) \ +do { \ + (ptr) = kmalloc((size), GFP_ATOMIC); \ + LIBCFS_ALLOC_POST(ptr, size); \ +} while (0) /** * allocate memory for specified CPU partition * \a cptab != NULL, \a cpt is CPU partition id of \a cptab * \a cptab == NULL, \a cpt is HW NUMA node id */ -#define LIBCFS_CPT_ALLOC_GFP(ptr, cptab, cpt, size, mask) \ +#define LIBCFS_CPT_ALLOC(ptr, cptab, cpt, size) \ do { \ - LIBCFS_ALLOC_PRE((size), (mask)); \ - (ptr) = (size) <= LIBCFS_VMALLOC_SIZE ? \ - kmalloc_node((size), (mask), cfs_cpt_spread_node(cptab, cpt)) :\ - vmalloc_node(size, cfs_cpt_spread_node(cptab, cpt)); \ + LASSERT(!in_interrupt()); \ + (ptr) = kvmalloc_node((size), GFP_NOFS, cfs_cpt_spread_node(cptab, cpt)); \ LIBCFS_ALLOC_POST((ptr), (size)); \ } while (0) -/** default numa allocator */ -#define LIBCFS_CPT_ALLOC(ptr, cptab, cpt, size) \ - LIBCFS_CPT_ALLOC_GFP(ptr, cptab, cpt, size, GFP_NOFS) - #define LIBCFS_FREE(ptr, size) \ do { \ if (unlikely(!(ptr))) { \ _______________________________________________ lustre-devel mailing list lustre-devel at lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From neilb at suse.com Thu Dec 14 04:43:43 2017 From: neilb at suse.com (NeilBrown) Date: Thu, 14 Dec 2017 15:43:43 +1100 Subject: [lustre-devel] [PATCH] staging: lustre: lnet: Fix recent breakage from list_for_each conversion Message-ID: <87a7ylor5s.fsf@notabene.neil.brown.name> Commit 8e55b6fd0660 ("staging: lustre: lnet: replace list_for_each with list_for_each_entry") was intended to be an idempotent change, but actually broke the behavior of ksocknal_add_peer() causing mounts to fail. The fact that it caused an existing "route2 = NULL;" to become redundant could have been a clue. The fact that the loop body set the new loop variable to NULL might also have been a clue The original code relied on "route2" being NULL if nothing was found. The new code would always set route2 to a non-NULL value if the list was empty, and would likely crash if the list was not empty. Restore correct functionality by using code-flow rather the value of "route2" to determine whether to use on old route, or to add a new one. Fixes: 8e55b6fd0660 ("staging: lustre: lnet: replace list_for_each with list_for_each_entry") Signed-off-by: NeilBrown --- .../staging/lustre/lnet/klnds/socklnd/socklnd.c | 23 ++++++++++------------ 1 file changed, 10 insertions(+), 13 deletions(-) diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c index 9c92a83f214a..634c0cc7909f 100644 --- a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c +++ b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c @@ -487,21 +487,18 @@ ksocknal_add_peer(struct lnet_ni *ni, struct lnet_process_id id, __u32 ipaddr, ksocknal_nid2peerlist(id.nid)); } - route2 = NULL; list_for_each_entry(route2, &peer->ksnp_routes, ksnr_list) { - if (route2->ksnr_ipaddr == ipaddr) - break; - - route2 = NULL; - } - if (!route2) { - ksocknal_add_route_locked(peer, route); - route->ksnr_share_count++; - } else { - ksocknal_route_decref(route); - route2->ksnr_share_count++; + if (route2->ksnr_ipaddr == ipaddr) { + /* Route already exists, use the old one */ + ksocknal_route_decref(route); + route2->ksnr_share_count++; + goto out; + } } - + /* Route doesn't already exist, add the new one */ + ksocknal_add_route_locked(peer, route); + route->ksnr_share_count++; +out: write_unlock_bh(&ksocknal_data.ksnd_global_lock); return 0; -- 2.14.0.rc0.dirty -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 832 bytes Desc: not available URL: From neilb at suse.com Mon Dec 18 00:46:30 2017 From: neilb at suse.com (NeilBrown) Date: Mon, 18 Dec 2017 11:46:30 +1100 Subject: [lustre-devel] [PATCH SERIES 1: 00/15] staging:lustre: convert most LIBCFS*ALLOC to k*malloc Message-ID: <151355781721.6200.2136335532722530242.stgit@noble> Lustre has some "convenience" macros for allocating and freeing memory. They: - warn if called from interrupt context - use kvmalloc - assume GFP_NOFS (which kvmalloc doesn't support) - print an error if the allocation fails - initialize the memory to zeroes. though the _ATOMIC version skips the first three. - kmalloc family functions already produce the warning. - kvmalloc is best kept for allocations which might be large, and where GFP_KERNEL is permitted - Assuming GFP_NOFS does hurt much for small allocation - though it increases the chance of failure a little - but is unnecessary in many cases and shouldn't be assumed. - Giving an error on failure can be achieved with CONFIG_SLAB_DEBUG. - Initializing to zeroes, where needed, can be done with __GFP_ZERO or kzalloc() So having these "convenience" functions tends to obscure the intention of the code by reducing the variety of calls (homogenising the code). This series converts many of the calls to kmalloc or kvmalloc or similar, and converts the corresponding LIBCFS_FREE() calls to kfree() or kvfree(). The LIBCFS_CPT_ALLOC() calls have not been changed as they are a little less straight forward, and deserve closer analysis before a clean conversion is possible. This series does not remove the zeroing in all cases where is isn't needed, but does remove it in some. Similarly GFP_NOFS is left is some cases where it might not be necessary. These omissions can be rectified later following proper analysis. I plan to send out several sets of lustre patches today (Christmas holiday reading :-) so I've labeled this "PATCH SERIES 1" to it is easier to apply them in the right order, if there are acceptable. Thanks, NeilBrown --- NeilBrown (15): staging: lustre: lnet-lib: opencode some alloc/free functions. staging: lustre: lnet: discard CFS_ALLOC_PTR staging: lustre: replace simple cases of LIBCFS_ALLOC with kzalloc. staging: lustre: lnet: switch to cpumask_var_t staging: lustre: lnet: selftest: don't allocate small strings. staging: lustre: lnet: use kmalloc/kvmalloc in router_proc staging: lustre: change some LIBCFS_ALLOC calls to k?alloc(GFP_KERNEL) staging: lustre: Convert more LIBCFS_ALLOC allocation to direct GFP_KERNEL staging: lustre: more LIBCFS_ALLOC conversions to GFP_KERNEL allocations. staging: lustre: more conversions to GFP_KERNEL allocations. staging: lustre: lnet-route: use kmalloc for small allocation staging: lustre: use kmalloc for allocating ksock_tx staging: lustre: cfs_percpt_alloc: use kvmalloc(GFP_KERNEL) staging: lustre: opencode LIBCFS_ALLOC_ATOMIC calls. staging: lustre: remove LIBCFS_ALLOC and LIBCFS_ALLOC_ATOMIC .../lustre/include/linux/libcfs/libcfs_cpu.h | 4 .../lustre/include/linux/libcfs/libcfs_private.h | 22 -- .../lustre/include/linux/libcfs/libcfs_string.h | 4 .../lustre/include/linux/libcfs/linux/linux-cpu.h | 4 .../staging/lustre/include/linux/lnet/lib-lnet.h | 63 ------- .../staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c | 55 +++--- .../staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c | 4 .../lustre/lnet/klnds/o2iblnd/o2iblnd_modparams.c | 4 .../staging/lustre/lnet/klnds/socklnd/socklnd.c | 46 ++--- .../staging/lustre/lnet/klnds/socklnd/socklnd_cb.c | 6 - .../lustre/lnet/klnds/socklnd/socklnd_proto.c | 8 - drivers/staging/lustre/lnet/libcfs/hash.c | 18 +- drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c | 6 - drivers/staging/lustre/lnet/libcfs/libcfs_lock.c | 6 - drivers/staging/lustre/lnet/libcfs/libcfs_mem.c | 15 +- drivers/staging/lustre/lnet/libcfs/libcfs_string.c | 12 + .../staging/lustre/lnet/libcfs/linux/linux-cpu.c | 126 ++++++------- .../lustre/lnet/libcfs/linux/linux-module.c | 4 drivers/staging/lustre/lnet/libcfs/module.c | 9 - drivers/staging/lustre/lnet/libcfs/workitem.c | 8 - drivers/staging/lustre/lnet/lnet/api-ni.c | 25 +-- drivers/staging/lustre/lnet/lnet/config.c | 42 ++-- drivers/staging/lustre/lnet/lnet/lib-eq.c | 15 +- drivers/staging/lustre/lnet/lnet/lib-md.c | 8 - drivers/staging/lustre/lnet/lnet/lib-me.c | 10 + drivers/staging/lustre/lnet/lnet/lib-move.c | 24 +-- drivers/staging/lustre/lnet/lnet/lib-msg.c | 6 - drivers/staging/lustre/lnet/lnet/lib-ptl.c | 2 drivers/staging/lustre/lnet/lnet/lib-socket.c | 14 + drivers/staging/lustre/lnet/lnet/net_fault.c | 10 + drivers/staging/lustre/lnet/lnet/nidstrings.c | 8 - drivers/staging/lustre/lnet/lnet/peer.c | 2 drivers/staging/lustre/lnet/lnet/router.c | 31 +-- drivers/staging/lustre/lnet/lnet/router_proc.c | 40 ++-- drivers/staging/lustre/lnet/selftest/conctl.c | 191 ++++---------------- drivers/staging/lustre/lnet/selftest/conrpc.c | 10 + drivers/staging/lustre/lnet/selftest/console.c | 83 ++++----- drivers/staging/lustre/lnet/selftest/framework.c | 26 +-- drivers/staging/lustre/lnet/selftest/module.c | 7 - drivers/staging/lustre/lnet/selftest/rpc.c | 16 +- drivers/staging/lustre/lnet/selftest/selftest.h | 2 .../lustre/lustre/obdclass/lprocfs_status.c | 21 +- 42 files changed, 381 insertions(+), 636 deletions(-) -- Signature From neilb at suse.com Mon Dec 18 00:46:30 2017 From: neilb at suse.com (NeilBrown) Date: Mon, 18 Dec 2017 11:46:30 +1100 Subject: [lustre-devel] [PATCH 01/15] staging: lustre: lnet-lib: opencode some alloc/free functions. In-Reply-To: <151355781721.6200.2136335532722530242.stgit@noble> References: <151355781721.6200.2136335532722530242.stgit@noble> Message-ID: <151355799021.6200.6628779311633006988.stgit@noble> These functions just call LIBCFS_ALLOC() which in-turn calls kvmalloc(). In none of these cases is the 'vmalloc' option needed. LIBCFS_ALLOC also produces a warning if NULL is returned, but that can be provided with CONFIG_SLAB_DEBUG. LIBCFS_ALLOC zeros the memory, so we need to use __GFP_ZERO too. So with one exception where the alloc function is not trivial, open-code the alloc and free functions using kmalloc and kfree. Note that the 'size' used in lnet_md_alloc() is limited and less than a page because LNET_MAX_IOV is 256, so kvmalloc is not needed. Signed-off-by: NeilBrown --- .../staging/lustre/include/linux/lnet/lib-lnet.h | 63 -------------------- drivers/staging/lustre/lnet/lnet/api-ni.c | 4 + drivers/staging/lustre/lnet/lnet/lib-eq.c | 6 +- drivers/staging/lustre/lnet/lnet/lib-md.c | 8 +-- drivers/staging/lustre/lnet/lnet/lib-me.c | 10 ++- drivers/staging/lustre/lnet/lnet/lib-move.c | 18 +++--- drivers/staging/lustre/lnet/lnet/lib-msg.c | 6 +- drivers/staging/lustre/lnet/lnet/lib-ptl.c | 2 - 8 files changed, 28 insertions(+), 89 deletions(-) diff --git a/drivers/staging/lustre/include/linux/lnet/lib-lnet.h b/drivers/staging/lustre/include/linux/lnet/lib-lnet.h index c1626726fa05..df4c72507a15 100644 --- a/drivers/staging/lustre/include/linux/lnet/lib-lnet.h +++ b/drivers/staging/lustre/include/linux/lnet/lib-lnet.h @@ -181,21 +181,6 @@ lnet_net_lock_current(void) #define MAX_PORTALS 64 -static inline struct lnet_eq * -lnet_eq_alloc(void) -{ - struct lnet_eq *eq; - - LIBCFS_ALLOC(eq, sizeof(*eq)); - return eq; -} - -static inline void -lnet_eq_free(struct lnet_eq *eq) -{ - LIBCFS_FREE(eq, sizeof(*eq)); -} - static inline struct lnet_libmd * lnet_md_alloc(struct lnet_md *umd) { @@ -211,7 +196,7 @@ lnet_md_alloc(struct lnet_md *umd) size = offsetof(struct lnet_libmd, md_iov.iov[niov]); } - LIBCFS_ALLOC(md, size); + md = kzalloc(size, GFP_NOFS); if (md) { /* Set here in case of early free */ @@ -223,52 +208,6 @@ lnet_md_alloc(struct lnet_md *umd) return md; } -static inline void -lnet_md_free(struct lnet_libmd *md) -{ - unsigned int size; - - if (md->md_options & LNET_MD_KIOV) - size = offsetof(struct lnet_libmd, md_iov.kiov[md->md_niov]); - else - size = offsetof(struct lnet_libmd, md_iov.iov[md->md_niov]); - - LIBCFS_FREE(md, size); -} - -static inline struct lnet_me * -lnet_me_alloc(void) -{ - struct lnet_me *me; - - LIBCFS_ALLOC(me, sizeof(*me)); - return me; -} - -static inline void -lnet_me_free(struct lnet_me *me) -{ - LIBCFS_FREE(me, sizeof(*me)); -} - -static inline struct lnet_msg * -lnet_msg_alloc(void) -{ - struct lnet_msg *msg; - - LIBCFS_ALLOC(msg, sizeof(*msg)); - - /* no need to zero, LIBCFS_ALLOC does for us */ - return msg; -} - -static inline void -lnet_msg_free(struct lnet_msg *msg) -{ - LASSERT(!msg->msg_onactivelist); - LIBCFS_FREE(msg, sizeof(*msg)); -} - struct lnet_libhandle *lnet_res_lh_lookup(struct lnet_res_container *rec, __u64 cookie); void lnet_res_lh_initialize(struct lnet_res_container *rec, diff --git a/drivers/staging/lustre/lnet/lnet/api-ni.c b/drivers/staging/lustre/lnet/lnet/api-ni.c index 7caff290c146..30d0999118c7 100644 --- a/drivers/staging/lustre/lnet/lnet/api-ni.c +++ b/drivers/staging/lustre/lnet/lnet/api-ni.c @@ -384,10 +384,10 @@ lnet_res_container_cleanup(struct lnet_res_container *rec) list_del_init(e); if (rec->rec_type == LNET_COOKIE_TYPE_EQ) { - lnet_eq_free(list_entry(e, struct lnet_eq, eq_list)); + kfree(list_entry(e, struct lnet_eq, eq_list)); } else if (rec->rec_type == LNET_COOKIE_TYPE_MD) { - lnet_md_free(list_entry(e, struct lnet_libmd, md_list)); + kfree(list_entry(e, struct lnet_libmd, md_list)); } else { /* NB: Active MEs should be attached on portals */ LBUG(); diff --git a/drivers/staging/lustre/lnet/lnet/lib-eq.c b/drivers/staging/lustre/lnet/lnet/lib-eq.c index daf744277003..7a4d1f7a693e 100644 --- a/drivers/staging/lustre/lnet/lnet/lib-eq.c +++ b/drivers/staging/lustre/lnet/lnet/lib-eq.c @@ -90,7 +90,7 @@ LNetEQAlloc(unsigned int count, lnet_eq_handler_t callback, if (!count && callback == LNET_EQ_HANDLER_NONE) return -EINVAL; - eq = lnet_eq_alloc(); + eq = kzalloc(sizeof(*eq), GFP_NOFS); if (!eq) return -ENOMEM; @@ -138,7 +138,7 @@ LNetEQAlloc(unsigned int count, lnet_eq_handler_t callback, if (eq->eq_refs) cfs_percpt_free(eq->eq_refs); - lnet_eq_free(eq); + kfree(eq); return -ENOMEM; } EXPORT_SYMBOL(LNetEQAlloc); @@ -197,7 +197,7 @@ LNetEQFree(struct lnet_handle_eq eqh) lnet_res_lh_invalidate(&eq->eq_lh); list_del(&eq->eq_list); - lnet_eq_free(eq); + kfree(eq); out: lnet_eq_wait_unlock(); lnet_res_unlock(LNET_LOCK_EX); diff --git a/drivers/staging/lustre/lnet/lnet/lib-md.c b/drivers/staging/lustre/lnet/lnet/lib-md.c index ac5b9593d597..8a22514aaf71 100644 --- a/drivers/staging/lustre/lnet/lnet/lib-md.c +++ b/drivers/staging/lustre/lnet/lnet/lib-md.c @@ -81,7 +81,7 @@ lnet_md_unlink(struct lnet_libmd *md) LASSERT(!list_empty(&md->md_list)); list_del_init(&md->md_list); - lnet_md_free(md); + kfree(md); } static int @@ -173,7 +173,7 @@ lnet_md_link(struct lnet_libmd *md, struct lnet_handle_eq eq_handle, int cpt) /* * NB we are passed an allocated, but inactive md. * if we return success, caller may lnet_md_unlink() it. - * otherwise caller may only lnet_md_free() it. + * otherwise caller may only kfree() it. */ /* * This implementation doesn't know how to create START events or @@ -329,7 +329,7 @@ LNetMDAttach(struct lnet_handle_me meh, struct lnet_md umd, out_unlock: lnet_res_unlock(cpt); out_free: - lnet_md_free(md); + kfree(md); return rc; } EXPORT_SYMBOL(LNetMDAttach); @@ -390,7 +390,7 @@ LNetMDBind(struct lnet_md umd, enum lnet_unlink unlink, out_unlock: lnet_res_unlock(cpt); out_free: - lnet_md_free(md); + kfree(md); return rc; } diff --git a/drivers/staging/lustre/lnet/lnet/lib-me.c b/drivers/staging/lustre/lnet/lnet/lib-me.c index dd5d3cf6d3e2..672e37bdd045 100644 --- a/drivers/staging/lustre/lnet/lnet/lib-me.c +++ b/drivers/staging/lustre/lnet/lnet/lib-me.c @@ -90,7 +90,7 @@ LNetMEAttach(unsigned int portal, if (!mtable) /* can't match portal type */ return -EPERM; - me = lnet_me_alloc(); + me = kzalloc(sizeof(*me), GFP_NOFS); if (!me) return -ENOMEM; @@ -157,7 +157,7 @@ LNetMEInsert(struct lnet_handle_me current_meh, if (pos == LNET_INS_LOCAL) return -EPERM; - new_me = lnet_me_alloc(); + new_me = kzalloc(sizeof(*new_me), GFP_NOFS); if (!new_me) return -ENOMEM; @@ -167,7 +167,7 @@ LNetMEInsert(struct lnet_handle_me current_meh, current_me = lnet_handle2me(¤t_meh); if (!current_me) { - lnet_me_free(new_me); + kfree(new_me); lnet_res_unlock(cpt); return -ENOENT; @@ -178,7 +178,7 @@ LNetMEInsert(struct lnet_handle_me current_meh, ptl = the_lnet.ln_portals[current_me->me_portal]; if (lnet_ptl_is_unique(ptl)) { /* nosense to insertion on unique portal */ - lnet_me_free(new_me); + kfree(new_me); lnet_res_unlock(cpt); return -EPERM; } @@ -270,5 +270,5 @@ lnet_me_unlink(struct lnet_me *me) } lnet_res_lh_invalidate(&me->me_lh); - lnet_me_free(me); + kfree(me); } diff --git a/drivers/staging/lustre/lnet/lnet/lib-move.c b/drivers/staging/lustre/lnet/lnet/lib-move.c index 68d16ffec980..137e3ab970dc 100644 --- a/drivers/staging/lustre/lnet/lnet/lib-move.c +++ b/drivers/staging/lustre/lnet/lnet/lib-move.c @@ -1769,7 +1769,7 @@ lnet_parse(struct lnet_ni *ni, struct lnet_hdr *hdr, lnet_nid_t from_nid, goto drop; } - msg = lnet_msg_alloc(); + msg = kzalloc(sizeof(*msg), GFP_NOFS); if (!msg) { CERROR("%s, src %s: Dropping %s (out of memory)\n", libcfs_nid2str(from_nid), libcfs_nid2str(src_nid), @@ -1777,7 +1777,7 @@ lnet_parse(struct lnet_ni *ni, struct lnet_hdr *hdr, lnet_nid_t from_nid, goto drop; } - /* msg zeroed in lnet_msg_alloc; + /* msg zeroed by kzalloc() * i.e. flags all clear, pointers NULL etc */ msg->msg_type = type; @@ -1812,7 +1812,7 @@ lnet_parse(struct lnet_ni *ni, struct lnet_hdr *hdr, lnet_nid_t from_nid, CERROR("%s, src %s: Dropping %s (error %d looking up sender)\n", libcfs_nid2str(from_nid), libcfs_nid2str(src_nid), lnet_msgtyp2str(type), rc); - lnet_msg_free(msg); + kfree(msg); if (rc == -ESHUTDOWN) /* We are shutting down. Don't do anything more */ return 0; @@ -2010,7 +2010,7 @@ LNetPut(lnet_nid_t self, struct lnet_handle_md mdh, enum lnet_ack_req ack, return -EIO; } - msg = lnet_msg_alloc(); + msg = kzalloc(sizeof(*msg), GFP_NOFS); if (!msg) { CERROR("Dropping PUT to %s: ENOMEM on struct lnet_msg\n", libcfs_id2str(target)); @@ -2031,7 +2031,7 @@ LNetPut(lnet_nid_t self, struct lnet_handle_md mdh, enum lnet_ack_req ack, md->md_me->me_portal); lnet_res_unlock(cpt); - lnet_msg_free(msg); + kfree(msg); return -ENOENT; } @@ -2086,7 +2086,7 @@ lnet_create_reply_msg(struct lnet_ni *ni, struct lnet_msg *getmsg) * CAVEAT EMPTOR: 'getmsg' is the original GET, which is freed when * lnet_finalize() is called on it, so the LND must call this first */ - struct lnet_msg *msg = lnet_msg_alloc(); + struct lnet_msg *msg = kzalloc(sizeof(*msg), GFP_NOFS); struct lnet_libmd *getmd = getmsg->msg_md; struct lnet_process_id peer_id = getmsg->msg_target; int cpt; @@ -2147,7 +2147,7 @@ lnet_create_reply_msg(struct lnet_ni *ni, struct lnet_msg *getmsg) lnet_net_unlock(cpt); if (msg) - lnet_msg_free(msg); + kfree(msg); return NULL; } @@ -2215,7 +2215,7 @@ LNetGet(lnet_nid_t self, struct lnet_handle_md mdh, return -EIO; } - msg = lnet_msg_alloc(); + msg = kzalloc(sizeof(*msg), GFP_NOFS); if (!msg) { CERROR("Dropping GET to %s: ENOMEM on struct lnet_msg\n", libcfs_id2str(target)); @@ -2236,7 +2236,7 @@ LNetGet(lnet_nid_t self, struct lnet_handle_md mdh, lnet_res_unlock(cpt); - lnet_msg_free(msg); + kfree(msg); return -ENOENT; } diff --git a/drivers/staging/lustre/lnet/lnet/lib-msg.c b/drivers/staging/lustre/lnet/lnet/lib-msg.c index c72ef05b2420..ff6c43323fb5 100644 --- a/drivers/staging/lustre/lnet/lnet/lib-msg.c +++ b/drivers/staging/lustre/lnet/lnet/lib-msg.c @@ -433,7 +433,7 @@ lnet_complete_msg_locked(struct lnet_msg *msg, int cpt) } lnet_msg_decommit(msg, cpt, status); - lnet_msg_free(msg); + kfree(msg); return 0; } @@ -466,7 +466,7 @@ lnet_finalize(struct lnet_ni *ni, struct lnet_msg *msg, int status) if (!msg->msg_tx_committed && !msg->msg_rx_committed) { /* not committed to network yet */ LASSERT(!msg->msg_onactivelist); - lnet_msg_free(msg); + kfree(msg); return; } @@ -546,7 +546,7 @@ lnet_msg_container_cleanup(struct lnet_msg_container *container) LASSERT(msg->msg_onactivelist); msg->msg_onactivelist = 0; list_del(&msg->msg_activelist); - lnet_msg_free(msg); + kfree(msg); count++; } diff --git a/drivers/staging/lustre/lnet/lnet/lib-ptl.c b/drivers/staging/lustre/lnet/lnet/lib-ptl.c index 8ae93bf6fd1b..519cfebaaa88 100644 --- a/drivers/staging/lustre/lnet/lnet/lib-ptl.c +++ b/drivers/staging/lustre/lnet/lnet/lib-ptl.c @@ -771,7 +771,7 @@ lnet_ptl_cleanup(struct lnet_portal *ptl) struct lnet_me, me_list); CERROR("Active ME %p on exit\n", me); list_del(&me->me_list); - lnet_me_free(me); + kfree(me); } } /* the extra entry is for MEs with ignore bits */ From neilb at suse.com Mon Dec 18 00:46:30 2017 From: neilb at suse.com (NeilBrown) Date: Mon, 18 Dec 2017 11:46:30 +1100 Subject: [lustre-devel] [PATCH 02/15] staging: lustre: lnet: discard CFS_ALLOC_PTR In-Reply-To: <151355781721.6200.2136335532722530242.stgit@noble> References: <151355781721.6200.2136335532722530242.stgit@noble> Message-ID: <151355799026.6200.16032725626950825179.stgit@noble> These trivial wrappers hurt readability and as they use kvmalloc, they are overly generic. So discard them and use kmalloc/kfree as is normal in Linux. Signed-off-by: NeilBrown --- .../lustre/include/linux/libcfs/libcfs_private.h | 3 --- drivers/staging/lustre/lnet/lnet/net_fault.c | 10 +++++----- 2 files changed, 5 insertions(+), 8 deletions(-) diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_private.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_private.h index 940200ee632e..d230c7f7cced 100644 --- a/drivers/staging/lustre/include/linux/libcfs/libcfs_private.h +++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_private.h @@ -215,9 +215,6 @@ do { \ #define LASSERT_ATOMIC_ZERO(a) LASSERT_ATOMIC_EQ(a, 0) #define LASSERT_ATOMIC_POS(a) LASSERT_ATOMIC_GT(a, 0) -#define CFS_ALLOC_PTR(ptr) LIBCFS_ALLOC(ptr, sizeof(*(ptr))) -#define CFS_FREE_PTR(ptr) LIBCFS_FREE(ptr, sizeof(*(ptr))) - /* implication */ #define ergo(a, b) (!(a) || (b)) /* logical equivalence */ diff --git a/drivers/staging/lustre/lnet/lnet/net_fault.c b/drivers/staging/lustre/lnet/lnet/net_fault.c index 5a5d1811ffbe..0318e64c413f 100644 --- a/drivers/staging/lustre/lnet/lnet/net_fault.c +++ b/drivers/staging/lustre/lnet/lnet/net_fault.c @@ -161,7 +161,7 @@ lnet_drop_rule_add(struct lnet_fault_attr *attr) if (lnet_fault_attr_validate(attr)) return -EINVAL; - CFS_ALLOC_PTR(rule); + rule = kzalloc(sizeof(*rule), GFP_NOFS); if (!rule) return -ENOMEM; @@ -223,7 +223,7 @@ lnet_drop_rule_del(lnet_nid_t src, lnet_nid_t dst) rule->dr_attr.u.drop.da_interval); list_del(&rule->dr_link); - CFS_FREE_PTR(rule); + kfree(rule); n++; } @@ -452,7 +452,7 @@ delay_rule_decref(struct lnet_delay_rule *rule) LASSERT(list_empty(&rule->dl_msg_list)); LASSERT(list_empty(&rule->dl_link)); - CFS_FREE_PTR(rule); + kfree(rule); } } @@ -738,7 +738,7 @@ lnet_delay_rule_add(struct lnet_fault_attr *attr) if (lnet_fault_attr_validate(attr)) return -EINVAL; - CFS_ALLOC_PTR(rule); + rule = kzalloc(sizeof(*rule), GFP_NOFS); if (!rule) return -ENOMEM; @@ -792,7 +792,7 @@ lnet_delay_rule_add(struct lnet_fault_attr *attr) return 0; failed: mutex_unlock(&delay_dd.dd_mutex); - CFS_FREE_PTR(rule); + kfree(rule); return rc; } From neilb at suse.com Mon Dec 18 00:46:30 2017 From: neilb at suse.com (NeilBrown) Date: Mon, 18 Dec 2017 11:46:30 +1100 Subject: [lustre-devel] [PATCH 03/15] staging: lustre: replace simple cases of LIBCFS_ALLOC with kzalloc. In-Reply-To: <151355781721.6200.2136335532722530242.stgit@noble> References: <151355781721.6200.2136335532722530242.stgit@noble> Message-ID: <151355799030.6200.8100283223734974485.stgit@noble> All usages of the form LIBCFS_ALLOC(variable, sizeof(variable)) or LIBCFS_ALLOC(variable, sizeof(variable's-type)) are changed to variable = kzalloc(sizeof(...), GFP_NOFS); Similarly, all LIBCFS_FREE(variable, sizeof(variable)) become kfree(variable); None of these need the vmalloc option, or any of the other minor benefits of LIBCFS_ALLOC(). Signed-off-by: NeilBrown --- .../staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c | 38 ++++++++++---------- .../staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c | 4 +- .../lustre/lnet/klnds/o2iblnd/o2iblnd_modparams.c | 4 +- .../staging/lustre/lnet/klnds/socklnd/socklnd.c | 22 ++++++------ .../staging/lustre/lnet/klnds/socklnd/socklnd_cb.c | 2 + .../lustre/lnet/klnds/socklnd/socklnd_proto.c | 8 ++-- drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c | 4 +- drivers/staging/lustre/lnet/libcfs/libcfs_lock.c | 6 ++- drivers/staging/lustre/lnet/libcfs/libcfs_string.c | 10 +++-- .../staging/lustre/lnet/libcfs/linux/linux-cpu.c | 15 ++++---- drivers/staging/lustre/lnet/libcfs/workitem.c | 8 ++-- drivers/staging/lustre/lnet/lnet/api-ni.c | 4 +- drivers/staging/lustre/lnet/lnet/config.c | 6 ++- drivers/staging/lustre/lnet/lnet/lib-move.c | 6 ++- drivers/staging/lustre/lnet/lnet/nidstrings.c | 8 ++-- drivers/staging/lustre/lnet/lnet/peer.c | 2 + drivers/staging/lustre/lnet/lnet/router.c | 26 ++++++-------- drivers/staging/lustre/lnet/lnet/router_proc.c | 6 ++- drivers/staging/lustre/lnet/selftest/conrpc.c | 10 +++-- drivers/staging/lustre/lnet/selftest/console.c | 26 +++++++------- drivers/staging/lustre/lnet/selftest/framework.c | 22 ++++++------ drivers/staging/lustre/lnet/selftest/rpc.c | 12 +++--- 22 files changed, 124 insertions(+), 125 deletions(-) diff --git a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c index 8024843521ab..039f53d787e4 100644 --- a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c +++ b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c @@ -367,7 +367,7 @@ void kiblnd_destroy_peer(struct kib_peer *peer) LASSERT(kiblnd_peer_idle(peer)); LASSERT(list_empty(&peer->ibp_tx_queue)); - LIBCFS_FREE(peer, sizeof(*peer)); + kfree(peer); /* * NB a peer's connections keep a reference on their peer until @@ -776,7 +776,7 @@ struct kib_conn *kiblnd_create_conn(struct kib_peer *peer, struct rdma_cm_id *cm goto failed_2; } - LIBCFS_FREE(init_qp_attr, sizeof(*init_qp_attr)); + kfree(init_qp_attr); /* 1 ref for caller and each rxmsg */ atomic_set(&conn->ibc_refcount, 1 + IBLND_RX_MSGS(conn)); @@ -828,7 +828,7 @@ struct kib_conn *kiblnd_create_conn(struct kib_peer *peer, struct rdma_cm_id *cm failed_2: kiblnd_destroy_conn(conn, true); failed_1: - LIBCFS_FREE(init_qp_attr, sizeof(*init_qp_attr)); + kfree(init_qp_attr); failed_0: return NULL; } @@ -883,7 +883,7 @@ void kiblnd_destroy_conn(struct kib_conn *conn, bool free_conn) } if (conn->ibc_connvars) - LIBCFS_FREE(conn->ibc_connvars, sizeof(*conn->ibc_connvars)); + kfree(conn->ibc_connvars); if (conn->ibc_hdev) kiblnd_hdev_decref(conn->ibc_hdev); @@ -897,7 +897,7 @@ void kiblnd_destroy_conn(struct kib_conn *conn, bool free_conn) atomic_dec(&net->ibn_nconns); } - LIBCFS_FREE(conn, sizeof(*conn)); + kfree(conn); } int kiblnd_close_peer_conns_locked(struct kib_peer *peer, int why) @@ -1299,7 +1299,7 @@ static void kiblnd_destroy_fmr_pool(struct kib_fmr_pool *fpo) frd_list) { list_del(&frd->frd_list); ib_dereg_mr(frd->frd_mr); - LIBCFS_FREE(frd, sizeof(*frd)); + kfree(frd); i++; } if (i < fpo->fast_reg.fpo_pool_size) @@ -1310,7 +1310,7 @@ static void kiblnd_destroy_fmr_pool(struct kib_fmr_pool *fpo) if (fpo->fpo_hdev) kiblnd_hdev_decref(fpo->fpo_hdev); - LIBCFS_FREE(fpo, sizeof(*fpo)); + kfree(fpo); } static void kiblnd_destroy_fmr_pool_list(struct list_head *head) @@ -1405,14 +1405,14 @@ static int kiblnd_alloc_freg_pool(struct kib_fmr_poolset *fps, struct kib_fmr_po out_middle: if (frd->frd_mr) ib_dereg_mr(frd->frd_mr); - LIBCFS_FREE(frd, sizeof(*frd)); + kfree(frd); out: list_for_each_entry_safe(frd, tmp, &fpo->fast_reg.fpo_pool_list, frd_list) { list_del(&frd->frd_list); ib_dereg_mr(frd->frd_mr); - LIBCFS_FREE(frd, sizeof(*frd)); + kfree(frd); } return rc; @@ -1464,7 +1464,7 @@ static int kiblnd_create_fmr_pool(struct kib_fmr_poolset *fps, out_fpo: kiblnd_hdev_decref(fpo->fpo_hdev); - LIBCFS_FREE(fpo, sizeof(*fpo)); + kfree(fpo); return rc; } @@ -2011,7 +2011,7 @@ static void kiblnd_destroy_tx_pool(struct kib_pool *pool) pool->po_size * sizeof(struct kib_tx)); out: kiblnd_fini_pool(pool); - LIBCFS_FREE(tpo, sizeof(*tpo)); + kfree(tpo); } static int kiblnd_tx_pool_size(int ncpts) @@ -2043,7 +2043,7 @@ static int kiblnd_create_tx_pool(struct kib_poolset *ps, int size, npg = DIV_ROUND_UP(size * IBLND_MSG_SIZE, PAGE_SIZE); if (kiblnd_alloc_pages(&tpo->tpo_tx_pages, ps->ps_cpt, npg)) { CERROR("Can't allocate tx pages: %d\n", npg); - LIBCFS_FREE(tpo, sizeof(*tpo)); + kfree(tpo); return -ENOMEM; } @@ -2263,7 +2263,7 @@ void kiblnd_hdev_destroy(struct kib_hca_dev *hdev) if (hdev->ibh_cmid) rdma_destroy_id(hdev->ibh_cmid); - LIBCFS_FREE(hdev, sizeof(*hdev)); + kfree(hdev); } /* DUMMY */ @@ -2392,7 +2392,7 @@ int kiblnd_dev_failover(struct kib_dev *dev) goto out; } - LIBCFS_ALLOC(hdev, sizeof(*hdev)); + kdev = kzalloc(sizeof(*hdev), GFP_NOFS); if (!hdev) { CERROR("Failed to allocate kib_hca_dev\n"); rdma_destroy_id(cmid); @@ -2471,7 +2471,7 @@ void kiblnd_destroy_dev(struct kib_dev *dev) if (dev->ibd_hdev) kiblnd_hdev_decref(dev->ibd_hdev); - LIBCFS_FREE(dev, sizeof(*dev)); + kfree(dev); } static struct kib_dev *kiblnd_create_dev(char *ifname) @@ -2495,7 +2495,7 @@ static struct kib_dev *kiblnd_create_dev(char *ifname) return NULL; } - LIBCFS_ALLOC(dev, sizeof(*dev)); + dev = kzalloc(sizeof(*dev), GFP_NOFS); if (!dev) return NULL; @@ -2517,7 +2517,7 @@ static struct kib_dev *kiblnd_create_dev(char *ifname) rc = kiblnd_dev_failover(dev); if (rc) { CERROR("Can't initialize device: %d\n", rc); - LIBCFS_FREE(dev, sizeof(*dev)); + kfree(dev); return NULL; } @@ -2648,7 +2648,7 @@ static void kiblnd_shutdown(struct lnet_ni *ni) net->ibn_init = IBLND_INIT_NOTHING; ni->ni_data = NULL; - LIBCFS_FREE(net, sizeof(*net)); + kfree(net); out: if (list_empty(&kiblnd_data.kib_devs)) @@ -2865,7 +2865,7 @@ static int kiblnd_startup(struct lnet_ni *ni) return rc; } - LIBCFS_ALLOC(net, sizeof(*net)); + net = kzalloc(sizeof(*net), GFP_NOFS); ni->ni_data = net; if (!net) goto net_failed; diff --git a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c index 40e3af5d8b04..9b3328c5d1e7 100644 --- a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c +++ b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c @@ -2124,7 +2124,7 @@ kiblnd_connreq_done(struct kib_conn *conn, int status) (conn->ibc_state == IBLND_CONN_PASSIVE_WAIT && peer->ibp_accepting > 0)); - LIBCFS_FREE(conn->ibc_connvars, sizeof(*conn->ibc_connvars)); + kfree(conn->ibc_connvars); conn->ibc_connvars = NULL; if (status) { @@ -3363,7 +3363,7 @@ kiblnd_connd(void *arg) reconn += kiblnd_reconnect_peer(conn->ibc_peer); kiblnd_peer_decref(conn->ibc_peer); - LIBCFS_FREE(conn, sizeof(*conn)); + kfree(conn); spin_lock_irqsave(lock, flags); } diff --git a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_modparams.c b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_modparams.c index a71b765215ad..b9235400bf1d 100644 --- a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_modparams.c +++ b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_modparams.c @@ -181,8 +181,8 @@ int kiblnd_tunables_setup(struct lnet_ni *ni) * defaulted */ if (!ni->ni_lnd_tunables) { - LIBCFS_ALLOC(ni->ni_lnd_tunables, - sizeof(*ni->ni_lnd_tunables)); + ni->ni_lnd_tunables = kzalloc(sizeof(*ni->ni_lnd_tunables), + GFP_NOFS); if (!ni->ni_lnd_tunables) return -ENOMEM; diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c index 8267119ccc8e..51157ae14a9e 100644 --- a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c +++ b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c @@ -66,7 +66,7 @@ ksocknal_create_route(__u32 ipaddr, int port) { struct ksock_route *route; - LIBCFS_ALLOC(route, sizeof(*route)); + route = kzalloc(sizeof(*route), GFP_NOFS); if (!route) return NULL; @@ -93,7 +93,7 @@ ksocknal_destroy_route(struct ksock_route *route) if (route->ksnr_peer) ksocknal_peer_decref(route->ksnr_peer); - LIBCFS_FREE(route, sizeof(*route)); + kfree(route); } static int @@ -132,7 +132,7 @@ ksocknal_create_peer(struct ksock_peer **peerp, struct lnet_ni *ni, if (net->ksnn_shutdown) { spin_unlock_bh(&net->ksnn_lock); - LIBCFS_FREE(peer, sizeof(*peer)); + kfree(peer); CERROR("Can't create peer: network shutdown\n"); return -ESHUTDOWN; } @@ -160,7 +160,7 @@ ksocknal_destroy_peer(struct ksock_peer *peer) LASSERT(list_empty(&peer->ksnp_tx_queue)); LASSERT(list_empty(&peer->ksnp_zc_req_list)); - LIBCFS_FREE(peer, sizeof(*peer)); + kfree(peer); /* * NB a peer's connections and routes keep a reference on their peer @@ -985,7 +985,7 @@ ksocknal_accept(struct lnet_ni *ni, struct socket *sock) rc = lnet_sock_getaddr(sock, 1, &peer_ip, &peer_port); LASSERT(!rc); /* we succeeded before */ - LIBCFS_ALLOC(cr, sizeof(*cr)); + cr = kzalloc(sizeof(*cr), GFP_NOFS); if (!cr) { LCONSOLE_ERROR_MSG(0x12f, "Dropping connection request from %pI4h: memory exhausted\n", &peer_ip); @@ -1043,7 +1043,7 @@ ksocknal_create_conn(struct lnet_ni *ni, struct ksock_route *route, LASSERT(active == (type != SOCKLND_CONN_NONE)); - LIBCFS_ALLOC(conn, sizeof(*conn)); + conn = kzalloc(sizeof(*conn), GFP_NOFS); if (!conn) { rc = -ENOMEM; goto failed_0; @@ -1419,7 +1419,7 @@ ksocknal_create_conn(struct lnet_ni *ni, struct ksock_route *route, LIBCFS_FREE(hello, offsetof(struct ksock_hello_msg, kshm_ips[LNET_MAX_INTERFACES])); - LIBCFS_FREE(conn, sizeof(*conn)); + kfree(conn); failed_0: sock_release(sock); @@ -1716,7 +1716,7 @@ ksocknal_destroy_conn(struct ksock_conn *conn) ksocknal_peer_decref(conn->ksnc_peer); - LIBCFS_FREE(conn, sizeof(*conn)); + kfree(conn); } int @@ -2622,7 +2622,7 @@ ksocknal_shutdown(struct lnet_ni *ni) } list_del(&net->ksnn_list); - LIBCFS_FREE(net, sizeof(*net)); + kfree(net); ksocknal_data.ksnd_nnets--; if (!ksocknal_data.ksnd_nnets) @@ -2815,7 +2815,7 @@ ksocknal_startup(struct lnet_ni *ni) return rc; } - LIBCFS_ALLOC(net, sizeof(*net)); + net = kzalloc(sizeof(*net), GFP_NOFS); if (!net) goto fail_0; @@ -2877,7 +2877,7 @@ ksocknal_startup(struct lnet_ni *ni) return 0; fail_1: - LIBCFS_FREE(net, sizeof(*net)); + kfree(net); fail_0: if (!ksocknal_data.ksnd_nnets) ksocknal_base_shutdown(); diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c index 27c56d5ae4e5..994b6693c6b7 100644 --- a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c +++ b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c @@ -2117,7 +2117,7 @@ ksocknal_connd(void *arg) ksocknal_create_conn(cr->ksncr_ni, NULL, cr->ksncr_sock, SOCKLND_CONN_NONE); lnet_ni_decref(cr->ksncr_ni); - LIBCFS_FREE(cr, sizeof(*cr)); + kfree(cr); spin_lock_bh(connd_lock); } diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_proto.c b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_proto.c index d827f770e831..05982dac781c 100644 --- a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_proto.c +++ b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_proto.c @@ -467,7 +467,7 @@ ksocknal_send_hello_v1(struct ksock_conn *conn, struct ksock_hello_msg *hello) BUILD_BUG_ON(sizeof(struct lnet_magicversion) != offsetof(struct lnet_hdr, src_nid)); - LIBCFS_ALLOC(hdr, sizeof(*hdr)); + hdr = kzalloc(sizeof(*hdr), GFP_NOFS); if (!hdr) { CERROR("Can't allocate struct lnet_hdr\n"); return -ENOMEM; @@ -526,7 +526,7 @@ ksocknal_send_hello_v1(struct ksock_conn *conn, struct ksock_hello_msg *hello) &conn->ksnc_ipaddr, conn->ksnc_port); } out: - LIBCFS_FREE(hdr, sizeof(*hdr)); + kfree(hdr); return rc; } @@ -582,7 +582,7 @@ ksocknal_recv_hello_v1(struct ksock_conn *conn, struct ksock_hello_msg *hello, int rc; int i; - LIBCFS_ALLOC(hdr, sizeof(*hdr)); + hdr = kzalloc(sizeof(*hdr), GFP_NOFS); if (!hdr) { CERROR("Can't allocate struct lnet_hdr\n"); return -ENOMEM; @@ -644,7 +644,7 @@ ksocknal_recv_hello_v1(struct ksock_conn *conn, struct ksock_hello_msg *hello, } } out: - LIBCFS_FREE(hdr, sizeof(*hdr)); + kfree(hdr); return rc; } diff --git a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c index e3a4c67a66b5..d05c3932b3b9 100644 --- a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c +++ b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c @@ -51,7 +51,7 @@ cfs_cpt_table_alloc(unsigned int ncpt) return NULL; } - LIBCFS_ALLOC(cptab, sizeof(*cptab)); + cptab = kzalloc(sizeof(*cptab), GFP_NOFS); if (cptab) { cptab->ctb_version = CFS_CPU_VERSION_MAGIC; node_set(0, cptab->ctb_nodemask); @@ -67,7 +67,7 @@ cfs_cpt_table_free(struct cfs_cpt_table *cptab) { LASSERT(cptab->ctb_version == CFS_CPU_VERSION_MAGIC); - LIBCFS_FREE(cptab, sizeof(*cptab)); + kfree(cptab); } EXPORT_SYMBOL(cfs_cpt_table_free); diff --git a/drivers/staging/lustre/lnet/libcfs/libcfs_lock.c b/drivers/staging/lustre/lnet/libcfs/libcfs_lock.c index f6a0040f4ab1..670ad5a34224 100644 --- a/drivers/staging/lustre/lnet/libcfs/libcfs_lock.c +++ b/drivers/staging/lustre/lnet/libcfs/libcfs_lock.c @@ -38,7 +38,7 @@ cfs_percpt_lock_free(struct cfs_percpt_lock *pcl) LASSERT(!pcl->pcl_locked); cfs_percpt_free(pcl->pcl_locks); - LIBCFS_FREE(pcl, sizeof(*pcl)); + kfree(pcl); } EXPORT_SYMBOL(cfs_percpt_lock_free); @@ -58,14 +58,14 @@ cfs_percpt_lock_create(struct cfs_cpt_table *cptab, int i; /* NB: cptab can be NULL, pcl will be for HW CPUs on that case */ - LIBCFS_ALLOC(pcl, sizeof(*pcl)); + pcl = kzalloc(sizeof(*pcl), GFP_NOFS); if (!pcl) return NULL; pcl->pcl_cptab = cptab; pcl->pcl_locks = cfs_percpt_alloc(cptab, sizeof(*lock)); if (!pcl->pcl_locks) { - LIBCFS_FREE(pcl, sizeof(*pcl)); + kfree(pcl); return NULL; } diff --git a/drivers/staging/lustre/lnet/libcfs/libcfs_string.c b/drivers/staging/lustre/lnet/libcfs/libcfs_string.c index bcac5074bf80..0c62855349e3 100644 --- a/drivers/staging/lustre/lnet/libcfs/libcfs_string.c +++ b/drivers/staging/lustre/lnet/libcfs/libcfs_string.c @@ -280,7 +280,7 @@ cfs_range_expr_parse(struct cfs_lstr *src, unsigned int min, unsigned int max, struct cfs_range_expr *re; struct cfs_lstr tok; - LIBCFS_ALLOC(re, sizeof(*re)); + re = kzalloc(sizeof(*re), GFP_NOFS); if (!re) return -ENOMEM; @@ -333,7 +333,7 @@ cfs_range_expr_parse(struct cfs_lstr *src, unsigned int min, unsigned int max, return 0; failed: - LIBCFS_FREE(re, sizeof(*re)); + kfree(re); return -EINVAL; } @@ -488,10 +488,10 @@ cfs_expr_list_free(struct cfs_expr_list *expr_list) expr = list_entry(expr_list->el_exprs.next, struct cfs_range_expr, re_link); list_del(&expr->re_link); - LIBCFS_FREE(expr, sizeof(*expr)); + kfree(expr); } - LIBCFS_FREE(expr_list, sizeof(*expr_list)); + kfree(expr_list); } EXPORT_SYMBOL(cfs_expr_list_free); @@ -510,7 +510,7 @@ cfs_expr_list_parse(char *str, int len, unsigned int min, unsigned int max, struct cfs_lstr src; int rc; - LIBCFS_ALLOC(expr_list, sizeof(*expr_list)); + expr_list = kzalloc(sizeof(*expr_list), GFP_NOFS); if (!expr_list) return -ENOMEM; diff --git a/drivers/staging/lustre/lnet/libcfs/linux/linux-cpu.c b/drivers/staging/lustre/lnet/libcfs/linux/linux-cpu.c index 51823ce71773..9a0f6cce1a1d 100644 --- a/drivers/staging/lustre/lnet/libcfs/linux/linux-cpu.c +++ b/drivers/staging/lustre/lnet/libcfs/linux/linux-cpu.c @@ -103,8 +103,7 @@ cfs_cpt_table_free(struct cfs_cpt_table *cptab) struct cfs_cpu_partition *part = &cptab->ctb_parts[i]; if (part->cpt_nodemask) { - LIBCFS_FREE(part->cpt_nodemask, - sizeof(*part->cpt_nodemask)); + kfree(part->cpt_nodemask); } if (part->cpt_cpumask) @@ -117,11 +116,11 @@ cfs_cpt_table_free(struct cfs_cpt_table *cptab) } if (cptab->ctb_nodemask) - LIBCFS_FREE(cptab->ctb_nodemask, sizeof(*cptab->ctb_nodemask)); + kfree(cptab->ctb_nodemask); if (cptab->ctb_cpumask) LIBCFS_FREE(cptab->ctb_cpumask, cpumask_size()); - LIBCFS_FREE(cptab, sizeof(*cptab)); + kfree(cptab); } EXPORT_SYMBOL(cfs_cpt_table_free); @@ -131,14 +130,15 @@ cfs_cpt_table_alloc(unsigned int ncpt) struct cfs_cpt_table *cptab; int i; - LIBCFS_ALLOC(cptab, sizeof(*cptab)); + cptab = kzalloc(sizeof(*cptab), GFP_NOFS); if (!cptab) return NULL; cptab->ctb_nparts = ncpt; LIBCFS_ALLOC(cptab->ctb_cpumask, cpumask_size()); - LIBCFS_ALLOC(cptab->ctb_nodemask, sizeof(*cptab->ctb_nodemask)); + cptab->ctb_nodemask = kzalloc(sizeof(*cptab->ctb_nodemask), + GFP_NOFS); if (!cptab->ctb_cpumask || !cptab->ctb_nodemask) goto failed; @@ -159,7 +159,8 @@ cfs_cpt_table_alloc(unsigned int ncpt) struct cfs_cpu_partition *part = &cptab->ctb_parts[i]; LIBCFS_ALLOC(part->cpt_cpumask, cpumask_size()); - LIBCFS_ALLOC(part->cpt_nodemask, sizeof(*part->cpt_nodemask)); + part->cpt_nodemask = kzalloc(sizeof(*part->cpt_nodemask), + GFP_NOFS); if (!part->cpt_cpumask || !part->cpt_nodemask) goto failed; } diff --git a/drivers/staging/lustre/lnet/libcfs/workitem.c b/drivers/staging/lustre/lnet/libcfs/workitem.c index 6a05d9bab8dc..74a9595dc730 100644 --- a/drivers/staging/lustre/lnet/libcfs/workitem.c +++ b/drivers/staging/lustre/lnet/libcfs/workitem.c @@ -328,7 +328,7 @@ cfs_wi_sched_destroy(struct cfs_wi_sched *sched) spin_unlock(&cfs_wi_data.wi_glock); LASSERT(!sched->ws_nscheduled); - LIBCFS_FREE(sched, sizeof(*sched)); + kfree(sched); } EXPORT_SYMBOL(cfs_wi_sched_destroy); @@ -344,12 +344,12 @@ cfs_wi_sched_create(char *name, struct cfs_cpt_table *cptab, LASSERT(!cptab || cpt == CFS_CPT_ANY || (cpt >= 0 && cpt < cfs_cpt_number(cptab))); - LIBCFS_ALLOC(sched, sizeof(*sched)); + sched = kzalloc(sizeof(*sched), GFP_NOFS); if (!sched) return -ENOMEM; if (strlen(name) > sizeof(sched->ws_name) - 1) { - LIBCFS_FREE(sched, sizeof(*sched)); + kfree(sched); return -E2BIG; } strncpy(sched->ws_name, name, sizeof(sched->ws_name)); @@ -458,7 +458,7 @@ cfs_wi_shutdown(void) } list_for_each_entry_safe(sched, temp, &cfs_wi_data.wi_scheds, ws_list) { list_del(&sched->ws_list); - LIBCFS_FREE(sched, sizeof(*sched)); + kfree(sched); } cfs_wi_data.wi_stopping = 0; diff --git a/drivers/staging/lustre/lnet/lnet/api-ni.c b/drivers/staging/lustre/lnet/lnet/api-ni.c index 30d0999118c7..e8f623190133 100644 --- a/drivers/staging/lustre/lnet/lnet/api-ni.c +++ b/drivers/staging/lustre/lnet/lnet/api-ni.c @@ -1280,8 +1280,8 @@ lnet_startup_lndni(struct lnet_ni *ni, struct lnet_ioctl_config_data *conf) lnd_tunables = (struct lnet_ioctl_config_lnd_tunables *)conf->cfg_bulk; if (lnd_tunables) { - LIBCFS_ALLOC(ni->ni_lnd_tunables, - sizeof(*ni->ni_lnd_tunables)); + ni->ni_lnd_tunables = kzalloc(sizeof(*ni->ni_lnd_tunables), + GFP_NOFS); if (!ni->ni_lnd_tunables) { mutex_unlock(&the_lnet.ln_lnd_mutex); rc = -ENOMEM; diff --git a/drivers/staging/lustre/lnet/lnet/config.c b/drivers/staging/lustre/lnet/lnet/config.c index 0cf0f4f99435..30fabfbc6898 100644 --- a/drivers/staging/lustre/lnet/lnet/config.c +++ b/drivers/staging/lustre/lnet/lnet/config.c @@ -107,7 +107,7 @@ lnet_ni_free(struct lnet_ni *ni) cfs_expr_list_values_free(ni->ni_cpts, ni->ni_ncpts); if (ni->ni_lnd_tunables) - LIBCFS_FREE(ni->ni_lnd_tunables, sizeof(*ni->ni_lnd_tunables)); + kfree(ni->ni_lnd_tunables); for (i = 0; i < LNET_MAX_INTERFACES && ni->ni_interfaces[i]; i++) { LIBCFS_FREE(ni->ni_interfaces[i], @@ -118,7 +118,7 @@ lnet_ni_free(struct lnet_ni *ni) if (ni->ni_net_ns) put_net(ni->ni_net_ns); - LIBCFS_FREE(ni, sizeof(*ni)); + kfree(ni); } struct lnet_ni * @@ -135,7 +135,7 @@ lnet_ni_alloc(__u32 net, struct cfs_expr_list *el, struct list_head *nilist) return NULL; } - LIBCFS_ALLOC(ni, sizeof(*ni)); + ni = kzalloc(sizeof(*ni), GFP_NOFS); if (!ni) { CERROR("Out of memory creating network %s\n", libcfs_net2str(net)); diff --git a/drivers/staging/lustre/lnet/lnet/lib-move.c b/drivers/staging/lustre/lnet/lnet/lib-move.c index 137e3ab970dc..d724c4c73ecc 100644 --- a/drivers/staging/lustre/lnet/lnet/lib-move.c +++ b/drivers/staging/lustre/lnet/lnet/lib-move.c @@ -57,7 +57,7 @@ lnet_fail_nid(lnet_nid_t nid, unsigned int threshold) /* NB: use lnet_net_lock(0) to serialize operations on test peers */ if (threshold) { /* Adding a new entry */ - LIBCFS_ALLOC(tp, sizeof(*tp)); + tp = kzalloc(sizeof(*tp), GFP_NOFS); if (!tp) return -ENOMEM; @@ -90,7 +90,7 @@ lnet_fail_nid(lnet_nid_t nid, unsigned int threshold) list_for_each_entry_safe(tp, temp, &cull, tp_list) { list_del(&tp->tp_list); - LIBCFS_FREE(tp, sizeof(*tp)); + kfree(tp); } return 0; } @@ -149,7 +149,7 @@ fail_peer(lnet_nid_t nid, int outgoing) list_for_each_entry_safe(tp, temp, &cull, tp_list) { list_del(&tp->tp_list); - LIBCFS_FREE(tp, sizeof(*tp)); + kfree(tp); } return fail; diff --git a/drivers/staging/lustre/lnet/lnet/nidstrings.c b/drivers/staging/lustre/lnet/lnet/nidstrings.c index 05b120c2d45a..3aba1421c741 100644 --- a/drivers/staging/lustre/lnet/lnet/nidstrings.c +++ b/drivers/staging/lustre/lnet/lnet/nidstrings.c @@ -166,7 +166,7 @@ parse_addrange(const struct cfs_lstr *src, struct nidrange *nidrange) return 0; } - LIBCFS_ALLOC(addrrange, sizeof(struct addrrange)); + addrrange = kzalloc(sizeof(struct addrrange), GFP_NOFS); if (!addrrange) return -ENOMEM; list_add_tail(&addrrange->ar_link, &nidrange->nr_addrranges); @@ -225,7 +225,7 @@ add_nidrange(const struct cfs_lstr *src, return nr; } - LIBCFS_ALLOC(nr, sizeof(struct nidrange)); + nr = kzalloc(sizeof(struct nidrange), GFP_NOFS); if (!nr) return NULL; list_add_tail(&nr->nr_link, nidlist); @@ -286,7 +286,7 @@ free_addrranges(struct list_head *list) cfs_expr_list_free_list(&ar->ar_numaddr_ranges); list_del(&ar->ar_link); - LIBCFS_FREE(ar, sizeof(struct addrrange)); + kfree(ar); } } @@ -308,7 +308,7 @@ cfs_free_nidlist(struct list_head *list) nr = list_entry(pos, struct nidrange, nr_link); free_addrranges(&nr->nr_addrranges); list_del(pos); - LIBCFS_FREE(nr, sizeof(struct nidrange)); + kfree(nr); } } EXPORT_SYMBOL(cfs_free_nidlist); diff --git a/drivers/staging/lustre/lnet/lnet/peer.c b/drivers/staging/lustre/lnet/lnet/peer.c index 5e94ad349454..19fcbcf0f642 100644 --- a/drivers/staging/lustre/lnet/lnet/peer.c +++ b/drivers/staging/lustre/lnet/lnet/peer.c @@ -212,7 +212,7 @@ lnet_peer_tables_cleanup(struct lnet_ni *ni) list_for_each_entry_safe(lp, temp, &deathrow, lp_hashlist) { list_del(&lp->lp_hashlist); - LIBCFS_FREE(lp, sizeof(*lp)); + kfree(lp); } } diff --git a/drivers/staging/lustre/lnet/lnet/router.c b/drivers/staging/lustre/lnet/lnet/router.c index 88283ca3f860..86f28152f7b3 100644 --- a/drivers/staging/lustre/lnet/lnet/router.c +++ b/drivers/staging/lustre/lnet/lnet/router.c @@ -318,15 +318,15 @@ lnet_add_route(__u32 net, __u32 hops, lnet_nid_t gateway, return -EEXIST; /* Assume net, route, all new */ - LIBCFS_ALLOC(route, sizeof(*route)); - LIBCFS_ALLOC(rnet, sizeof(*rnet)); + route = kzalloc(sizeof(*route), GFP_NOFS); + rnet = kzalloc(sizeof(*rnet), GFP_NOFS); if (!route || !rnet) { CERROR("Out of memory creating route %s %d %s\n", libcfs_net2str(net), hops, libcfs_nid2str(gateway)); if (route) - LIBCFS_FREE(route, sizeof(*route)); + kfree(route); if (rnet) - LIBCFS_FREE(rnet, sizeof(*rnet)); + kfree(rnet); return -ENOMEM; } @@ -342,8 +342,8 @@ lnet_add_route(__u32 net, __u32 hops, lnet_nid_t gateway, if (rc) { lnet_net_unlock(LNET_LOCK_EX); - LIBCFS_FREE(route, sizeof(*route)); - LIBCFS_FREE(rnet, sizeof(*rnet)); + kfree(route); + kfree(rnet); if (rc == -EHOSTUNREACH) /* gateway is not on a local net */ return rc; /* ignore the route entry */ @@ -398,11 +398,11 @@ lnet_add_route(__u32 net, __u32 hops, lnet_nid_t gateway, if (!add_route) { rc = -EEXIST; - LIBCFS_FREE(route, sizeof(*route)); + kfree(route); } if (rnet != rnet2) - LIBCFS_FREE(rnet, sizeof(*rnet)); + kfree(rnet); /* indicate to startup the router checker if configured */ wake_up(&the_lnet.ln_rc_waitq); @@ -520,10 +520,8 @@ lnet_del_route(__u32 net, lnet_nid_t gw_nid) lnet_net_unlock(LNET_LOCK_EX); - LIBCFS_FREE(route, sizeof(*route)); - - if (rnet) - LIBCFS_FREE(rnet, sizeof(*rnet)); + kfree(route); + kfree(rnet); rc = 0; lnet_net_lock(LNET_LOCK_EX); @@ -894,7 +892,7 @@ lnet_destroy_rc_data(struct lnet_rc_data *rcd) if (rcd->rcd_pinginfo) LIBCFS_FREE(rcd->rcd_pinginfo, LNET_PINGINFO_SIZE); - LIBCFS_FREE(rcd, sizeof(*rcd)); + kfree(rcd); } static struct lnet_rc_data * @@ -908,7 +906,7 @@ lnet_create_rc_data_locked(struct lnet_peer *gateway) lnet_net_unlock(gateway->lp_cpt); - LIBCFS_ALLOC(rcd, sizeof(*rcd)); + rcd = kzalloc(sizeof(*rcd), GFP_NOFS); if (!rcd) goto out; diff --git a/drivers/staging/lustre/lnet/lnet/router_proc.c b/drivers/staging/lustre/lnet/lnet/router_proc.c index d32d653edcb0..06e20b314520 100644 --- a/drivers/staging/lustre/lnet/lnet/router_proc.c +++ b/drivers/staging/lustre/lnet/lnet/router_proc.c @@ -91,13 +91,13 @@ static int __proc_lnet_stats(void *data, int write, /* read */ - LIBCFS_ALLOC(ctrs, sizeof(*ctrs)); + ctrs = kzalloc(sizeof(*ctrs), GFP_NOFS); if (!ctrs) return -ENOMEM; LIBCFS_ALLOC(tmpstr, tmpsiz); if (!tmpstr) { - LIBCFS_FREE(ctrs, sizeof(*ctrs)); + kfree(ctrs); return -ENOMEM; } @@ -119,7 +119,7 @@ static int __proc_lnet_stats(void *data, int write, tmpstr + pos, "\n"); LIBCFS_FREE(tmpstr, tmpsiz); - LIBCFS_FREE(ctrs, sizeof(*ctrs)); + kfree(ctrs); return rc; } diff --git a/drivers/staging/lustre/lnet/selftest/conrpc.c b/drivers/staging/lustre/lnet/selftest/conrpc.c index 6a0f770e0e24..7aa515c34594 100644 --- a/drivers/staging/lustre/lnet/selftest/conrpc.c +++ b/drivers/staging/lustre/lnet/selftest/conrpc.c @@ -129,7 +129,7 @@ lstcon_rpc_prep(struct lstcon_node *nd, int service, unsigned int feats, spin_unlock(&console_session.ses_rpc_lock); if (!crpc) { - LIBCFS_ALLOC(crpc, sizeof(*crpc)); + crpc = kzalloc(sizeof(*crpc), GFP_NOFS); if (!crpc) return -ENOMEM; } @@ -140,7 +140,7 @@ lstcon_rpc_prep(struct lstcon_node *nd, int service, unsigned int feats, return 0; } - LIBCFS_FREE(crpc, sizeof(*crpc)); + kfree(crpc); return rc; } @@ -250,7 +250,7 @@ lstcon_rpc_trans_prep(struct list_head *translist, int transop, } /* create a trans group */ - LIBCFS_ALLOC(trans, sizeof(*trans)); + trans = kzalloc(sizeof(*trans), GFP_NOFS); if (!trans) return -ENOMEM; @@ -585,7 +585,7 @@ lstcon_rpc_trans_destroy(struct lstcon_rpc_trans *trans) CDEBUG(D_NET, "Transaction %s destroyed with %d pending RPCs\n", lstcon_rpc_trans_name(trans->tas_opc), count); - LIBCFS_FREE(trans, sizeof(*trans)); + kfree(trans); } int @@ -1369,7 +1369,7 @@ lstcon_rpc_cleanup_wait(void) list_for_each_entry_safe(crpc, temp, &zlist, crp_link) { list_del(&crpc->crp_link); - LIBCFS_FREE(crpc, sizeof(struct lstcon_rpc)); + kfree(crpc); } } diff --git a/drivers/staging/lustre/lnet/selftest/console.c b/drivers/staging/lustre/lnet/selftest/console.c index a2662638d599..edf5e59a4351 100644 --- a/drivers/staging/lustre/lnet/selftest/console.c +++ b/drivers/staging/lustre/lnet/selftest/console.c @@ -166,7 +166,7 @@ lstcon_ndlink_find(struct list_head *hash, struct lnet_process_id id, if (rc) return rc; - LIBCFS_ALLOC(ndl, sizeof(struct lstcon_ndlink)); + ndl = kzalloc(sizeof(struct lstcon_ndlink), GFP_NOFS); if (!ndl) { lstcon_node_put(nd); return -ENOMEM; @@ -190,7 +190,7 @@ lstcon_ndlink_release(struct lstcon_ndlink *ndl) list_del(&ndl->ndl_hlink); /* delete from hash */ lstcon_node_put(ndl->ndl_node); - LIBCFS_FREE(ndl, sizeof(*ndl)); + kfree(ndl); } static int @@ -807,7 +807,7 @@ lstcon_group_info(char *name, struct lstcon_ndlist_ent __user *gents_p, } /* non-verbose query */ - LIBCFS_ALLOC(gentp, sizeof(struct lstcon_ndlist_ent)); + gentp = kzalloc(sizeof(struct lstcon_ndlist_ent), GFP_NOFS); if (!gentp) { CERROR("Can't allocate ndlist_ent\n"); lstcon_group_decref(grp); @@ -821,7 +821,7 @@ lstcon_group_info(char *name, struct lstcon_ndlist_ent __user *gents_p, rc = copy_to_user(gents_p, gentp, sizeof(struct lstcon_ndlist_ent)) ? -EFAULT : 0; - LIBCFS_FREE(gentp, sizeof(struct lstcon_ndlist_ent)); + kfree(gentp); lstcon_group_decref(grp); @@ -856,7 +856,7 @@ lstcon_batch_add(char *name) return rc; } - LIBCFS_ALLOC(bat, sizeof(struct lstcon_batch)); + bat = kzalloc(sizeof(struct lstcon_batch), GFP_NOFS); if (!bat) { CERROR("Can't allocate descriptor for batch %s\n", name); return -ENOMEM; @@ -866,7 +866,7 @@ lstcon_batch_add(char *name) sizeof(struct list_head) * LST_NODE_HASHSIZE); if (!bat->bat_cli_hash) { CERROR("Can't allocate hash for batch %s\n", name); - LIBCFS_FREE(bat, sizeof(struct lstcon_batch)); + kfree(bat); return -ENOMEM; } @@ -876,7 +876,7 @@ lstcon_batch_add(char *name) if (!bat->bat_srv_hash) { CERROR("Can't allocate hash for batch %s\n", name); LIBCFS_FREE(bat->bat_cli_hash, LST_NODE_HASHSIZE); - LIBCFS_FREE(bat, sizeof(struct lstcon_batch)); + kfree(bat); return -ENOMEM; } @@ -884,7 +884,7 @@ lstcon_batch_add(char *name) if (strlen(name) > sizeof(bat->bat_name) - 1) { LIBCFS_FREE(bat->bat_srv_hash, LST_NODE_HASHSIZE); LIBCFS_FREE(bat->bat_cli_hash, LST_NODE_HASHSIZE); - LIBCFS_FREE(bat, sizeof(struct lstcon_batch)); + kfree(bat); return -E2BIG; } strncpy(bat->bat_name, name, sizeof(bat->bat_name)); @@ -971,7 +971,7 @@ lstcon_batch_info(char *name, struct lstcon_test_batch_ent __user *ent_up, } /* non-verbose query */ - LIBCFS_ALLOC(entp, sizeof(struct lstcon_test_batch_ent)); + entp = kzalloc(sizeof(struct lstcon_test_batch_ent), GFP_NOFS); if (!entp) return -ENOMEM; @@ -993,7 +993,7 @@ lstcon_batch_info(char *name, struct lstcon_test_batch_ent __user *ent_up, rc = copy_to_user(ent_up, entp, sizeof(struct lstcon_test_batch_ent)) ? -EFAULT : 0; - LIBCFS_FREE(entp, sizeof(struct lstcon_test_batch_ent)); + kfree(entp); return rc; } @@ -1138,7 +1138,7 @@ lstcon_batch_destroy(struct lstcon_batch *bat) sizeof(struct list_head) * LST_NODE_HASHSIZE); LIBCFS_FREE(bat->bat_srv_hash, sizeof(struct list_head) * LST_NODE_HASHSIZE); - LIBCFS_FREE(bat, sizeof(struct lstcon_batch)); + kfree(bat); } static int @@ -1790,7 +1790,7 @@ lstcon_session_info(struct lst_sid __user *sid_up, int __user *key_up, if (console_session.ses_state != LST_SESSION_ACTIVE) return -ESRCH; - LIBCFS_ALLOC(entp, sizeof(*entp)); + entp = kzalloc(sizeof(*entp), GFP_NOFS); if (!entp) return -ENOMEM; @@ -1807,7 +1807,7 @@ lstcon_session_info(struct lst_sid __user *sid_up, int __user *key_up, copy_to_user(name_up, console_session.ses_name, len)) rc = -EFAULT; - LIBCFS_FREE(entp, sizeof(*entp)); + kfree(entp); return rc; } diff --git a/drivers/staging/lustre/lnet/selftest/framework.c b/drivers/staging/lustre/lnet/selftest/framework.c index fe889607ff3f..b8d95a20b2eb 100644 --- a/drivers/staging/lustre/lnet/selftest/framework.c +++ b/drivers/staging/lustre/lnet/selftest/framework.c @@ -143,7 +143,7 @@ sfw_register_test(struct srpc_service *service, return -EEXIST; } - LIBCFS_ALLOC(tsc, sizeof(struct sfw_test_case)); + tsc = kzalloc(sizeof(struct sfw_test_case), GFP_NOFS); if (!tsc) return -ENOMEM; @@ -344,7 +344,7 @@ sfw_bid2batch(struct lst_bid bid) if (bat) return bat; - LIBCFS_ALLOC(bat, sizeof(struct sfw_batch)); + bat = kzalloc(sizeof(struct sfw_batch), GFP_NOFS); if (!bat) return NULL; @@ -447,7 +447,7 @@ sfw_make_session(struct srpc_mksn_reqst *request, struct srpc_mksn_reply *reply) } /* brand new or create by force */ - LIBCFS_ALLOC(sn, sizeof(struct sfw_session)); + sn = kzalloc(sizeof(struct sfw_session), GFP_NOFS); if (!sn) { CERROR("dropping RPC mksn under memory pressure\n"); return -ENOMEM; @@ -632,7 +632,7 @@ sfw_destroy_test_instance(struct sfw_test_instance *tsi) tsu = list_entry(tsi->tsi_units.next, struct sfw_test_unit, tsu_list); list_del(&tsu->tsu_list); - LIBCFS_FREE(tsu, sizeof(*tsu)); + kfree(tsu); } while (!list_empty(&tsi->tsi_free_rpcs)) { @@ -644,7 +644,7 @@ sfw_destroy_test_instance(struct sfw_test_instance *tsi) clean: sfw_unload_test(tsi); - LIBCFS_FREE(tsi, sizeof(*tsi)); + kfree(tsi); } static void @@ -662,7 +662,7 @@ sfw_destroy_batch(struct sfw_batch *tsb) sfw_destroy_test_instance(tsi); } - LIBCFS_FREE(tsb, sizeof(struct sfw_batch)); + kfree(tsb); } void @@ -680,7 +680,7 @@ sfw_destroy_session(struct sfw_session *sn) sfw_destroy_batch(batch); } - LIBCFS_FREE(sn, sizeof(*sn)); + kfree(sn); atomic_dec(&sfw_data.fw_nzombies); } @@ -740,7 +740,7 @@ sfw_add_test_instance(struct sfw_batch *tsb, struct srpc_server_rpc *rpc) int i; int rc; - LIBCFS_ALLOC(tsi, sizeof(*tsi)); + tsi = kzalloc(sizeof(*tsi), GFP_NOFS); if (!tsi) { CERROR("Can't allocate test instance for batch: %llu\n", tsb->bat_id.bat_id); @@ -763,7 +763,7 @@ sfw_add_test_instance(struct sfw_batch *tsb, struct srpc_server_rpc *rpc) rc = sfw_load_test(tsi); if (rc) { - LIBCFS_FREE(tsi, sizeof(*tsi)); + kfree(tsi); return rc; } @@ -795,7 +795,7 @@ sfw_add_test_instance(struct sfw_batch *tsb, struct srpc_server_rpc *rpc) sfw_unpack_id(id); for (j = 0; j < tsi->tsi_concur; j++) { - LIBCFS_ALLOC(tsu, sizeof(struct sfw_test_unit)); + tsu = kzalloc(sizeof(struct sfw_test_unit), GFP_NOFS); if (!tsu) { rc = -ENOMEM; CERROR("Can't allocate tsu for %d\n", @@ -1785,6 +1785,6 @@ sfw_shutdown(void) srpc_wait_service_shutdown(tsc->tsc_srv_service); list_del(&tsc->tsc_list); - LIBCFS_FREE(tsc, sizeof(*tsc)); + kfree(tsc); } } diff --git a/drivers/staging/lustre/lnet/selftest/rpc.c b/drivers/staging/lustre/lnet/selftest/rpc.c index ab7e8a8e58b8..7061f19f047f 100644 --- a/drivers/staging/lustre/lnet/selftest/rpc.c +++ b/drivers/staging/lustre/lnet/selftest/rpc.c @@ -214,7 +214,7 @@ srpc_service_fini(struct srpc_service *svc) buf = list_entry(q->next, struct srpc_buffer, buf_list); list_del(&buf->buf_list); - LIBCFS_FREE(buf, sizeof(*buf)); + kfree(buf); } } @@ -225,7 +225,7 @@ srpc_service_fini(struct srpc_service *svc) struct srpc_server_rpc, srpc_list); list_del(&rpc->srpc_list); - LIBCFS_FREE(rpc, sizeof(*rpc)); + kfree(rpc); } } @@ -508,7 +508,7 @@ __must_hold(&scd->scd_lock) list_del(&buf->buf_list); spin_unlock(&scd->scd_lock); - LIBCFS_FREE(buf, sizeof(*buf)); + kfree(buf); spin_lock(&scd->scd_lock); return rc; @@ -535,7 +535,7 @@ srpc_add_buffer(struct swi_workitem *wi) spin_unlock(&scd->scd_lock); - LIBCFS_ALLOC(buf, sizeof(*buf)); + buf = kzalloc(sizeof(*buf), GFP_NOFS); if (!buf) { CERROR("Failed to add new buf to service: %s\n", scd->scd_svc->sv_name); @@ -547,7 +547,7 @@ srpc_add_buffer(struct swi_workitem *wi) spin_lock(&scd->scd_lock); if (scd->scd_svc->sv_shuttingdown) { spin_unlock(&scd->scd_lock); - LIBCFS_FREE(buf, sizeof(*buf)); + kfree(buf); spin_lock(&scd->scd_lock); rc = -ESHUTDOWN; @@ -725,7 +725,7 @@ __must_hold(&scd->scd_lock) } spin_unlock(&scd->scd_lock); - LIBCFS_FREE(buf, sizeof(*buf)); + kfree(buf); spin_lock(&scd->scd_lock); } From neilb at suse.com Mon Dec 18 00:46:30 2017 From: neilb at suse.com (NeilBrown) Date: Mon, 18 Dec 2017 11:46:30 +1100 Subject: [lustre-devel] [PATCH 04/15] staging: lustre: lnet: switch to cpumask_var_t In-Reply-To: <151355781721.6200.2136335532722530242.stgit@noble> References: <151355781721.6200.2136335532722530242.stgit@noble> Message-ID: <151355799036.6200.12786177156592779397.stgit@noble> So that we can use the common cpumask allocation functions, switch to cpumask_var_t. We need to be careful not the free a cpumask_var_t until the variable has been initialized, and it cannot be initialized directly. So we must be sure either that it is filled with zeros, or that zalloc_cpumask_var() has been called on it. Signed-off-by: NeilBrown --- .../lustre/include/linux/libcfs/libcfs_cpu.h | 4 - .../lustre/include/linux/libcfs/linux/linux-cpu.h | 4 - .../staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c | 6 + drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c | 2 .../staging/lustre/lnet/libcfs/linux/linux-cpu.c | 92 +++++++++----------- 5 files changed, 49 insertions(+), 59 deletions(-) diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h index 6d132f941281..61bce77fddd6 100644 --- a/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h +++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h @@ -79,7 +79,7 @@ /** * return cpumask of CPU partition \a cpt */ -cpumask_t *cfs_cpt_cpumask(struct cfs_cpt_table *cptab, int cpt); +cpumask_var_t *cfs_cpt_cpumask(struct cfs_cpt_table *cptab, int cpt); /** * print string information of cpt-table */ @@ -96,7 +96,7 @@ struct cfs_cpt_table { u64 ctb_version; }; -static inline cpumask_t * +static inline cpumask_var_t * cfs_cpt_cpumask(struct cfs_cpt_table *cptab, int cpt) { return NULL; diff --git a/drivers/staging/lustre/include/linux/libcfs/linux/linux-cpu.h b/drivers/staging/lustre/include/linux/libcfs/linux/linux-cpu.h index 854c84358ab4..6035376f2830 100644 --- a/drivers/staging/lustre/include/linux/libcfs/linux/linux-cpu.h +++ b/drivers/staging/lustre/include/linux/libcfs/linux/linux-cpu.h @@ -49,7 +49,7 @@ /** virtual processing unit */ struct cfs_cpu_partition { /* CPUs mask for this partition */ - cpumask_t *cpt_cpumask; + cpumask_var_t cpt_cpumask; /* nodes mask for this partition */ nodemask_t *cpt_nodemask; /* spread rotor for NUMA allocator */ @@ -69,7 +69,7 @@ struct cfs_cpt_table { /* shadow HW CPU to CPU partition ID */ int *ctb_cpu2cpt; /* all cpus in this partition table */ - cpumask_t *ctb_cpumask; + cpumask_var_t ctb_cpumask; /* all nodes in this partition table */ nodemask_t *ctb_nodemask; }; diff --git a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c index 039f53d787e4..f9bdf1877794 100644 --- a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c +++ b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c @@ -596,7 +596,7 @@ static void kiblnd_setup_mtu_locked(struct rdma_cm_id *cmid) static int kiblnd_get_completion_vector(struct kib_conn *conn, int cpt) { - cpumask_t *mask; + cpumask_var_t *mask; int vectors; int off; int i; @@ -611,8 +611,8 @@ static int kiblnd_get_completion_vector(struct kib_conn *conn, int cpt) return 0; /* hash NID to CPU id in this partition... */ - off = do_div(nid, cpumask_weight(mask)); - for_each_cpu(i, mask) { + off = do_div(nid, cpumask_weight(*mask)); + for_each_cpu(i, *mask) { if (!off--) return i % vectors; } diff --git a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c index d05c3932b3b9..76291a350406 100644 --- a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c +++ b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c @@ -113,7 +113,7 @@ cfs_cpt_nodemask(struct cfs_cpt_table *cptab, int cpt) { return &cptab->ctb_nodemask; } -EXPORT_SYMBOL(cfs_cpt_cpumask); +EXPORT_SYMBOL(cfs_cpt_nodemask); int cfs_cpt_set_cpu(struct cfs_cpt_table *cptab, int cpt, int cpu) diff --git a/drivers/staging/lustre/lnet/libcfs/linux/linux-cpu.c b/drivers/staging/lustre/lnet/libcfs/linux/linux-cpu.c index 9a0f6cce1a1d..44e44a999648 100644 --- a/drivers/staging/lustre/lnet/libcfs/linux/linux-cpu.c +++ b/drivers/staging/lustre/lnet/libcfs/linux/linux-cpu.c @@ -72,7 +72,7 @@ struct cfs_cpt_data { /* mutex to protect cpt_cpumask */ struct mutex cpt_mutex; /* scratch buffer for set/unset_node */ - cpumask_t *cpt_cpumask; + cpumask_var_t cpt_cpumask; }; static struct cfs_cpt_data cpt_data; @@ -106,8 +106,7 @@ cfs_cpt_table_free(struct cfs_cpt_table *cptab) kfree(part->cpt_nodemask); } - if (part->cpt_cpumask) - LIBCFS_FREE(part->cpt_cpumask, cpumask_size()); + free_cpumask_var(part->cpt_cpumask); } if (cptab->ctb_parts) { @@ -117,8 +116,7 @@ cfs_cpt_table_free(struct cfs_cpt_table *cptab) if (cptab->ctb_nodemask) kfree(cptab->ctb_nodemask); - if (cptab->ctb_cpumask) - LIBCFS_FREE(cptab->ctb_cpumask, cpumask_size()); + free_cpumask_var(cptab->ctb_cpumask); kfree(cptab); } @@ -136,11 +134,10 @@ cfs_cpt_table_alloc(unsigned int ncpt) cptab->ctb_nparts = ncpt; - LIBCFS_ALLOC(cptab->ctb_cpumask, cpumask_size()); cptab->ctb_nodemask = kzalloc(sizeof(*cptab->ctb_nodemask), GFP_NOFS); - - if (!cptab->ctb_cpumask || !cptab->ctb_nodemask) + if (!zalloc_cpumask_var(&cptab->ctb_cpumask, GFP_NOFS) || + !cptab->ctb_nodemask) goto failed; LIBCFS_ALLOC(cptab->ctb_cpu2cpt, @@ -158,10 +155,10 @@ cfs_cpt_table_alloc(unsigned int ncpt) for (i = 0; i < ncpt; i++) { struct cfs_cpu_partition *part = &cptab->ctb_parts[i]; - LIBCFS_ALLOC(part->cpt_cpumask, cpumask_size()); part->cpt_nodemask = kzalloc(sizeof(*part->cpt_nodemask), GFP_NOFS); - if (!part->cpt_cpumask || !part->cpt_nodemask) + if (!zalloc_cpumask_var(&part->cpt_cpumask, GFP_NOFS) || + !part->cpt_nodemask) goto failed; } @@ -252,13 +249,13 @@ cfs_cpt_online(struct cfs_cpt_table *cptab, int cpt) } EXPORT_SYMBOL(cfs_cpt_online); -cpumask_t * +cpumask_var_t * cfs_cpt_cpumask(struct cfs_cpt_table *cptab, int cpt) { LASSERT(cpt == CFS_CPT_ANY || (cpt >= 0 && cpt < cptab->ctb_nparts)); return cpt == CFS_CPT_ANY ? - cptab->ctb_cpumask : cptab->ctb_parts[cpt].cpt_cpumask; + &cptab->ctb_cpumask : &cptab->ctb_parts[cpt].cpt_cpumask; } EXPORT_SYMBOL(cfs_cpt_cpumask); @@ -406,7 +403,6 @@ EXPORT_SYMBOL(cfs_cpt_unset_cpumask); int cfs_cpt_set_node(struct cfs_cpt_table *cptab, int cpt, int node) { - cpumask_t *mask; int rc; if (node < 0 || node >= MAX_NUMNODES) { @@ -417,10 +413,9 @@ cfs_cpt_set_node(struct cfs_cpt_table *cptab, int cpt, int node) mutex_lock(&cpt_data.cpt_mutex); - mask = cpt_data.cpt_cpumask; - cfs_node_to_cpumask(node, mask); + cfs_node_to_cpumask(node, cpt_data.cpt_cpumask); - rc = cfs_cpt_set_cpumask(cptab, cpt, mask); + rc = cfs_cpt_set_cpumask(cptab, cpt, cpt_data.cpt_cpumask); mutex_unlock(&cpt_data.cpt_mutex); @@ -431,8 +426,6 @@ EXPORT_SYMBOL(cfs_cpt_set_node); void cfs_cpt_unset_node(struct cfs_cpt_table *cptab, int cpt, int node) { - cpumask_t *mask; - if (node < 0 || node >= MAX_NUMNODES) { CDEBUG(D_INFO, "Invalid NUMA id %d for CPU partition %d\n", node, cpt); @@ -441,10 +434,9 @@ cfs_cpt_unset_node(struct cfs_cpt_table *cptab, int cpt, int node) mutex_lock(&cpt_data.cpt_mutex); - mask = cpt_data.cpt_cpumask; - cfs_node_to_cpumask(node, mask); + cfs_node_to_cpumask(node, cpt_data.cpt_cpumask); - cfs_cpt_unset_cpumask(cptab, cpt, mask); + cfs_cpt_unset_cpumask(cptab, cpt, cpt_data.cpt_cpumask); mutex_unlock(&cpt_data.cpt_mutex); } @@ -559,7 +551,7 @@ EXPORT_SYMBOL(cfs_cpt_of_cpu); int cfs_cpt_bind(struct cfs_cpt_table *cptab, int cpt) { - cpumask_t *cpumask; + cpumask_var_t *cpumask; nodemask_t *nodemask; int rc; int i; @@ -567,24 +559,24 @@ cfs_cpt_bind(struct cfs_cpt_table *cptab, int cpt) LASSERT(cpt == CFS_CPT_ANY || (cpt >= 0 && cpt < cptab->ctb_nparts)); if (cpt == CFS_CPT_ANY) { - cpumask = cptab->ctb_cpumask; + cpumask = &cptab->ctb_cpumask; nodemask = cptab->ctb_nodemask; } else { - cpumask = cptab->ctb_parts[cpt].cpt_cpumask; + cpumask = &cptab->ctb_parts[cpt].cpt_cpumask; nodemask = cptab->ctb_parts[cpt].cpt_nodemask; } - if (cpumask_any_and(cpumask, cpu_online_mask) >= nr_cpu_ids) { + if (cpumask_any_and(*cpumask, cpu_online_mask) >= nr_cpu_ids) { CERROR("No online CPU found in CPU partition %d, did someone do CPU hotplug on system? You might need to reload Lustre modules to keep system working well.\n", cpt); return -EINVAL; } for_each_online_cpu(i) { - if (cpumask_test_cpu(i, cpumask)) + if (cpumask_test_cpu(i, *cpumask)) continue; - rc = set_cpus_allowed_ptr(current, cpumask); + rc = set_cpus_allowed_ptr(current, *cpumask); set_mems_allowed(*nodemask); if (!rc) schedule(); /* switch to allowed CPU */ @@ -605,8 +597,8 @@ static int cfs_cpt_choose_ncpus(struct cfs_cpt_table *cptab, int cpt, cpumask_t *node, int number) { - cpumask_t *socket = NULL; - cpumask_t *core = NULL; + cpumask_var_t socket; + cpumask_var_t core; int rc = 0; int cpu; @@ -624,13 +616,17 @@ cfs_cpt_choose_ncpus(struct cfs_cpt_table *cptab, int cpt, return 0; } - /* allocate scratch buffer */ - LIBCFS_ALLOC(socket, cpumask_size()); - LIBCFS_ALLOC(core, cpumask_size()); - if (!socket || !core) { + /* + * Allocate scratch buffers + * As we cannot initialize a cpumask_var_t, we need + * to alloc both before we can risk trying to free either + */ + if (!zalloc_cpumask_var(&socket, GFP_NOFS)) + rc = -ENOMEM; + if (!zalloc_cpumask_var(&core, GFP_NOFS)) rc = -ENOMEM; + if (rc) goto out; - } while (!cpumask_empty(node)) { cpu = cpumask_first(node); @@ -668,10 +664,8 @@ cfs_cpt_choose_ncpus(struct cfs_cpt_table *cptab, int cpt, } out: - if (socket) - LIBCFS_FREE(socket, cpumask_size()); - if (core) - LIBCFS_FREE(core, cpumask_size()); + free_cpumask_var(socket); + free_cpumask_var(core); return rc; } @@ -724,7 +718,7 @@ static struct cfs_cpt_table * cfs_cpt_table_create(int ncpt) { struct cfs_cpt_table *cptab = NULL; - cpumask_t *mask = NULL; + cpumask_var_t mask; int cpt = 0; int num; int rc; @@ -757,8 +751,7 @@ cfs_cpt_table_create(int ncpt) goto failed; } - LIBCFS_ALLOC(mask, cpumask_size()); - if (!mask) { + if (!zalloc_cpumask_var(&mask, GFP_NOFS)){ CERROR("Failed to allocate scratch cpumask\n"); goto failed; } @@ -785,7 +778,7 @@ cfs_cpt_table_create(int ncpt) rc = cfs_cpt_choose_ncpus(cptab, cpt, mask, n); if (rc < 0) - goto failed; + goto failed_mask; LASSERT(num >= cpumask_weight(part->cpt_cpumask)); if (num == cpumask_weight(part->cpt_cpumask)) @@ -798,20 +791,19 @@ cfs_cpt_table_create(int ncpt) CERROR("Expect %d(%d) CPU partitions but got %d(%d), CPU hotplug/unplug while setting?\n", cptab->ctb_nparts, num, cpt, cpumask_weight(cptab->ctb_parts[ncpt - 1].cpt_cpumask)); - goto failed; + goto failed_mask; } - LIBCFS_FREE(mask, cpumask_size()); + free_cpumask_var(mask); return cptab; + failed_mask: + free_cpumask_var(mask); failed: CERROR("Failed to setup CPU-partition-table with %d CPU-partitions, online HW nodes: %d, HW cpus: %d.\n", ncpt, num_online_nodes(), num_online_cpus()); - if (mask) - LIBCFS_FREE(mask, cpumask_size()); - if (cptab) cfs_cpt_table_free(cptab); @@ -1014,8 +1006,7 @@ cfs_cpu_fini(void) cpuhp_remove_state_nocalls(lustre_cpu_online); cpuhp_remove_state_nocalls(CPUHP_LUSTRE_CFS_DEAD); #endif - if (cpt_data.cpt_cpumask) - LIBCFS_FREE(cpt_data.cpt_cpumask, cpumask_size()); + free_cpumask_var(cpt_data.cpt_cpumask); } int @@ -1027,8 +1018,7 @@ cfs_cpu_init(void) memset(&cpt_data, 0, sizeof(cpt_data)); - LIBCFS_ALLOC(cpt_data.cpt_cpumask, cpumask_size()); - if (!cpt_data.cpt_cpumask) { + if (!zalloc_cpumask_var(&cpt_data.cpt_cpumask, GFP_NOFS)) { CERROR("Failed to allocate scratch buffer\n"); return -1; } From neilb at suse.com Mon Dec 18 00:46:30 2017 From: neilb at suse.com (NeilBrown) Date: Mon, 18 Dec 2017 11:46:30 +1100 Subject: [lustre-devel] [PATCH 05/15] staging: lustre: lnet: selftest: don't allocate small strings. In-Reply-To: <151355781721.6200.2136335532722530242.stgit@noble> References: <151355781721.6200.2136335532722530242.stgit@noble> Message-ID: <151355799039.6200.3383435277341826480.stgit@noble> All of the "name" buffers here are at most LST_NAME_SIZE+1 bytes, so 33 bytes at most. They are only used temporarily during the life of the function that allocates them. So it is much simpler to just allocate on the stack. Worst case is lst_tet_add_ioct(), which allocates 3 for these which 99 bytes on the stack, instead of the 24 that would have been allocated for 64-bit pointers. Signed-off-by: NeilBrown --- drivers/staging/lustre/lnet/selftest/conctl.c | 180 ++++--------------------- 1 file changed, 29 insertions(+), 151 deletions(-) diff --git a/drivers/staging/lustre/lnet/selftest/conctl.c b/drivers/staging/lustre/lnet/selftest/conctl.c index 082c0afacf23..442a18ddd41f 100644 --- a/drivers/staging/lustre/lnet/selftest/conctl.c +++ b/drivers/staging/lustre/lnet/selftest/conctl.c @@ -45,7 +45,7 @@ static int lst_session_new_ioctl(struct lstio_session_new_args *args) { - char *name; + char name[LST_NAME_SIZE + 1]; int rc; if (!args->lstio_ses_idp || /* address for output sid */ @@ -55,13 +55,8 @@ lst_session_new_ioctl(struct lstio_session_new_args *args) args->lstio_ses_nmlen > LST_NAME_SIZE) return -EINVAL; - LIBCFS_ALLOC(name, args->lstio_ses_nmlen + 1); - if (!name) - return -ENOMEM; - if (copy_from_user(name, args->lstio_ses_namep, args->lstio_ses_nmlen)) { - LIBCFS_FREE(name, args->lstio_ses_nmlen + 1); return -EFAULT; } @@ -74,7 +69,6 @@ lst_session_new_ioctl(struct lstio_session_new_args *args) args->lstio_ses_force, args->lstio_ses_idp); - LIBCFS_FREE(name, args->lstio_ses_nmlen + 1); return rc; } @@ -112,7 +106,7 @@ lst_session_info_ioctl(struct lstio_session_info_args *args) static int lst_debug_ioctl(struct lstio_debug_args *args) { - char *name = NULL; + char name[LST_NAME_SIZE + 1]; int client = 1; int rc; @@ -128,16 +122,10 @@ lst_debug_ioctl(struct lstio_debug_args *args) return -EINVAL; if (args->lstio_dbg_namep) { - LIBCFS_ALLOC(name, args->lstio_dbg_nmlen + 1); - if (!name) - return -ENOMEM; if (copy_from_user(name, args->lstio_dbg_namep, - args->lstio_dbg_nmlen)) { - LIBCFS_FREE(name, args->lstio_dbg_nmlen + 1); - + args->lstio_dbg_nmlen)) return -EFAULT; - } name[args->lstio_dbg_nmlen] = 0; } @@ -154,7 +142,7 @@ lst_debug_ioctl(struct lstio_debug_args *args) client = 0; /* fall through */ case LST_OPC_BATCHCLI: - if (!name) + if (!args->lstio_dbg_namep) goto out; rc = lstcon_batch_debug(args->lstio_dbg_timeout, @@ -162,7 +150,7 @@ lst_debug_ioctl(struct lstio_debug_args *args) break; case LST_OPC_GROUP: - if (!name) + if (!args->lstio_dbg_namep) goto out; rc = lstcon_group_debug(args->lstio_dbg_timeout, @@ -185,16 +173,13 @@ lst_debug_ioctl(struct lstio_debug_args *args) } out: - if (name) - LIBCFS_FREE(name, args->lstio_dbg_nmlen + 1); - return rc; } static int lst_group_add_ioctl(struct lstio_group_add_args *args) { - char *name; + char name[LST_NAME_SIZE + 1]; int rc; if (args->lstio_grp_key != console_session.ses_key) @@ -205,22 +190,14 @@ lst_group_add_ioctl(struct lstio_group_add_args *args) args->lstio_grp_nmlen > LST_NAME_SIZE) return -EINVAL; - LIBCFS_ALLOC(name, args->lstio_grp_nmlen + 1); - if (!name) - return -ENOMEM; - if (copy_from_user(name, args->lstio_grp_namep, - args->lstio_grp_nmlen)) { - LIBCFS_FREE(name, args->lstio_grp_nmlen); + args->lstio_grp_nmlen)) return -EFAULT; - } name[args->lstio_grp_nmlen] = 0; rc = lstcon_group_add(name); - LIBCFS_FREE(name, args->lstio_grp_nmlen + 1); - return rc; } @@ -228,7 +205,7 @@ static int lst_group_del_ioctl(struct lstio_group_del_args *args) { int rc; - char *name; + char name[LST_NAME_SIZE + 1]; if (args->lstio_grp_key != console_session.ses_key) return -EACCES; @@ -238,22 +215,14 @@ lst_group_del_ioctl(struct lstio_group_del_args *args) args->lstio_grp_nmlen > LST_NAME_SIZE) return -EINVAL; - LIBCFS_ALLOC(name, args->lstio_grp_nmlen + 1); - if (!name) - return -ENOMEM; - if (copy_from_user(name, args->lstio_grp_namep, - args->lstio_grp_nmlen)) { - LIBCFS_FREE(name, args->lstio_grp_nmlen + 1); + args->lstio_grp_nmlen)) return -EFAULT; - } name[args->lstio_grp_nmlen] = 0; rc = lstcon_group_del(name); - LIBCFS_FREE(name, args->lstio_grp_nmlen + 1); - return rc; } @@ -261,7 +230,7 @@ static int lst_group_update_ioctl(struct lstio_group_update_args *args) { int rc; - char *name; + char name[LST_NAME_SIZE + 1]; if (args->lstio_grp_key != console_session.ses_key) return -EACCES; @@ -272,15 +241,9 @@ lst_group_update_ioctl(struct lstio_group_update_args *args) args->lstio_grp_nmlen > LST_NAME_SIZE) return -EINVAL; - LIBCFS_ALLOC(name, args->lstio_grp_nmlen + 1); - if (!name) - return -ENOMEM; - if (copy_from_user(name, args->lstio_grp_namep, - args->lstio_grp_nmlen)) { - LIBCFS_FREE(name, args->lstio_grp_nmlen + 1); + args->lstio_grp_nmlen)) return -EFAULT; - } name[args->lstio_grp_nmlen] = 0; @@ -309,8 +272,6 @@ lst_group_update_ioctl(struct lstio_group_update_args *args) break; } - LIBCFS_FREE(name, args->lstio_grp_nmlen + 1); - return rc; } @@ -319,7 +280,7 @@ lst_nodes_add_ioctl(struct lstio_group_nodes_args *args) { unsigned int feats; int rc; - char *name; + char name[LST_NAME_SIZE + 1]; if (args->lstio_grp_key != console_session.ses_key) return -EACCES; @@ -333,16 +294,9 @@ lst_nodes_add_ioctl(struct lstio_group_nodes_args *args) args->lstio_grp_nmlen > LST_NAME_SIZE) return -EINVAL; - LIBCFS_ALLOC(name, args->lstio_grp_nmlen + 1); - if (!name) - return -ENOMEM; - if (copy_from_user(name, args->lstio_grp_namep, - args->lstio_grp_nmlen)) { - LIBCFS_FREE(name, args->lstio_grp_nmlen + 1); - + args->lstio_grp_nmlen)) return -EFAULT; - } name[args->lstio_grp_nmlen] = 0; @@ -350,7 +304,6 @@ lst_nodes_add_ioctl(struct lstio_group_nodes_args *args) args->lstio_grp_idsp, &feats, args->lstio_grp_resultp); - LIBCFS_FREE(name, args->lstio_grp_nmlen + 1); if (!rc && copy_to_user(args->lstio_grp_featp, &feats, sizeof(feats))) { return -EINVAL; @@ -379,7 +332,7 @@ lst_group_list_ioctl(struct lstio_group_list_args *args) static int lst_group_info_ioctl(struct lstio_group_info_args *args) { - char *name; + char name[LST_NAME_SIZE + 1]; int ndent; int index; int rc; @@ -411,23 +364,15 @@ lst_group_info_ioctl(struct lstio_group_info_args *args) return -EINVAL; } - LIBCFS_ALLOC(name, args->lstio_grp_nmlen + 1); - if (!name) - return -ENOMEM; - if (copy_from_user(name, args->lstio_grp_namep, - args->lstio_grp_nmlen)) { - LIBCFS_FREE(name, args->lstio_grp_nmlen + 1); + args->lstio_grp_nmlen)) return -EFAULT; - } name[args->lstio_grp_nmlen] = 0; rc = lstcon_group_info(name, args->lstio_grp_entp, &index, &ndent, args->lstio_grp_dentsp); - LIBCFS_FREE(name, args->lstio_grp_nmlen + 1); - if (rc) return rc; @@ -443,7 +388,7 @@ static int lst_batch_add_ioctl(struct lstio_batch_add_args *args) { int rc; - char *name; + char name[LST_NAME_SIZE + 1]; if (args->lstio_bat_key != console_session.ses_key) return -EACCES; @@ -453,22 +398,14 @@ lst_batch_add_ioctl(struct lstio_batch_add_args *args) args->lstio_bat_nmlen > LST_NAME_SIZE) return -EINVAL; - LIBCFS_ALLOC(name, args->lstio_bat_nmlen + 1); - if (!name) - return -ENOMEM; - if (copy_from_user(name, args->lstio_bat_namep, - args->lstio_bat_nmlen)) { - LIBCFS_FREE(name, args->lstio_bat_nmlen + 1); + args->lstio_bat_nmlen)) return -EFAULT; - } name[args->lstio_bat_nmlen] = 0; rc = lstcon_batch_add(name); - LIBCFS_FREE(name, args->lstio_bat_nmlen + 1); - return rc; } @@ -476,7 +413,7 @@ static int lst_batch_run_ioctl(struct lstio_batch_run_args *args) { int rc; - char *name; + char name[LST_NAME_SIZE + 1]; if (args->lstio_bat_key != console_session.ses_key) return -EACCES; @@ -486,23 +423,15 @@ lst_batch_run_ioctl(struct lstio_batch_run_args *args) args->lstio_bat_nmlen > LST_NAME_SIZE) return -EINVAL; - LIBCFS_ALLOC(name, args->lstio_bat_nmlen + 1); - if (!name) - return -ENOMEM; - if (copy_from_user(name, args->lstio_bat_namep, - args->lstio_bat_nmlen)) { - LIBCFS_FREE(name, args->lstio_bat_nmlen + 1); + args->lstio_bat_nmlen)) return -EFAULT; - } name[args->lstio_bat_nmlen] = 0; rc = lstcon_batch_run(name, args->lstio_bat_timeout, args->lstio_bat_resultp); - LIBCFS_FREE(name, args->lstio_bat_nmlen + 1); - return rc; } @@ -510,7 +439,7 @@ static int lst_batch_stop_ioctl(struct lstio_batch_stop_args *args) { int rc; - char *name; + char name[LST_NAME_SIZE + 1]; if (args->lstio_bat_key != console_session.ses_key) return -EACCES; @@ -521,30 +450,22 @@ lst_batch_stop_ioctl(struct lstio_batch_stop_args *args) args->lstio_bat_nmlen > LST_NAME_SIZE) return -EINVAL; - LIBCFS_ALLOC(name, args->lstio_bat_nmlen + 1); - if (!name) - return -ENOMEM; - if (copy_from_user(name, args->lstio_bat_namep, - args->lstio_bat_nmlen)) { - LIBCFS_FREE(name, args->lstio_bat_nmlen + 1); + args->lstio_bat_nmlen)) return -EFAULT; - } name[args->lstio_bat_nmlen] = 0; rc = lstcon_batch_stop(name, args->lstio_bat_force, args->lstio_bat_resultp); - LIBCFS_FREE(name, args->lstio_bat_nmlen + 1); - return rc; } static int lst_batch_query_ioctl(struct lstio_batch_query_args *args) { - char *name; + char name[LST_NAME_SIZE + 1]; int rc; if (args->lstio_bat_key != console_session.ses_key) @@ -559,15 +480,9 @@ lst_batch_query_ioctl(struct lstio_batch_query_args *args) if (args->lstio_bat_testidx < 0) return -EINVAL; - LIBCFS_ALLOC(name, args->lstio_bat_nmlen + 1); - if (!name) - return -ENOMEM; - if (copy_from_user(name, args->lstio_bat_namep, - args->lstio_bat_nmlen)) { - LIBCFS_FREE(name, args->lstio_bat_nmlen + 1); + args->lstio_bat_nmlen)) return -EFAULT; - } name[args->lstio_bat_nmlen] = 0; @@ -577,8 +492,6 @@ lst_batch_query_ioctl(struct lstio_batch_query_args *args) args->lstio_bat_timeout, args->lstio_bat_resultp); - LIBCFS_FREE(name, args->lstio_bat_nmlen + 1); - return rc; } @@ -602,7 +515,7 @@ lst_batch_list_ioctl(struct lstio_batch_list_args *args) static int lst_batch_info_ioctl(struct lstio_batch_info_args *args) { - char *name; + char name[LST_NAME_SIZE + 1]; int rc; int index; int ndent; @@ -634,15 +547,9 @@ lst_batch_info_ioctl(struct lstio_batch_info_args *args) return -EINVAL; } - LIBCFS_ALLOC(name, args->lstio_bat_nmlen + 1); - if (!name) - return -ENOMEM; - if (copy_from_user(name, args->lstio_bat_namep, - args->lstio_bat_nmlen)) { - LIBCFS_FREE(name, args->lstio_bat_nmlen + 1); + args->lstio_bat_nmlen)) return -EFAULT; - } name[args->lstio_bat_nmlen] = 0; @@ -650,8 +557,6 @@ lst_batch_info_ioctl(struct lstio_batch_info_args *args) args->lstio_bat_server, args->lstio_bat_testidx, &index, &ndent, args->lstio_bat_dentsp); - LIBCFS_FREE(name, args->lstio_bat_nmlen + 1); - if (rc) return rc; @@ -667,7 +572,7 @@ static int lst_stat_query_ioctl(struct lstio_stat_args *args) { int rc; - char *name = NULL; + char name[LST_NAME_SIZE + 1]; /* TODO: not finished */ if (args->lstio_sta_key != console_session.ses_key) @@ -689,10 +594,6 @@ lst_stat_query_ioctl(struct lstio_stat_args *args) args->lstio_sta_nmlen > LST_NAME_SIZE) return -EINVAL; - LIBCFS_ALLOC(name, args->lstio_sta_nmlen + 1); - if (!name) - return -ENOMEM; - rc = copy_from_user(name, args->lstio_sta_namep, args->lstio_sta_nmlen); if (!rc) @@ -704,16 +605,14 @@ lst_stat_query_ioctl(struct lstio_stat_args *args) rc = -EINVAL; } - if (name) - LIBCFS_FREE(name, args->lstio_sta_nmlen + 1); return rc; } static int lst_test_add_ioctl(struct lstio_test_args *args) { - char *batch_name; - char *src_name = NULL; - char *dst_name = NULL; + char batch_name[LST_NAME_SIZE + 1]; + char src_name[LST_NAME_SIZE + 1]; + char dst_name[LST_NAME_SIZE + 1]; void *param = NULL; int ret = 0; int rc = -ENOMEM; @@ -748,18 +647,6 @@ static int lst_test_add_ioctl(struct lstio_test_args *args) if (!args->lstio_tes_param && args->lstio_tes_param_len) return -EINVAL; - LIBCFS_ALLOC(batch_name, args->lstio_tes_bat_nmlen + 1); - if (!batch_name) - return rc; - - LIBCFS_ALLOC(src_name, args->lstio_tes_sgrp_nmlen + 1); - if (!src_name) - goto out; - - LIBCFS_ALLOC(dst_name, args->lstio_tes_dgrp_nmlen + 1); - if (!dst_name) - goto out; - if (args->lstio_tes_param) { LIBCFS_ALLOC(param, args->lstio_tes_param_len); if (!param) @@ -791,15 +678,6 @@ static int lst_test_add_ioctl(struct lstio_test_args *args) rc = (copy_to_user(args->lstio_tes_retp, &ret, sizeof(ret))) ? -EFAULT : 0; out: - if (batch_name) - LIBCFS_FREE(batch_name, args->lstio_tes_bat_nmlen + 1); - - if (src_name) - LIBCFS_FREE(src_name, args->lstio_tes_sgrp_nmlen + 1); - - if (dst_name) - LIBCFS_FREE(dst_name, args->lstio_tes_dgrp_nmlen + 1); - if (param) LIBCFS_FREE(param, args->lstio_tes_param_len); From neilb at suse.com Mon Dec 18 00:46:30 2017 From: neilb at suse.com (NeilBrown) Date: Mon, 18 Dec 2017 11:46:30 +1100 Subject: [lustre-devel] [PATCH 08/15] staging: lustre: Convert more LIBCFS_ALLOC allocation to direct GFP_KERNEL In-Reply-To: <151355781721.6200.2136335532722530242.stgit@noble> References: <151355781721.6200.2136335532722530242.stgit@noble> Message-ID: <151355799049.6200.16488920506903028598.stgit@noble> None of these need to be GFP_NOFS, so use GFP_KERNEL explicitly with kmalloc(), kvmalloc(), or kvmalloc_array(). Changing matching LIBCFS_FREE() to kfree() or kvfree() Signed-off-by: NeilBrown --- .../lustre/lnet/libcfs/linux/linux-module.c | 4 +- drivers/staging/lustre/lnet/libcfs/module.c | 9 ++--- drivers/staging/lustre/lnet/lnet/api-ni.c | 17 ++++------ drivers/staging/lustre/lnet/lnet/config.c | 34 ++++++++------------ 4 files changed, 26 insertions(+), 38 deletions(-) diff --git a/drivers/staging/lustre/lnet/libcfs/linux/linux-module.c b/drivers/staging/lustre/lnet/libcfs/linux/linux-module.c index b5746230ab31..ddf625669bff 100644 --- a/drivers/staging/lustre/lnet/libcfs/linux/linux-module.c +++ b/drivers/staging/lustre/lnet/libcfs/linux/linux-module.c @@ -146,7 +146,7 @@ int libcfs_ioctl_getdata(struct libcfs_ioctl_hdr **hdr_pp, return -EINVAL; } - LIBCFS_ALLOC(*hdr_pp, hdr.ioc_len); + *hdr_pp = kvmalloc(hdr.ioc_len, GFP_KERNEL); if (!*hdr_pp) return -ENOMEM; @@ -164,7 +164,7 @@ int libcfs_ioctl_getdata(struct libcfs_ioctl_hdr **hdr_pp, return 0; free: - LIBCFS_FREE(*hdr_pp, hdr.ioc_len); + kvfree(*hdr_pp); return err; } diff --git a/drivers/staging/lustre/lnet/libcfs/module.c b/drivers/staging/lustre/lnet/libcfs/module.c index 4ead55920e79..ff4b0cec1bbe 100644 --- a/drivers/staging/lustre/lnet/libcfs/module.c +++ b/drivers/staging/lustre/lnet/libcfs/module.c @@ -156,7 +156,7 @@ int libcfs_ioctl(unsigned long cmd, void __user *uparam) break; } } out: - LIBCFS_FREE(hdr, hdr->ioc_len); + kvfree(hdr); return err; } @@ -302,7 +302,7 @@ static int __proc_cpt_table(void *data, int write, LASSERT(cfs_cpt_table); while (1) { - LIBCFS_ALLOC(buf, len); + buf = kmalloc(len, GFP_KERNEL); if (!buf) return -ENOMEM; @@ -311,7 +311,7 @@ static int __proc_cpt_table(void *data, int write, break; if (rc == -EFBIG) { - LIBCFS_FREE(buf, len); + kfree(buf); len <<= 1; continue; } @@ -325,8 +325,7 @@ static int __proc_cpt_table(void *data, int write, rc = cfs_trace_copyout_string(buffer, nob, buf + pos, NULL); out: - if (buf) - LIBCFS_FREE(buf, len); + kfree(buf); return rc; } diff --git a/drivers/staging/lustre/lnet/lnet/api-ni.c b/drivers/staging/lustre/lnet/lnet/api-ni.c index e8f623190133..f201d2d1eb02 100644 --- a/drivers/staging/lustre/lnet/lnet/api-ni.c +++ b/drivers/staging/lustre/lnet/lnet/api-ni.c @@ -108,7 +108,8 @@ lnet_create_remote_nets_table(void) LASSERT(!the_lnet.ln_remote_nets_hash); LASSERT(the_lnet.ln_remote_nets_hbits > 0); - LIBCFS_ALLOC(hash, LNET_REMOTE_NETS_HASH_SIZE * sizeof(*hash)); + hash = kvmalloc_node(LNET_REMOTE_NETS_HASH_SIZE, sizeof(*hash), + GFP_KERNEL); if (!hash) { CERROR("Failed to create remote nets hash table\n"); return -ENOMEM; @@ -131,9 +132,7 @@ lnet_destroy_remote_nets_table(void) for (i = 0; i < LNET_REMOTE_NETS_HASH_SIZE; i++) LASSERT(list_empty(&the_lnet.ln_remote_nets_hash[i])); - LIBCFS_FREE(the_lnet.ln_remote_nets_hash, - LNET_REMOTE_NETS_HASH_SIZE * - sizeof(the_lnet.ln_remote_nets_hash[0])); + kvfree(the_lnet.ln_remote_nets_hash); the_lnet.ln_remote_nets_hash = NULL; } @@ -831,7 +830,7 @@ lnet_ping_info_create(int num_ni) unsigned int infosz; infosz = offsetof(struct lnet_ping_info, pi_ni[num_ni]); - LIBCFS_ALLOC(ping_info, infosz); + ping_info = kvzalloc(infosz, GFP_KERNEL); if (!ping_info) { CERROR("Can't allocate ping info[%d]\n", num_ni); return NULL; @@ -864,9 +863,7 @@ lnet_get_ni_count(void) static inline void lnet_ping_info_free(struct lnet_ping_info *pinfo) { - LIBCFS_FREE(pinfo, - offsetof(struct lnet_ping_info, - pi_ni[pinfo->pi_nnis])); + kvfree(pinfo); } static void @@ -2160,7 +2157,7 @@ static int lnet_ping(struct lnet_process_id id, int timeout_ms, if (id.pid == LNET_PID_ANY) id.pid = LNET_PID_LUSTRE; - LIBCFS_ALLOC(info, infosz); + info = kzalloc(infosz, GFP_KERNEL); if (!info) return -ENOMEM; @@ -2310,6 +2307,6 @@ static int lnet_ping(struct lnet_process_id id, int timeout_ms, LASSERT(!rc2); out_0: - LIBCFS_FREE(info, infosz); + kfree(info); return rc; } diff --git a/drivers/staging/lustre/lnet/lnet/config.c b/drivers/staging/lustre/lnet/lnet/config.c index 66a222e5220b..6430183b2986 100644 --- a/drivers/staging/lustre/lnet/lnet/config.c +++ b/drivers/staging/lustre/lnet/lnet/config.c @@ -109,10 +109,8 @@ lnet_ni_free(struct lnet_ni *ni) if (ni->ni_lnd_tunables) kfree(ni->ni_lnd_tunables); - for (i = 0; i < LNET_MAX_INTERFACES && ni->ni_interfaces[i]; i++) { - LIBCFS_FREE(ni->ni_interfaces[i], - strlen(ni->ni_interfaces[i]) + 1); - } + for (i = 0; i < LNET_MAX_INTERFACES && ni->ni_interfaces[i]; i++) + kfree(ni->ni_interfaces[i]); /* release reference to net namespace */ if (ni->ni_net_ns) @@ -198,7 +196,6 @@ int lnet_parse_networks(struct list_head *nilist, char *networks) { struct cfs_expr_list *el = NULL; - int tokensize; char *tokens; char *str; char *tmp; @@ -219,15 +216,12 @@ lnet_parse_networks(struct list_head *nilist, char *networks) return -EINVAL; } - tokensize = strlen(networks) + 1; - - LIBCFS_ALLOC(tokens, tokensize); + tokens = kstrdup(networks, GFP_KERNEL); if (!tokens) { CERROR("Can't allocate net tokens\n"); return -ENOMEM; } - memcpy(tokens, networks, tokensize); tmp = tokens; str = tokens; @@ -349,14 +343,11 @@ lnet_parse_networks(struct list_head *nilist, char *networks) * The newly allocated ni_interfaces[] can be * freed when freeing the NI */ - LIBCFS_ALLOC(ni->ni_interfaces[niface], - strlen(iface) + 1); + ni->ni_interfaces[niface] = kstrdup(iface, GFP_KERNEL); if (!ni->ni_interfaces[niface]) { CERROR("Can't allocate net interface name\n"); goto failed; } - strncpy(ni->ni_interfaces[niface], iface, - strlen(iface)); niface++; iface = comma; } while (iface); @@ -384,7 +375,7 @@ lnet_parse_networks(struct list_head *nilist, char *networks) list_for_each(temp_node, nilist) nnets++; - LIBCFS_FREE(tokens, tokensize); + kfree(tokens); return nnets; failed_syntax: @@ -400,7 +391,7 @@ lnet_parse_networks(struct list_head *nilist, char *networks) if (el) cfs_expr_list_free(el); - LIBCFS_FREE(tokens, tokensize); + kfree(tokens); return -EINVAL; } @@ -424,7 +415,7 @@ lnet_new_text_buf(int str_len) return NULL; } - LIBCFS_ALLOC(ltb, nob); + ltb = kmalloc(nob, GFP_KERNEL); if (!ltb) return NULL; @@ -438,7 +429,7 @@ static void lnet_free_text_buf(struct lnet_text_buf *ltb) { lnet_tbnob -= ltb->ltb_size; - LIBCFS_FREE(ltb, ltb->ltb_size); + kfree(ltb); } static void @@ -1156,7 +1147,7 @@ lnet_ipaddr_enumerate(__u32 **ipaddrsp) if (nif <= 0) return nif; - LIBCFS_ALLOC(ipaddrs, nif * sizeof(*ipaddrs)); + ipaddrs = kzalloc(nif * sizeof(*ipaddrs), GFP_KERNEL); if (!ipaddrs) { CERROR("Can't allocate ipaddrs[%d]\n", nif); lnet_ipif_free_enumeration(ifnames, nif); @@ -1189,7 +1180,8 @@ lnet_ipaddr_enumerate(__u32 **ipaddrsp) *ipaddrsp = ipaddrs; } else { if (nip > 0) { - LIBCFS_ALLOC(ipaddrs2, nip * sizeof(*ipaddrs2)); + ipaddrs2 = kzalloc(nip * sizeof(*ipaddrs2), + GFP_KERNEL); if (!ipaddrs2) { CERROR("Can't allocate ipaddrs[%d]\n", nip); nip = -ENOMEM; @@ -1200,7 +1192,7 @@ lnet_ipaddr_enumerate(__u32 **ipaddrsp) rc = nip; } } - LIBCFS_FREE(ipaddrs, nip * sizeof(*ipaddrs)); + kfree(ipaddrs); } return nip; } @@ -1226,7 +1218,7 @@ lnet_parse_ip2nets(char **networksp, char *ip2nets) } rc = lnet_match_networks(networksp, ip2nets, ipaddrs, nip); - LIBCFS_FREE(ipaddrs, nip * sizeof(*ipaddrs)); + kfree(ipaddrs); if (rc < 0) { LCONSOLE_ERROR_MSG(0x119, "Error %d parsing ip2nets\n", rc); From neilb at suse.com Mon Dec 18 00:46:30 2017 From: neilb at suse.com (NeilBrown) Date: Mon, 18 Dec 2017 11:46:30 +1100 Subject: [lustre-devel] [PATCH 06/15] staging: lustre: lnet: use kmalloc/kvmalloc in router_proc In-Reply-To: <151355781721.6200.2136335532722530242.stgit@noble> References: <151355781721.6200.2136335532722530242.stgit@noble> Message-ID: <151355799042.6200.15576640392787632589.stgit@noble> The buffers allocated in router_proc are to temporarily hold strings created for procfs files. So they do not need to be zeroed and are safe to use GFP_KERNEL. So use kmalloc() directly except in two cases where it isn't trivial to confirm that the size is always small. In those cases, use kvmalloc(). Signed-off-by: NeilBrown --- drivers/staging/lustre/lnet/lnet/router_proc.c | 34 ++++++++++++------------ 1 file changed, 17 insertions(+), 17 deletions(-) diff --git a/drivers/staging/lustre/lnet/lnet/router_proc.c b/drivers/staging/lustre/lnet/lnet/router_proc.c index 06e20b314520..8d6d6b4d6619 100644 --- a/drivers/staging/lustre/lnet/lnet/router_proc.c +++ b/drivers/staging/lustre/lnet/lnet/router_proc.c @@ -95,7 +95,7 @@ static int __proc_lnet_stats(void *data, int write, if (!ctrs) return -ENOMEM; - LIBCFS_ALLOC(tmpstr, tmpsiz); + tmpstr = kmalloc(tmpsiz, GFP_KERNEL); if (!tmpstr) { kfree(ctrs); return -ENOMEM; @@ -118,7 +118,7 @@ static int __proc_lnet_stats(void *data, int write, rc = cfs_trace_copyout_string(buffer, nob, tmpstr + pos, "\n"); - LIBCFS_FREE(tmpstr, tmpsiz); + kfree(tmpstr); kfree(ctrs); return rc; } @@ -151,7 +151,7 @@ static int proc_lnet_routes(struct ctl_table *table, int write, if (!*lenp) return 0; - LIBCFS_ALLOC(tmpstr, tmpsiz); + tmpstr = kmalloc(tmpsiz, GFP_KERNEL); if (!tmpstr) return -ENOMEM; @@ -183,7 +183,7 @@ static int proc_lnet_routes(struct ctl_table *table, int write, if (ver != LNET_PROC_VERSION(the_lnet.ln_remote_nets_version)) { lnet_net_unlock(0); - LIBCFS_FREE(tmpstr, tmpsiz); + kfree(tmpstr); return -ESTALE; } @@ -248,7 +248,7 @@ static int proc_lnet_routes(struct ctl_table *table, int write, } } - LIBCFS_FREE(tmpstr, tmpsiz); + kfree(tmpstr); if (!rc) *lenp = len; @@ -275,7 +275,7 @@ static int proc_lnet_routers(struct ctl_table *table, int write, if (!*lenp) return 0; - LIBCFS_ALLOC(tmpstr, tmpsiz); + tmpstr = kmalloc(tmpsiz, GFP_KERNEL); if (!tmpstr) return -ENOMEM; @@ -303,7 +303,7 @@ static int proc_lnet_routers(struct ctl_table *table, int write, if (ver != LNET_PROC_VERSION(the_lnet.ln_routers_version)) { lnet_net_unlock(0); - LIBCFS_FREE(tmpstr, tmpsiz); + kfree(tmpstr); return -ESTALE; } @@ -385,7 +385,7 @@ static int proc_lnet_routers(struct ctl_table *table, int write, } } - LIBCFS_FREE(tmpstr, tmpsiz); + kfree(tmpstr); if (!rc) *lenp = len; @@ -418,7 +418,7 @@ static int proc_lnet_peers(struct ctl_table *table, int write, return 0; } - LIBCFS_ALLOC(tmpstr, tmpsiz); + tmpstr = kmalloc(tmpsiz, GFP_KERNEL); if (!tmpstr) return -ENOMEM; @@ -448,7 +448,7 @@ static int proc_lnet_peers(struct ctl_table *table, int write, if (ver != LNET_PROC_VERSION(ptable->pt_version)) { lnet_net_unlock(cpt); - LIBCFS_FREE(tmpstr, tmpsiz); + kfree(tmpstr); return -ESTALE; } @@ -556,7 +556,7 @@ static int proc_lnet_peers(struct ctl_table *table, int write, *ppos = LNET_PROC_POS_MAKE(cpt, ver, hash, hoff); } - LIBCFS_FREE(tmpstr, tmpsiz); + kfree(tmpstr); if (!rc) *lenp = len; @@ -579,7 +579,7 @@ static int __proc_lnet_buffers(void *data, int write, /* (4 %d) * 4 * LNET_CPT_NUMBER */ tmpsiz = 64 * (LNET_NRBPOOLS + 1) * LNET_CPT_NUMBER; - LIBCFS_ALLOC(tmpstr, tmpsiz); + tmpstr = kvmalloc(tmpsiz, GFP_KERNEL); if (!tmpstr) return -ENOMEM; @@ -618,7 +618,7 @@ static int __proc_lnet_buffers(void *data, int write, rc = cfs_trace_copyout_string(buffer, nob, tmpstr + pos, NULL); - LIBCFS_FREE(tmpstr, tmpsiz); + kvfree(tmpstr); return rc; } @@ -643,7 +643,7 @@ static int proc_lnet_nis(struct ctl_table *table, int write, if (!*lenp) return 0; - LIBCFS_ALLOC(tmpstr, tmpsiz); + tmpstr = kvmalloc(tmpsiz, GFP_KERNEL); if (!tmpstr) return -ENOMEM; @@ -744,7 +744,7 @@ static int proc_lnet_nis(struct ctl_table *table, int write, *ppos += 1; } - LIBCFS_FREE(tmpstr, tmpsiz); + kvfree(tmpstr); if (!rc) *lenp = len; @@ -795,7 +795,7 @@ static int __proc_lnet_portal_rotor(void *data, int write, int rc; int i; - LIBCFS_ALLOC(buf, buf_len); + buf = kmalloc(buf_len, GFP_KERNEL); if (!buf) return -ENOMEM; @@ -843,7 +843,7 @@ static int __proc_lnet_portal_rotor(void *data, int write, } lnet_res_unlock(0); out: - LIBCFS_FREE(buf, buf_len); + kfree(buf); return rc; } From neilb at suse.com Mon Dec 18 00:46:30 2017 From: neilb at suse.com (NeilBrown) Date: Mon, 18 Dec 2017 11:46:30 +1100 Subject: [lustre-devel] [PATCH 11/15] staging: lustre: lnet-route: use kmalloc for small allocation In-Reply-To: <151355781721.6200.2136335532722530242.stgit@noble> References: <151355781721.6200.2136335532722530242.stgit@noble> Message-ID: <151355799060.6200.12666775662373842922.stgit@noble> This allocation is reasonably small. As the function is called "*_locked", it might not be safe to perform a GFP_KERNEL allocation, so be safe and use GFP_NOFS. Signed-off-by: NeilBrown --- drivers/staging/lustre/lnet/lnet/router.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/drivers/staging/lustre/lnet/lnet/router.c b/drivers/staging/lustre/lnet/lnet/router.c index 86f28152f7b3..c40aa79baf5c 100644 --- a/drivers/staging/lustre/lnet/lnet/router.c +++ b/drivers/staging/lustre/lnet/lnet/router.c @@ -889,8 +889,7 @@ lnet_destroy_rc_data(struct lnet_rc_data *rcd) lnet_net_unlock(cpt); } - if (rcd->rcd_pinginfo) - LIBCFS_FREE(rcd->rcd_pinginfo, LNET_PINGINFO_SIZE); + kfree(rcd->rcd_pinginfo); kfree(rcd); } @@ -913,7 +912,7 @@ lnet_create_rc_data_locked(struct lnet_peer *gateway) LNetInvalidateMDHandle(&rcd->rcd_mdh); INIT_LIST_HEAD(&rcd->rcd_list); - LIBCFS_ALLOC(pi, LNET_PINGINFO_SIZE); + pi = kzalloc(LNET_PINGINFO_SIZE, GFP_NOFS); if (!pi) goto out; From neilb at suse.com Mon Dec 18 00:46:30 2017 From: neilb at suse.com (NeilBrown) Date: Mon, 18 Dec 2017 11:46:30 +1100 Subject: [lustre-devel] [PATCH 09/15] staging: lustre: more LIBCFS_ALLOC conversions to GFP_KERNEL allocations. In-Reply-To: <151355781721.6200.2136335532722530242.stgit@noble> References: <151355781721.6200.2136335532722530242.stgit@noble> Message-ID: <151355799053.6200.9171627143037737175.stgit@noble> None of these need GFP_NOFS so allocate directly. Change matching LIBCFS_FREE() to kfree() or kvfree(). Signed-off-by: NeilBrown --- .../staging/lustre/lnet/libcfs/linux/linux-cpu.c | 19 +++++++------------ drivers/staging/lustre/lnet/lnet/lib-eq.c | 9 ++++----- drivers/staging/lustre/lnet/lnet/lib-socket.c | 14 +++++++------- drivers/staging/lustre/lnet/selftest/conctl.c | 11 +++++------ 4 files changed, 23 insertions(+), 30 deletions(-) diff --git a/drivers/staging/lustre/lnet/libcfs/linux/linux-cpu.c b/drivers/staging/lustre/lnet/libcfs/linux/linux-cpu.c index 44e44a999648..e9156bf05ed4 100644 --- a/drivers/staging/lustre/lnet/libcfs/linux/linux-cpu.c +++ b/drivers/staging/lustre/lnet/libcfs/linux/linux-cpu.c @@ -93,11 +93,7 @@ cfs_cpt_table_free(struct cfs_cpt_table *cptab) { int i; - if (cptab->ctb_cpu2cpt) { - LIBCFS_FREE(cptab->ctb_cpu2cpt, - num_possible_cpus() * - sizeof(cptab->ctb_cpu2cpt[0])); - } + kvfree(cptab->ctb_cpu2cpt); for (i = 0; cptab->ctb_parts && i < cptab->ctb_nparts; i++) { struct cfs_cpu_partition *part = &cptab->ctb_parts[i]; @@ -109,10 +105,7 @@ cfs_cpt_table_free(struct cfs_cpt_table *cptab) free_cpumask_var(part->cpt_cpumask); } - if (cptab->ctb_parts) { - LIBCFS_FREE(cptab->ctb_parts, - cptab->ctb_nparts * sizeof(cptab->ctb_parts[0])); - } + kvfree(cptab->ctb_parts); if (cptab->ctb_nodemask) kfree(cptab->ctb_nodemask); @@ -140,15 +133,17 @@ cfs_cpt_table_alloc(unsigned int ncpt) !cptab->ctb_nodemask) goto failed; - LIBCFS_ALLOC(cptab->ctb_cpu2cpt, - num_possible_cpus() * sizeof(cptab->ctb_cpu2cpt[0])); + cptab->ctb_cpu2cpt = kvmalloc_array(num_possible_cpus(), + sizeof(cptab->ctb_cpu2cpt[0]), + GFP_KERNEL); if (!cptab->ctb_cpu2cpt) goto failed; memset(cptab->ctb_cpu2cpt, -1, num_possible_cpus() * sizeof(cptab->ctb_cpu2cpt[0])); - LIBCFS_ALLOC(cptab->ctb_parts, ncpt * sizeof(cptab->ctb_parts[0])); + cptab->ctb_parts = kvmalloc_array(ncpt, sizeof(cptab->ctb_parts[0]), + GFP_KERNEL); if (!cptab->ctb_parts) goto failed; diff --git a/drivers/staging/lustre/lnet/lnet/lib-eq.c b/drivers/staging/lustre/lnet/lnet/lib-eq.c index 7a4d1f7a693e..a173b69e2f92 100644 --- a/drivers/staging/lustre/lnet/lnet/lib-eq.c +++ b/drivers/staging/lustre/lnet/lnet/lib-eq.c @@ -95,7 +95,8 @@ LNetEQAlloc(unsigned int count, lnet_eq_handler_t callback, return -ENOMEM; if (count) { - LIBCFS_ALLOC(eq->eq_events, count * sizeof(struct lnet_event)); + eq->eq_events = kvmalloc_array(count, sizeof(struct lnet_event), + GFP_KERNEL | __GFP_ZERO); if (!eq->eq_events) goto failed; /* @@ -132,8 +133,7 @@ LNetEQAlloc(unsigned int count, lnet_eq_handler_t callback, return 0; failed: - if (eq->eq_events) - LIBCFS_FREE(eq->eq_events, count * sizeof(struct lnet_event)); + kvfree(eq->eq_events); if (eq->eq_refs) cfs_percpt_free(eq->eq_refs); @@ -202,8 +202,7 @@ LNetEQFree(struct lnet_handle_eq eqh) lnet_eq_wait_unlock(); lnet_res_unlock(LNET_LOCK_EX); - if (events) - LIBCFS_FREE(events, size * sizeof(struct lnet_event)); + kvfree(events); if (refs) cfs_percpt_free(refs); diff --git a/drivers/staging/lustre/lnet/lnet/lib-socket.c b/drivers/staging/lustre/lnet/lnet/lib-socket.c index 539a26444f31..d3c35887598c 100644 --- a/drivers/staging/lustre/lnet/lnet/lib-socket.c +++ b/drivers/staging/lustre/lnet/lnet/lib-socket.c @@ -174,7 +174,7 @@ lnet_ipif_enumerate(char ***namesp) nalloc); } - LIBCFS_ALLOC(ifr, nalloc * sizeof(*ifr)); + ifr = kzalloc(nalloc * sizeof(*ifr), GFP_KERNEL); if (!ifr) { CERROR("ENOMEM enumerating up to %d interfaces\n", nalloc); @@ -199,14 +199,14 @@ lnet_ipif_enumerate(char ***namesp) if (nfound < nalloc || toobig) break; - LIBCFS_FREE(ifr, nalloc * sizeof(*ifr)); + kfree(ifr); nalloc *= 2; } if (!nfound) goto out1; - LIBCFS_ALLOC(names, nfound * sizeof(*names)); + names = kzalloc(nfound * sizeof(*names), GFP_KERNEL); if (!names) { rc = -ENOMEM; goto out1; @@ -222,7 +222,7 @@ lnet_ipif_enumerate(char ***namesp) goto out2; } - LIBCFS_ALLOC(names[i], IFNAMSIZ); + names[i] = kmalloc(IFNAMSIZ, GFP_KERNEL); if (!names[i]) { rc = -ENOMEM; goto out2; @@ -239,7 +239,7 @@ lnet_ipif_enumerate(char ***namesp) if (rc < 0) lnet_ipif_free_enumeration(names, nfound); out1: - LIBCFS_FREE(ifr, nalloc * sizeof(*ifr)); + kfree(ifr); out0: return rc; } @@ -253,9 +253,9 @@ lnet_ipif_free_enumeration(char **names, int n) LASSERT(n > 0); for (i = 0; i < n && names[i]; i++) - LIBCFS_FREE(names[i], IFNAMSIZ); + kfree(names[i]); - LIBCFS_FREE(names, n * sizeof(*names)); + kfree(names); } EXPORT_SYMBOL(lnet_ipif_free_enumeration); diff --git a/drivers/staging/lustre/lnet/selftest/conctl.c b/drivers/staging/lustre/lnet/selftest/conctl.c index 442a18ddd41f..34ba440b3c02 100644 --- a/drivers/staging/lustre/lnet/selftest/conctl.c +++ b/drivers/staging/lustre/lnet/selftest/conctl.c @@ -648,7 +648,7 @@ static int lst_test_add_ioctl(struct lstio_test_args *args) return -EINVAL; if (args->lstio_tes_param) { - LIBCFS_ALLOC(param, args->lstio_tes_param_len); + param = kmalloc(args->lstio_tes_param_len, GFP_KERNEL); if (!param) goto out; if (copy_from_user(param, args->lstio_tes_param, @@ -678,8 +678,7 @@ static int lst_test_add_ioctl(struct lstio_test_args *args) rc = (copy_to_user(args->lstio_tes_retp, &ret, sizeof(ret))) ? -EFAULT : 0; out: - if (param) - LIBCFS_FREE(param, args->lstio_tes_param_len); + kfree(param); return rc; } @@ -702,13 +701,13 @@ lstcon_ioctl_entry(unsigned int cmd, struct libcfs_ioctl_hdr *hdr) if (data->ioc_plen1 > PAGE_SIZE) return -EINVAL; - LIBCFS_ALLOC(buf, data->ioc_plen1); + buf = kmalloc(data->ioc_plen1, GFP_KERNEL); if (!buf) return -ENOMEM; /* copy in parameter */ if (copy_from_user(buf, data->ioc_pbuf1, data->ioc_plen1)) { - LIBCFS_FREE(buf, data->ioc_plen1); + kfree(buf); return -EFAULT; } @@ -798,7 +797,7 @@ lstcon_ioctl_entry(unsigned int cmd, struct libcfs_ioctl_hdr *hdr) out: mutex_unlock(&console_session.ses_mutex); - LIBCFS_FREE(buf, data->ioc_plen1); + kfree(buf); return rc; } From neilb at suse.com Mon Dec 18 00:46:30 2017 From: neilb at suse.com (NeilBrown) Date: Mon, 18 Dec 2017 11:46:30 +1100 Subject: [lustre-devel] [PATCH 10/15] staging: lustre: more conversions to GFP_KERNEL allocations. In-Reply-To: <151355781721.6200.2136335532722530242.stgit@noble> References: <151355781721.6200.2136335532722530242.stgit@noble> Message-ID: <151355799056.6200.17314874815893234845.stgit@noble> These are not called from filesystem context, so use GFP_KERNEL, not LIBCFS_ALLOC(). Signed-off-by: NeilBrown --- drivers/staging/lustre/lnet/selftest/console.c | 57 +++++++++----------- drivers/staging/lustre/lnet/selftest/framework.c | 4 + drivers/staging/lustre/lnet/selftest/module.c | 7 +- drivers/staging/lustre/lnet/selftest/rpc.c | 4 + drivers/staging/lustre/lnet/selftest/selftest.h | 2 - .../lustre/lustre/obdclass/lprocfs_status.c | 14 ++--- 6 files changed, 40 insertions(+), 48 deletions(-) diff --git a/drivers/staging/lustre/lnet/selftest/console.c b/drivers/staging/lustre/lnet/selftest/console.c index edf5e59a4351..1acd5cb324b1 100644 --- a/drivers/staging/lustre/lnet/selftest/console.c +++ b/drivers/staging/lustre/lnet/selftest/console.c @@ -88,7 +88,7 @@ lstcon_node_find(struct lnet_process_id id, struct lstcon_node **ndpp, if (!create) return -ENOENT; - LIBCFS_ALLOC(*ndpp, sizeof(**ndpp) + sizeof(*ndl)); + *ndpp = kzalloc(sizeof(**ndpp) + sizeof(*ndl), GFP_KERNEL); if (!*ndpp) return -ENOMEM; @@ -133,7 +133,7 @@ lstcon_node_put(struct lstcon_node *nd) list_del(&ndl->ndl_link); list_del(&ndl->ndl_hlink); - LIBCFS_FREE(nd, sizeof(*nd) + sizeof(*ndl)); + kfree(nd); } static int @@ -199,16 +199,16 @@ lstcon_group_alloc(char *name, struct lstcon_group **grpp) struct lstcon_group *grp; int i; - LIBCFS_ALLOC(grp, offsetof(struct lstcon_group, - grp_ndl_hash[LST_NODE_HASHSIZE])); + grp = kmalloc(offsetof(struct lstcon_group, + grp_ndl_hash[LST_NODE_HASHSIZE]), + GFP_KERNEL); if (!grp) return -ENOMEM; grp->grp_ref = 1; if (name) { if (strlen(name) > sizeof(grp->grp_name) - 1) { - LIBCFS_FREE(grp, offsetof(struct lstcon_group, - grp_ndl_hash[LST_NODE_HASHSIZE])); + kfree(grp); return -E2BIG; } strncpy(grp->grp_name, name, sizeof(grp->grp_name)); @@ -263,8 +263,7 @@ lstcon_group_decref(struct lstcon_group *grp) for (i = 0; i < LST_NODE_HASHSIZE; i++) LASSERT(list_empty(&grp->grp_ndl_hash[i])); - LIBCFS_FREE(grp, offsetof(struct lstcon_group, - grp_ndl_hash[LST_NODE_HASHSIZE])); + kfree(grp); } static int @@ -862,8 +861,8 @@ lstcon_batch_add(char *name) return -ENOMEM; } - LIBCFS_ALLOC(bat->bat_cli_hash, - sizeof(struct list_head) * LST_NODE_HASHSIZE); + bat->bat_cli_hash = kmalloc(sizeof(struct list_head) * LST_NODE_HASHSIZE, + GFP_KERNEL); if (!bat->bat_cli_hash) { CERROR("Can't allocate hash for batch %s\n", name); kfree(bat); @@ -871,19 +870,19 @@ lstcon_batch_add(char *name) return -ENOMEM; } - LIBCFS_ALLOC(bat->bat_srv_hash, - sizeof(struct list_head) * LST_NODE_HASHSIZE); + bat->bat_srv_hash = kmalloc(sizeof(struct list_head) * LST_NODE_HASHSIZE, + GFP_KERNEL); if (!bat->bat_srv_hash) { CERROR("Can't allocate hash for batch %s\n", name); - LIBCFS_FREE(bat->bat_cli_hash, LST_NODE_HASHSIZE); + kfree(bat->bat_cli_hash); kfree(bat); return -ENOMEM; } if (strlen(name) > sizeof(bat->bat_name) - 1) { - LIBCFS_FREE(bat->bat_srv_hash, LST_NODE_HASHSIZE); - LIBCFS_FREE(bat->bat_cli_hash, LST_NODE_HASHSIZE); + kfree(bat->bat_srv_hash); + kfree(bat->bat_cli_hash); kfree(bat); return -E2BIG; } @@ -1107,8 +1106,7 @@ lstcon_batch_destroy(struct lstcon_batch *bat) lstcon_group_decref(test->tes_src_grp); lstcon_group_decref(test->tes_dst_grp); - LIBCFS_FREE(test, offsetof(struct lstcon_test, - tes_param[test->tes_paramlen])); + kfree(test); } LASSERT(list_empty(&bat->bat_trans_list)); @@ -1134,10 +1132,8 @@ lstcon_batch_destroy(struct lstcon_batch *bat) LASSERT(list_empty(&bat->bat_srv_hash[i])); } - LIBCFS_FREE(bat->bat_cli_hash, - sizeof(struct list_head) * LST_NODE_HASHSIZE); - LIBCFS_FREE(bat->bat_srv_hash, - sizeof(struct list_head) * LST_NODE_HASHSIZE); + kfree(bat->bat_cli_hash); + kfree(bat->bat_srv_hash); kfree(bat); } @@ -1311,7 +1307,8 @@ lstcon_test_add(char *batch_name, int type, int loop, if (dst_grp->grp_userland) *retp = 1; - LIBCFS_ALLOC(test, offsetof(struct lstcon_test, tes_param[paramlen])); + test = kzalloc(offsetof(struct lstcon_test, tes_param[paramlen]), + GFP_KERNEL); if (!test) { CERROR("Can't allocate test descriptor\n"); rc = -ENOMEM; @@ -1357,8 +1354,7 @@ lstcon_test_add(char *batch_name, int type, int loop, /* hold groups so nobody can change them */ return rc; out: - if (test) - LIBCFS_FREE(test, offsetof(struct lstcon_test, tes_param[paramlen])); + kfree(test); if (dst_grp) lstcon_group_decref(dst_grp); @@ -2027,8 +2023,8 @@ lstcon_console_init(void) INIT_LIST_HEAD(&console_session.ses_bat_list); INIT_LIST_HEAD(&console_session.ses_trans_list); - LIBCFS_ALLOC(console_session.ses_ndl_hash, - sizeof(struct list_head) * LST_GLOBAL_HASHSIZE); + console_session.ses_ndl_hash = + kmalloc(sizeof(struct list_head) * LST_GLOBAL_HASHSIZE, GFP_KERNEL); if (!console_session.ses_ndl_hash) return -ENOMEM; @@ -2041,8 +2037,7 @@ lstcon_console_init(void) rc = srpc_add_service(&lstcon_acceptor_service); LASSERT(rc != -EBUSY); if (rc) { - LIBCFS_FREE(console_session.ses_ndl_hash, - sizeof(struct list_head) * LST_GLOBAL_HASHSIZE); + kfree(console_session.ses_ndl_hash); return rc; } @@ -2064,8 +2059,7 @@ lstcon_console_init(void) srpc_shutdown_service(&lstcon_acceptor_service); srpc_remove_service(&lstcon_acceptor_service); - LIBCFS_FREE(console_session.ses_ndl_hash, - sizeof(struct list_head) * LST_GLOBAL_HASHSIZE); + kfree(console_session.ses_ndl_hash); srpc_wait_service_shutdown(&lstcon_acceptor_service); @@ -2099,8 +2093,7 @@ lstcon_console_fini(void) for (i = 0; i < LST_NODE_HASHSIZE; i++) LASSERT(list_empty(&console_session.ses_ndl_hash[i])); - LIBCFS_FREE(console_session.ses_ndl_hash, - sizeof(struct list_head) * LST_GLOBAL_HASHSIZE); + kfree(console_session.ses_ndl_hash); srpc_wait_service_shutdown(&lstcon_acceptor_service); diff --git a/drivers/staging/lustre/lnet/selftest/framework.c b/drivers/staging/lustre/lnet/selftest/framework.c index b8d95a20b2eb..4592ece98d95 100644 --- a/drivers/staging/lustre/lnet/selftest/framework.c +++ b/drivers/staging/lustre/lnet/selftest/framework.c @@ -639,7 +639,7 @@ sfw_destroy_test_instance(struct sfw_test_instance *tsi) rpc = list_entry(tsi->tsi_free_rpcs.next, struct srpc_client_rpc, crpc_list); list_del(&rpc->crpc_list); - LIBCFS_FREE(rpc, srpc_client_rpc_size(rpc)); + kfree(rpc); } clean: @@ -1767,7 +1767,7 @@ sfw_shutdown(void) struct srpc_client_rpc, crpc_list); list_del(&rpc->crpc_list); - LIBCFS_FREE(rpc, srpc_client_rpc_size(rpc)); + kfree(rpc); } for (i = 0; ; i++) { diff --git a/drivers/staging/lustre/lnet/selftest/module.c b/drivers/staging/lustre/lnet/selftest/module.c index 1d44d912f014..ba4b6145c953 100644 --- a/drivers/staging/lustre/lnet/selftest/module.c +++ b/drivers/staging/lustre/lnet/selftest/module.c @@ -72,9 +72,7 @@ lnet_selftest_exit(void) continue; cfs_wi_sched_destroy(lst_sched_test[i]); } - LIBCFS_FREE(lst_sched_test, - sizeof(lst_sched_test[0]) * - cfs_cpt_number(lnet_cpt_table())); + kvfree(lst_sched_test); lst_sched_test = NULL; /* fall through */ case LST_INIT_WI_SERIAL: @@ -103,7 +101,8 @@ lnet_selftest_init(void) lst_init_step = LST_INIT_WI_SERIAL; nscheds = cfs_cpt_number(lnet_cpt_table()); - LIBCFS_ALLOC(lst_sched_test, sizeof(lst_sched_test[0]) * nscheds); + lst_sched_test = kvmalloc_array(nscheds, sizeof(lst_sched_test[0]), + GFP_KERNEL | __GFP_ZERO); if (!lst_sched_test) goto error; diff --git a/drivers/staging/lustre/lnet/selftest/rpc.c b/drivers/staging/lustre/lnet/selftest/rpc.c index 7061f19f047f..e195b9ee544c 100644 --- a/drivers/staging/lustre/lnet/selftest/rpc.c +++ b/drivers/staging/lustre/lnet/selftest/rpc.c @@ -1322,8 +1322,8 @@ srpc_create_client_rpc(struct lnet_process_id peer, int service, { struct srpc_client_rpc *rpc; - LIBCFS_ALLOC(rpc, offsetof(struct srpc_client_rpc, - crpc_bulk.bk_iovs[nbulkiov])); + rpc = kzalloc(offsetof(struct srpc_client_rpc, + crpc_bulk.bk_iovs[nbulkiov]), GFP_KERNEL); if (!rpc) return NULL; diff --git a/drivers/staging/lustre/lnet/selftest/selftest.h b/drivers/staging/lustre/lnet/selftest/selftest.h index 8c10f0f149d5..b23a953d8efe 100644 --- a/drivers/staging/lustre/lnet/selftest/selftest.h +++ b/drivers/staging/lustre/lnet/selftest/selftest.h @@ -516,7 +516,7 @@ srpc_destroy_client_rpc(struct srpc_client_rpc *rpc) LASSERT(!atomic_read(&rpc->crpc_refcount)); if (!rpc->crpc_fini) - LIBCFS_FREE(rpc, srpc_client_rpc_size(rpc)); + kfree(rpc); else (*rpc->crpc_fini)(rpc); } diff --git a/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c b/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c index 05d71f568837..85483a38c6c4 100644 --- a/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c +++ b/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c @@ -1137,7 +1137,8 @@ struct lprocfs_stats *lprocfs_alloc_stats(unsigned int num, num_entry = num_possible_cpus(); /* alloc percpu pointers for all possible cpu slots */ - LIBCFS_ALLOC(stats, offsetof(typeof(*stats), ls_percpu[num_entry])); + stats = kvzalloc(offsetof(typeof(*stats), ls_percpu[num_entry]), + GFP_KERNEL); if (!stats) return NULL; @@ -1146,8 +1147,9 @@ struct lprocfs_stats *lprocfs_alloc_stats(unsigned int num, spin_lock_init(&stats->ls_lock); /* alloc num of counter headers */ - LIBCFS_ALLOC(stats->ls_cnt_header, - stats->ls_num * sizeof(struct lprocfs_counter_header)); + stats->ls_cnt_header = kvmalloc_array(stats->ls_num, + sizeof(struct lprocfs_counter_header), + GFP_KERNEL | __GFP_ZERO); if (!stats->ls_cnt_header) goto fail; @@ -1193,10 +1195,8 @@ void lprocfs_free_stats(struct lprocfs_stats **statsh) for (i = 0; i < num_entry; i++) if (stats->ls_percpu[i]) LIBCFS_FREE(stats->ls_percpu[i], percpusize); - if (stats->ls_cnt_header) - LIBCFS_FREE(stats->ls_cnt_header, stats->ls_num * - sizeof(struct lprocfs_counter_header)); - LIBCFS_FREE(stats, offsetof(typeof(*stats), ls_percpu[num_entry])); + kvfree(stats->ls_cnt_header); + kvfree(stats); } EXPORT_SYMBOL(lprocfs_free_stats); From neilb at suse.com Mon Dec 18 00:46:30 2017 From: neilb at suse.com (NeilBrown) Date: Mon, 18 Dec 2017 11:46:30 +1100 Subject: [lustre-devel] [PATCH 07/15] staging: lustre: change some LIBCFS_ALLOC calls to k?alloc(GFP_KERNEL) In-Reply-To: <151355781721.6200.2136335532722530242.stgit@noble> References: <151355781721.6200.2136335532722530242.stgit@noble> Message-ID: <151355799045.6200.566350694041212400.stgit@noble> When an allocation happens from process context rather than filesystem context, it is best to use GFP_KERNEL rather than LIBCFS_ALLOC() which always uses GFP_NOFS. This include initialization during, or prior to, mount, and code run from separate worker threads. So for some of these cases, switch to kmalloc, kvmalloc, or kvmalloc_array() as appropriate. In some cases we preserve __GFP_ZERO (via kzalloc/kvzalloc), but in others it is clear that allocated memory is immediately initialized. In each case, the matching LIBCFS_FREE() is converted to kfree() or kvfree() This is just a subset of location that need changing. As there are quite a lot, I've broken them up into several ad-hoc sets to avoid review-fatigue. Signed-off-by: NeilBrown --- .../lustre/include/linux/libcfs/libcfs_string.h | 4 ++-- .../staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c | 11 ++++------ .../staging/lustre/lnet/klnds/socklnd/socklnd.c | 22 ++++++++------------ drivers/staging/lustre/lnet/libcfs/hash.c | 18 +++++++--------- drivers/staging/lustre/lnet/libcfs/libcfs_mem.c | 9 ++++---- drivers/staging/lustre/lnet/libcfs/libcfs_string.c | 2 +- drivers/staging/lustre/lnet/lnet/config.c | 2 +- 7 files changed, 29 insertions(+), 39 deletions(-) diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_string.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_string.h index 1191764c431a..c1375733ff31 100644 --- a/drivers/staging/lustre/include/linux/libcfs/libcfs_string.h +++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_string.h @@ -86,11 +86,11 @@ static inline void cfs_expr_list_values_free(u32 *values, int num) { /* - * This array is allocated by LIBCFS_ALLOC(), so it shouldn't be freed + * This array is allocated by kvalloc(), so it shouldn't be freed * by OBD_FREE() if it's called by module other than libcfs & LNet, * otherwise we will see fake memory leak */ - LIBCFS_FREE(values, num * sizeof(values[0])); + kvfree(values); } void cfs_expr_list_free(struct cfs_expr_list *expr_list); diff --git a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c index f9bdf1877794..f59c20a870a8 100644 --- a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c +++ b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c @@ -2577,11 +2577,7 @@ static void kiblnd_base_shutdown(void) break; } - if (kiblnd_data.kib_peers) { - LIBCFS_FREE(kiblnd_data.kib_peers, - sizeof(struct list_head) * - kiblnd_data.kib_peer_hash_size); - } + kvfree(kiblnd_data.kib_peers); if (kiblnd_data.kib_scheds) cfs_percpt_free(kiblnd_data.kib_scheds); @@ -2673,8 +2669,9 @@ static int kiblnd_base_startup(void) INIT_LIST_HEAD(&kiblnd_data.kib_failed_devs); kiblnd_data.kib_peer_hash_size = IBLND_PEER_HASH_SIZE; - LIBCFS_ALLOC(kiblnd_data.kib_peers, - sizeof(struct list_head) * kiblnd_data.kib_peer_hash_size); + kiblnd_data.kib_peers = kvmalloc_array(kiblnd_data.kib_peer_hash_size, + sizeof(struct list_head), + GFP_KERNEL); if (!kiblnd_data.kib_peers) goto failed; for (i = 0; i < kiblnd_data.kib_peer_hash_size; i++) diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c index 51157ae14a9e..dc63ed2ceb97 100644 --- a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c +++ b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c @@ -1070,8 +1070,9 @@ ksocknal_create_conn(struct lnet_ni *ni, struct ksock_route *route, conn->ksnc_tx_carrier = NULL; atomic_set(&conn->ksnc_tx_nob, 0); - LIBCFS_ALLOC(hello, offsetof(struct ksock_hello_msg, - kshm_ips[LNET_MAX_INTERFACES])); + hello = kvzalloc(offsetof(struct ksock_hello_msg, + kshm_ips[LNET_MAX_INTERFACES]), + GFP_KERNEL); if (!hello) { rc = -ENOMEM; goto failed_1; @@ -1334,8 +1335,7 @@ ksocknal_create_conn(struct lnet_ni *ni, struct ksock_route *route, rc = ksocknal_send_hello(ni, conn, peerid.nid, hello); } - LIBCFS_FREE(hello, offsetof(struct ksock_hello_msg, - kshm_ips[LNET_MAX_INTERFACES])); + kvfree(hello); /* * setup the socket AFTER I've received hello (it disables @@ -1415,9 +1415,7 @@ ksocknal_create_conn(struct lnet_ni *ni, struct ksock_route *route, ksocknal_peer_decref(peer); failed_1: - if (hello) - LIBCFS_FREE(hello, offsetof(struct ksock_hello_msg, - kshm_ips[LNET_MAX_INTERFACES])); + kvfree(hello); kfree(conn); @@ -2269,9 +2267,7 @@ ksocknal_free_buffers(void) cfs_percpt_free(ksocknal_data.ksnd_sched_info); } - LIBCFS_FREE(ksocknal_data.ksnd_peers, - sizeof(struct list_head) * - ksocknal_data.ksnd_peer_hash_size); + kvfree(ksocknal_data.ksnd_peers); spin_lock(&ksocknal_data.ksnd_tx_lock); @@ -2401,9 +2397,9 @@ ksocknal_base_startup(void) memset(&ksocknal_data, 0, sizeof(ksocknal_data)); /* zero pointers */ ksocknal_data.ksnd_peer_hash_size = SOCKNAL_PEER_HASH_SIZE; - LIBCFS_ALLOC(ksocknal_data.ksnd_peers, - sizeof(struct list_head) * - ksocknal_data.ksnd_peer_hash_size); + ksocknal_data.ksnd_peers = kvmalloc_array(ksocknal_data.ksnd_peer_hash_size, + sizeof(struct list_head), + GFP_KERNEL); if (!ksocknal_data.ksnd_peers) return -ENOMEM; diff --git a/drivers/staging/lustre/lnet/libcfs/hash.c b/drivers/staging/lustre/lnet/libcfs/hash.c index f4f67d2b301e..aabe29eef85c 100644 --- a/drivers/staging/lustre/lnet/libcfs/hash.c +++ b/drivers/staging/lustre/lnet/libcfs/hash.c @@ -864,12 +864,10 @@ cfs_hash_buckets_free(struct cfs_hash_bucket **buckets, { int i; - for (i = prev_size; i < size; i++) { - if (buckets[i]) - LIBCFS_FREE(buckets[i], bkt_size); - } + for (i = prev_size; i < size; i++) + kfree(buckets[i]); - LIBCFS_FREE(buckets, sizeof(buckets[0]) * size); + kvfree(buckets); } /* @@ -889,7 +887,7 @@ cfs_hash_buckets_realloc(struct cfs_hash *hs, struct cfs_hash_bucket **old_bkts, if (old_bkts && old_size == new_size) return old_bkts; - LIBCFS_ALLOC(new_bkts, sizeof(new_bkts[0]) * new_size); + new_bkts = kvmalloc_array(new_size, sizeof(new_bkts[0]), GFP_KERNEL); if (!new_bkts) return NULL; @@ -902,7 +900,7 @@ cfs_hash_buckets_realloc(struct cfs_hash *hs, struct cfs_hash_bucket **old_bkts, struct hlist_head *hhead; struct cfs_hash_bd bd; - LIBCFS_ALLOC(new_bkts[i], cfs_hash_bkt_size(hs)); + new_bkts[i] = kzalloc(cfs_hash_bkt_size(hs), GFP_KERNEL); if (!new_bkts[i]) { cfs_hash_buckets_free(new_bkts, cfs_hash_bkt_size(hs), old_size, new_size); @@ -1023,7 +1021,7 @@ cfs_hash_create(char *name, unsigned int cur_bits, unsigned int max_bits, len = !(flags & CFS_HASH_BIGNAME) ? CFS_HASH_NAME_LEN : CFS_HASH_BIGNAME_LEN; - LIBCFS_ALLOC(hs, offsetof(struct cfs_hash, hs_name[len])); + hs = kzalloc(offsetof(struct cfs_hash, hs_name[len]), GFP_KERNEL); if (!hs) return NULL; @@ -1055,7 +1053,7 @@ cfs_hash_create(char *name, unsigned int cur_bits, unsigned int max_bits, if (hs->hs_buckets) return hs; - LIBCFS_FREE(hs, offsetof(struct cfs_hash, hs_name[len])); + kfree(hs); return NULL; } EXPORT_SYMBOL(cfs_hash_create); @@ -1118,7 +1116,7 @@ cfs_hash_destroy(struct cfs_hash *hs) 0, CFS_HASH_NBKT(hs)); i = cfs_hash_with_bigname(hs) ? CFS_HASH_BIGNAME_LEN : CFS_HASH_NAME_LEN; - LIBCFS_FREE(hs, offsetof(struct cfs_hash, hs_name[i])); + kfree(hs); } struct cfs_hash *cfs_hash_getref(struct cfs_hash *hs) diff --git a/drivers/staging/lustre/lnet/libcfs/libcfs_mem.c b/drivers/staging/lustre/lnet/libcfs/libcfs_mem.c index df93d8f77ea2..23734cfb5d44 100644 --- a/drivers/staging/lustre/lnet/libcfs/libcfs_mem.c +++ b/drivers/staging/lustre/lnet/libcfs/libcfs_mem.c @@ -130,10 +130,9 @@ cfs_array_free(void *vars) if (!arr->va_ptrs[i]) continue; - LIBCFS_FREE(arr->va_ptrs[i], arr->va_size); + kvfree(arr->va_ptrs[i]); } - LIBCFS_FREE(arr, offsetof(struct cfs_var_array, - va_ptrs[arr->va_count])); + kvfree(arr); } EXPORT_SYMBOL(cfs_array_free); @@ -148,7 +147,7 @@ cfs_array_alloc(int count, unsigned int size) struct cfs_var_array *arr; int i; - LIBCFS_ALLOC(arr, offsetof(struct cfs_var_array, va_ptrs[count])); + arr = kvmalloc(offsetof(struct cfs_var_array, va_ptrs[count]), GFP_KERNEL); if (!arr) return NULL; @@ -156,7 +155,7 @@ cfs_array_alloc(int count, unsigned int size) arr->va_size = size; for (i = 0; i < count; i++) { - LIBCFS_ALLOC(arr->va_ptrs[i], size); + arr->va_ptrs[i] = kvzalloc(size, GFP_KERNEL); if (!arr->va_ptrs[i]) { cfs_array_free((void *)&arr->va_ptrs[0]); diff --git a/drivers/staging/lustre/lnet/libcfs/libcfs_string.c b/drivers/staging/lustre/lnet/libcfs/libcfs_string.c index 0c62855349e3..b1d8faa3f7aa 100644 --- a/drivers/staging/lustre/lnet/libcfs/libcfs_string.c +++ b/drivers/staging/lustre/lnet/libcfs/libcfs_string.c @@ -457,7 +457,7 @@ cfs_expr_list_values(struct cfs_expr_list *expr_list, int max, u32 **valpp) return -EINVAL; } - LIBCFS_ALLOC(val, sizeof(val[0]) * count); + val = kvmalloc_array(count, sizeof(val[0]), GFP_KERNEL | __GFP_ZERO); if (!val) return -ENOMEM; diff --git a/drivers/staging/lustre/lnet/lnet/config.c b/drivers/staging/lustre/lnet/lnet/config.c index 30fabfbc6898..66a222e5220b 100644 --- a/drivers/staging/lustre/lnet/lnet/config.c +++ b/drivers/staging/lustre/lnet/lnet/config.c @@ -170,7 +170,7 @@ lnet_ni_alloc(__u32 net, struct cfs_expr_list *el, struct list_head *nilist) LASSERT(rc <= LNET_CPT_NUMBER); if (rc == LNET_CPT_NUMBER) { - LIBCFS_FREE(ni->ni_cpts, rc * sizeof(ni->ni_cpts[0])); + cfs_expr_list_values_free(ni->ni_cpts, LNET_CPT_NUMBER); ni->ni_cpts = NULL; } From neilb at suse.com Mon Dec 18 00:46:30 2017 From: neilb at suse.com (NeilBrown) Date: Mon, 18 Dec 2017 11:46:30 +1100 Subject: [lustre-devel] [PATCH 12/15] staging: lustre: use kmalloc for allocating ksock_tx In-Reply-To: <151355781721.6200.2136335532722530242.stgit@noble> References: <151355781721.6200.2136335532722530242.stgit@noble> Message-ID: <151355799064.6200.14626350790590057942.stgit@noble> The size of the data structure is primarily controlled by the iovec size, which is limited to 256. Entries in this vector are 12 bytes, so the whole will always fit in a page. So it is safe to use kmalloc (kvmalloc not needed). So replace LIBCFS_ALLOC with kmalloc. Signed-off-by: NeilBrown --- .../staging/lustre/lnet/klnds/socklnd/socklnd.c | 2 +- .../staging/lustre/lnet/klnds/socklnd/socklnd_cb.c | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c index dc63ed2ceb97..7dba949a95a7 100644 --- a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c +++ b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c @@ -2282,7 +2282,7 @@ ksocknal_free_buffers(void) list_for_each_entry_safe(tx, temp, &zlist, tx_list) { list_del(&tx->tx_list); - LIBCFS_FREE(tx, tx->tx_desc_size); + kfree(tx); } } else { spin_unlock(&ksocknal_data.ksnd_tx_lock); diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c index 994b6693c6b7..11fd3a36424f 100644 --- a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c +++ b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c @@ -46,7 +46,7 @@ ksocknal_alloc_tx(int type, int size) } if (!tx) - LIBCFS_ALLOC(tx, size); + tx = kzalloc(size, GFP_NOFS); if (!tx) return NULL; @@ -102,7 +102,7 @@ ksocknal_free_tx(struct ksock_tx *tx) spin_unlock(&ksocknal_data.ksnd_tx_lock); } else { - LIBCFS_FREE(tx, tx->tx_desc_size); + kfree(tx); } } From neilb at suse.com Mon Dec 18 00:46:30 2017 From: neilb at suse.com (NeilBrown) Date: Mon, 18 Dec 2017 11:46:30 +1100 Subject: [lustre-devel] [PATCH 15/15] staging: lustre: remove LIBCFS_ALLOC and LIBCFS_ALLOC_ATOMIC In-Reply-To: <151355781721.6200.2136335532722530242.stgit@noble> References: <151355781721.6200.2136335532722530242.stgit@noble> Message-ID: <151355799074.6200.8015775299754662811.stgit@noble> These are no longer used. Signed-off-by: NeilBrown --- .../lustre/include/linux/libcfs/libcfs_private.h | 19 ------------------- 1 file changed, 19 deletions(-) diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_private.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_private.h index d230c7f7cced..35ed07ff7c71 100644 --- a/drivers/staging/lustre/include/linux/libcfs/libcfs_private.h +++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_private.h @@ -85,25 +85,6 @@ do { \ } \ } while (0) -/** - * default allocator - */ -#define LIBCFS_ALLOC(ptr, size) \ -do { \ - LASSERT(!in_interrupt()); \ - (ptr) = kvmalloc((size), GFP_NOFS); \ - LIBCFS_ALLOC_POST((ptr), (size)); \ -} while (0) - -/** - * non-sleeping allocator - */ -#define LIBCFS_ALLOC_ATOMIC(ptr, size) \ -do { \ - (ptr) = kmalloc((size), GFP_ATOMIC); \ - LIBCFS_ALLOC_POST(ptr, size); \ -} while (0) - /** * allocate memory for specified CPU partition * \a cptab != NULL, \a cpt is CPU partition id of \a cptab From neilb at suse.com Mon Dec 18 00:46:30 2017 From: neilb at suse.com (NeilBrown) Date: Mon, 18 Dec 2017 11:46:30 +1100 Subject: [lustre-devel] [PATCH 13/15] staging: lustre: cfs_percpt_alloc: use kvmalloc(GFP_KERNEL) In-Reply-To: <151355781721.6200.2136335532722530242.stgit@noble> References: <151355781721.6200.2136335532722530242.stgit@noble> Message-ID: <151355799068.6200.10665584403840611965.stgit@noble> this allocation is called from several places, but all are during initialization, so GFP_NOFS is not needed. So use kvmalloc and GFP_KERNEL. Signed-off-by: NeilBrown --- drivers/staging/lustre/lnet/libcfs/libcfs_mem.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/drivers/staging/lustre/lnet/libcfs/libcfs_mem.c b/drivers/staging/lustre/lnet/libcfs/libcfs_mem.c index 23734cfb5d44..8e2b4f1db0a1 100644 --- a/drivers/staging/lustre/lnet/libcfs/libcfs_mem.c +++ b/drivers/staging/lustre/lnet/libcfs/libcfs_mem.c @@ -54,8 +54,7 @@ cfs_percpt_free(void *vars) LIBCFS_FREE(arr->va_ptrs[i], arr->va_size); } - LIBCFS_FREE(arr, offsetof(struct cfs_var_array, - va_ptrs[arr->va_count])); + kvfree(arr); } EXPORT_SYMBOL(cfs_percpt_free); @@ -79,7 +78,8 @@ cfs_percpt_alloc(struct cfs_cpt_table *cptab, unsigned int size) count = cfs_cpt_number(cptab); - LIBCFS_ALLOC(arr, offsetof(struct cfs_var_array, va_ptrs[count])); + arr = kvzalloc(offsetof(struct cfs_var_array, va_ptrs[count]), + GFP_KERNEL); if (!arr) return NULL; From neilb at suse.com Mon Dec 18 00:46:30 2017 From: neilb at suse.com (NeilBrown) Date: Mon, 18 Dec 2017 11:46:30 +1100 Subject: [lustre-devel] [PATCH 14/15] staging: lustre: opencode LIBCFS_ALLOC_ATOMIC calls. In-Reply-To: <151355781721.6200.2136335532722530242.stgit@noble> References: <151355781721.6200.2136335532722530242.stgit@noble> Message-ID: <151355799072.6200.1169111940083599054.stgit@noble> Just call kzalloc(GFP_ATOMIC) directly. We don't need the warning on failure. Signed-off-by: NeilBrown --- .../lustre/lustre/obdclass/lprocfs_status.c | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c b/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c index 85483a38c6c4..e1f4ef2bddd4 100644 --- a/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c +++ b/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c @@ -1093,7 +1093,7 @@ int lprocfs_stats_alloc_one(struct lprocfs_stats *stats, unsigned int cpuid) LASSERT((stats->ls_flags & LPROCFS_STATS_FLAG_NOPERCPU) == 0); percpusize = lprocfs_stats_counter_size(stats); - LIBCFS_ALLOC_ATOMIC(stats->ls_percpu[cpuid], percpusize); + stats->ls_percpu[cpuid] = kzalloc(percpusize, GFP_ATOMIC); if (stats->ls_percpu[cpuid]) { rc = 0; if (unlikely(stats->ls_biggest_alloc_num <= cpuid)) { @@ -1156,7 +1156,7 @@ struct lprocfs_stats *lprocfs_alloc_stats(unsigned int num, if ((flags & LPROCFS_STATS_FLAG_NOPERCPU) != 0) { /* contains only one set counters */ percpusize = lprocfs_stats_counter_size(stats); - LIBCFS_ALLOC_ATOMIC(stats->ls_percpu[0], percpusize); + stats->ls_percpu[0] = kzalloc(percpusize, GFP_ATOMIC); if (!stats->ls_percpu[0]) goto fail; stats->ls_biggest_alloc_num = 1; @@ -1193,8 +1193,7 @@ void lprocfs_free_stats(struct lprocfs_stats **statsh) percpusize = lprocfs_stats_counter_size(stats); for (i = 0; i < num_entry; i++) - if (stats->ls_percpu[i]) - LIBCFS_FREE(stats->ls_percpu[i], percpusize); + kfree(stats->ls_percpu[i]); kvfree(stats->ls_cnt_header); kvfree(stats); } From neilb at suse.com Mon Dec 18 01:13:20 2017 From: neilb at suse.com (NeilBrown) Date: Mon, 18 Dec 2017 12:13:20 +1100 Subject: [lustre-devel] [PATCH SERIES 2: 0/4] staging:lustre: improve mounting code Message-ID: <151355938405.10227.11903161187102828189.stgit@noble> These patches make improvements to the lustre code for supporting the mount system call. There is a user-visible change: "lustre" now appears in /proc/filesystem with the "nodev" flag present. Previously it didn't have this flag, which was incorrect. There is also a bugfix: there is a possible module-unload race that could trigger a crash. Thanks, NeilBrown --- NeilBrown (4): staging: lustre: obdclass: discard FS_NEEDS_DEV flag. staging: lustre: obdclass: remove pointless struct lustre_mount_data2 staging: lustre: obdclass: remove vfsmount option from client_fill_super staging: lustre: obd_mount: fix possible race with module unload. .../staging/lustre/lustre/include/lustre_disk.h | 6 +- .../staging/lustre/lustre/llite/llite_internal.h | 2 - drivers/staging/lustre/lustre/llite/llite_lib.c | 7 +- drivers/staging/lustre/lustre/llite/super25.c | 6 +- drivers/staging/lustre/lustre/obdclass/obd_mount.c | 63 +++++++++----------- 5 files changed, 38 insertions(+), 46 deletions(-) -- Signature From neilb at suse.com Mon Dec 18 01:13:20 2017 From: neilb at suse.com (NeilBrown) Date: Mon, 18 Dec 2017 12:13:20 +1100 Subject: [lustre-devel] [PATCH 2/4] staging: lustre: obdclass: remove pointless struct lustre_mount_data2 In-Reply-To: <151355938405.10227.11903161187102828189.stgit@noble> References: <151355938405.10227.11903161187102828189.stgit@noble> Message-ID: <151355960074.10227.13015552450143881635.stgit@noble> This is used to pass a void* and NULL to lustre_fill_super(). It is easier just to pass the void*. Signed-off-by: NeilBrown --- drivers/staging/lustre/lustre/obdclass/obd_mount.c | 19 ++++--------------- 1 file changed, 4 insertions(+), 15 deletions(-) diff --git a/drivers/staging/lustre/lustre/obdclass/obd_mount.c b/drivers/staging/lustre/lustre/obdclass/obd_mount.c index d49dc72ccba2..c98c7716a910 100644 --- a/drivers/staging/lustre/lustre/obdclass/obd_mount.c +++ b/drivers/staging/lustre/lustre/obdclass/obd_mount.c @@ -1107,20 +1107,14 @@ static int lmd_parse(char *options, struct lustre_mount_data *lmd) return -EINVAL; } -struct lustre_mount_data2 { - void *lmd2_data; - struct vfsmount *lmd2_mnt; -}; - /** This is the entry point for the mount call into Lustre. * This is called when a server or client is mounted, * and this is where we start setting things up. * @param data Mount options (e.g. -o flock,abort_recov) */ -static int lustre_fill_super(struct super_block *sb, void *data, int silent) +static int lustre_fill_super(struct super_block *sb, void *lmd2_data, int silent) { struct lustre_mount_data *lmd; - struct lustre_mount_data2 *lmd2 = data; struct lustre_sb_info *lsi; int rc; @@ -1143,7 +1137,7 @@ static int lustre_fill_super(struct super_block *sb, void *data, int silent) obd_zombie_barrier(); /* Figure out the lmd from the mount options */ - if (lmd_parse((lmd2->lmd2_data), lmd)) { + if (lmd_parse(lmd2_data, lmd)) { lustre_put_lsi(sb); rc = -EINVAL; goto out; @@ -1165,7 +1159,7 @@ static int lustre_fill_super(struct super_block *sb, void *data, int silent) } /* Connect and start */ /* (should always be ll_fill_super) */ - rc = (*client_fill_super)(sb, lmd2->lmd2_mnt); + rc = (*client_fill_super)(sb, NULL); /* c_f_s will call lustre_common_put_super on failure */ } } else { @@ -1209,12 +1203,7 @@ EXPORT_SYMBOL(lustre_register_kill_super_cb); static struct dentry *lustre_mount(struct file_system_type *fs_type, int flags, const char *devname, void *data) { - struct lustre_mount_data2 lmd2 = { - .lmd2_data = data, - .lmd2_mnt = NULL - }; - - return mount_nodev(fs_type, flags, &lmd2, lustre_fill_super); + return mount_nodev(fs_type, flags, data, lustre_fill_super); } static void lustre_kill_super(struct super_block *sb) From neilb at suse.com Mon Dec 18 01:13:20 2017 From: neilb at suse.com (NeilBrown) Date: Mon, 18 Dec 2017 12:13:20 +1100 Subject: [lustre-devel] [PATCH 1/4] staging: lustre: obdclass: discard FS_NEEDS_DEV flag. In-Reply-To: <151355938405.10227.11903161187102828189.stgit@noble> References: <151355938405.10227.11903161187102828189.stgit@noble> Message-ID: <151355960071.10227.7678385187334068429.stgit@noble> Lustre mounts do not need a dev, as we can see from lustre_mount() calling mount_nodev(). So remove the flag which could cause confusion elsewhere. Also format lustre_fs_types so that fields line up. Signed-off-by: NeilBrown --- drivers/staging/lustre/lustre/obdclass/obd_mount.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/drivers/staging/lustre/lustre/obdclass/obd_mount.c b/drivers/staging/lustre/lustre/obdclass/obd_mount.c index 2a79a223b98a..d49dc72ccba2 100644 --- a/drivers/staging/lustre/lustre/obdclass/obd_mount.c +++ b/drivers/staging/lustre/lustre/obdclass/obd_mount.c @@ -1230,11 +1230,11 @@ static void lustre_kill_super(struct super_block *sb) /** Register the "lustre" fs type */ static struct file_system_type lustre_fs_type = { - .owner = THIS_MODULE, - .name = "lustre", - .mount = lustre_mount, - .kill_sb = lustre_kill_super, - .fs_flags = FS_REQUIRES_DEV | FS_RENAME_DOES_D_MOVE, + .owner = THIS_MODULE, + .name = "lustre", + .mount = lustre_mount, + .kill_sb = lustre_kill_super, + .fs_flags = FS_RENAME_DOES_D_MOVE, }; MODULE_ALIAS_FS("lustre"); From neilb at suse.com Mon Dec 18 01:13:20 2017 From: neilb at suse.com (NeilBrown) Date: Mon, 18 Dec 2017 12:13:20 +1100 Subject: [lustre-devel] [PATCH 4/4] staging: lustre: obd_mount: fix possible race with module unload. In-Reply-To: <151355938405.10227.11903161187102828189.stgit@noble> References: <151355938405.10227.11903161187102828189.stgit@noble> Message-ID: <151355960081.10227.4261861922935137504.stgit@noble> lustre_fill_super() calls client_fill_super() without holding a reference to the module containing client_fill_super. If that module is unloaded at a bad time, this can crash. To be able to get a reference to the module using try_get_module(), we need a pointer to the module. So replace lustre_register_client_fill_super() and lustre_register_kill_super_cb() with a single lustre_register_super_ops() which also passed a module pointer. Then use a spinlock to ensure the module pointer isn't removed while try_module_get() is running, and use try_module_get() to ensure we have a reference before calling client_fill_super(). Signed-off-by: NeilBrown --- .../staging/lustre/lustre/include/lustre_disk.h | 5 ++- drivers/staging/lustre/lustre/llite/super25.c | 6 +--- drivers/staging/lustre/lustre/obdclass/obd_mount.c | 30 +++++++++++++------- 3 files changed, 24 insertions(+), 17 deletions(-) diff --git a/drivers/staging/lustre/lustre/include/lustre_disk.h b/drivers/staging/lustre/lustre/include/lustre_disk.h index 5089ec02dcad..100e993ab00b 100644 --- a/drivers/staging/lustre/lustre/include/lustre_disk.h +++ b/drivers/staging/lustre/lustre/include/lustre_disk.h @@ -141,8 +141,9 @@ struct lustre_sb_info { /* obd_mount.c */ int lustre_start_mgc(struct super_block *sb); -void lustre_register_client_fill_super(int (*cfs)(struct super_block *sb)); -void lustre_register_kill_super_cb(void (*cfs)(struct super_block *sb)); +void lustre_register_super_ops(struct module *mod, + int (*cfs)(struct super_block *sb), + void (*ksc)(struct super_block *sb)); int lustre_common_put_super(struct super_block *sb); int mgc_fsname2resid(char *fsname, struct ldlm_res_id *res_id, int type); diff --git a/drivers/staging/lustre/lustre/llite/super25.c b/drivers/staging/lustre/lustre/llite/super25.c index 0bda111a096e..10105339790e 100644 --- a/drivers/staging/lustre/lustre/llite/super25.c +++ b/drivers/staging/lustre/lustre/llite/super25.c @@ -159,8 +159,7 @@ static int __init lustre_init(void) if (rc != 0) goto out_inode_fini_env; - lustre_register_client_fill_super(ll_fill_super); - lustre_register_kill_super_cb(ll_kill_super); + lustre_register_super_ops(THIS_MODULE, ll_fill_super, ll_kill_super); lustre_register_client_process_config(ll_process_config); return 0; @@ -181,8 +180,7 @@ static int __init lustre_init(void) static void __exit lustre_exit(void) { - lustre_register_client_fill_super(NULL); - lustre_register_kill_super_cb(NULL); + lustre_register_super_ops(NULL, NULL, NULL); lustre_register_client_process_config(NULL); debugfs_remove(llite_root); diff --git a/drivers/staging/lustre/lustre/obdclass/obd_mount.c b/drivers/staging/lustre/lustre/obdclass/obd_mount.c index 8068cbbc8390..acc1ea773c9c 100644 --- a/drivers/staging/lustre/lustre/obdclass/obd_mount.c +++ b/drivers/staging/lustre/lustre/obdclass/obd_mount.c @@ -49,8 +49,9 @@ #include #include +static DEFINE_SPINLOCK(client_lock); +static struct module *client_mod; static int (*client_fill_super)(struct super_block *sb); - static void (*kill_super_cb)(struct super_block *sb); /**************** config llog ********************/ @@ -1143,10 +1144,15 @@ static int lustre_fill_super(struct super_block *sb, void *lmd2_data, int silent } if (lmd_is_client(lmd)) { + bool have_client = false; CDEBUG(D_MOUNT, "Mounting client %s\n", lmd->lmd_profile); if (!client_fill_super) request_module("lustre"); - if (!client_fill_super) { + spin_lock(&client_lock); + if (client_fill_super && try_module_get(client_mod)) + have_client = true; + spin_unlock(&client_lock); + if (!have_client) { LCONSOLE_ERROR_MSG(0x165, "Nothing registered for client mount! Is the 'lustre' module loaded?\n"); lustre_put_lsi(sb); rc = -ENODEV; @@ -1159,7 +1165,9 @@ static int lustre_fill_super(struct super_block *sb, void *lmd2_data, int silent /* Connect and start */ /* (should always be ll_fill_super) */ rc = (*client_fill_super)(sb); - /* c_f_s will call lustre_common_put_super on failure */ + /* c_f_s will call lustre_common_put_super on failure, otherwise + * c_f_s will have taken another reference to the module */ + module_put(client_mod); } } else { CERROR("This is client-side-only module, cannot handle server mount.\n"); @@ -1185,17 +1193,17 @@ static int lustre_fill_super(struct super_block *sb, void *lmd2_data, int silent /* We can't call ll_fill_super by name because it lives in a module that * must be loaded after this one. */ -void lustre_register_client_fill_super(int (*cfs)(struct super_block *sb)) +void lustre_register_super_ops(struct module *mod, + int (*cfs)(struct super_block *sb), + void (*ksc)(struct super_block *sb)) { + spin_lock(&client_lock); + client_mod = mod; client_fill_super = cfs; + kill_super_cb = ksc; + spin_unlock(&client_lock); } -EXPORT_SYMBOL(lustre_register_client_fill_super); - -void lustre_register_kill_super_cb(void (*cfs)(struct super_block *sb)) -{ - kill_super_cb = cfs; -} -EXPORT_SYMBOL(lustre_register_kill_super_cb); +EXPORT_SYMBOL(lustre_register_super_ops); /***************** FS registration ******************/ static struct dentry *lustre_mount(struct file_system_type *fs_type, int flags, From neilb at suse.com Mon Dec 18 01:13:20 2017 From: neilb at suse.com (NeilBrown) Date: Mon, 18 Dec 2017 12:13:20 +1100 Subject: [lustre-devel] [PATCH 3/4] staging: lustre: obdclass: remove vfsmount option from client_fill_super In-Reply-To: <151355938405.10227.11903161187102828189.stgit@noble> References: <151355938405.10227.11903161187102828189.stgit@noble> Message-ID: <151355960078.10227.13624890998823000934.stgit@noble> This arg is always NULL and is never used. So discard it from this and related functions. Signed-off-by: NeilBrown --- .../staging/lustre/lustre/include/lustre_disk.h | 3 +-- .../staging/lustre/lustre/llite/llite_internal.h | 2 +- drivers/staging/lustre/lustre/llite/llite_lib.c | 7 +++---- drivers/staging/lustre/lustre/obdclass/obd_mount.c | 8 +++----- 4 files changed, 8 insertions(+), 12 deletions(-) diff --git a/drivers/staging/lustre/lustre/include/lustre_disk.h b/drivers/staging/lustre/lustre/include/lustre_disk.h index 8f1a22527006..5089ec02dcad 100644 --- a/drivers/staging/lustre/lustre/include/lustre_disk.h +++ b/drivers/staging/lustre/lustre/include/lustre_disk.h @@ -141,8 +141,7 @@ struct lustre_sb_info { /* obd_mount.c */ int lustre_start_mgc(struct super_block *sb); -void lustre_register_client_fill_super(int (*cfs)(struct super_block *sb, - struct vfsmount *mnt)); +void lustre_register_client_fill_super(int (*cfs)(struct super_block *sb)); void lustre_register_kill_super_cb(void (*cfs)(struct super_block *sb)); int lustre_common_put_super(struct super_block *sb); diff --git a/drivers/staging/lustre/lustre/llite/llite_internal.h b/drivers/staging/lustre/lustre/llite/llite_internal.h index b133fd00c08c..99e17d0cb3d6 100644 --- a/drivers/staging/lustre/lustre/llite/llite_internal.h +++ b/drivers/staging/lustre/lustre/llite/llite_internal.h @@ -792,7 +792,7 @@ int ll_revalidate_it_finish(struct ptlrpc_request *request, extern struct super_operations lustre_super_operations; void ll_lli_init(struct ll_inode_info *lli); -int ll_fill_super(struct super_block *sb, struct vfsmount *mnt); +int ll_fill_super(struct super_block *sb); void ll_put_super(struct super_block *sb); void ll_kill_super(struct super_block *sb); struct inode *ll_inode_from_resource_lock(struct ldlm_lock *lock); diff --git a/drivers/staging/lustre/lustre/llite/llite_lib.c b/drivers/staging/lustre/lustre/llite/llite_lib.c index e84719662edf..6735a6f006d2 100644 --- a/drivers/staging/lustre/lustre/llite/llite_lib.c +++ b/drivers/staging/lustre/lustre/llite/llite_lib.c @@ -146,8 +146,7 @@ static void ll_free_sbi(struct super_block *sb) kfree(sbi); } -static int client_common_fill_super(struct super_block *sb, char *md, char *dt, - struct vfsmount *mnt) +static int client_common_fill_super(struct super_block *sb, char *md, char *dt) { struct inode *root = NULL; struct ll_sb_info *sbi = ll_s2sbi(sb); @@ -867,7 +866,7 @@ void ll_lli_init(struct ll_inode_info *lli) mutex_init(&lli->lli_layout_mutex); } -int ll_fill_super(struct super_block *sb, struct vfsmount *mnt) +int ll_fill_super(struct super_block *sb) { struct lustre_profile *lprof = NULL; struct lustre_sb_info *lsi = s2lsi(sb); @@ -944,7 +943,7 @@ int ll_fill_super(struct super_block *sb, struct vfsmount *mnt) } /* connections, registrations, sb setup */ - err = client_common_fill_super(sb, md, dt, mnt); + err = client_common_fill_super(sb, md, dt); if (!err) sbi->ll_client_common_fill_super_succeeded = 1; diff --git a/drivers/staging/lustre/lustre/obdclass/obd_mount.c b/drivers/staging/lustre/lustre/obdclass/obd_mount.c index c98c7716a910..8068cbbc8390 100644 --- a/drivers/staging/lustre/lustre/obdclass/obd_mount.c +++ b/drivers/staging/lustre/lustre/obdclass/obd_mount.c @@ -49,8 +49,7 @@ #include #include -static int (*client_fill_super)(struct super_block *sb, - struct vfsmount *mnt); +static int (*client_fill_super)(struct super_block *sb); static void (*kill_super_cb)(struct super_block *sb); @@ -1159,7 +1158,7 @@ static int lustre_fill_super(struct super_block *sb, void *lmd2_data, int silent } /* Connect and start */ /* (should always be ll_fill_super) */ - rc = (*client_fill_super)(sb, NULL); + rc = (*client_fill_super)(sb); /* c_f_s will call lustre_common_put_super on failure */ } } else { @@ -1186,8 +1185,7 @@ static int lustre_fill_super(struct super_block *sb, void *lmd2_data, int silent /* We can't call ll_fill_super by name because it lives in a module that * must be loaded after this one. */ -void lustre_register_client_fill_super(int (*cfs)(struct super_block *sb, - struct vfsmount *mnt)) +void lustre_register_client_fill_super(int (*cfs)(struct super_block *sb)) { client_fill_super = cfs; } From neilb at suse.com Mon Dec 18 01:25:19 2017 From: neilb at suse.com (NeilBrown) Date: Mon, 18 Dec 2017 12:25:19 +1100 Subject: [lustre-devel] [PATCH SERIES 3: 0/4] staging:lustre: remove workitem code Message-ID: <151356013394.15912.9645528414310488005.stgit@noble> Lustre has a "workitem" subsystem with much the same functionality as the Linux workqueue subsystem. This patch converts all users of workitem to workqueue, then removes workitem. This requires that "apply_workqueue_attrs" be exported to modules, so this intro and the patch which does the EXPORT_SYMBOL_GPL are cc:ed to Tejun and Lai. Thanks, NeilBrown --- NeilBrown (4): staging: lustre: libcfs: use a workqueue for rehash work. staging: lustre: libcfs: remove wi_data from cfs_workitem staging: lustre: lnet: convert selftest to use workqueues staging: lustre: libcfs: remove workitem code. .../staging/lustre/include/linux/libcfs/libcfs.h | 3 .../lustre/include/linux/libcfs/libcfs_hash.h | 6 .../lustre/include/linux/libcfs/libcfs_workitem.h | 107 ----- drivers/staging/lustre/lnet/libcfs/Makefile | 2 drivers/staging/lustre/lnet/libcfs/hash.c | 82 +--- drivers/staging/lustre/lnet/libcfs/module.c | 27 - drivers/staging/lustre/lnet/libcfs/workitem.c | 466 -------------------- drivers/staging/lustre/lnet/selftest/framework.c | 14 - drivers/staging/lustre/lnet/selftest/module.c | 39 +- drivers/staging/lustre/lnet/selftest/rpc.c | 71 +-- drivers/staging/lustre/lnet/selftest/selftest.h | 44 +- kernel/workqueue.c | 1 12 files changed, 111 insertions(+), 751 deletions(-) delete mode 100644 drivers/staging/lustre/include/linux/libcfs/libcfs_workitem.h delete mode 100644 drivers/staging/lustre/lnet/libcfs/workitem.c -- Signature From neilb at suse.com Mon Dec 18 01:25:19 2017 From: neilb at suse.com (NeilBrown) Date: Mon, 18 Dec 2017 12:25:19 +1100 Subject: [lustre-devel] [PATCH 2/4] staging: lustre: libcfs: remove wi_data from cfs_workitem In-Reply-To: <151356013394.15912.9645528414310488005.stgit@noble> References: <151356013394.15912.9645528414310488005.stgit@noble> Message-ID: <151356031915.15912.4949412570489146428.stgit@noble> In every case, the value passed via wi_data can be determined from the cfs_workitem pointer using container_of(). So use container_of(), and discard wi_data. Signed-off-by: NeilBrown --- .../lustre/include/linux/libcfs/libcfs_workitem.h | 5 +---- drivers/staging/lustre/lnet/selftest/framework.c | 4 ++-- drivers/staging/lustre/lnet/selftest/rpc.c | 10 +++++----- drivers/staging/lustre/lnet/selftest/selftest.h | 6 +++--- 4 files changed, 11 insertions(+), 14 deletions(-) diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_workitem.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_workitem.h index fc780f608e57..ddaca33a13ac 100644 --- a/drivers/staging/lustre/include/linux/libcfs/libcfs_workitem.h +++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_workitem.h @@ -75,8 +75,6 @@ struct cfs_workitem { struct list_head wi_list; /** working function */ cfs_wi_action_t wi_action; - /** arg for working function */ - void *wi_data; /** in running */ unsigned short wi_running:1; /** scheduled */ @@ -84,13 +82,12 @@ struct cfs_workitem { }; static inline void -cfs_wi_init(struct cfs_workitem *wi, void *data, cfs_wi_action_t action) +cfs_wi_init(struct cfs_workitem *wi, cfs_wi_action_t action) { INIT_LIST_HEAD(&wi->wi_list); wi->wi_running = 0; wi->wi_scheduled = 0; - wi->wi_data = data; wi->wi_action = action; } diff --git a/drivers/staging/lustre/lnet/selftest/framework.c b/drivers/staging/lustre/lnet/selftest/framework.c index 4592ece98d95..2e1126552e18 100644 --- a/drivers/staging/lustre/lnet/selftest/framework.c +++ b/drivers/staging/lustre/lnet/selftest/framework.c @@ -944,7 +944,7 @@ sfw_create_test_rpc(struct sfw_test_unit *tsu, struct lnet_process_id peer, static int sfw_run_test(struct swi_workitem *wi) { - struct sfw_test_unit *tsu = wi->swi_workitem.wi_data; + struct sfw_test_unit *tsu = container_of(wi, struct sfw_test_unit, tsu_worker); struct sfw_test_instance *tsi = tsu->tsu_instance; struct srpc_client_rpc *rpc = NULL; @@ -1016,7 +1016,7 @@ sfw_run_batch(struct sfw_batch *tsb) atomic_inc(&tsi->tsi_nactive); tsu->tsu_loop = tsi->tsi_loop; wi = &tsu->tsu_worker; - swi_init_workitem(wi, tsu, sfw_run_test, + swi_init_workitem(wi, sfw_run_test, lst_sched_test[lnet_cpt_of_nid(tsu->tsu_dest.nid)]); swi_schedule_workitem(wi); } diff --git a/drivers/staging/lustre/lnet/selftest/rpc.c b/drivers/staging/lustre/lnet/selftest/rpc.c index e195b9ee544c..4ebb5a1107be 100644 --- a/drivers/staging/lustre/lnet/selftest/rpc.c +++ b/drivers/staging/lustre/lnet/selftest/rpc.c @@ -176,7 +176,7 @@ srpc_init_server_rpc(struct srpc_server_rpc *rpc, struct srpc_buffer *buffer) { memset(rpc, 0, sizeof(*rpc)); - swi_init_workitem(&rpc->srpc_wi, rpc, srpc_handle_rpc, + swi_init_workitem(&rpc->srpc_wi, srpc_handle_rpc, srpc_serv_is_framework(scd->scd_svc) ? lst_sched_serial : lst_sched_test[scd->scd_cpt]); @@ -280,7 +280,7 @@ srpc_service_init(struct srpc_service *svc) * NB: don't use lst_sched_serial for adding buffer, * see details in srpc_service_add_buffers() */ - swi_init_workitem(&scd->scd_buf_wi, scd, + swi_init_workitem(&scd->scd_buf_wi, srpc_add_buffer, lst_sched_test[i]); if (i && srpc_serv_is_framework(svc)) { @@ -517,7 +517,7 @@ __must_hold(&scd->scd_lock) int srpc_add_buffer(struct swi_workitem *wi) { - struct srpc_service_cd *scd = wi->swi_workitem.wi_data; + struct srpc_service_cd *scd = container_of(wi, struct srpc_service_cd, scd_buf_wi); struct srpc_buffer *buf; int rc = 0; @@ -968,7 +968,7 @@ srpc_server_rpc_done(struct srpc_server_rpc *rpc, int status) int srpc_handle_rpc(struct swi_workitem *wi) { - struct srpc_server_rpc *rpc = wi->swi_workitem.wi_data; + struct srpc_server_rpc *rpc = container_of(wi, struct srpc_server_rpc, srpc_wi); struct srpc_service_cd *scd = rpc->srpc_scd; struct srpc_service *sv = scd->scd_svc; struct srpc_event *ev = &rpc->srpc_ev; @@ -1188,7 +1188,7 @@ srpc_send_rpc(struct swi_workitem *wi) LASSERT(wi); - rpc = wi->swi_workitem.wi_data; + rpc = container_of(wi, struct srpc_client_rpc, crpc_wi); LASSERT(rpc); LASSERT(wi == &rpc->crpc_wi); diff --git a/drivers/staging/lustre/lnet/selftest/selftest.h b/drivers/staging/lustre/lnet/selftest/selftest.h index b23a953d8efe..465417263ef1 100644 --- a/drivers/staging/lustre/lnet/selftest/selftest.h +++ b/drivers/staging/lustre/lnet/selftest/selftest.h @@ -476,13 +476,13 @@ swi_wi_action(struct cfs_workitem *wi) } static inline void -swi_init_workitem(struct swi_workitem *swi, void *data, +swi_init_workitem(struct swi_workitem *swi, swi_action_t action, struct cfs_wi_sched *sched) { swi->swi_sched = sched; swi->swi_action = action; swi->swi_state = SWI_STATE_NEWBORN; - cfs_wi_init(&swi->swi_workitem, data, swi_wi_action); + cfs_wi_init(&swi->swi_workitem, swi_wi_action); } static inline void @@ -533,7 +533,7 @@ srpc_init_client_rpc(struct srpc_client_rpc *rpc, struct lnet_process_id peer, crpc_bulk.bk_iovs[nbulkiov])); INIT_LIST_HEAD(&rpc->crpc_list); - swi_init_workitem(&rpc->crpc_wi, rpc, srpc_send_rpc, + swi_init_workitem(&rpc->crpc_wi, srpc_send_rpc, lst_sched_test[lnet_cpt_of_nid(peer.nid)]); spin_lock_init(&rpc->crpc_lock); atomic_set(&rpc->crpc_refcount, 1); /* 1 ref for caller */ From neilb at suse.com Mon Dec 18 01:25:19 2017 From: neilb at suse.com (NeilBrown) Date: Mon, 18 Dec 2017 12:25:19 +1100 Subject: [lustre-devel] [PATCH 1/4] staging: lustre: libcfs: use a workqueue for rehash work. In-Reply-To: <151356013394.15912.9645528414310488005.stgit@noble> References: <151356013394.15912.9645528414310488005.stgit@noble> Message-ID: <151356031911.15912.4212899869566992358.stgit@noble> lustre has a work-item queuing scheme that provides the same functionality as linux work_queues. To make the code easier for linux devs to follow, change to use work_queues. Signed-off-by: NeilBrown --- .../staging/lustre/include/linux/libcfs/libcfs.h | 2 .../lustre/include/linux/libcfs/libcfs_hash.h | 6 + drivers/staging/lustre/lnet/libcfs/hash.c | 82 +++++--------------- drivers/staging/lustre/lnet/libcfs/module.c | 16 ++-- 4 files changed, 31 insertions(+), 75 deletions(-) diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs.h b/drivers/staging/lustre/include/linux/libcfs/libcfs.h index 6ad8867e5451..dcdc05dde6b8 100644 --- a/drivers/staging/lustre/include/linux/libcfs/libcfs.h +++ b/drivers/staging/lustre/include/linux/libcfs/libcfs.h @@ -126,7 +126,7 @@ extern struct miscdevice libcfs_dev; */ extern char lnet_debug_log_upcall[1024]; -extern struct cfs_wi_sched *cfs_sched_rehash; +extern struct workqueue_struct *cfs_rehash_wq; struct lnet_debugfs_symlink_def { char *name; diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_hash.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_hash.h index 5a27220cc608..0506f1d45757 100644 --- a/drivers/staging/lustre/include/linux/libcfs/libcfs_hash.h +++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_hash.h @@ -248,7 +248,7 @@ struct cfs_hash { /** # of iterators (caller of cfs_hash_for_each_*) */ u32 hs_iterators; /** rehash workitem */ - struct cfs_workitem hs_rehash_wi; + struct work_struct hs_rehash_work; /** refcount on this hash table */ atomic_t hs_refcount; /** rehash buckets-table */ @@ -265,7 +265,7 @@ struct cfs_hash { /** bits when we found the max depth */ unsigned int hs_dep_bits; /** workitem to output max depth */ - struct cfs_workitem hs_dep_wi; + struct work_struct hs_dep_work; #endif /** name of htable */ char hs_name[0]; @@ -738,7 +738,7 @@ u64 cfs_hash_size_get(struct cfs_hash *hs); */ void cfs_hash_rehash_cancel_locked(struct cfs_hash *hs); void cfs_hash_rehash_cancel(struct cfs_hash *hs); -int cfs_hash_rehash(struct cfs_hash *hs, int do_rehash); +void cfs_hash_rehash(struct cfs_hash *hs, int do_rehash); void cfs_hash_rehash_key(struct cfs_hash *hs, const void *old_key, void *new_key, struct hlist_node *hnode); diff --git a/drivers/staging/lustre/lnet/libcfs/hash.c b/drivers/staging/lustre/lnet/libcfs/hash.c index aabe29eef85c..f7b3c9306456 100644 --- a/drivers/staging/lustre/lnet/libcfs/hash.c +++ b/drivers/staging/lustre/lnet/libcfs/hash.c @@ -114,7 +114,7 @@ module_param(warn_on_depth, uint, 0644); MODULE_PARM_DESC(warn_on_depth, "warning when hash depth is high."); #endif -struct cfs_wi_sched *cfs_sched_rehash; +struct workqueue_struct *cfs_rehash_wq; static inline void cfs_hash_nl_lock(union cfs_hash_lock *lock, int exclusive) {} @@ -519,7 +519,7 @@ cfs_hash_bd_dep_record(struct cfs_hash *hs, struct cfs_hash_bd *bd, int dep_cur) hs->hs_dep_bits = hs->hs_cur_bits; spin_unlock(&hs->hs_dep_lock); - cfs_wi_schedule(cfs_sched_rehash, &hs->hs_dep_wi); + queue_work(cfs_rehash_wq, &hs->hs_dep_work); # endif } @@ -937,12 +937,12 @@ cfs_hash_buckets_realloc(struct cfs_hash *hs, struct cfs_hash_bucket **old_bkts, * @flags - CFS_HASH_REHASH enable synamic hash resizing * - CFS_HASH_SORT enable chained hash sort */ -static int cfs_hash_rehash_worker(struct cfs_workitem *wi); +static void cfs_hash_rehash_worker(struct work_struct *work); #if CFS_HASH_DEBUG_LEVEL >= CFS_HASH_DEBUG_1 -static int cfs_hash_dep_print(struct cfs_workitem *wi) +static void cfs_hash_dep_print(struct work_struct *work) { - struct cfs_hash *hs = container_of(wi, struct cfs_hash, hs_dep_wi); + struct cfs_hash *hs = container_of(work, struct cfs_hash, hs_dep_work); int dep; int bkt; int off; @@ -966,21 +966,12 @@ static int cfs_hash_dep_print(struct cfs_workitem *wi) static void cfs_hash_depth_wi_init(struct cfs_hash *hs) { spin_lock_init(&hs->hs_dep_lock); - cfs_wi_init(&hs->hs_dep_wi, hs, cfs_hash_dep_print); + INIT_WORK(&hs->hs_dep_work, cfs_hash_dep_print); } static void cfs_hash_depth_wi_cancel(struct cfs_hash *hs) { - if (cfs_wi_deschedule(cfs_sched_rehash, &hs->hs_dep_wi)) - return; - - spin_lock(&hs->hs_dep_lock); - while (hs->hs_dep_bits) { - spin_unlock(&hs->hs_dep_lock); - cond_resched(); - spin_lock(&hs->hs_dep_lock); - } - spin_unlock(&hs->hs_dep_lock); + cancel_work_sync(&hs->hs_dep_work); } #else /* CFS_HASH_DEBUG_LEVEL < CFS_HASH_DEBUG_1 */ @@ -1042,7 +1033,7 @@ cfs_hash_create(char *name, unsigned int cur_bits, unsigned int max_bits, hs->hs_ops = ops; hs->hs_extra_bytes = extra_bytes; hs->hs_rehash_bits = 0; - cfs_wi_init(&hs->hs_rehash_wi, hs, cfs_hash_rehash_worker); + INIT_WORK(&hs->hs_rehash_work, cfs_hash_rehash_worker); cfs_hash_depth_wi_init(hs); if (cfs_hash_with_rehash(hs)) @@ -1362,6 +1353,7 @@ cfs_hash_for_each_enter(struct cfs_hash *hs) cfs_hash_lock(hs, 1); hs->hs_iterators++; + cfs_hash_unlock(hs, 1); /* NB: iteration is mostly called by service thread, * we tend to cancel pending rehash-request, instead of @@ -1369,8 +1361,7 @@ cfs_hash_for_each_enter(struct cfs_hash *hs) * after iteration */ if (cfs_hash_is_rehashing(hs)) - cfs_hash_rehash_cancel_locked(hs); - cfs_hash_unlock(hs, 1); + cfs_hash_rehash_cancel(hs); } static void @@ -1771,43 +1762,14 @@ EXPORT_SYMBOL(cfs_hash_for_each_key); * this approach assumes a reasonably uniform hashing function. The * theta thresholds for @hs are tunable via cfs_hash_set_theta(). */ -void -cfs_hash_rehash_cancel_locked(struct cfs_hash *hs) -{ - int i; - - /* need hold cfs_hash_lock(hs, 1) */ - LASSERT(cfs_hash_with_rehash(hs) && - !cfs_hash_with_no_lock(hs)); - - if (!cfs_hash_is_rehashing(hs)) - return; - - if (cfs_wi_deschedule(cfs_sched_rehash, &hs->hs_rehash_wi)) { - hs->hs_rehash_bits = 0; - return; - } - - for (i = 2; cfs_hash_is_rehashing(hs); i++) { - cfs_hash_unlock(hs, 1); - /* raise console warning while waiting too long */ - CDEBUG(is_power_of_2(i >> 3) ? D_WARNING : D_INFO, - "hash %s is still rehashing, rescheded %d\n", - hs->hs_name, i - 1); - cond_resched(); - cfs_hash_lock(hs, 1); - } -} - void cfs_hash_rehash_cancel(struct cfs_hash *hs) { - cfs_hash_lock(hs, 1); - cfs_hash_rehash_cancel_locked(hs); - cfs_hash_unlock(hs, 1); + LASSERT(cfs_hash_with_rehash(hs)); + cancel_work_sync(&hs->hs_rehash_work); } -int +void cfs_hash_rehash(struct cfs_hash *hs, int do_rehash) { int rc; @@ -1819,21 +1781,21 @@ cfs_hash_rehash(struct cfs_hash *hs, int do_rehash) rc = cfs_hash_rehash_bits(hs); if (rc <= 0) { cfs_hash_unlock(hs, 1); - return rc; + return; } hs->hs_rehash_bits = rc; if (!do_rehash) { /* launch and return */ - cfs_wi_schedule(cfs_sched_rehash, &hs->hs_rehash_wi); + queue_work(cfs_rehash_wq, &hs->hs_rehash_work); cfs_hash_unlock(hs, 1); - return 0; + return; } /* rehash right now */ cfs_hash_unlock(hs, 1); - return cfs_hash_rehash_worker(&hs->hs_rehash_wi); + cfs_hash_rehash_worker(&hs->hs_rehash_work); } static int @@ -1867,10 +1829,10 @@ cfs_hash_rehash_bd(struct cfs_hash *hs, struct cfs_hash_bd *old) return c; } -static int -cfs_hash_rehash_worker(struct cfs_workitem *wi) +static void +cfs_hash_rehash_worker(struct work_struct *work) { - struct cfs_hash *hs = container_of(wi, struct cfs_hash, hs_rehash_wi); + struct cfs_hash *hs = container_of(work, struct cfs_hash, hs_rehash_work); struct cfs_hash_bucket **bkts; struct cfs_hash_bd bd; unsigned int old_size; @@ -1954,8 +1916,6 @@ cfs_hash_rehash_worker(struct cfs_workitem *wi) hs->hs_cur_bits = hs->hs_rehash_bits; out: hs->hs_rehash_bits = 0; - if (rc == -ESRCH) /* never be scheduled again */ - cfs_wi_exit(cfs_sched_rehash, wi); bsize = cfs_hash_bkt_size(hs); cfs_hash_unlock(hs, 1); /* can't refer to @hs anymore because it could be destroyed */ @@ -1963,8 +1923,6 @@ cfs_hash_rehash_worker(struct cfs_workitem *wi) cfs_hash_buckets_free(bkts, bsize, new_size, old_size); if (rc) CDEBUG(D_INFO, "early quit of rehashing: %d\n", rc); - /* return 1 only if cfs_wi_exit is called */ - return rc == -ESRCH; } /** diff --git a/drivers/staging/lustre/lnet/libcfs/module.c b/drivers/staging/lustre/lnet/libcfs/module.c index ff4b0cec1bbe..0254593b7f01 100644 --- a/drivers/staging/lustre/lnet/libcfs/module.c +++ b/drivers/staging/lustre/lnet/libcfs/module.c @@ -553,12 +553,10 @@ static int libcfs_init(void) goto cleanup_deregister; } - /* max to 4 threads, should be enough for rehash */ - rc = min(cfs_cpt_weight(cfs_cpt_table, CFS_CPT_ANY), 4); - rc = cfs_wi_sched_create("cfs_rh", cfs_cpt_table, CFS_CPT_ANY, - rc, &cfs_sched_rehash); - if (rc) { - CERROR("Startup workitem scheduler: error: %d\n", rc); + cfs_rehash_wq = alloc_workqueue("cfs_rh", WQ_SYSFS, 4); + if (!cfs_rehash_wq) { + CERROR("Failed to start rehash workqueue.\n"); + rc = -ENOMEM; goto cleanup_deregister; } @@ -589,9 +587,9 @@ static void libcfs_exit(void) lustre_remove_debugfs(); - if (cfs_sched_rehash) { - cfs_wi_sched_destroy(cfs_sched_rehash); - cfs_sched_rehash = NULL; + if (cfs_rehash_wq) { + destroy_workqueue(cfs_rehash_wq); + cfs_rehash_wq = NULL; } cfs_crypto_unregister(); From neilb at suse.com Mon Dec 18 01:25:19 2017 From: neilb at suse.com (NeilBrown) Date: Mon, 18 Dec 2017 12:25:19 +1100 Subject: [lustre-devel] [PATCH 4/4] staging: lustre: libcfs: remove workitem code. In-Reply-To: <151356013394.15912.9645528414310488005.stgit@noble> References: <151356013394.15912.9645528414310488005.stgit@noble> Message-ID: <151356031923.15912.14983671872492844978.stgit@noble> There are now no users. workqueues are doing the job that this used to do. Signed-off-by: NeilBrown --- .../staging/lustre/include/linux/libcfs/libcfs.h | 1 .../lustre/include/linux/libcfs/libcfs_workitem.h | 104 ---- drivers/staging/lustre/lnet/libcfs/Makefile | 2 drivers/staging/lustre/lnet/libcfs/module.c | 11 drivers/staging/lustre/lnet/libcfs/workitem.c | 466 -------------------- 5 files changed, 2 insertions(+), 582 deletions(-) delete mode 100644 drivers/staging/lustre/include/linux/libcfs/libcfs_workitem.h delete mode 100644 drivers/staging/lustre/lnet/libcfs/workitem.c diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs.h b/drivers/staging/lustre/include/linux/libcfs/libcfs.h index dcdc05dde6b8..f2ba83ee5362 100644 --- a/drivers/staging/lustre/include/linux/libcfs/libcfs.h +++ b/drivers/staging/lustre/include/linux/libcfs/libcfs.h @@ -45,7 +45,6 @@ #include #include #include -#include #include #include #include diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_workitem.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_workitem.h deleted file mode 100644 index ddaca33a13ac..000000000000 --- a/drivers/staging/lustre/include/linux/libcfs/libcfs_workitem.h +++ /dev/null @@ -1,104 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0 -/* - * GPL HEADER START - * - * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. - * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License version 2 only, - * as published by the Free Software Foundation. - * - * This program is distributed in the hope that it will be useful, but - * WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - * General Public License version 2 for more details (a copy is included - * in the LICENSE file that accompanied this code). - * - * You should have received a copy of the GNU General Public License - * version 2 along with this program; If not, see - * http://www.gnu.org/licenses/gpl-2.0.html - * - * GPL HEADER END - */ -/* - * Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved. - * Use is subject to license terms. - * - * Copyright (c) 2012, Intel Corporation. - */ -/* - * This file is part of Lustre, http://www.lustre.org/ - * Lustre is a trademark of Sun Microsystems, Inc. - * - * libcfs/include/libcfs/libcfs_workitem.h - * - * Author: Isaac Huang - * Liang Zhen - * - * A workitems is deferred work with these semantics: - * - a workitem always runs in thread context. - * - a workitem can be concurrent with other workitems but is strictly - * serialized with respect to itself. - * - no CPU affinity, a workitem does not necessarily run on the same CPU - * that schedules it. However, this might change in the future. - * - if a workitem is scheduled again before it has a chance to run, it - * runs only once. - * - if a workitem is scheduled while it runs, it runs again after it - * completes; this ensures that events occurring while other events are - * being processed receive due attention. This behavior also allows a - * workitem to reschedule itself. - * - * Usage notes: - * - a workitem can sleep but it should be aware of how that sleep might - * affect others. - * - a workitem runs inside a kernel thread so there's no user space to access. - * - do not use a workitem if the scheduling latency can't be tolerated. - * - * When wi_action returns non-zero, it means the workitem has either been - * freed or reused and workitem scheduler won't touch it any more. - */ - -#ifndef __LIBCFS_WORKITEM_H__ -#define __LIBCFS_WORKITEM_H__ - -struct cfs_wi_sched; - -void cfs_wi_sched_destroy(struct cfs_wi_sched *sched); -int cfs_wi_sched_create(char *name, struct cfs_cpt_table *cptab, int cpt, - int nthrs, struct cfs_wi_sched **sched_pp); - -struct cfs_workitem; - -typedef int (*cfs_wi_action_t) (struct cfs_workitem *); -struct cfs_workitem { - /** chain on runq or rerunq */ - struct list_head wi_list; - /** working function */ - cfs_wi_action_t wi_action; - /** in running */ - unsigned short wi_running:1; - /** scheduled */ - unsigned short wi_scheduled:1; -}; - -static inline void -cfs_wi_init(struct cfs_workitem *wi, cfs_wi_action_t action) -{ - INIT_LIST_HEAD(&wi->wi_list); - - wi->wi_running = 0; - wi->wi_scheduled = 0; - wi->wi_action = action; -} - -void cfs_wi_schedule(struct cfs_wi_sched *sched, struct cfs_workitem *wi); -int cfs_wi_deschedule(struct cfs_wi_sched *sched, struct cfs_workitem *wi); -void cfs_wi_exit(struct cfs_wi_sched *sched, struct cfs_workitem *wi); - -int cfs_wi_startup(void); -void cfs_wi_shutdown(void); - -/** # workitem scheduler loops before reschedule */ -#define CFS_WI_RESCHED 128 - -#endif /* __LIBCFS_WORKITEM_H__ */ diff --git a/drivers/staging/lustre/lnet/libcfs/Makefile b/drivers/staging/lustre/lnet/libcfs/Makefile index 1607570ef8de..51b529434332 100644 --- a/drivers/staging/lustre/lnet/libcfs/Makefile +++ b/drivers/staging/lustre/lnet/libcfs/Makefile @@ -15,7 +15,7 @@ libcfs-linux-objs += linux-mem.o libcfs-linux-objs := $(addprefix linux/,$(libcfs-linux-objs)) libcfs-all-objs := debug.o fail.o module.o tracefile.o \ - libcfs_string.o hash.o prng.o workitem.o \ + libcfs_string.o hash.o prng.o \ libcfs_cpu.o libcfs_mem.o libcfs_lock.o libcfs-objs := $(libcfs-linux-objs) $(libcfs-all-objs) diff --git a/drivers/staging/lustre/lnet/libcfs/module.c b/drivers/staging/lustre/lnet/libcfs/module.c index 0254593b7f01..1d3e8b925d0a 100644 --- a/drivers/staging/lustre/lnet/libcfs/module.c +++ b/drivers/staging/lustre/lnet/libcfs/module.c @@ -547,12 +547,6 @@ static int libcfs_init(void) goto cleanup_cpu; } - rc = cfs_wi_startup(); - if (rc) { - CERROR("initialize workitem: error %d\n", rc); - goto cleanup_deregister; - } - cfs_rehash_wq = alloc_workqueue("cfs_rh", WQ_SYSFS, 4); if (!cfs_rehash_wq) { CERROR("Failed to start rehash workqueue.\n"); @@ -563,15 +557,13 @@ static int libcfs_init(void) rc = cfs_crypto_register(); if (rc) { CERROR("cfs_crypto_register: error %d\n", rc); - goto cleanup_wi; + goto cleanup_deregister; } lustre_insert_debugfs(lnet_table, lnet_debugfs_symlinks); CDEBUG(D_OTHER, "portals setup OK\n"); return 0; - cleanup_wi: - cfs_wi_shutdown(); cleanup_deregister: misc_deregister(&libcfs_dev); cleanup_cpu: @@ -593,7 +585,6 @@ static void libcfs_exit(void) } cfs_crypto_unregister(); - cfs_wi_shutdown(); misc_deregister(&libcfs_dev); diff --git a/drivers/staging/lustre/lnet/libcfs/workitem.c b/drivers/staging/lustre/lnet/libcfs/workitem.c deleted file mode 100644 index 74a9595dc730..000000000000 --- a/drivers/staging/lustre/lnet/libcfs/workitem.c +++ /dev/null @@ -1,466 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0 -/* - * GPL HEADER START - * - * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. - * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License version 2 only, - * as published by the Free Software Foundation. - * - * This program is distributed in the hope that it will be useful, but - * WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - * General Public License version 2 for more details (a copy is included - * in the LICENSE file that accompanied this code). - * - * You should have received a copy of the GNU General Public License - * version 2 along with this program; If not, see - * http://www.gnu.org/licenses/gpl-2.0.html - * - * GPL HEADER END - */ -/* - * Copyright (c) 2007, 2010, Oracle and/or its affiliates. All rights reserved. - * Use is subject to license terms. - * - * Copyright (c) 2011, 2012, Intel Corporation. - */ -/* - * This file is part of Lustre, http://www.lustre.org/ - * Lustre is a trademark of Sun Microsystems, Inc. - * - * libcfs/libcfs/workitem.c - * - * Author: Isaac Huang - * Liang Zhen - */ - -#define DEBUG_SUBSYSTEM S_LNET - -#include - -#define CFS_WS_NAME_LEN 16 - -struct cfs_wi_sched { - /* chain on global list */ - struct list_head ws_list; - /** serialised workitems */ - spinlock_t ws_lock; - /** where schedulers sleep */ - wait_queue_head_t ws_waitq; - /** concurrent workitems */ - struct list_head ws_runq; - /** - * rescheduled running-workitems, a workitem can be rescheduled - * while running in wi_action(), but we don't to execute it again - * unless it returns from wi_action(), so we put it on ws_rerunq - * while rescheduling, and move it to runq after it returns - * from wi_action() - */ - struct list_head ws_rerunq; - /** CPT-table for this scheduler */ - struct cfs_cpt_table *ws_cptab; - /** CPT id for affinity */ - int ws_cpt; - /** number of scheduled workitems */ - int ws_nscheduled; - /** started scheduler thread, protected by cfs_wi_data::wi_glock */ - unsigned int ws_nthreads:30; - /** shutting down, protected by cfs_wi_data::wi_glock */ - unsigned int ws_stopping:1; - /** serialize starting thread, protected by cfs_wi_data::wi_glock */ - unsigned int ws_starting:1; - /** scheduler name */ - char ws_name[CFS_WS_NAME_LEN]; -}; - -static struct cfs_workitem_data { - /** serialize */ - spinlock_t wi_glock; - /** list of all schedulers */ - struct list_head wi_scheds; - /** WI module is initialized */ - int wi_init; - /** shutting down the whole WI module */ - int wi_stopping; -} cfs_wi_data; - -static inline int -cfs_wi_sched_cansleep(struct cfs_wi_sched *sched) -{ - spin_lock(&sched->ws_lock); - if (sched->ws_stopping) { - spin_unlock(&sched->ws_lock); - return 0; - } - - if (!list_empty(&sched->ws_runq)) { - spin_unlock(&sched->ws_lock); - return 0; - } - spin_unlock(&sched->ws_lock); - return 1; -} - -/* XXX: - * 0. it only works when called from wi->wi_action. - * 1. when it returns no one shall try to schedule the workitem. - */ -void -cfs_wi_exit(struct cfs_wi_sched *sched, struct cfs_workitem *wi) -{ - LASSERT(!in_interrupt()); /* because we use plain spinlock */ - LASSERT(!sched->ws_stopping); - - spin_lock(&sched->ws_lock); - - LASSERT(wi->wi_running); - if (wi->wi_scheduled) { /* cancel pending schedules */ - LASSERT(!list_empty(&wi->wi_list)); - list_del_init(&wi->wi_list); - - LASSERT(sched->ws_nscheduled > 0); - sched->ws_nscheduled--; - } - - LASSERT(list_empty(&wi->wi_list)); - - wi->wi_scheduled = 1; /* LBUG future schedule attempts */ - spin_unlock(&sched->ws_lock); -} -EXPORT_SYMBOL(cfs_wi_exit); - -/** - * cancel schedule request of workitem \a wi - */ -int -cfs_wi_deschedule(struct cfs_wi_sched *sched, struct cfs_workitem *wi) -{ - int rc; - - LASSERT(!in_interrupt()); /* because we use plain spinlock */ - LASSERT(!sched->ws_stopping); - - /* - * return 0 if it's running already, otherwise return 1, which - * means the workitem will not be scheduled and will not have - * any race with wi_action. - */ - spin_lock(&sched->ws_lock); - - rc = !(wi->wi_running); - - if (wi->wi_scheduled) { /* cancel pending schedules */ - LASSERT(!list_empty(&wi->wi_list)); - list_del_init(&wi->wi_list); - - LASSERT(sched->ws_nscheduled > 0); - sched->ws_nscheduled--; - - wi->wi_scheduled = 0; - } - - LASSERT(list_empty(&wi->wi_list)); - - spin_unlock(&sched->ws_lock); - return rc; -} -EXPORT_SYMBOL(cfs_wi_deschedule); - -/* - * Workitem scheduled with (serial == 1) is strictly serialised not only with - * itself, but also with others scheduled this way. - * - * Now there's only one static serialised queue, but in the future more might - * be added, and even dynamic creation of serialised queues might be supported. - */ -void -cfs_wi_schedule(struct cfs_wi_sched *sched, struct cfs_workitem *wi) -{ - LASSERT(!in_interrupt()); /* because we use plain spinlock */ - LASSERT(!sched->ws_stopping); - - spin_lock(&sched->ws_lock); - - if (!wi->wi_scheduled) { - LASSERT(list_empty(&wi->wi_list)); - - wi->wi_scheduled = 1; - sched->ws_nscheduled++; - if (!wi->wi_running) { - list_add_tail(&wi->wi_list, &sched->ws_runq); - wake_up(&sched->ws_waitq); - } else { - list_add(&wi->wi_list, &sched->ws_rerunq); - } - } - - LASSERT(!list_empty(&wi->wi_list)); - spin_unlock(&sched->ws_lock); -} -EXPORT_SYMBOL(cfs_wi_schedule); - -static int cfs_wi_scheduler(void *arg) -{ - struct cfs_wi_sched *sched = (struct cfs_wi_sched *)arg; - - cfs_block_allsigs(); - - /* CPT affinity scheduler? */ - if (sched->ws_cptab) - if (cfs_cpt_bind(sched->ws_cptab, sched->ws_cpt)) - CWARN("Unable to bind %s on CPU partition %d\n", - sched->ws_name, sched->ws_cpt); - - spin_lock(&cfs_wi_data.wi_glock); - - LASSERT(sched->ws_starting == 1); - sched->ws_starting--; - sched->ws_nthreads++; - - spin_unlock(&cfs_wi_data.wi_glock); - - spin_lock(&sched->ws_lock); - - while (!sched->ws_stopping) { - int nloops = 0; - int rc; - struct cfs_workitem *wi; - - while (!list_empty(&sched->ws_runq) && - nloops < CFS_WI_RESCHED) { - wi = list_entry(sched->ws_runq.next, - struct cfs_workitem, wi_list); - LASSERT(wi->wi_scheduled && !wi->wi_running); - - list_del_init(&wi->wi_list); - - LASSERT(sched->ws_nscheduled > 0); - sched->ws_nscheduled--; - - wi->wi_running = 1; - wi->wi_scheduled = 0; - - spin_unlock(&sched->ws_lock); - nloops++; - - rc = (*wi->wi_action)(wi); - - spin_lock(&sched->ws_lock); - if (rc) /* WI should be dead, even be freed! */ - continue; - - wi->wi_running = 0; - if (list_empty(&wi->wi_list)) - continue; - - LASSERT(wi->wi_scheduled); - /* wi is rescheduled, should be on rerunq now, we - * move it to runq so it can run action now - */ - list_move_tail(&wi->wi_list, &sched->ws_runq); - } - - if (!list_empty(&sched->ws_runq)) { - spin_unlock(&sched->ws_lock); - /* don't sleep because some workitems still - * expect me to come back soon - */ - cond_resched(); - spin_lock(&sched->ws_lock); - continue; - } - - spin_unlock(&sched->ws_lock); - rc = wait_event_interruptible_exclusive(sched->ws_waitq, - !cfs_wi_sched_cansleep(sched)); - spin_lock(&sched->ws_lock); - } - - spin_unlock(&sched->ws_lock); - - spin_lock(&cfs_wi_data.wi_glock); - sched->ws_nthreads--; - spin_unlock(&cfs_wi_data.wi_glock); - - return 0; -} - -void -cfs_wi_sched_destroy(struct cfs_wi_sched *sched) -{ - int i; - - LASSERT(cfs_wi_data.wi_init); - LASSERT(!cfs_wi_data.wi_stopping); - - spin_lock(&cfs_wi_data.wi_glock); - if (sched->ws_stopping) { - CDEBUG(D_INFO, "%s is in progress of stopping\n", - sched->ws_name); - spin_unlock(&cfs_wi_data.wi_glock); - return; - } - - LASSERT(!list_empty(&sched->ws_list)); - sched->ws_stopping = 1; - - spin_unlock(&cfs_wi_data.wi_glock); - - i = 2; - wake_up_all(&sched->ws_waitq); - - spin_lock(&cfs_wi_data.wi_glock); - while (sched->ws_nthreads > 0) { - CDEBUG(is_power_of_2(++i) ? D_WARNING : D_NET, - "waiting for %d threads of WI sched[%s] to terminate\n", - sched->ws_nthreads, sched->ws_name); - - spin_unlock(&cfs_wi_data.wi_glock); - set_current_state(TASK_UNINTERRUPTIBLE); - schedule_timeout(cfs_time_seconds(1) / 20); - spin_lock(&cfs_wi_data.wi_glock); - } - - list_del(&sched->ws_list); - - spin_unlock(&cfs_wi_data.wi_glock); - LASSERT(!sched->ws_nscheduled); - - kfree(sched); -} -EXPORT_SYMBOL(cfs_wi_sched_destroy); - -int -cfs_wi_sched_create(char *name, struct cfs_cpt_table *cptab, - int cpt, int nthrs, struct cfs_wi_sched **sched_pp) -{ - struct cfs_wi_sched *sched; - int rc; - - LASSERT(cfs_wi_data.wi_init); - LASSERT(!cfs_wi_data.wi_stopping); - LASSERT(!cptab || cpt == CFS_CPT_ANY || - (cpt >= 0 && cpt < cfs_cpt_number(cptab))); - - sched = kzalloc(sizeof(*sched), GFP_NOFS); - if (!sched) - return -ENOMEM; - - if (strlen(name) > sizeof(sched->ws_name) - 1) { - kfree(sched); - return -E2BIG; - } - strncpy(sched->ws_name, name, sizeof(sched->ws_name)); - - sched->ws_cptab = cptab; - sched->ws_cpt = cpt; - - spin_lock_init(&sched->ws_lock); - init_waitqueue_head(&sched->ws_waitq); - INIT_LIST_HEAD(&sched->ws_runq); - INIT_LIST_HEAD(&sched->ws_rerunq); - INIT_LIST_HEAD(&sched->ws_list); - - rc = 0; - while (nthrs > 0) { - char name[16]; - struct task_struct *task; - - spin_lock(&cfs_wi_data.wi_glock); - while (sched->ws_starting > 0) { - spin_unlock(&cfs_wi_data.wi_glock); - schedule(); - spin_lock(&cfs_wi_data.wi_glock); - } - - sched->ws_starting++; - spin_unlock(&cfs_wi_data.wi_glock); - - if (sched->ws_cptab && sched->ws_cpt >= 0) { - snprintf(name, sizeof(name), "%s_%02d_%02u", - sched->ws_name, sched->ws_cpt, - sched->ws_nthreads); - } else { - snprintf(name, sizeof(name), "%s_%02u", - sched->ws_name, sched->ws_nthreads); - } - - task = kthread_run(cfs_wi_scheduler, sched, "%s", name); - if (!IS_ERR(task)) { - nthrs--; - continue; - } - rc = PTR_ERR(task); - - CERROR("Failed to create thread for WI scheduler %s: %d\n", - name, rc); - - spin_lock(&cfs_wi_data.wi_glock); - - /* make up for cfs_wi_sched_destroy */ - list_add(&sched->ws_list, &cfs_wi_data.wi_scheds); - sched->ws_starting--; - - spin_unlock(&cfs_wi_data.wi_glock); - - cfs_wi_sched_destroy(sched); - return rc; - } - spin_lock(&cfs_wi_data.wi_glock); - list_add(&sched->ws_list, &cfs_wi_data.wi_scheds); - spin_unlock(&cfs_wi_data.wi_glock); - - *sched_pp = sched; - return 0; -} -EXPORT_SYMBOL(cfs_wi_sched_create); - -int -cfs_wi_startup(void) -{ - memset(&cfs_wi_data, 0, sizeof(cfs_wi_data)); - - spin_lock_init(&cfs_wi_data.wi_glock); - INIT_LIST_HEAD(&cfs_wi_data.wi_scheds); - cfs_wi_data.wi_init = 1; - - return 0; -} - -void -cfs_wi_shutdown(void) -{ - struct cfs_wi_sched *sched; - struct cfs_wi_sched *temp; - - spin_lock(&cfs_wi_data.wi_glock); - cfs_wi_data.wi_stopping = 1; - spin_unlock(&cfs_wi_data.wi_glock); - - /* nobody should contend on this list */ - list_for_each_entry(sched, &cfs_wi_data.wi_scheds, ws_list) { - sched->ws_stopping = 1; - wake_up_all(&sched->ws_waitq); - } - - list_for_each_entry(sched, &cfs_wi_data.wi_scheds, ws_list) { - spin_lock(&cfs_wi_data.wi_glock); - - while (sched->ws_nthreads) { - spin_unlock(&cfs_wi_data.wi_glock); - set_current_state(TASK_UNINTERRUPTIBLE); - schedule_timeout(cfs_time_seconds(1) / 20); - spin_lock(&cfs_wi_data.wi_glock); - } - spin_unlock(&cfs_wi_data.wi_glock); - } - list_for_each_entry_safe(sched, temp, &cfs_wi_data.wi_scheds, ws_list) { - list_del(&sched->ws_list); - kfree(sched); - } - - cfs_wi_data.wi_stopping = 0; - cfs_wi_data.wi_init = 0; -} From neilb at suse.com Mon Dec 18 01:25:19 2017 From: neilb at suse.com (NeilBrown) Date: Mon, 18 Dec 2017 12:25:19 +1100 Subject: [lustre-devel] [PATCH 3/4] staging: lustre: lnet: convert selftest to use workqueues In-Reply-To: <151356013394.15912.9645528414310488005.stgit@noble> References: <151356013394.15912.9645528414310488005.stgit@noble> Message-ID: <151356031919.15912.12406337913485020577.stgit@noble> Instead of the cfs workitem library, use workqueues. As lnet wants to provide a cpu mask of allowed cpus, it needs to be a WQ_UNBOUND work queue so that tasks can run on cpus other than where they were submitted. apply_workqueue_atts needs to be exported for lustre to use it. Signed-off-by: NeilBrown --- drivers/staging/lustre/lnet/selftest/framework.c | 10 +--- drivers/staging/lustre/lnet/selftest/module.c | 39 ++++++++------ drivers/staging/lustre/lnet/selftest/rpc.c | 61 +++++++++------------- drivers/staging/lustre/lnet/selftest/selftest.h | 40 ++++++-------- kernel/workqueue.c | 1 5 files changed, 69 insertions(+), 82 deletions(-) diff --git a/drivers/staging/lustre/lnet/selftest/framework.c b/drivers/staging/lustre/lnet/selftest/framework.c index 2e1126552e18..c7697f66f663 100644 --- a/drivers/staging/lustre/lnet/selftest/framework.c +++ b/drivers/staging/lustre/lnet/selftest/framework.c @@ -941,15 +941,13 @@ sfw_create_test_rpc(struct sfw_test_unit *tsu, struct lnet_process_id peer, return 0; } -static int +static void sfw_run_test(struct swi_workitem *wi) { struct sfw_test_unit *tsu = container_of(wi, struct sfw_test_unit, tsu_worker); struct sfw_test_instance *tsi = tsu->tsu_instance; struct srpc_client_rpc *rpc = NULL; - LASSERT(wi == &tsu->tsu_worker); - if (tsi->tsi_ops->tso_prep_rpc(tsu, tsu->tsu_dest, &rpc)) { LASSERT(!rpc); goto test_done; @@ -975,7 +973,7 @@ sfw_run_test(struct swi_workitem *wi) rpc->crpc_timeout = rpc_timeout; srpc_post_rpc(rpc); spin_unlock(&rpc->crpc_lock); - return 0; + return; test_done: /* @@ -985,9 +983,7 @@ sfw_run_test(struct swi_workitem *wi) * - my batch is still active; no one can run it again now. * Cancel pending schedules and prevent future schedule attempts: */ - swi_exit_workitem(wi); sfw_test_unit_done(tsu); - return 1; } static int @@ -1017,7 +1013,7 @@ sfw_run_batch(struct sfw_batch *tsb) tsu->tsu_loop = tsi->tsi_loop; wi = &tsu->tsu_worker; swi_init_workitem(wi, sfw_run_test, - lst_sched_test[lnet_cpt_of_nid(tsu->tsu_dest.nid)]); + lst_test_wq[lnet_cpt_of_nid(tsu->tsu_dest.nid)]); swi_schedule_workitem(wi); } } diff --git a/drivers/staging/lustre/lnet/selftest/module.c b/drivers/staging/lustre/lnet/selftest/module.c index ba4b6145c953..aa6bfd5baf2f 100644 --- a/drivers/staging/lustre/lnet/selftest/module.c +++ b/drivers/staging/lustre/lnet/selftest/module.c @@ -47,8 +47,8 @@ enum { static int lst_init_step = LST_INIT_NONE; -struct cfs_wi_sched *lst_sched_serial; -struct cfs_wi_sched **lst_sched_test; +struct workqueue_struct *lst_serial_wq; +struct workqueue_struct **lst_test_wq; static void lnet_selftest_exit(void) @@ -68,16 +68,16 @@ lnet_selftest_exit(void) case LST_INIT_WI_TEST: for (i = 0; i < cfs_cpt_number(lnet_cpt_table()); i++) { - if (!lst_sched_test[i]) + if (!lst_test_wq[i]) continue; - cfs_wi_sched_destroy(lst_sched_test[i]); + destroy_workqueue(lst_test_wq[i]); } - kvfree(lst_sched_test); - lst_sched_test = NULL; + kvfree(lst_test_wq); + lst_test_wq = NULL; /* fall through */ case LST_INIT_WI_SERIAL: - cfs_wi_sched_destroy(lst_sched_serial); - lst_sched_serial = NULL; + destroy_workqueue(lst_serial_wq); + lst_serial_wq = NULL; case LST_INIT_NONE: break; default: @@ -92,33 +92,40 @@ lnet_selftest_init(void) int rc; int i; - rc = cfs_wi_sched_create("lst_s", lnet_cpt_table(), CFS_CPT_ANY, - 1, &lst_sched_serial); - if (rc) { + lst_serial_wq = alloc_ordered_workqueue("lst_s", 0); + if (!lst_serial_wq) { CERROR("Failed to create serial WI scheduler for LST\n"); return rc; } lst_init_step = LST_INIT_WI_SERIAL; nscheds = cfs_cpt_number(lnet_cpt_table()); - lst_sched_test = kvmalloc_array(nscheds, sizeof(lst_sched_test[0]), + lst_test_wq = kvmalloc_array(nscheds, sizeof(lst_test_wq[0]), GFP_KERNEL | __GFP_ZERO); - if (!lst_sched_test) + if (!lst_test_wq) goto error; lst_init_step = LST_INIT_WI_TEST; for (i = 0; i < nscheds; i++) { int nthrs = cfs_cpt_weight(lnet_cpt_table(), i); + struct workqueue_attrs attrs; /* reserve at least one CPU for LND */ nthrs = max(nthrs - 1, 1); - rc = cfs_wi_sched_create("lst_t", lnet_cpt_table(), i, - nthrs, &lst_sched_test[i]); - if (rc) { + lst_test_wq[i] = alloc_workqueue("lst_t", WQ_UNBOUND, nthrs); + if (!lst_test_wq[i]) { CWARN("Failed to create CPU partition affinity WI scheduler %d for LST\n", i); goto error; } + attrs.nice = 0; + #ifdef CONFIG_CPUMASK_OFFSTACK + attrs.cpumask = lnet_cpt_table()->ctb_parts[i].cpt_cpumask; + #else + cpumask_copy(attrs.cpumask, lnet_cpt_table()->ctb_parts[i].cpt_cpumask); + #endif + attrs.no_numa = false; + apply_workqueue_attrs(lst_test_wq[i], &attrs); } rc = srpc_startup(); diff --git a/drivers/staging/lustre/lnet/selftest/rpc.c b/drivers/staging/lustre/lnet/selftest/rpc.c index 4ebb5a1107be..b515138dca2c 100644 --- a/drivers/staging/lustre/lnet/selftest/rpc.c +++ b/drivers/staging/lustre/lnet/selftest/rpc.c @@ -68,7 +68,7 @@ srpc_serv_portal(int svc_id) } /* forward ref's */ -int srpc_handle_rpc(struct swi_workitem *wi); +void srpc_handle_rpc(struct swi_workitem *wi); void srpc_get_counters(struct srpc_counters *cnt) { @@ -178,7 +178,7 @@ srpc_init_server_rpc(struct srpc_server_rpc *rpc, memset(rpc, 0, sizeof(*rpc)); swi_init_workitem(&rpc->srpc_wi, srpc_handle_rpc, srpc_serv_is_framework(scd->scd_svc) ? - lst_sched_serial : lst_sched_test[scd->scd_cpt]); + lst_serial_wq : lst_test_wq[scd->scd_cpt]); rpc->srpc_ev.ev_fired = 1; /* no event expected now */ @@ -242,7 +242,7 @@ srpc_service_nrpcs(struct srpc_service *svc) max(nrpcs, SFW_FRWK_WI_MIN) : max(nrpcs, SFW_TEST_WI_MIN); } -int srpc_add_buffer(struct swi_workitem *wi); +void srpc_add_buffer(struct swi_workitem *wi); static int srpc_service_init(struct srpc_service *svc) @@ -277,11 +277,11 @@ srpc_service_init(struct srpc_service *svc) scd->scd_ev.ev_type = SRPC_REQUEST_RCVD; /* - * NB: don't use lst_sched_serial for adding buffer, + * NB: don't use lst_serial_wq for adding buffer, * see details in srpc_service_add_buffers() */ swi_init_workitem(&scd->scd_buf_wi, - srpc_add_buffer, lst_sched_test[i]); + srpc_add_buffer, lst_test_wq[i]); if (i && srpc_serv_is_framework(svc)) { /* @@ -514,7 +514,7 @@ __must_hold(&scd->scd_lock) return rc; } -int +void srpc_add_buffer(struct swi_workitem *wi) { struct srpc_service_cd *scd = container_of(wi, struct srpc_service_cd, scd_buf_wi); @@ -573,7 +573,6 @@ srpc_add_buffer(struct swi_workitem *wi) } spin_unlock(&scd->scd_lock); - return 0; } int @@ -605,15 +604,15 @@ srpc_service_add_buffers(struct srpc_service *sv, int nbuffer) spin_lock(&scd->scd_lock); /* * NB: srpc_service_add_buffers() can be called inside - * thread context of lst_sched_serial, and we don't normally + * thread context of lst_serial_wq, and we don't normally * allow to sleep inside thread context of WI scheduler * because it will block current scheduler thread from doing * anything else, even worse, it could deadlock if it's * waiting on result from another WI of the same scheduler. * However, it's safe at here because scd_buf_wi is scheduled - * by thread in a different WI scheduler (lst_sched_test), + * by thread in a different WI scheduler (lst_test_wq), * so we don't have any risk of deadlock, though this could - * block all WIs pending on lst_sched_serial for a moment + * block all WIs pending on lst_serial_wq for a moment * which is not good but not fatal. */ lst_wait_until(scd->scd_buf_err || @@ -660,11 +659,9 @@ srpc_finish_service(struct srpc_service *sv) LASSERT(sv->sv_shuttingdown); /* srpc_shutdown_service called */ cfs_percpt_for_each(scd, i, sv->sv_cpt_data) { + swi_cancel_workitem(&scd->scd_buf_wi); + spin_lock(&scd->scd_lock); - if (!swi_deschedule_workitem(&scd->scd_buf_wi)) { - spin_unlock(&scd->scd_lock); - return 0; - } if (scd->scd_buf_nposted > 0) { CDEBUG(D_NET, "waiting for %d posted buffers to unlink\n", @@ -680,11 +677,9 @@ srpc_finish_service(struct srpc_service *sv) rpc = list_entry(scd->scd_rpc_active.next, struct srpc_server_rpc, srpc_list); - CNETERR("Active RPC %p on shutdown: sv %s, peer %s, wi %s scheduled %d running %d, ev fired %d type %d status %d lnet %d\n", + CNETERR("Active RPC %p on shutdown: sv %s, peer %s, wi %s, ev fired %d type %d status %d lnet %d\n", rpc, sv->sv_name, libcfs_id2str(rpc->srpc_peer), swi_state2str(rpc->srpc_wi.swi_state), - rpc->srpc_wi.swi_workitem.wi_scheduled, - rpc->srpc_wi.swi_workitem.wi_running, rpc->srpc_ev.ev_fired, rpc->srpc_ev.ev_type, rpc->srpc_ev.ev_status, rpc->srpc_ev.ev_lnet); spin_unlock(&scd->scd_lock); @@ -947,7 +942,6 @@ srpc_server_rpc_done(struct srpc_server_rpc *rpc, int status) * Cancel pending schedules and prevent future schedule attempts: */ LASSERT(rpc->srpc_ev.ev_fired); - swi_exit_workitem(&rpc->srpc_wi); if (!sv->sv_shuttingdown && !list_empty(&scd->scd_buf_blocked)) { buffer = list_entry(scd->scd_buf_blocked.next, @@ -965,7 +959,7 @@ srpc_server_rpc_done(struct srpc_server_rpc *rpc, int status) } /* handles an incoming RPC */ -int +void srpc_handle_rpc(struct swi_workitem *wi) { struct srpc_server_rpc *rpc = container_of(wi, struct srpc_server_rpc, srpc_wi); @@ -987,9 +981,8 @@ srpc_handle_rpc(struct swi_workitem *wi) if (ev->ev_fired) { /* no more event, OK to finish */ srpc_server_rpc_done(rpc, -ESHUTDOWN); - return 1; } - return 0; + return; } spin_unlock(&scd->scd_lock); @@ -1007,7 +1000,7 @@ srpc_handle_rpc(struct swi_workitem *wi) if (!msg->msg_magic) { /* moaned already in srpc_lnet_ev_handler */ srpc_server_rpc_done(rpc, EBADMSG); - return 1; + return; } srpc_unpack_msg_hdr(msg); @@ -1023,7 +1016,7 @@ srpc_handle_rpc(struct swi_workitem *wi) LASSERT(!reply->status || !rpc->srpc_bulk); if (rc) { srpc_server_rpc_done(rpc, rc); - return 1; + return; } } @@ -1032,7 +1025,7 @@ srpc_handle_rpc(struct swi_workitem *wi) if (rpc->srpc_bulk) { rc = srpc_do_bulk(rpc); if (!rc) - return 0; /* wait for bulk */ + return; /* wait for bulk */ LASSERT(ev->ev_fired); ev->ev_status = rc; @@ -1050,16 +1043,16 @@ srpc_handle_rpc(struct swi_workitem *wi) if (rc) { srpc_server_rpc_done(rpc, rc); - return 1; + return; } } wi->swi_state = SWI_STATE_REPLY_SUBMITTED; rc = srpc_send_reply(rpc); if (!rc) - return 0; /* wait for reply */ + return; /* wait for reply */ srpc_server_rpc_done(rpc, rc); - return 1; + return; case SWI_STATE_REPLY_SUBMITTED: if (!ev->ev_fired) { @@ -1072,10 +1065,8 @@ srpc_handle_rpc(struct swi_workitem *wi) wi->swi_state = SWI_STATE_DONE; srpc_server_rpc_done(rpc, ev->ev_status); - return 1; + return; } - - return 0; } static void @@ -1170,7 +1161,6 @@ srpc_client_rpc_done(struct srpc_client_rpc *rpc, int status) * Cancel pending schedules and prevent future schedule attempts: */ LASSERT(!srpc_event_pending(rpc)); - swi_exit_workitem(wi); spin_unlock(&rpc->crpc_lock); @@ -1178,7 +1168,7 @@ srpc_client_rpc_done(struct srpc_client_rpc *rpc, int status) } /* sends an outgoing RPC */ -int +void srpc_send_rpc(struct swi_workitem *wi) { int rc = 0; @@ -1214,7 +1204,7 @@ srpc_send_rpc(struct swi_workitem *wi) rc = srpc_prepare_reply(rpc); if (rc) { srpc_client_rpc_done(rpc, rc); - return 1; + return; } rc = srpc_prepare_bulk(rpc); @@ -1291,7 +1281,7 @@ srpc_send_rpc(struct swi_workitem *wi) wi->swi_state = SWI_STATE_DONE; srpc_client_rpc_done(rpc, rc); - return 1; + return; } if (rc) { @@ -1308,10 +1298,9 @@ srpc_send_rpc(struct swi_workitem *wi) if (!srpc_event_pending(rpc)) { srpc_client_rpc_done(rpc, -EINTR); - return 1; + return; } } - return 0; } struct srpc_client_rpc * diff --git a/drivers/staging/lustre/lnet/selftest/selftest.h b/drivers/staging/lustre/lnet/selftest/selftest.h index 465417263ef1..ad04534f000c 100644 --- a/drivers/staging/lustre/lnet/selftest/selftest.h +++ b/drivers/staging/lustre/lnet/selftest/selftest.h @@ -169,11 +169,11 @@ struct srpc_buffer { }; struct swi_workitem; -typedef int (*swi_action_t) (struct swi_workitem *); +typedef void (*swi_action_t) (struct swi_workitem *); struct swi_workitem { - struct cfs_wi_sched *swi_sched; - struct cfs_workitem swi_workitem; + struct workqueue_struct *swi_wq; + struct work_struct swi_work; swi_action_t swi_action; int swi_state; }; @@ -444,7 +444,7 @@ void srpc_free_bulk(struct srpc_bulk *bk); struct srpc_bulk *srpc_alloc_bulk(int cpt, unsigned int off, unsigned int bulk_npg, unsigned int bulk_len, int sink); -int srpc_send_rpc(struct swi_workitem *wi); +void srpc_send_rpc(struct swi_workitem *wi); int srpc_send_reply(struct srpc_server_rpc *rpc); int srpc_add_service(struct srpc_service *sv); int srpc_remove_service(struct srpc_service *sv); @@ -456,8 +456,8 @@ void srpc_service_remove_buffers(struct srpc_service *sv, int nbuffer); void srpc_get_counters(struct srpc_counters *cnt); void srpc_set_counters(const struct srpc_counters *cnt); -extern struct cfs_wi_sched *lst_sched_serial; -extern struct cfs_wi_sched **lst_sched_test; +extern struct workqueue_struct *lst_serial_wq; +extern struct workqueue_struct **lst_test_wq; static inline int srpc_serv_is_framework(struct srpc_service *svc) @@ -465,42 +465,36 @@ srpc_serv_is_framework(struct srpc_service *svc) return svc->sv_id < SRPC_FRAMEWORK_SERVICE_MAX_ID; } -static inline int -swi_wi_action(struct cfs_workitem *wi) +static void +swi_wi_action(struct work_struct *wi) { struct swi_workitem *swi; - swi = container_of(wi, struct swi_workitem, swi_workitem); + swi = container_of(wi, struct swi_workitem, swi_work); - return swi->swi_action(swi); + swi->swi_action(swi); } static inline void swi_init_workitem(struct swi_workitem *swi, - swi_action_t action, struct cfs_wi_sched *sched) + swi_action_t action, struct workqueue_struct *wq) { - swi->swi_sched = sched; + swi->swi_wq = wq; swi->swi_action = action; swi->swi_state = SWI_STATE_NEWBORN; - cfs_wi_init(&swi->swi_workitem, swi_wi_action); + INIT_WORK(&swi->swi_work, swi_wi_action); } static inline void swi_schedule_workitem(struct swi_workitem *wi) { - cfs_wi_schedule(wi->swi_sched, &wi->swi_workitem); -} - -static inline void -swi_exit_workitem(struct swi_workitem *swi) -{ - cfs_wi_exit(swi->swi_sched, &swi->swi_workitem); + queue_work(wi->swi_wq, &wi->swi_work); } static inline int -swi_deschedule_workitem(struct swi_workitem *swi) +swi_cancel_workitem(struct swi_workitem *swi) { - return cfs_wi_deschedule(swi->swi_sched, &swi->swi_workitem); + return cancel_work_sync(&swi->swi_work); } int sfw_startup(void); @@ -534,7 +528,7 @@ srpc_init_client_rpc(struct srpc_client_rpc *rpc, struct lnet_process_id peer, INIT_LIST_HEAD(&rpc->crpc_list); swi_init_workitem(&rpc->crpc_wi, srpc_send_rpc, - lst_sched_test[lnet_cpt_of_nid(peer.nid)]); + lst_test_wq[lnet_cpt_of_nid(peer.nid)]); spin_lock_init(&rpc->crpc_lock); atomic_set(&rpc->crpc_refcount, 1); /* 1 ref for caller */ diff --git a/kernel/workqueue.c b/kernel/workqueue.c index 8fdb710bfdd7..024fd99bf7ea 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -3806,6 +3806,7 @@ int apply_workqueue_attrs(struct workqueue_struct *wq, return ret; } +EXPORT_SYMBOL_GPL(apply_workqueue_attrs); /** * wq_update_unbound_numa - update NUMA affinity of a wq for CPU hot[un]plug From neilb at suse.com Mon Dec 18 01:41:42 2017 From: neilb at suse.com (NeilBrown) Date: Mon, 18 Dec 2017 12:41:42 +1100 Subject: [lustre-devel] [PATCH SERIES 4: 0/4] staging: lustre: use standard prng Message-ID: <151356119565.24290.5570928075046338898.stgit@noble> Lustre has its own internal PRNG code. This adds nothing of value to the Linux standard prng code, so switch over to using the standard interfaces. This adds a few callers to add_device_randomness(), which helps everyone, and removes unnecessary code. Thanks, NeilBrown --- NeilBrown (4): staging: lustre: replace cfs_rand() with prandom_u32_max() staging: lustre: replace cfs_srand() calls with add_device_randomness(). staging: lustre: replace cfs_get_random_bytes calls with get_random_byte() staging: lustre: libcfs: remove prng .../staging/lustre/include/linux/libcfs/libcfs.h | 10 - drivers/staging/lustre/lnet/libcfs/Makefile | 2 drivers/staging/lustre/lnet/libcfs/fail.c | 2 drivers/staging/lustre/lnet/libcfs/prng.c | 137 -------------------- drivers/staging/lustre/lnet/lnet/net_fault.c | 38 +++--- drivers/staging/lustre/lnet/lnet/router.c | 19 +-- drivers/staging/lustre/lustre/include/obd_class.h | 2 drivers/staging/lustre/lustre/llite/super25.c | 17 +- drivers/staging/lustre/lustre/mgc/mgc_request.c | 4 - .../lustre/lustre/obdclass/lustre_handles.c | 9 - drivers/staging/lustre/lustre/ptlrpc/client.c | 2 11 files changed, 42 insertions(+), 200 deletions(-) delete mode 100644 drivers/staging/lustre/lnet/libcfs/prng.c -- Signature From neilb at suse.com Mon Dec 18 01:41:42 2017 From: neilb at suse.com (NeilBrown) Date: Mon, 18 Dec 2017 12:41:42 +1100 Subject: [lustre-devel] [PATCH 1/4] staging: lustre: replace cfs_rand() with prandom_u32_max() In-Reply-To: <151356119565.24290.5570928075046338898.stgit@noble> References: <151356119565.24290.5570928075046338898.stgit@noble> Message-ID: <151356130236.24290.3477673707432595018.stgit@noble> All occurrences of cfs_rand() % X are replaced with prandom_u32_max(X) cfs_rand() is a simple Linear Congruential PRNG. prandom_u32_max() is at least as random, is seeded with more randomness, and uses cpu-local state to avoid cross-cpu issues. This is the first step is discarding the libcfs prng with the standard linux prng. Signed-off-by: NeilBrown --- drivers/staging/lustre/lnet/libcfs/fail.c | 2 + drivers/staging/lustre/lnet/lnet/net_fault.c | 38 ++++++++++++----------- drivers/staging/lustre/lnet/lnet/router.c | 4 +- drivers/staging/lustre/lustre/mgc/mgc_request.c | 4 +- 4 files changed, 25 insertions(+), 23 deletions(-) diff --git a/drivers/staging/lustre/lnet/libcfs/fail.c b/drivers/staging/lustre/lnet/libcfs/fail.c index 5d501beeb622..39439b303d65 100644 --- a/drivers/staging/lustre/lnet/libcfs/fail.c +++ b/drivers/staging/lustre/lnet/libcfs/fail.c @@ -61,7 +61,7 @@ int __cfs_fail_check_set(u32 id, u32 value, int set) /* Fail 1/cfs_fail_val times */ if (cfs_fail_loc & CFS_FAIL_RAND) { - if (cfs_fail_val < 2 || cfs_rand() % cfs_fail_val > 0) + if (cfs_fail_val < 2 || prandom_u32_max(cfs_fail_val) > 0) return 0; } diff --git a/drivers/staging/lustre/lnet/lnet/net_fault.c b/drivers/staging/lustre/lnet/lnet/net_fault.c index 0318e64c413f..e3468cef273b 100644 --- a/drivers/staging/lustre/lnet/lnet/net_fault.c +++ b/drivers/staging/lustre/lnet/lnet/net_fault.c @@ -170,10 +170,10 @@ lnet_drop_rule_add(struct lnet_fault_attr *attr) rule->dr_attr = *attr; if (attr->u.drop.da_interval) { rule->dr_time_base = cfs_time_shift(attr->u.drop.da_interval); - rule->dr_drop_time = cfs_time_shift(cfs_rand() % - attr->u.drop.da_interval); + rule->dr_drop_time = cfs_time_shift( + prandom_u32_max(attr->u.drop.da_interval)); } else { - rule->dr_drop_at = cfs_rand() % attr->u.drop.da_rate; + rule->dr_drop_at = prandom_u32_max(attr->u.drop.da_rate); } lnet_net_lock(LNET_LOCK_EX); @@ -277,10 +277,10 @@ lnet_drop_rule_reset(void) memset(&rule->dr_stat, 0, sizeof(rule->dr_stat)); if (attr->u.drop.da_rate) { - rule->dr_drop_at = cfs_rand() % attr->u.drop.da_rate; + rule->dr_drop_at = prandom_u32_max(attr->u.drop.da_rate); } else { - rule->dr_drop_time = cfs_time_shift(cfs_rand() % - attr->u.drop.da_interval); + rule->dr_drop_time = cfs_time_shift( + prandom_u32_max(attr->u.drop.da_interval)); rule->dr_time_base = cfs_time_shift(attr->u.drop.da_interval); } spin_unlock(&rule->dr_lock); @@ -315,8 +315,8 @@ drop_rule_match(struct lnet_drop_rule *rule, lnet_nid_t src, rule->dr_time_base = now; rule->dr_drop_time = rule->dr_time_base + - cfs_time_seconds(cfs_rand() % - attr->u.drop.da_interval); + cfs_time_seconds( + prandom_u32_max(attr->u.drop.da_interval)); rule->dr_time_base += cfs_time_seconds(attr->u.drop.da_interval); CDEBUG(D_NET, "Drop Rule %s->%s: next drop : %lu\n", @@ -330,7 +330,7 @@ drop_rule_match(struct lnet_drop_rule *rule, lnet_nid_t src, if (!do_div(rule->dr_stat.fs_count, attr->u.drop.da_rate)) { rule->dr_drop_at = rule->dr_stat.fs_count + - cfs_rand() % attr->u.drop.da_rate; + prandom_u32_max(attr->u.drop.da_rate); CDEBUG(D_NET, "Drop Rule %s->%s: next drop: %lu\n", libcfs_nid2str(attr->fa_src), libcfs_nid2str(attr->fa_dst), rule->dr_drop_at); @@ -483,8 +483,9 @@ delay_rule_match(struct lnet_delay_rule *rule, lnet_nid_t src, rule->dl_time_base = now; rule->dl_delay_time = rule->dl_time_base + - cfs_time_seconds(cfs_rand() % - attr->u.delay.la_interval); + cfs_time_seconds( + prandom_u32_max( + attr->u.delay.la_interval)); rule->dl_time_base += cfs_time_seconds(attr->u.delay.la_interval); CDEBUG(D_NET, "Delay Rule %s->%s: next delay : %lu\n", @@ -498,7 +499,7 @@ delay_rule_match(struct lnet_delay_rule *rule, lnet_nid_t src, /* generate the next random rate sequence */ if (!do_div(rule->dl_stat.fs_count, attr->u.delay.la_rate)) { rule->dl_delay_at = rule->dl_stat.fs_count + - cfs_rand() % attr->u.delay.la_rate; + prandom_u32_max(attr->u.delay.la_rate); CDEBUG(D_NET, "Delay Rule %s->%s: next delay: %lu\n", libcfs_nid2str(attr->fa_src), libcfs_nid2str(attr->fa_dst), rule->dl_delay_at); @@ -771,10 +772,10 @@ lnet_delay_rule_add(struct lnet_fault_attr *attr) rule->dl_attr = *attr; if (attr->u.delay.la_interval) { rule->dl_time_base = cfs_time_shift(attr->u.delay.la_interval); - rule->dl_delay_time = cfs_time_shift(cfs_rand() % - attr->u.delay.la_interval); + rule->dl_delay_time = cfs_time_shift( + prandom_u32_max(attr->u.delay.la_interval)); } else { - rule->dl_delay_at = cfs_rand() % attr->u.delay.la_rate; + rule->dl_delay_at = prandom_u32_max(attr->u.delay.la_rate); } rule->dl_msg_send = -1; @@ -920,10 +921,11 @@ lnet_delay_rule_reset(void) memset(&rule->dl_stat, 0, sizeof(rule->dl_stat)); if (attr->u.delay.la_rate) { - rule->dl_delay_at = cfs_rand() % attr->u.delay.la_rate; + rule->dl_delay_at = prandom_u32_max(attr->u.delay.la_rate); } else { - rule->dl_delay_time = cfs_time_shift(cfs_rand() % - attr->u.delay.la_interval); + rule->dl_delay_time = + cfs_time_shift(prandom_u32_max( + attr->u.delay.la_interval)); rule->dl_time_base = cfs_time_shift(attr->u.delay.la_interval); } spin_unlock(&rule->dl_lock); diff --git a/drivers/staging/lustre/lnet/lnet/router.c b/drivers/staging/lustre/lnet/lnet/router.c index c40aa79baf5c..e5c9b29e199f 100644 --- a/drivers/staging/lustre/lnet/lnet/router.c +++ b/drivers/staging/lustre/lnet/lnet/router.c @@ -277,8 +277,8 @@ lnet_add_route_to_rnet(struct lnet_remotenet *rnet, struct lnet_route *route) len++; } - /* len+1 positions to add a new entry, also prevents division by 0 */ - offset = cfs_rand() % (len + 1); + /* len+1 positions to add a new entry */ + offset = prandom_u32_max(len + 1); list_for_each(e, &rnet->lrn_routes) { if (!offset) break; diff --git a/drivers/staging/lustre/lustre/mgc/mgc_request.c b/drivers/staging/lustre/lustre/mgc/mgc_request.c index 77fa8fea0249..79ff85feab64 100644 --- a/drivers/staging/lustre/lustre/mgc/mgc_request.c +++ b/drivers/staging/lustre/lustre/mgc/mgc_request.c @@ -523,7 +523,7 @@ static void do_requeue(struct config_llog_data *cld) * in order to not flood the MGS. */ #define MGC_TIMEOUT_MIN_SECONDS 5 -#define MGC_TIMEOUT_RAND_CENTISEC 0x1ff /* ~500 */ +#define MGC_TIMEOUT_RAND_CENTISEC 500 static int mgc_requeue_thread(void *data) { @@ -537,7 +537,7 @@ static int mgc_requeue_thread(void *data) while (!(rq_state & RQ_STOP)) { struct l_wait_info lwi; struct config_llog_data *cld, *cld_prev; - int rand = cfs_rand() & MGC_TIMEOUT_RAND_CENTISEC; + int rand = prandom_u32_max(MGC_TIMEOUT_RAND_CENTISEC); int to; /* Any new or requeued lostlocks will change the state */ From neilb at suse.com Mon Dec 18 01:41:42 2017 From: neilb at suse.com (NeilBrown) Date: Mon, 18 Dec 2017 12:41:42 +1100 Subject: [lustre-devel] [PATCH 2/4] staging: lustre: replace cfs_srand() calls with add_device_randomness(). In-Reply-To: <151356119565.24290.5570928075046338898.stgit@noble> References: <151356119565.24290.5570928075046338898.stgit@noble> Message-ID: <151356130240.24290.14619191557989403391.stgit@noble> The only places that cfs_srand is called, the random bits are mixed with bits from get_random_bytes(). So it is equally effective to add entropy to either pool. So we can replace calls to cfs_srand() with calls that add the entropy with add_device_randomness(). That function adds time-based entropy, so we can discard the ktime_get_ts64 calls. One location in lustre_handles.c only adds timebased entropy. This cannot improve the entropy provided by get_random_bytes(), so just discard that call. Signed-off-by: NeilBrown --- drivers/staging/lustre/lnet/lnet/router.c | 15 ++++++--------- drivers/staging/lustre/lustre/llite/super25.c | 17 +++++++---------- .../lustre/lustre/obdclass/lustre_handles.c | 7 ------- 3 files changed, 13 insertions(+), 26 deletions(-) diff --git a/drivers/staging/lustre/lnet/lnet/router.c b/drivers/staging/lustre/lnet/lnet/router.c index e5c9b29e199f..80a7e8a88acb 100644 --- a/drivers/staging/lustre/lnet/lnet/router.c +++ b/drivers/staging/lustre/lnet/lnet/router.c @@ -238,28 +238,25 @@ lnet_find_net_locked(__u32 net) static void lnet_shuffle_seed(void) { static int seeded; - __u32 lnd_type, seed[2]; - struct timespec64 ts; struct lnet_ni *ni; if (seeded) return; - cfs_get_random_bytes(seed, sizeof(seed)); - /* * Nodes with small feet have little entropy * the NID for this node gives the most entropy in the low bits */ list_for_each_entry(ni, &the_lnet.ln_nis, ni_list) { - lnd_type = LNET_NETTYP(LNET_NIDNET(ni->ni_nid)); + __u32 lnd_type, seed; - if (lnd_type != LOLND) - seed[0] ^= (LNET_NIDADDR(ni->ni_nid) | lnd_type); + lnd_type = LNET_NETTYP(LNET_NIDNET(ni->ni_nid)); + if (lnd_type != LOLND) { + seed = (LNET_NIDADDR(ni->ni_nid) | lnd_type); + add_device_randomness(&seed, sizeof(seed)); + } } - ktime_get_ts64(&ts); - cfs_srand(ts.tv_sec ^ seed[0], ts.tv_nsec ^ seed[1]); seeded = 1; } diff --git a/drivers/staging/lustre/lustre/llite/super25.c b/drivers/staging/lustre/lustre/llite/super25.c index 10105339790e..9b0bb3541a84 100644 --- a/drivers/staging/lustre/lustre/llite/super25.c +++ b/drivers/staging/lustre/lustre/llite/super25.c @@ -86,8 +86,7 @@ MODULE_ALIAS_FS("lustre"); static int __init lustre_init(void) { struct lnet_process_id lnet_id; - struct timespec64 ts; - int i, rc, seed[2]; + int i, rc; BUILD_BUG_ON(sizeof(LUSTRE_VOLATILE_HDR) != LUSTRE_VOLATILE_HDR_LEN + 1); @@ -126,22 +125,20 @@ static int __init lustre_init(void) goto out_debugfs; } - cfs_get_random_bytes(seed, sizeof(seed)); - /* Nodes with small feet have little entropy. The NID for this * node gives the most entropy in the low bits */ for (i = 0;; i++) { + u32 seed; + if (LNetGetId(i, &lnet_id) == -ENOENT) break; - - if (LNET_NETTYP(LNET_NIDNET(lnet_id.nid)) != LOLND) - seed[0] ^= LNET_NIDADDR(lnet_id.nid); + if (LNET_NETTYP(LNET_NIDNET(lnet_id.nid)) != LOLND) { + seed = LNET_NIDADDR(lnet_id.nid); + add_device_randomness(&seed, sizeof(seed)); + } } - ktime_get_ts64(&ts); - cfs_srand(ts.tv_sec ^ seed[0], ts.tv_nsec ^ seed[1]); - rc = vvp_global_init(); if (rc != 0) goto out_sysfs; diff --git a/drivers/staging/lustre/lustre/obdclass/lustre_handles.c b/drivers/staging/lustre/lustre/obdclass/lustre_handles.c index 71329adc0318..d1b6c2f134d7 100644 --- a/drivers/staging/lustre/lustre/obdclass/lustre_handles.c +++ b/drivers/staging/lustre/lustre/obdclass/lustre_handles.c @@ -181,8 +181,6 @@ EXPORT_SYMBOL(class_handle_free_cb); int class_handle_init(void) { struct handle_bucket *bucket; - struct timespec64 ts; - int seed[2]; LASSERT(!handle_hash); @@ -198,11 +196,6 @@ int class_handle_init(void) spin_lock_init(&bucket->lock); } - /** bug 21430: add randomness to the initial base */ - cfs_get_random_bytes(seed, sizeof(seed)); - ktime_get_ts64(&ts); - cfs_srand(ts.tv_sec ^ seed[0], ts.tv_nsec ^ seed[1]); - cfs_get_random_bytes(&handle_base, sizeof(handle_base)); LASSERT(handle_base != 0ULL); From neilb at suse.com Mon Dec 18 01:41:42 2017 From: neilb at suse.com (NeilBrown) Date: Mon, 18 Dec 2017 12:41:42 +1100 Subject: [lustre-devel] [PATCH 3/4] staging: lustre: replace cfs_get_random_bytes calls with get_random_byte() In-Reply-To: <151356119565.24290.5570928075046338898.stgit@noble> References: <151356119565.24290.5570928075046338898.stgit@noble> Message-ID: <151356130244.24290.391082297032618393.stgit@noble> The cfs_get_random_bytes() interface adds nothing of value to get_random_byte() (which it uses internally). So just use the standard interface. Signed-off-by: NeilBrown --- drivers/staging/lustre/lustre/include/obd_class.h | 2 +- .../lustre/lustre/obdclass/lustre_handles.c | 2 +- drivers/staging/lustre/lustre/ptlrpc/client.c | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/drivers/staging/lustre/lustre/include/obd_class.h b/drivers/staging/lustre/lustre/include/obd_class.h index 67c535c5aa98..25db7ec6ecd0 100644 --- a/drivers/staging/lustre/lustre/include/obd_class.h +++ b/drivers/staging/lustre/lustre/include/obd_class.h @@ -1562,7 +1562,7 @@ int class_procfs_init(void); int class_procfs_clean(void); /* prng.c */ -#define ll_generate_random_uuid(uuid_out) cfs_get_random_bytes(uuid_out, sizeof(class_uuid_t)) +#define ll_generate_random_uuid(uuid_out) get_random_bytes(uuid_out, sizeof(class_uuid_t)) /* statfs_pack.c */ struct kstatfs; diff --git a/drivers/staging/lustre/lustre/obdclass/lustre_handles.c b/drivers/staging/lustre/lustre/obdclass/lustre_handles.c index d1b6c2f134d7..2d6da2431a09 100644 --- a/drivers/staging/lustre/lustre/obdclass/lustre_handles.c +++ b/drivers/staging/lustre/lustre/obdclass/lustre_handles.c @@ -196,7 +196,7 @@ int class_handle_init(void) spin_lock_init(&bucket->lock); } - cfs_get_random_bytes(&handle_base, sizeof(handle_base)); + get_random_bytes(&handle_base, sizeof(handle_base)); LASSERT(handle_base != 0ULL); return 0; diff --git a/drivers/staging/lustre/lustre/ptlrpc/client.c b/drivers/staging/lustre/lustre/ptlrpc/client.c index 2a9f2f2ebaa8..bac4b2304bad 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/client.c +++ b/drivers/staging/lustre/lustre/ptlrpc/client.c @@ -3067,7 +3067,7 @@ void ptlrpc_init_xid(void) spin_lock_init(&ptlrpc_last_xid_lock); if (now < YEAR_2004) { - cfs_get_random_bytes(&ptlrpc_last_xid, sizeof(ptlrpc_last_xid)); + get_random_bytes(&ptlrpc_last_xid, sizeof(ptlrpc_last_xid)); ptlrpc_last_xid >>= 2; ptlrpc_last_xid |= (1ULL << 61); } else { From neilb at suse.com Mon Dec 18 01:41:42 2017 From: neilb at suse.com (NeilBrown) Date: Mon, 18 Dec 2017 12:41:42 +1100 Subject: [lustre-devel] [PATCH 4/4] staging: lustre: libcfs: remove prng In-Reply-To: <151356119565.24290.5570928075046338898.stgit@noble> References: <151356119565.24290.5570928075046338898.stgit@noble> Message-ID: <151356130248.24290.2371669046568173867.stgit@noble> The cfs prng is no longer used, so discard it. Signed-off-by: NeilBrown --- .../staging/lustre/include/linux/libcfs/libcfs.h | 10 - drivers/staging/lustre/lnet/libcfs/Makefile | 2 drivers/staging/lustre/lnet/libcfs/prng.c | 137 -------------------- 3 files changed, 1 insertion(+), 148 deletions(-) delete mode 100644 drivers/staging/lustre/lnet/libcfs/prng.c diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs.h b/drivers/staging/lustre/include/linux/libcfs/libcfs.h index f2ba83ee5362..ca3472cc952f 100644 --- a/drivers/staging/lustre/include/linux/libcfs/libcfs.h +++ b/drivers/staging/lustre/include/linux/libcfs/libcfs.h @@ -73,16 +73,6 @@ sigset_t cfs_block_sigsinv(unsigned long sigs); void cfs_restore_sigs(sigset_t sigset); void cfs_clear_sigpending(void); -/* - * Random number handling - */ - -/* returns a random 32-bit integer */ -unsigned int cfs_rand(void); -/* seed the generator */ -void cfs_srand(unsigned int seed1, unsigned int seed2); -void cfs_get_random_bytes(void *buf, int size); - struct libcfs_ioctl_handler { struct list_head item; int (*handle_ioctl)(unsigned int cmd, struct libcfs_ioctl_hdr *hdr); diff --git a/drivers/staging/lustre/lnet/libcfs/Makefile b/drivers/staging/lustre/lnet/libcfs/Makefile index 51b529434332..730f2c675047 100644 --- a/drivers/staging/lustre/lnet/libcfs/Makefile +++ b/drivers/staging/lustre/lnet/libcfs/Makefile @@ -15,7 +15,7 @@ libcfs-linux-objs += linux-mem.o libcfs-linux-objs := $(addprefix linux/,$(libcfs-linux-objs)) libcfs-all-objs := debug.o fail.o module.o tracefile.o \ - libcfs_string.o hash.o prng.o \ + libcfs_string.o hash.o \ libcfs_cpu.o libcfs_mem.o libcfs_lock.o libcfs-objs := $(libcfs-linux-objs) $(libcfs-all-objs) diff --git a/drivers/staging/lustre/lnet/libcfs/prng.c b/drivers/staging/lustre/lnet/libcfs/prng.c deleted file mode 100644 index f47cf67a92e3..000000000000 --- a/drivers/staging/lustre/lnet/libcfs/prng.c +++ /dev/null @@ -1,137 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0 -/* - * GPL HEADER START - * - * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. - * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License version 2 only, - * as published by the Free Software Foundation. - * - * This program is distributed in the hope that it will be useful, but - * WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - * General Public License version 2 for more details (a copy is included - * in the LICENSE file that accompanied this code). - * - * You should have received a copy of the GNU General Public License - * version 2 along with this program; If not, see - * http://www.gnu.org/licenses/gpl-2.0.html - * - * GPL HEADER END - */ -/* - * Copyright (c) 2007, 2010, Oracle and/or its affiliates. All rights reserved. - * Use is subject to license terms. - */ -/* - * This file is part of Lustre, http://www.lustre.org/ - * Lustre is a trademark of Sun Microsystems, Inc. - * - * libcfs/libcfs/prng.c - * - * concatenation of following two 16-bit multiply with carry generators - * x(n)=a*x(n-1)+carry mod 2^16 and y(n)=b*y(n-1)+carry mod 2^16, - * number and carry packed within the same 32 bit integer. - * algorithm recommended by Marsaglia - */ - -#include - -/* - * From: George Marsaglia - * Newsgroups: sci.math - * Subject: Re: A RANDOM NUMBER GENERATOR FOR C - * Date: Tue, 30 Sep 1997 05:29:35 -0700 - * - * You may replace the two constants 36969 and 18000 by any - * pair of distinct constants from this list: - * 18000 18030 18273 18513 18879 19074 19098 19164 19215 19584 - * 19599 19950 20088 20508 20544 20664 20814 20970 21153 21243 - * 21423 21723 21954 22125 22188 22293 22860 22938 22965 22974 - * 23109 23124 23163 23208 23508 23520 23553 23658 23865 24114 - * 24219 24660 24699 24864 24948 25023 25308 25443 26004 26088 - * 26154 26550 26679 26838 27183 27258 27753 27795 27810 27834 - * 27960 28320 28380 28689 28710 28794 28854 28959 28980 29013 - * 29379 29889 30135 30345 30459 30714 30903 30963 31059 31083 - * (or any other 16-bit constants k for which both k*2^16-1 - * and k*2^15-1 are prime) - */ - -#define RANDOM_CONST_A 18030 -#define RANDOM_CONST_B 29013 - -static unsigned int seed_x = 521288629; -static unsigned int seed_y = 362436069; - -/** - * cfs_rand - creates new seeds - * - * First it creates new seeds from the previous seeds. Then it generates a - * new pseudo random number for use. - * - * Returns a pseudo-random 32-bit integer - */ -unsigned int cfs_rand(void) -{ - seed_x = RANDOM_CONST_A * (seed_x & 65535) + (seed_x >> 16); - seed_y = RANDOM_CONST_B * (seed_y & 65535) + (seed_y >> 16); - - return ((seed_x << 16) + (seed_y & 65535)); -} -EXPORT_SYMBOL(cfs_rand); - -/** - * cfs_srand - sets the initial seed - * @seed1 : (seed_x) should have the most entropy in the low bits of the word - * @seed2 : (seed_y) should have the most entropy in the high bits of the word - * - * Replaces the original seeds with new values. Used to generate a new pseudo - * random numbers. - */ -void cfs_srand(unsigned int seed1, unsigned int seed2) -{ - if (seed1) - seed_x = seed1; /* use default seeds if parameter is 0 */ - if (seed2) - seed_y = seed2; -} -EXPORT_SYMBOL(cfs_srand); - -/** - * cfs_get_random_bytes - generate a bunch of random numbers - * @buf : buffer to fill with random numbers - * @size: size of passed in buffer - * - * Fills a buffer with random bytes - */ -void cfs_get_random_bytes(void *buf, int size) -{ - int *p = buf; - int rem, tmp; - - LASSERT(size >= 0); - - rem = min((int)((unsigned long)buf & (sizeof(int) - 1)), size); - if (rem) { - get_random_bytes(&tmp, sizeof(tmp)); - tmp ^= cfs_rand(); - memcpy(buf, &tmp, rem); - p = buf + rem; - size -= rem; - } - - while (size >= sizeof(int)) { - get_random_bytes(&tmp, sizeof(tmp)); - *p = cfs_rand() ^ tmp; - size -= sizeof(int); - p++; - } - buf = p; - if (size) { - get_random_bytes(&tmp, sizeof(tmp)); - tmp ^= cfs_rand(); - memcpy(buf, &tmp, size); - } -} -EXPORT_SYMBOL(cfs_get_random_bytes); From neilb at suse.com Mon Dec 18 07:13:13 2017 From: neilb at suse.com (NeilBrown) Date: Mon, 18 Dec 2017 18:13:13 +1100 Subject: [lustre-devel] [PATCH 08/15] staging: lustre: Convert more LIBCFS_ALLOC allocation to direct GFP_KERNEL In-Reply-To: <151355799049.6200.16488920506903028598.stgit@noble> References: <151355781721.6200.2136335532722530242.stgit@noble> <151355799049.6200.16488920506903028598.stgit@noble> Message-ID: <87y3m0ld9y.fsf@notabene.neil.brown.name> Sorry, it seems I mustn't have tested the file version of this patch. That "kvmalloc_node()" was clearly wrong. This version passes testing. Thanks, NeilBrown "git am" will ignore everything before this line: -----------------8<----------------- None of these need to be GFP_NOFS, so use GFP_KERNEL explicitly with kmalloc(), kvmalloc(), or kvmalloc_array(). Changing matching LIBCFS_FREE() to kfree() or kvfree() Signed-off-by: NeilBrown --- .../lustre/lnet/libcfs/linux/linux-module.c | 4 +-- drivers/staging/lustre/lnet/libcfs/module.c | 9 +++--- drivers/staging/lustre/lnet/lnet/api-ni.c | 17 +++++------ drivers/staging/lustre/lnet/lnet/config.c | 34 +++++++++------------- 4 files changed, 26 insertions(+), 38 deletions(-) diff --git a/drivers/staging/lustre/lnet/libcfs/linux/linux-module.c b/drivers/staging/lustre/lnet/libcfs/linux/linux-module.c index b5746230ab31..ddf625669bff 100644 --- a/drivers/staging/lustre/lnet/libcfs/linux/linux-module.c +++ b/drivers/staging/lustre/lnet/libcfs/linux/linux-module.c @@ -146,7 +146,7 @@ int libcfs_ioctl_getdata(struct libcfs_ioctl_hdr **hdr_pp, return -EINVAL; } - LIBCFS_ALLOC(*hdr_pp, hdr.ioc_len); + *hdr_pp = kvmalloc(hdr.ioc_len, GFP_KERNEL); if (!*hdr_pp) return -ENOMEM; @@ -164,7 +164,7 @@ int libcfs_ioctl_getdata(struct libcfs_ioctl_hdr **hdr_pp, return 0; free: - LIBCFS_FREE(*hdr_pp, hdr.ioc_len); + kvfree(*hdr_pp); return err; } diff --git a/drivers/staging/lustre/lnet/libcfs/module.c b/drivers/staging/lustre/lnet/libcfs/module.c index 4ead55920e79..1bd33497a7c0 100644 --- a/drivers/staging/lustre/lnet/libcfs/module.c +++ b/drivers/staging/lustre/lnet/libcfs/module.c @@ -156,7 +156,7 @@ int libcfs_ioctl(unsigned long cmd, void __user *uparam) break; } } out: - LIBCFS_FREE(hdr, hdr->ioc_len); + kvfree(hdr); return err; } @@ -302,7 +302,7 @@ static int __proc_cpt_table(void *data, int write, LASSERT(cfs_cpt_table); while (1) { - LIBCFS_ALLOC(buf, len); + buf = kzalloc(len, GFP_KERNEL); if (!buf) return -ENOMEM; @@ -311,7 +311,7 @@ static int __proc_cpt_table(void *data, int write, break; if (rc == -EFBIG) { - LIBCFS_FREE(buf, len); + kfree(buf); len <<= 1; continue; } @@ -325,8 +325,7 @@ static int __proc_cpt_table(void *data, int write, rc = cfs_trace_copyout_string(buffer, nob, buf + pos, NULL); out: - if (buf) - LIBCFS_FREE(buf, len); + kfree(buf); return rc; } diff --git a/drivers/staging/lustre/lnet/lnet/api-ni.c b/drivers/staging/lustre/lnet/lnet/api-ni.c index e8f623190133..6a1fb0397604 100644 --- a/drivers/staging/lustre/lnet/lnet/api-ni.c +++ b/drivers/staging/lustre/lnet/lnet/api-ni.c @@ -108,7 +108,8 @@ lnet_create_remote_nets_table(void) LASSERT(!the_lnet.ln_remote_nets_hash); LASSERT(the_lnet.ln_remote_nets_hbits > 0); - LIBCFS_ALLOC(hash, LNET_REMOTE_NETS_HASH_SIZE * sizeof(*hash)); + hash = kvmalloc_array(LNET_REMOTE_NETS_HASH_SIZE, sizeof(*hash), + GFP_KERNEL); if (!hash) { CERROR("Failed to create remote nets hash table\n"); return -ENOMEM; @@ -131,9 +132,7 @@ lnet_destroy_remote_nets_table(void) for (i = 0; i < LNET_REMOTE_NETS_HASH_SIZE; i++) LASSERT(list_empty(&the_lnet.ln_remote_nets_hash[i])); - LIBCFS_FREE(the_lnet.ln_remote_nets_hash, - LNET_REMOTE_NETS_HASH_SIZE * - sizeof(the_lnet.ln_remote_nets_hash[0])); + kvfree(the_lnet.ln_remote_nets_hash); the_lnet.ln_remote_nets_hash = NULL; } @@ -831,7 +830,7 @@ lnet_ping_info_create(int num_ni) unsigned int infosz; infosz = offsetof(struct lnet_ping_info, pi_ni[num_ni]); - LIBCFS_ALLOC(ping_info, infosz); + ping_info = kvzalloc(infosz, GFP_KERNEL); if (!ping_info) { CERROR("Can't allocate ping info[%d]\n", num_ni); return NULL; @@ -864,9 +863,7 @@ lnet_get_ni_count(void) static inline void lnet_ping_info_free(struct lnet_ping_info *pinfo) { - LIBCFS_FREE(pinfo, - offsetof(struct lnet_ping_info, - pi_ni[pinfo->pi_nnis])); + kvfree(pinfo); } static void @@ -2160,7 +2157,7 @@ static int lnet_ping(struct lnet_process_id id, int timeout_ms, if (id.pid == LNET_PID_ANY) id.pid = LNET_PID_LUSTRE; - LIBCFS_ALLOC(info, infosz); + info = kzalloc(infosz, GFP_KERNEL); if (!info) return -ENOMEM; @@ -2310,6 +2307,6 @@ static int lnet_ping(struct lnet_process_id id, int timeout_ms, LASSERT(!rc2); out_0: - LIBCFS_FREE(info, infosz); + kfree(info); return rc; } diff --git a/drivers/staging/lustre/lnet/lnet/config.c b/drivers/staging/lustre/lnet/lnet/config.c index 66a222e5220b..fd53c74766a7 100644 --- a/drivers/staging/lustre/lnet/lnet/config.c +++ b/drivers/staging/lustre/lnet/lnet/config.c @@ -109,10 +109,8 @@ lnet_ni_free(struct lnet_ni *ni) if (ni->ni_lnd_tunables) kfree(ni->ni_lnd_tunables); - for (i = 0; i < LNET_MAX_INTERFACES && ni->ni_interfaces[i]; i++) { - LIBCFS_FREE(ni->ni_interfaces[i], - strlen(ni->ni_interfaces[i]) + 1); - } + for (i = 0; i < LNET_MAX_INTERFACES && ni->ni_interfaces[i]; i++) + kfree(ni->ni_interfaces[i]); /* release reference to net namespace */ if (ni->ni_net_ns) @@ -198,7 +196,6 @@ int lnet_parse_networks(struct list_head *nilist, char *networks) { struct cfs_expr_list *el = NULL; - int tokensize; char *tokens; char *str; char *tmp; @@ -219,15 +216,12 @@ lnet_parse_networks(struct list_head *nilist, char *networks) return -EINVAL; } - tokensize = strlen(networks) + 1; - - LIBCFS_ALLOC(tokens, tokensize); + tokens = kstrdup(networks, GFP_KERNEL); if (!tokens) { CERROR("Can't allocate net tokens\n"); return -ENOMEM; } - memcpy(tokens, networks, tokensize); tmp = tokens; str = tokens; @@ -349,14 +343,11 @@ lnet_parse_networks(struct list_head *nilist, char *networks) * The newly allocated ni_interfaces[] can be * freed when freeing the NI */ - LIBCFS_ALLOC(ni->ni_interfaces[niface], - strlen(iface) + 1); + ni->ni_interfaces[niface] = kstrdup(iface, GFP_KERNEL); if (!ni->ni_interfaces[niface]) { CERROR("Can't allocate net interface name\n"); goto failed; } - strncpy(ni->ni_interfaces[niface], iface, - strlen(iface)); niface++; iface = comma; } while (iface); @@ -384,7 +375,7 @@ lnet_parse_networks(struct list_head *nilist, char *networks) list_for_each(temp_node, nilist) nnets++; - LIBCFS_FREE(tokens, tokensize); + kfree(tokens); return nnets; failed_syntax: @@ -400,7 +391,7 @@ lnet_parse_networks(struct list_head *nilist, char *networks) if (el) cfs_expr_list_free(el); - LIBCFS_FREE(tokens, tokensize); + kfree(tokens); return -EINVAL; } @@ -424,7 +415,7 @@ lnet_new_text_buf(int str_len) return NULL; } - LIBCFS_ALLOC(ltb, nob); + ltb = kzalloc(nob, GFP_KERNEL); if (!ltb) return NULL; @@ -438,7 +429,7 @@ static void lnet_free_text_buf(struct lnet_text_buf *ltb) { lnet_tbnob -= ltb->ltb_size; - LIBCFS_FREE(ltb, ltb->ltb_size); + kfree(ltb); } static void @@ -1156,7 +1147,7 @@ lnet_ipaddr_enumerate(__u32 **ipaddrsp) if (nif <= 0) return nif; - LIBCFS_ALLOC(ipaddrs, nif * sizeof(*ipaddrs)); + ipaddrs = kzalloc(nif * sizeof(*ipaddrs), GFP_KERNEL); if (!ipaddrs) { CERROR("Can't allocate ipaddrs[%d]\n", nif); lnet_ipif_free_enumeration(ifnames, nif); @@ -1189,7 +1180,8 @@ lnet_ipaddr_enumerate(__u32 **ipaddrsp) *ipaddrsp = ipaddrs; } else { if (nip > 0) { - LIBCFS_ALLOC(ipaddrs2, nip * sizeof(*ipaddrs2)); + ipaddrs2 = kzalloc(nip * sizeof(*ipaddrs2), + GFP_KERNEL); if (!ipaddrs2) { CERROR("Can't allocate ipaddrs[%d]\n", nip); nip = -ENOMEM; @@ -1200,7 +1192,7 @@ lnet_ipaddr_enumerate(__u32 **ipaddrsp) rc = nip; } } - LIBCFS_FREE(ipaddrs, nip * sizeof(*ipaddrs)); + kfree(ipaddrs); } return nip; } @@ -1226,7 +1218,7 @@ lnet_parse_ip2nets(char **networksp, char *ip2nets) } rc = lnet_match_networks(networksp, ip2nets, ipaddrs, nip); - LIBCFS_FREE(ipaddrs, nip * sizeof(*ipaddrs)); + kfree(ipaddrs); if (rc < 0) { LCONSOLE_ERROR_MSG(0x119, "Error %d parsing ip2nets\n", rc); -- 2.14.0.rc0.dirty -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 832 bytes Desc: not available URL: From neilb at suse.com Mon Dec 18 07:17:59 2017 From: neilb at suse.com (NeilBrown) Date: Mon, 18 Dec 2017 18:17:59 +1100 Subject: [lustre-devel] [PATCH SERIES 5: 00/16] staging: lustre: use standard wait_event macros Message-ID: <151358127190.5099.12792810096274074963.stgit@noble> Lustre has l_wait_event() which is a complex macro that does similar things to the wait_event macro family in Linux. This patch series converts all l_wait_event to something more familiar to Linux developers. Some of the conversions are subtle. I think I've understood the code and got the conversion correct, but I've quite possibly messed up. If you only review one series, this is the one that demands review. I haven't converted the single use of l_wait_event_exclusive_head(). That needs more thought. This is the last series for today. I do have some more patches, but that are all individual patches that I hope to send as a set of ad-hoc patches tomorrow. Thanks, NeilBrown --- NeilBrown (16): staging: lustre: discard SVC_SIGNAL and related functions staging: lustre: replace simple cases of l_wait_event() with wait_event(). staging: lustre: discard cfs_time_seconds() staging: lustre: use wait_event_timeout() where appropriate. staging: lustre: introduce and use l_wait_event_abortable() staging: lustre: simplify l_wait_event when intr handler but no timeout. staging: lustre: simplify waiting in ldlm_completion_ast() staging: lustre: open code polling loop instead of using l_wait_event() staging: lustre: simplify waiting in ptlrpc_invalidate_import() staging: lustre: remove back_to_sleep() and use loops. staging: lustre: make polling loop in ptlrpc_unregister_bulk more obvious staging: lustre: use wait_event_timeout in ptlrpcd() staging: lustre: improve waiting in sptlrpc_req_refresh_ctx staging: lustre: use explicit poll loop in ptlrpc_service_unlink_rqbd staging: lustre: use explicit poll loop in ptlrpc_unregister_reply staging: lustre: remove l_wait_event from ptlrpc_set_wait .../lustre/include/linux/libcfs/libcfs_debug.h | 4 - .../lustre/include/linux/libcfs/libcfs_time.h | 2 .../lustre/include/linux/libcfs/linux/linux-time.h | 7 - .../staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c | 8 + .../staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c | 4 - .../staging/lustre/lnet/klnds/socklnd/socklnd.c | 6 - .../staging/lustre/lnet/klnds/socklnd/socklnd_cb.c | 22 ++- drivers/staging/lustre/lnet/libcfs/debug.c | 2 drivers/staging/lustre/lnet/libcfs/fail.c | 2 drivers/staging/lustre/lnet/libcfs/tracefile.c | 4 - drivers/staging/lustre/lnet/lnet/acceptor.c | 2 drivers/staging/lustre/lnet/lnet/api-ni.c | 4 - drivers/staging/lustre/lnet/lnet/lib-move.c | 4 - drivers/staging/lustre/lnet/lnet/net_fault.c | 14 +- drivers/staging/lustre/lnet/lnet/peer.c | 2 drivers/staging/lustre/lnet/lnet/router.c | 8 + drivers/staging/lustre/lnet/selftest/conrpc.c | 4 - drivers/staging/lustre/lnet/selftest/rpc.c | 2 drivers/staging/lustre/lnet/selftest/selftest.h | 2 drivers/staging/lustre/lnet/selftest/timer.c | 2 drivers/staging/lustre/lustre/include/lustre_dlm.h | 2 drivers/staging/lustre/lustre/include/lustre_lib.h | 126 ++++++++++++++++++-- drivers/staging/lustre/lustre/include/lustre_mdc.h | 2 drivers/staging/lustre/lustre/include/lustre_net.h | 8 - drivers/staging/lustre/lustre/ldlm/ldlm_flock.c | 30 +---- drivers/staging/lustre/lustre/ldlm/ldlm_lock.c | 14 +- drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c | 12 +- drivers/staging/lustre/lustre/ldlm/ldlm_pool.c | 17 +-- drivers/staging/lustre/lustre/ldlm/ldlm_request.c | 51 +++----- drivers/staging/lustre/lustre/ldlm/ldlm_resource.c | 14 +- drivers/staging/lustre/lustre/llite/llite_lib.c | 23 +--- drivers/staging/lustre/lustre/llite/statahead.c | 60 ++++------ drivers/staging/lustre/lustre/lov/lov_object.c | 6 - drivers/staging/lustre/lustre/lov/lov_request.c | 12 +- drivers/staging/lustre/lustre/mdc/mdc_request.c | 5 - drivers/staging/lustre/lustre/mgc/mgc_request.c | 19 +-- drivers/staging/lustre/lustre/obdclass/cl_io.c | 23 ++-- drivers/staging/lustre/lustre/obdclass/genops.c | 24 +--- drivers/staging/lustre/lustre/obdclass/llog_obd.c | 5 - .../staging/lustre/lustre/obdecho/echo_client.c | 2 drivers/staging/lustre/lustre/osc/osc_cache.c | 28 ++-- drivers/staging/lustre/lustre/osc/osc_object.c | 6 - drivers/staging/lustre/lustre/osc/osc_page.c | 6 - drivers/staging/lustre/lustre/osc/osc_request.c | 6 - drivers/staging/lustre/lustre/ptlrpc/client.c | 101 +++++++--------- drivers/staging/lustre/lustre/ptlrpc/events.c | 7 - drivers/staging/lustre/lustre/ptlrpc/import.c | 51 +++----- drivers/staging/lustre/lustre/ptlrpc/niobuf.c | 14 +- .../staging/lustre/lustre/ptlrpc/pack_generic.c | 9 + drivers/staging/lustre/lustre/ptlrpc/pinger.c | 28 ++-- .../staging/lustre/lustre/ptlrpc/ptlrpc_internal.h | 2 drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c | 17 +-- drivers/staging/lustre/lustre/ptlrpc/recover.c | 12 +- drivers/staging/lustre/lustre/ptlrpc/sec.c | 23 ++-- drivers/staging/lustre/lustre/ptlrpc/sec_gc.c | 23 +--- drivers/staging/lustre/lustre/ptlrpc/service.c | 40 +++--- 56 files changed, 446 insertions(+), 487 deletions(-) -- Signature From neilb at suse.com Mon Dec 18 07:17:59 2017 From: neilb at suse.com (NeilBrown) Date: Mon, 18 Dec 2017 18:17:59 +1100 Subject: [lustre-devel] [PATCH 01/16] staging: lustre: discard SVC_SIGNAL and related functions In-Reply-To: <151358127190.5099.12792810096274074963.stgit@noble> References: <151358127190.5099.12792810096274074963.stgit@noble> Message-ID: <151358147976.5099.8293758151714003196.stgit@noble> This flag is never set, so remove checks and remove the flag. Signed-off-by: NeilBrown --- drivers/staging/lustre/lustre/include/lustre_net.h | 6 ------ drivers/staging/lustre/lustre/ptlrpc/sec_gc.c | 4 +--- 2 files changed, 1 insertion(+), 9 deletions(-) diff --git a/drivers/staging/lustre/lustre/include/lustre_net.h b/drivers/staging/lustre/lustre/include/lustre_net.h index 3ff5de4770e8..4c665eca2467 100644 --- a/drivers/staging/lustre/lustre/include/lustre_net.h +++ b/drivers/staging/lustre/lustre/include/lustre_net.h @@ -1260,7 +1260,6 @@ enum { SVC_STARTING = 1 << 2, SVC_RUNNING = 1 << 3, SVC_EVENT = 1 << 4, - SVC_SIGNAL = 1 << 5, }; #define PTLRPC_THR_NAME_LEN 32 @@ -1333,11 +1332,6 @@ static inline int thread_is_event(struct ptlrpc_thread *thread) return !!(thread->t_flags & SVC_EVENT); } -static inline int thread_is_signal(struct ptlrpc_thread *thread) -{ - return !!(thread->t_flags & SVC_SIGNAL); -} - static inline void thread_clear_flags(struct ptlrpc_thread *thread, __u32 flags) { thread->t_flags &= ~flags; diff --git a/drivers/staging/lustre/lustre/ptlrpc/sec_gc.c b/drivers/staging/lustre/lustre/ptlrpc/sec_gc.c index 8d1e0edfcede..d85c8638c009 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/sec_gc.c +++ b/drivers/staging/lustre/lustre/ptlrpc/sec_gc.c @@ -153,7 +153,6 @@ static int sec_gc_main(void *arg) while (1) { struct ptlrpc_sec *sec; - thread_clear_flags(thread, SVC_SIGNAL); sec_process_ctx_list(); again: /* go through sec list do gc. @@ -184,8 +183,7 @@ static int sec_gc_main(void *arg) lwi = LWI_TIMEOUT(msecs_to_jiffies(SEC_GC_INTERVAL * MSEC_PER_SEC), NULL, NULL); l_wait_event(thread->t_ctl_waitq, - thread_is_stopping(thread) || - thread_is_signal(thread), + thread_is_stopping(thread), &lwi); if (thread_test_and_clear_flags(thread, SVC_STOPPING)) From neilb at suse.com Mon Dec 18 07:17:59 2017 From: neilb at suse.com (NeilBrown) Date: Mon, 18 Dec 2017 18:17:59 +1100 Subject: [lustre-devel] [PATCH 03/16] staging: lustre: discard cfs_time_seconds() In-Reply-To: <151358127190.5099.12792810096274074963.stgit@noble> References: <151358127190.5099.12792810096274074963.stgit@noble> Message-ID: <151358147986.5099.7604772313072719201.stgit@noble> cfs_time_seconds() converts a number of second to the matching number of jiffies. The standard way to do this in Linux is "* HZ". So discard cfs_time_seconds() and use "* HZ" instead. Signed-off-by: NeilBrown --- .../lustre/include/linux/libcfs/libcfs_debug.h | 4 ++-- .../lustre/include/linux/libcfs/libcfs_time.h | 2 +- .../lustre/include/linux/libcfs/linux/linux-time.h | 7 +----- .../staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c | 8 ++++--- .../staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c | 4 ++-- .../staging/lustre/lnet/klnds/socklnd/socklnd.c | 6 +++-- .../staging/lustre/lnet/klnds/socklnd/socklnd_cb.c | 22 ++++++++++---------- drivers/staging/lustre/lnet/libcfs/debug.c | 2 +- drivers/staging/lustre/lnet/libcfs/fail.c | 2 +- drivers/staging/lustre/lnet/libcfs/tracefile.c | 4 ++-- drivers/staging/lustre/lnet/lnet/acceptor.c | 2 +- drivers/staging/lustre/lnet/lnet/api-ni.c | 4 ++-- drivers/staging/lustre/lnet/lnet/lib-move.c | 4 ++-- drivers/staging/lustre/lnet/lnet/net_fault.c | 14 +++++-------- drivers/staging/lustre/lnet/lnet/peer.c | 2 +- drivers/staging/lustre/lnet/lnet/router.c | 8 ++++--- drivers/staging/lustre/lnet/selftest/conrpc.c | 4 ++-- drivers/staging/lustre/lnet/selftest/rpc.c | 2 +- drivers/staging/lustre/lnet/selftest/selftest.h | 2 +- drivers/staging/lustre/lnet/selftest/timer.c | 2 +- drivers/staging/lustre/lustre/include/lustre_dlm.h | 2 +- drivers/staging/lustre/lustre/include/lustre_mdc.h | 2 +- drivers/staging/lustre/lustre/include/lustre_net.h | 2 +- drivers/staging/lustre/lustre/ldlm/ldlm_lock.c | 2 +- drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c | 4 ++-- drivers/staging/lustre/lustre/ldlm/ldlm_pool.c | 2 +- drivers/staging/lustre/lustre/ldlm/ldlm_request.c | 2 +- drivers/staging/lustre/lustre/ldlm/ldlm_resource.c | 2 +- drivers/staging/lustre/lustre/llite/llite_lib.c | 4 ++-- drivers/staging/lustre/lustre/llite/statahead.c | 2 +- drivers/staging/lustre/lustre/lov/lov_request.c | 4 ++-- drivers/staging/lustre/lustre/mdc/mdc_request.c | 2 +- drivers/staging/lustre/lustre/mgc/mgc_request.c | 2 +- drivers/staging/lustre/lustre/obdclass/cl_io.c | 2 +- .../staging/lustre/lustre/obdecho/echo_client.c | 2 +- drivers/staging/lustre/lustre/osc/osc_cache.c | 4 ++-- drivers/staging/lustre/lustre/osc/osc_object.c | 2 +- drivers/staging/lustre/lustre/ptlrpc/client.c | 10 +++++---- drivers/staging/lustre/lustre/ptlrpc/events.c | 2 +- drivers/staging/lustre/lustre/ptlrpc/import.c | 15 ++++++-------- drivers/staging/lustre/lustre/ptlrpc/niobuf.c | 4 ++-- .../staging/lustre/lustre/ptlrpc/pack_generic.c | 2 +- drivers/staging/lustre/lustre/ptlrpc/pinger.c | 8 ++++--- drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c | 4 ++-- drivers/staging/lustre/lustre/ptlrpc/recover.c | 2 +- drivers/staging/lustre/lustre/ptlrpc/service.c | 8 ++++--- 46 files changed, 96 insertions(+), 106 deletions(-) diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_debug.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_debug.h index 1b98f0953afb..9290a19429e7 100644 --- a/drivers/staging/lustre/include/linux/libcfs/libcfs_debug.h +++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_debug.h @@ -66,8 +66,8 @@ extern unsigned int libcfs_panic_on_lbug; # define DEBUG_SUBSYSTEM S_UNDEFINED #endif -#define CDEBUG_DEFAULT_MAX_DELAY (cfs_time_seconds(600)) /* jiffies */ -#define CDEBUG_DEFAULT_MIN_DELAY ((cfs_time_seconds(1) + 1) / 2) /* jiffies */ +#define CDEBUG_DEFAULT_MAX_DELAY (600 * HZ) /* jiffies */ +#define CDEBUG_DEFAULT_MIN_DELAY ((HZ + 1) / 2) /* jiffies */ #define CDEBUG_DEFAULT_BACKOFF 2 struct cfs_debug_limit_state { unsigned long cdls_next; diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_time.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_time.h index 9699646decb9..c4f25be78268 100644 --- a/drivers/staging/lustre/include/linux/libcfs/libcfs_time.h +++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_time.h @@ -62,7 +62,7 @@ static inline int cfs_time_aftereq(unsigned long t1, unsigned long t2) static inline unsigned long cfs_time_shift(int seconds) { - return cfs_time_add(cfs_time_current(), cfs_time_seconds(seconds)); + return cfs_time_add(cfs_time_current(), seconds * HZ); } /* diff --git a/drivers/staging/lustre/include/linux/libcfs/linux/linux-time.h b/drivers/staging/lustre/include/linux/libcfs/linux/linux-time.h index aece13698eb4..805cb326af86 100644 --- a/drivers/staging/lustre/include/linux/libcfs/linux/linux-time.h +++ b/drivers/staging/lustre/include/linux/libcfs/linux/linux-time.h @@ -65,11 +65,6 @@ static inline unsigned long cfs_time_current(void) return jiffies; } -static inline long cfs_time_seconds(int seconds) -{ - return ((long)seconds) * msecs_to_jiffies(MSEC_PER_SEC); -} - static inline long cfs_duration_sec(long d) { return d / msecs_to_jiffies(MSEC_PER_SEC); @@ -85,7 +80,7 @@ static inline u64 cfs_time_add_64(u64 t, u64 d) static inline u64 cfs_time_shift_64(int seconds) { return cfs_time_add_64(cfs_time_current_64(), - cfs_time_seconds(seconds)); + seconds * HZ); } static inline int cfs_time_before_64(u64 t1, u64 t2) diff --git a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c index f59c20a870a8..ad59dd761b51 100644 --- a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c +++ b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c @@ -1220,7 +1220,7 @@ static struct kib_hca_dev *kiblnd_current_hdev(struct kib_dev *dev) CDEBUG(D_NET, "%s: Wait for failover\n", dev->ibd_ifname); set_current_state(TASK_INTERRUPTIBLE); - schedule_timeout(cfs_time_seconds(1) / 100); + schedule_timeout(HZ / 100); read_lock_irqsave(&kiblnd_data.kib_global_lock, flags); } @@ -1931,7 +1931,7 @@ struct list_head *kiblnd_pool_alloc_node(struct kib_poolset *ps) set_current_state(TASK_INTERRUPTIBLE); schedule_timeout(interval); - if (interval < cfs_time_seconds(1)) + if (interval < HZ) interval *= 2; goto again; @@ -2568,7 +2568,7 @@ static void kiblnd_base_shutdown(void) "Waiting for %d threads to terminate\n", atomic_read(&kiblnd_data.kib_nthreads)); set_current_state(TASK_UNINTERRUPTIBLE); - schedule_timeout(cfs_time_seconds(1)); + schedule_timeout(HZ); } /* fall through */ @@ -2619,7 +2619,7 @@ static void kiblnd_shutdown(struct lnet_ni *ni) libcfs_nid2str(ni->ni_nid), atomic_read(&net->ibn_npeers)); set_current_state(TASK_UNINTERRUPTIBLE); - schedule_timeout(cfs_time_seconds(1)); + schedule_timeout(HZ); } kiblnd_net_fini_pools(net); diff --git a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c index 9b3328c5d1e7..0b30c205e760 100644 --- a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c +++ b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c @@ -3726,8 +3726,8 @@ kiblnd_failover_thread(void *arg) add_wait_queue(&kiblnd_data.kib_failover_waitq, &wait); write_unlock_irqrestore(glock, flags); - rc = schedule_timeout(long_sleep ? cfs_time_seconds(10) : - cfs_time_seconds(1)); + rc = schedule_timeout(long_sleep ? 10 * HZ : + HZ); remove_wait_queue(&kiblnd_data.kib_failover_waitq, &wait); write_lock_irqsave(glock, flags); diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c index 7dba949a95a7..6ab876d8c744 100644 --- a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c +++ b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c @@ -1677,7 +1677,7 @@ ksocknal_destroy_conn(struct ksock_conn *conn) switch (conn->ksnc_rx_state) { case SOCKNAL_RX_LNET_PAYLOAD: last_rcv = conn->ksnc_rx_deadline - - cfs_time_seconds(*ksocknal_tunables.ksnd_timeout); + *ksocknal_tunables.ksnd_timeout * HZ; CERROR("Completing partial receive from %s[%d], ip %pI4h:%d, with error, wanted: %zd, left: %d, last alive is %ld secs ago\n", libcfs_id2str(conn->ksnc_peer->ksnp_id), conn->ksnc_type, &conn->ksnc_ipaddr, conn->ksnc_port, @@ -2361,7 +2361,7 @@ ksocknal_base_shutdown(void) ksocknal_data.ksnd_nthreads); read_unlock(&ksocknal_data.ksnd_global_lock); set_current_state(TASK_UNINTERRUPTIBLE); - schedule_timeout(cfs_time_seconds(1)); + schedule_timeout(HZ); read_lock(&ksocknal_data.ksnd_global_lock); } read_unlock(&ksocknal_data.ksnd_global_lock); @@ -2604,7 +2604,7 @@ ksocknal_shutdown(struct lnet_ni *ni) "waiting for %d peers to disconnect\n", net->ksnn_npeers); set_current_state(TASK_UNINTERRUPTIBLE); - schedule_timeout(cfs_time_seconds(1)); + schedule_timeout(HZ); ksocknal_debug_peerhash(ni); diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c index 11fd3a36424f..63e452f666bf 100644 --- a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c +++ b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c @@ -189,7 +189,7 @@ ksocknal_transmit(struct ksock_conn *conn, struct ksock_tx *tx) if (ksocknal_data.ksnd_stall_tx) { set_current_state(TASK_UNINTERRUPTIBLE); - schedule_timeout(cfs_time_seconds(ksocknal_data.ksnd_stall_tx)); + schedule_timeout(ksocknal_data.ksnd_stall_tx * HZ); } LASSERT(tx->tx_resid); @@ -294,7 +294,7 @@ ksocknal_receive(struct ksock_conn *conn) if (ksocknal_data.ksnd_stall_rx) { set_current_state(TASK_UNINTERRUPTIBLE); - schedule_timeout(cfs_time_seconds(ksocknal_data.ksnd_stall_rx)); + schedule_timeout(ksocknal_data.ksnd_stall_rx * HZ); } rc = ksocknal_connsock_addref(conn); @@ -1780,7 +1780,7 @@ ksocknal_connect(struct ksock_route *route) int rc = 0; deadline = cfs_time_add(cfs_time_current(), - cfs_time_seconds(*ksocknal_tunables.ksnd_timeout)); + *ksocknal_tunables.ksnd_timeout * HZ); write_lock_bh(&ksocknal_data.ksnd_global_lock); @@ -1878,7 +1878,7 @@ ksocknal_connect(struct ksock_route *route) * so min_reconnectms should be good heuristic */ route->ksnr_retry_interval = - cfs_time_seconds(*ksocknal_tunables.ksnd_min_reconnectms) / 1000; + *ksocknal_tunables.ksnd_min_reconnectms * HZ / 1000; route->ksnr_timeout = cfs_time_add(cfs_time_current(), route->ksnr_retry_interval); } @@ -1899,10 +1899,10 @@ ksocknal_connect(struct ksock_route *route) route->ksnr_retry_interval *= 2; route->ksnr_retry_interval = max(route->ksnr_retry_interval, - cfs_time_seconds(*ksocknal_tunables.ksnd_min_reconnectms) / 1000); + (long)*ksocknal_tunables.ksnd_min_reconnectms * HZ / 1000); route->ksnr_retry_interval = min(route->ksnr_retry_interval, - cfs_time_seconds(*ksocknal_tunables.ksnd_max_reconnectms) / 1000); + (long)*ksocknal_tunables.ksnd_max_reconnectms * HZ / 1000); LASSERT(route->ksnr_retry_interval); route->ksnr_timeout = cfs_time_add(cfs_time_current(), @@ -1972,7 +1972,7 @@ ksocknal_connd_check_start(time64_t sec, long *timeout) if (sec - ksocknal_data.ksnd_connd_failed_stamp <= 1) { /* may run out of resource, retry later */ - *timeout = cfs_time_seconds(1); + *timeout = HZ; return 0; } @@ -2031,8 +2031,8 @@ ksocknal_connd_check_stop(time64_t sec, long *timeout) val = (int)(ksocknal_data.ksnd_connd_starting_stamp + SOCKNAL_CONND_TIMEOUT - sec); - *timeout = (val > 0) ? cfs_time_seconds(val) : - cfs_time_seconds(SOCKNAL_CONND_TIMEOUT); + *timeout = (val > 0) ? val * HZ : + SOCKNAL_CONND_TIMEOUT * HZ; if (val > 0) return 0; @@ -2307,7 +2307,7 @@ ksocknal_send_keepalive_locked(struct ksock_peer *peer) if (*ksocknal_tunables.ksnd_keepalive <= 0 || time_before(cfs_time_current(), cfs_time_add(peer->ksnp_last_alive, - cfs_time_seconds(*ksocknal_tunables.ksnd_keepalive)))) + *ksocknal_tunables.ksnd_keepalive * HZ))) return 0; if (time_before(cfs_time_current(), peer->ksnp_send_keepalive)) @@ -2563,7 +2563,7 @@ ksocknal_reaper(void *arg) ksocknal_data.ksnd_peer_hash_size; } - deadline = cfs_time_add(deadline, cfs_time_seconds(p)); + deadline = cfs_time_add(deadline, p * HZ); } if (nenomem_conns) { diff --git a/drivers/staging/lustre/lnet/libcfs/debug.c b/drivers/staging/lustre/lnet/libcfs/debug.c index 551c45bf4108..c70d2ae29b11 100644 --- a/drivers/staging/lustre/lnet/libcfs/debug.c +++ b/drivers/staging/lustre/lnet/libcfs/debug.c @@ -113,7 +113,7 @@ static int param_set_delay_minmax(const char *val, if (rc) return -EINVAL; - d = cfs_time_seconds(sec) / 100; + d = sec * HZ / 100; if (d < min || d > max) return -EINVAL; diff --git a/drivers/staging/lustre/lnet/libcfs/fail.c b/drivers/staging/lustre/lnet/libcfs/fail.c index 39439b303d65..d3f1e866c6a7 100644 --- a/drivers/staging/lustre/lnet/libcfs/fail.c +++ b/drivers/staging/lustre/lnet/libcfs/fail.c @@ -134,7 +134,7 @@ int __cfs_fail_timeout_set(u32 id, u32 value, int ms, int set) CERROR("cfs_fail_timeout id %x sleeping for %dms\n", id, ms); set_current_state(TASK_UNINTERRUPTIBLE); - schedule_timeout(cfs_time_seconds(ms) / 1000); + schedule_timeout(ms * HZ / 1000); CERROR("cfs_fail_timeout id %x awake\n", id); } return ret; diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.c b/drivers/staging/lustre/lnet/libcfs/tracefile.c index da2844f37edf..8c33c8fa66e2 100644 --- a/drivers/staging/lustre/lnet/libcfs/tracefile.c +++ b/drivers/staging/lustre/lnet/libcfs/tracefile.c @@ -441,7 +441,7 @@ int libcfs_debug_vmsg2(struct libcfs_debug_msg_data *msgdata, if (cfs_time_after(cfs_time_current(), cdls->cdls_next + libcfs_console_max_delay + - cfs_time_seconds(10))) { + 10 * HZ)) { /* last timeout was a long time ago */ cdls->cdls_delay /= libcfs_console_backoff * 4; } else { @@ -1071,7 +1071,7 @@ static int tracefiled(void *arg) init_waitqueue_entry(&__wait, current); add_wait_queue(&tctl->tctl_waitq, &__wait); set_current_state(TASK_INTERRUPTIBLE); - schedule_timeout(cfs_time_seconds(1)); + schedule_timeout(HZ); remove_wait_queue(&tctl->tctl_waitq, &__wait); } complete(&tctl->tctl_stop); diff --git a/drivers/staging/lustre/lnet/lnet/acceptor.c b/drivers/staging/lustre/lnet/lnet/acceptor.c index ee85cab6f437..6c1f4941d4ba 100644 --- a/drivers/staging/lustre/lnet/lnet/acceptor.c +++ b/drivers/staging/lustre/lnet/lnet/acceptor.c @@ -365,7 +365,7 @@ lnet_acceptor(void *arg) if (rc != -EAGAIN) { CWARN("Accept error %d: pausing...\n", rc); set_current_state(TASK_UNINTERRUPTIBLE); - schedule_timeout(cfs_time_seconds(1)); + schedule_timeout(HZ); } continue; } diff --git a/drivers/staging/lustre/lnet/lnet/api-ni.c b/drivers/staging/lustre/lnet/lnet/api-ni.c index 6a1fb0397604..ddd37eae63c6 100644 --- a/drivers/staging/lustre/lnet/lnet/api-ni.c +++ b/drivers/staging/lustre/lnet/lnet/api-ni.c @@ -973,7 +973,7 @@ lnet_ping_md_unlink(struct lnet_ping_info *pinfo, while (pinfo->pi_features != LNET_PING_FEAT_INVAL) { CDEBUG(D_NET, "Still waiting for ping MD to unlink\n"); set_current_state(TASK_UNINTERRUPTIBLE); - schedule_timeout(cfs_time_seconds(1)); + schedule_timeout(HZ); } cfs_restore_sigs(blocked); @@ -1112,7 +1112,7 @@ lnet_clear_zombies_nis_locked(void) libcfs_nid2str(ni->ni_nid)); } set_current_state(TASK_UNINTERRUPTIBLE); - schedule_timeout(cfs_time_seconds(1)); + schedule_timeout(HZ); lnet_net_lock(LNET_LOCK_EX); continue; } diff --git a/drivers/staging/lustre/lnet/lnet/lib-move.c b/drivers/staging/lustre/lnet/lnet/lib-move.c index d724c4c73ecc..7fe7ae917273 100644 --- a/drivers/staging/lustre/lnet/lnet/lib-move.c +++ b/drivers/staging/lustre/lnet/lnet/lib-move.c @@ -524,7 +524,7 @@ lnet_peer_is_alive(struct lnet_peer *lp, unsigned long now) return 0; deadline = cfs_time_add(lp->lp_last_alive, - cfs_time_seconds(lp->lp_ni->ni_peertimeout)); + lp->lp_ni->ni_peertimeout * HZ); alive = cfs_time_after(deadline, now); /* Update obsolete lp_alive except for routers assumed to be dead @@ -562,7 +562,7 @@ lnet_peer_alive_locked(struct lnet_peer *lp) unsigned long next_query = cfs_time_add(lp->lp_last_query, - cfs_time_seconds(lnet_queryinterval)); + lnet_queryinterval * HZ); if (time_before(now, next_query)) { if (lp->lp_alive) diff --git a/drivers/staging/lustre/lnet/lnet/net_fault.c b/drivers/staging/lustre/lnet/lnet/net_fault.c index e3468cef273b..a63b7941d435 100644 --- a/drivers/staging/lustre/lnet/lnet/net_fault.c +++ b/drivers/staging/lustre/lnet/lnet/net_fault.c @@ -315,9 +315,8 @@ drop_rule_match(struct lnet_drop_rule *rule, lnet_nid_t src, rule->dr_time_base = now; rule->dr_drop_time = rule->dr_time_base + - cfs_time_seconds( - prandom_u32_max(attr->u.drop.da_interval)); - rule->dr_time_base += cfs_time_seconds(attr->u.drop.da_interval); + prandom_u32_max(attr->u.drop.da_interval) * HZ; + rule->dr_time_base += attr->u.drop.da_interval * HZ; CDEBUG(D_NET, "Drop Rule %s->%s: next drop : %lu\n", libcfs_nid2str(attr->fa_src), @@ -440,8 +439,7 @@ static struct delay_daemon_data delay_dd; static unsigned long round_timeout(unsigned long timeout) { - return cfs_time_seconds((unsigned int) - cfs_duration_sec(cfs_time_sub(timeout, 0)) + 1); + return (unsigned int)rounddown(timeout, HZ) + HZ; } static void @@ -483,10 +481,8 @@ delay_rule_match(struct lnet_delay_rule *rule, lnet_nid_t src, rule->dl_time_base = now; rule->dl_delay_time = rule->dl_time_base + - cfs_time_seconds( - prandom_u32_max( - attr->u.delay.la_interval)); - rule->dl_time_base += cfs_time_seconds(attr->u.delay.la_interval); + prandom_u32_max(attr->u.delay.la_interval) * HZ; + rule->dl_time_base += attr->u.delay.la_interval * HZ; CDEBUG(D_NET, "Delay Rule %s->%s: next delay : %lu\n", libcfs_nid2str(attr->fa_src), diff --git a/drivers/staging/lustre/lnet/lnet/peer.c b/drivers/staging/lustre/lnet/lnet/peer.c index 19fcbcf0f642..89610f768b4f 100644 --- a/drivers/staging/lustre/lnet/lnet/peer.c +++ b/drivers/staging/lustre/lnet/lnet/peer.c @@ -137,7 +137,7 @@ lnet_peer_table_deathrow_wait_locked(struct lnet_peer_table *ptable, ptable->pt_zombies); } set_current_state(TASK_UNINTERRUPTIBLE); - schedule_timeout(cfs_time_seconds(1) >> 1); + schedule_timeout(HZ >> 1); lnet_net_lock(cpt_locked); } } diff --git a/drivers/staging/lustre/lnet/lnet/router.c b/drivers/staging/lustre/lnet/lnet/router.c index 80a7e8a88acb..95ee9997b9b0 100644 --- a/drivers/staging/lustre/lnet/lnet/router.c +++ b/drivers/staging/lustre/lnet/lnet/router.c @@ -810,7 +810,7 @@ lnet_wait_known_routerstate(void) return; set_current_state(TASK_UNINTERRUPTIBLE); - schedule_timeout(cfs_time_seconds(1)); + schedule_timeout(HZ); } } @@ -1013,7 +1013,7 @@ lnet_ping_router_locked(struct lnet_peer *rtr) if (secs && !rtr->lp_ping_notsent && cfs_time_after(now, cfs_time_add(rtr->lp_ping_timestamp, - cfs_time_seconds(secs)))) { + secs * HZ))) { int rc; struct lnet_process_id id; struct lnet_handle_md mdh; @@ -1187,7 +1187,7 @@ lnet_prune_rc_data(int wait_unlink) CDEBUG(((i & (-i)) == i) ? D_WARNING : D_NET, "Waiting for rc buffers to unlink\n"); set_current_state(TASK_UNINTERRUPTIBLE); - schedule_timeout(cfs_time_seconds(1) / 4); + schedule_timeout(HZ / 4); lnet_net_lock(LNET_LOCK_EX); } @@ -1284,7 +1284,7 @@ lnet_router_checker(void *arg) else wait_event_interruptible_timeout(the_lnet.ln_rc_waitq, false, - cfs_time_seconds(1)); + HZ); } lnet_prune_rc_data(1); /* wait for UNLINK */ diff --git a/drivers/staging/lustre/lnet/selftest/conrpc.c b/drivers/staging/lustre/lnet/selftest/conrpc.c index 7aa515c34594..6dcc966b293b 100644 --- a/drivers/staging/lustre/lnet/selftest/conrpc.c +++ b/drivers/staging/lustre/lnet/selftest/conrpc.c @@ -359,7 +359,7 @@ lstcon_rpc_trans_postwait(struct lstcon_rpc_trans *trans, int timeout) rc = wait_event_interruptible_timeout(trans->tas_waitq, lstcon_rpc_trans_check(trans), - cfs_time_seconds(timeout)); + timeout * HZ); rc = (rc > 0) ? 0 : ((rc < 0) ? -EINTR : -ETIMEDOUT); mutex_lock(&console_session.ses_mutex); @@ -1350,7 +1350,7 @@ lstcon_rpc_cleanup_wait(void) CWARN("Session is shutting down, waiting for termination of transactions\n"); set_current_state(TASK_UNINTERRUPTIBLE); - schedule_timeout(cfs_time_seconds(1)); + schedule_timeout(HZ); mutex_lock(&console_session.ses_mutex); } diff --git a/drivers/staging/lustre/lnet/selftest/rpc.c b/drivers/staging/lustre/lnet/selftest/rpc.c index b515138dca2c..18e0f1b87fc1 100644 --- a/drivers/staging/lustre/lnet/selftest/rpc.c +++ b/drivers/staging/lustre/lnet/selftest/rpc.c @@ -1605,7 +1605,7 @@ srpc_startup(void) /* 1 second pause to avoid timestamp reuse */ set_current_state(TASK_UNINTERRUPTIBLE); - schedule_timeout(cfs_time_seconds(1)); + schedule_timeout(HZ); srpc_data.rpc_matchbits = ((__u64)ktime_get_real_seconds()) << 48; srpc_data.rpc_state = SRPC_STATE_NONE; diff --git a/drivers/staging/lustre/lnet/selftest/selftest.h b/drivers/staging/lustre/lnet/selftest/selftest.h index ad04534f000c..05466b85e1c0 100644 --- a/drivers/staging/lustre/lnet/selftest/selftest.h +++ b/drivers/staging/lustre/lnet/selftest/selftest.h @@ -575,7 +575,7 @@ swi_state2str(int state) #define selftest_wait_events() \ do { \ set_current_state(TASK_UNINTERRUPTIBLE); \ - schedule_timeout(cfs_time_seconds(1) / 10); \ + schedule_timeout(HZ / 10); \ } while (0) #define lst_wait_until(cond, lock, fmt, ...) \ diff --git a/drivers/staging/lustre/lnet/selftest/timer.c b/drivers/staging/lustre/lnet/selftest/timer.c index ab125a8524c5..9716afeb3c94 100644 --- a/drivers/staging/lustre/lnet/selftest/timer.c +++ b/drivers/staging/lustre/lnet/selftest/timer.c @@ -177,7 +177,7 @@ stt_timer_main(void *arg) rc = wait_event_timeout(stt_data.stt_waitq, stt_data.stt_shuttingdown, - cfs_time_seconds(STTIMER_SLOTTIME)); + STTIMER_SLOTTIME * HZ); } spin_lock(&stt_data.stt_lock); diff --git a/drivers/staging/lustre/lustre/include/lustre_dlm.h b/drivers/staging/lustre/lustre/include/lustre_dlm.h index e0b17052b2ea..239aa2b1268f 100644 --- a/drivers/staging/lustre/lustre/include/lustre_dlm.h +++ b/drivers/staging/lustre/lustre/include/lustre_dlm.h @@ -60,7 +60,7 @@ struct obd_device; #define OBD_LDLM_DEVICENAME "ldlm" #define LDLM_DEFAULT_LRU_SIZE (100 * num_online_cpus()) -#define LDLM_DEFAULT_MAX_ALIVE (cfs_time_seconds(3900)) /* 65 min */ +#define LDLM_DEFAULT_MAX_ALIVE (65 * 60 * HZ) /* 65 min */ #define LDLM_DEFAULT_PARALLEL_AST_LIMIT 1024 /** diff --git a/drivers/staging/lustre/lustre/include/lustre_mdc.h b/drivers/staging/lustre/lustre/include/lustre_mdc.h index 007e1ec3f0f4..a9c9992a2502 100644 --- a/drivers/staging/lustre/lustre/include/lustre_mdc.h +++ b/drivers/staging/lustre/lustre/include/lustre_mdc.h @@ -124,7 +124,7 @@ static inline void mdc_get_rpc_lock(struct mdc_rpc_lock *lck, */ while (unlikely(lck->rpcl_it == MDC_FAKE_RPCL_IT)) { mutex_unlock(&lck->rpcl_mutex); - schedule_timeout(cfs_time_seconds(1) / 4); + schedule_timeout(HZ / 4); goto again; } diff --git a/drivers/staging/lustre/lustre/include/lustre_net.h b/drivers/staging/lustre/lustre/include/lustre_net.h index 4c665eca2467..5a4434e7c85a 100644 --- a/drivers/staging/lustre/lustre/include/lustre_net.h +++ b/drivers/staging/lustre/lustre/include/lustre_net.h @@ -2262,7 +2262,7 @@ static inline int ptlrpc_send_limit_expired(struct ptlrpc_request *req) { if (req->rq_delay_limit != 0 && time_before(cfs_time_add(req->rq_queued_time, - cfs_time_seconds(req->rq_delay_limit)), + req->rq_delay_limit * HZ), cfs_time_current())) { return 1; } diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c b/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c index 975fabc73148..53500883f243 100644 --- a/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c @@ -1366,7 +1366,7 @@ enum ldlm_mode ldlm_lock_match(struct ldlm_namespace *ns, __u64 flags, } } - lwi = LWI_TIMEOUT_INTR(cfs_time_seconds(obd_timeout), + lwi = LWI_TIMEOUT_INTR(obd_timeout * HZ, NULL, LWI_ON_SIGNAL_NOOP, NULL); /* XXX FIXME see comment on CAN_MATCH in lustre_dlm.h */ diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c b/drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c index d9835418d340..a47ac6f0d288 100644 --- a/drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c @@ -163,7 +163,7 @@ static void ldlm_handle_cp_callback(struct ptlrpc_request *req, LDLM_DEBUG(lock, "client completion callback handler START"); if (OBD_FAIL_CHECK(OBD_FAIL_LDLM_CANCEL_BL_CB_RACE)) { - int to = cfs_time_seconds(1); + int to = HZ; while (to > 0) { set_current_state(TASK_INTERRUPTIBLE); @@ -327,7 +327,7 @@ static void ldlm_handle_gl_callback(struct ptlrpc_request *req, !lock->l_readers && !lock->l_writers && cfs_time_after(cfs_time_current(), cfs_time_add(lock->l_last_used, - cfs_time_seconds(10)))) { + 10 * HZ))) { unlock_res_and_lock(lock); if (ldlm_bl_to_thread_lock(ns, NULL, lock)) ldlm_handle_bl_callback(ns, NULL, lock); diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c b/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c index d562f90cee97..eb1f3d45c68c 100644 --- a/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c @@ -1008,7 +1008,7 @@ static int ldlm_pools_thread_main(void *arg) * Wait until the next check time, or until we're * stopped. */ - lwi = LWI_TIMEOUT(cfs_time_seconds(c_time), + lwi = LWI_TIMEOUT(c_time * HZ, NULL, NULL); l_wait_event(thread->t_ctl_waitq, thread_is_stopping(thread) || diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_request.c b/drivers/staging/lustre/lustre/ldlm/ldlm_request.c index 6aa37463db46..a244fa717134 100644 --- a/drivers/staging/lustre/lustre/ldlm/ldlm_request.c +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_request.c @@ -288,7 +288,7 @@ int ldlm_completion_ast(struct ldlm_lock *lock, __u64 flags, void *data) LDLM_DEBUG(lock, "waiting indefinitely because of NO_TIMEOUT"); lwi = LWI_INTR(interrupted_completion_wait, &lwd); } else { - lwi = LWI_TIMEOUT_INTR(cfs_time_seconds(timeout), + lwi = LWI_TIMEOUT_INTR(timeout * HZ, ldlm_expired_completion_wait, interrupted_completion_wait, &lwd); } diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_resource.c b/drivers/staging/lustre/lustre/ldlm/ldlm_resource.c index 9958533cc227..2e66825c8f4b 100644 --- a/drivers/staging/lustre/lustre/ldlm/ldlm_resource.c +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_resource.c @@ -799,7 +799,7 @@ static void cleanup_resource(struct ldlm_resource *res, struct list_head *q, LDLM_DEBUG(lock, "setting FL_LOCAL_ONLY"); if (lock->l_flags & LDLM_FL_FAIL_LOC) { set_current_state(TASK_UNINTERRUPTIBLE); - schedule_timeout(cfs_time_seconds(4)); + schedule_timeout(4 * HZ); set_current_state(TASK_RUNNING); } if (lock->l_completion_ast) diff --git a/drivers/staging/lustre/lustre/llite/llite_lib.c b/drivers/staging/lustre/lustre/llite/llite_lib.c index 6735a6f006d2..0a9183f271f5 100644 --- a/drivers/staging/lustre/lustre/llite/llite_lib.c +++ b/drivers/staging/lustre/lustre/llite/llite_lib.c @@ -2026,8 +2026,8 @@ void ll_umount_begin(struct super_block *sb) * to decrement mnt_cnt and hope to finish it within 10sec. */ init_waitqueue_head(&waitq); - lwi = LWI_TIMEOUT_INTERVAL(cfs_time_seconds(10), - cfs_time_seconds(1), NULL, NULL); + lwi = LWI_TIMEOUT_INTERVAL(10 * HZ, + HZ, NULL, NULL); l_wait_event(waitq, may_umount(sbi->ll_mnt.mnt), &lwi); schedule(); diff --git a/drivers/staging/lustre/lustre/llite/statahead.c b/drivers/staging/lustre/lustre/llite/statahead.c index 39040916a043..9f9897523374 100644 --- a/drivers/staging/lustre/lustre/llite/statahead.c +++ b/drivers/staging/lustre/lustre/llite/statahead.c @@ -1424,7 +1424,7 @@ static int revalidate_statahead_dentry(struct inode *dir, spin_lock(&lli->lli_sa_lock); sai->sai_index_wait = entry->se_index; spin_unlock(&lli->lli_sa_lock); - lwi = LWI_TIMEOUT_INTR(cfs_time_seconds(30), NULL, + lwi = LWI_TIMEOUT_INTR(30 * HZ, NULL, LWI_ON_SIGNAL_NOOP, NULL); rc = l_wait_event(sai->sai_waitq, sa_ready(entry), &lwi); if (rc < 0) { diff --git a/drivers/staging/lustre/lustre/lov/lov_request.c b/drivers/staging/lustre/lustre/lov/lov_request.c index cfa1d7f92b0f..fb3b7a7fa32a 100644 --- a/drivers/staging/lustre/lustre/lov/lov_request.c +++ b/drivers/staging/lustre/lustre/lov/lov_request.c @@ -126,8 +126,8 @@ static int lov_check_and_wait_active(struct lov_obd *lov, int ost_idx) mutex_unlock(&lov->lov_lock); init_waitqueue_head(&waitq); - lwi = LWI_TIMEOUT_INTERVAL(cfs_time_seconds(obd_timeout), - cfs_time_seconds(1), NULL, NULL); + lwi = LWI_TIMEOUT_INTERVAL(obd_timeout * HZ, + HZ, NULL, NULL); rc = l_wait_event(waitq, lov_check_set(lov, ost_idx), &lwi); if (tgt->ltd_active) diff --git a/drivers/staging/lustre/lustre/mdc/mdc_request.c b/drivers/staging/lustre/lustre/mdc/mdc_request.c index 03e55bca4ada..b12518ba5ae9 100644 --- a/drivers/staging/lustre/lustre/mdc/mdc_request.c +++ b/drivers/staging/lustre/lustre/mdc/mdc_request.c @@ -888,7 +888,7 @@ static int mdc_getpage(struct obd_export *exp, const struct lu_fid *fid, exp->exp_obd->obd_name, -EIO); return -EIO; } - lwi = LWI_TIMEOUT_INTR(cfs_time_seconds(resends), NULL, NULL, + lwi = LWI_TIMEOUT_INTR(resends * HZ, NULL, NULL, NULL); l_wait_event(waitq, 0, &lwi); diff --git a/drivers/staging/lustre/lustre/mgc/mgc_request.c b/drivers/staging/lustre/lustre/mgc/mgc_request.c index 81b101941eec..4c2e20ff95d7 100644 --- a/drivers/staging/lustre/lustre/mgc/mgc_request.c +++ b/drivers/staging/lustre/lustre/mgc/mgc_request.c @@ -1628,7 +1628,7 @@ int mgc_process_log(struct obd_device *mgc, struct config_llog_data *cld) if (rcl == -ESHUTDOWN && atomic_read(&mgc->u.cli.cl_mgc_refcount) > 0 && !retry) { - int secs = cfs_time_seconds(obd_timeout); + int secs = obd_timeout * HZ; struct obd_import *imp; struct l_wait_info lwi; diff --git a/drivers/staging/lustre/lustre/obdclass/cl_io.c b/drivers/staging/lustre/lustre/obdclass/cl_io.c index a3fb2bbde70f..d8be01b9257f 100644 --- a/drivers/staging/lustre/lustre/obdclass/cl_io.c +++ b/drivers/staging/lustre/lustre/obdclass/cl_io.c @@ -1097,7 +1097,7 @@ EXPORT_SYMBOL(cl_sync_io_init); int cl_sync_io_wait(const struct lu_env *env, struct cl_sync_io *anchor, long timeout) { - struct l_wait_info lwi = LWI_TIMEOUT_INTR(cfs_time_seconds(timeout), + struct l_wait_info lwi = LWI_TIMEOUT_INTR(timeout * HZ, NULL, NULL, NULL); int rc; diff --git a/drivers/staging/lustre/lustre/obdecho/echo_client.c b/drivers/staging/lustre/lustre/obdecho/echo_client.c index b9c1dc7e61b0..9c5ce5074b66 100644 --- a/drivers/staging/lustre/lustre/obdecho/echo_client.c +++ b/drivers/staging/lustre/lustre/obdecho/echo_client.c @@ -752,7 +752,7 @@ static struct lu_device *echo_device_free(const struct lu_env *env, spin_unlock(&ec->ec_lock); CERROR("echo_client still has objects at cleanup time, wait for 1 second\n"); set_current_state(TASK_UNINTERRUPTIBLE); - schedule_timeout(cfs_time_seconds(1)); + schedule_timeout(HZ); lu_site_purge(env, ed->ed_site, -1); spin_lock(&ec->ec_lock); } diff --git a/drivers/staging/lustre/lustre/osc/osc_cache.c b/drivers/staging/lustre/lustre/osc/osc_cache.c index d58a25a2a5b4..e6d99dc048ce 100644 --- a/drivers/staging/lustre/lustre/osc/osc_cache.c +++ b/drivers/staging/lustre/lustre/osc/osc_cache.c @@ -934,7 +934,7 @@ static int osc_extent_wait(const struct lu_env *env, struct osc_extent *ext, enum osc_extent_state state) { struct osc_object *obj = ext->oe_obj; - struct l_wait_info lwi = LWI_TIMEOUT_INTR(cfs_time_seconds(600), NULL, + struct l_wait_info lwi = LWI_TIMEOUT_INTR(600 * HZ, NULL, LWI_ON_SIGNAL_NOOP, NULL); int rc = 0; @@ -1571,7 +1571,7 @@ static int osc_enter_cache(const struct lu_env *env, struct client_obd *cli, struct l_wait_info lwi; int rc = -EDQUOT; - lwi = LWI_TIMEOUT_INTR(cfs_time_seconds(AT_OFF ? obd_timeout : at_max), + lwi = LWI_TIMEOUT_INTR((AT_OFF ? obd_timeout : at_max) * HZ, NULL, LWI_ON_SIGNAL_NOOP, NULL); OSC_DUMP_GRANT(D_CACHE, cli, "need:%d\n", bytes); diff --git a/drivers/staging/lustre/lustre/osc/osc_object.c b/drivers/staging/lustre/lustre/osc/osc_object.c index 1de25496a7d9..747060521135 100644 --- a/drivers/staging/lustre/lustre/osc/osc_object.c +++ b/drivers/staging/lustre/lustre/osc/osc_object.c @@ -328,7 +328,7 @@ int osc_object_is_contended(struct osc_object *obj) * ll_file_is_contended. */ retry_time = cfs_time_add(obj->oo_contention_time, - cfs_time_seconds(osc_contention_time)); + osc_contention_time * HZ); if (cfs_time_after(cur_time, retry_time)) { osc_object_clear_contended(obj); return 0; diff --git a/drivers/staging/lustre/lustre/ptlrpc/client.c b/drivers/staging/lustre/lustre/ptlrpc/client.c index bac4b2304bad..0ab13f8e5993 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/client.c +++ b/drivers/staging/lustre/lustre/ptlrpc/client.c @@ -766,7 +766,7 @@ int ptlrpc_request_bufs_pack(struct ptlrpc_request *request, * fail_loc */ set_current_state(TASK_UNINTERRUPTIBLE); - schedule_timeout(cfs_time_seconds(2)); + schedule_timeout(2 * HZ); set_current_state(TASK_RUNNING); } } @@ -2284,7 +2284,7 @@ int ptlrpc_set_wait(struct ptlrpc_request_set *set) * We still want to block for a limited time, * so we allow interrupts during the timeout. */ - lwi = LWI_TIMEOUT_INTR_ALL(cfs_time_seconds(1), + lwi = LWI_TIMEOUT_INTR_ALL(HZ, ptlrpc_expired_set, ptlrpc_interrupted_set, set); else @@ -2293,7 +2293,7 @@ int ptlrpc_set_wait(struct ptlrpc_request_set *set) * interrupts are allowed. Wait until all * complete, or an in-flight req times out. */ - lwi = LWI_TIMEOUT(cfs_time_seconds(timeout ? timeout : 1), + lwi = LWI_TIMEOUT((timeout ? timeout : 1) * HZ, ptlrpc_expired_set, set); rc = l_wait_event(set->set_waitq, ptlrpc_check_set(NULL, set), &lwi); @@ -2538,8 +2538,8 @@ static int ptlrpc_unregister_reply(struct ptlrpc_request *request, int async) * Network access will complete in finite time but the HUGE * timeout lets us CWARN for visibility of sluggish NALs */ - lwi = LWI_TIMEOUT_INTERVAL(cfs_time_seconds(LONG_UNLINK), - cfs_time_seconds(1), NULL, NULL); + lwi = LWI_TIMEOUT_INTERVAL(LONG_UNLINK * HZ, + HZ, NULL, NULL); rc = l_wait_event(*wq, !ptlrpc_client_recv_or_unlink(request), &lwi); if (rc == 0) { diff --git a/drivers/staging/lustre/lustre/ptlrpc/events.c b/drivers/staging/lustre/lustre/ptlrpc/events.c index 811b7ab3a582..71f7588570ef 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/events.c +++ b/drivers/staging/lustre/lustre/ptlrpc/events.c @@ -517,7 +517,7 @@ static void ptlrpc_ni_fini(void) /* Wait for a bit */ init_waitqueue_head(&waitq); - lwi = LWI_TIMEOUT(cfs_time_seconds(2), NULL, NULL); + lwi = LWI_TIMEOUT(2 * HZ, NULL, NULL); l_wait_event(waitq, 0, &lwi); break; } diff --git a/drivers/staging/lustre/lustre/ptlrpc/import.c b/drivers/staging/lustre/lustre/ptlrpc/import.c index 5b0f65536c29..0eba5f18bd3b 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/import.c +++ b/drivers/staging/lustre/lustre/ptlrpc/import.c @@ -307,9 +307,9 @@ void ptlrpc_invalidate_import(struct obd_import *imp) * have been locally cancelled by ptlrpc_abort_inflight. */ lwi = LWI_TIMEOUT_INTERVAL( - cfs_timeout_cap(cfs_time_seconds(timeout)), - (timeout > 1) ? cfs_time_seconds(1) : - cfs_time_seconds(1) / 2, + cfs_timeout_cap(timeout * HZ), + (timeout > 1) ? HZ : + HZ / 2, NULL, NULL); rc = l_wait_event(imp->imp_recovery_waitq, (atomic_read(&imp->imp_inflight) == 0), @@ -431,7 +431,7 @@ void ptlrpc_fail_import(struct obd_import *imp, __u32 conn_cnt) int ptlrpc_reconnect_import(struct obd_import *imp) { struct l_wait_info lwi; - int secs = cfs_time_seconds(obd_timeout); + int secs = obd_timeout * HZ; int rc; ptlrpc_pinger_force(imp); @@ -1508,14 +1508,13 @@ int ptlrpc_disconnect_import(struct obd_import *imp, int noclose) if (AT_OFF) { if (imp->imp_server_timeout) - timeout = cfs_time_seconds(obd_timeout / 2); + timeout = obd_timeout * HZ / 2; else - timeout = cfs_time_seconds(obd_timeout); + timeout = obd_timeout * HZ; } else { int idx = import_at_get_index(imp, imp->imp_client->cli_request_portal); - timeout = cfs_time_seconds( - at_get(&imp->imp_at.iat_service_estimate[idx])); + timeout = at_get(&imp->imp_at.iat_service_estimate[idx]) * HZ; } lwi = LWI_TIMEOUT_INTR(cfs_timeout_cap(timeout), diff --git a/drivers/staging/lustre/lustre/ptlrpc/niobuf.c b/drivers/staging/lustre/lustre/ptlrpc/niobuf.c index 047d712e850c..0c2ded721c49 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/niobuf.c +++ b/drivers/staging/lustre/lustre/ptlrpc/niobuf.c @@ -270,8 +270,8 @@ int ptlrpc_unregister_bulk(struct ptlrpc_request *req, int async) /* Network access will complete in finite time but the HUGE * timeout lets us CWARN for visibility of sluggish LNDs */ - lwi = LWI_TIMEOUT_INTERVAL(cfs_time_seconds(LONG_UNLINK), - cfs_time_seconds(1), NULL, NULL); + lwi = LWI_TIMEOUT_INTERVAL(LONG_UNLINK * HZ, + HZ, NULL, NULL); rc = l_wait_event(*wq, !ptlrpc_client_bulk_active(req), &lwi); if (rc == 0) { ptlrpc_rqphase_move(req, req->rq_next_phase); diff --git a/drivers/staging/lustre/lustre/ptlrpc/pack_generic.c b/drivers/staging/lustre/lustre/ptlrpc/pack_generic.c index a64e125df95f..c060d6f5015a 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/pack_generic.c +++ b/drivers/staging/lustre/lustre/ptlrpc/pack_generic.c @@ -267,7 +267,7 @@ lustre_get_emerg_rs(struct ptlrpc_service_part *svcpt) /* If we cannot get anything for some long time, we better * bail out instead of waiting infinitely */ - lwi = LWI_TIMEOUT(cfs_time_seconds(10), NULL, NULL); + lwi = LWI_TIMEOUT(10 * HZ, NULL, NULL); rc = l_wait_event(svcpt->scp_rep_waitq, !list_empty(&svcpt->scp_rep_idle), &lwi); if (rc != 0) diff --git a/drivers/staging/lustre/lustre/ptlrpc/pinger.c b/drivers/staging/lustre/lustre/ptlrpc/pinger.c index 4148a6661dcf..da3afda72e14 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/pinger.c +++ b/drivers/staging/lustre/lustre/ptlrpc/pinger.c @@ -141,7 +141,7 @@ static long pinger_check_timeout(unsigned long time) } mutex_unlock(&pinger_mutex); - return cfs_time_sub(cfs_time_add(time, cfs_time_seconds(timeout)), + return cfs_time_sub(cfs_time_add(time, timeout * HZ), cfs_time_current()); } @@ -247,7 +247,7 @@ static int ptlrpc_pinger_main(void *arg) if (imp->imp_pingable && imp->imp_next_ping && cfs_time_after(imp->imp_next_ping, cfs_time_add(this_ping, - cfs_time_seconds(PING_INTERVAL)))) + PING_INTERVAL * HZ))) ptlrpc_update_next_ping(imp, 0); } mutex_unlock(&pinger_mutex); @@ -264,10 +264,10 @@ static int ptlrpc_pinger_main(void *arg) CDEBUG(D_INFO, "next wakeup in " CFS_DURATION_T " (%ld)\n", time_to_next_wake, cfs_time_add(this_ping, - cfs_time_seconds(PING_INTERVAL))); + PING_INTERVAL * HZ)); if (time_to_next_wake > 0) { lwi = LWI_TIMEOUT(max_t(long, time_to_next_wake, - cfs_time_seconds(1)), + HZ), NULL, NULL); l_wait_event(thread->t_ctl_waitq, thread_is_stopping(thread) || diff --git a/drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c b/drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c index 8b865294d933..dad2f9290f70 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c +++ b/drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c @@ -230,7 +230,7 @@ void ptlrpcd_add_req(struct ptlrpc_request *req) spin_lock(&req->rq_lock); if (req->rq_invalid_rqset) { - struct l_wait_info lwi = LWI_TIMEOUT(cfs_time_seconds(5), + struct l_wait_info lwi = LWI_TIMEOUT(5 * HZ, back_to_sleep, NULL); req->rq_invalid_rqset = 0; @@ -438,7 +438,7 @@ static int ptlrpcd(void *arg) int timeout; timeout = ptlrpc_set_next_timeout(set); - lwi = LWI_TIMEOUT(cfs_time_seconds(timeout ? timeout : 1), + lwi = LWI_TIMEOUT((timeout ? timeout : 1) * HZ, ptlrpc_expired_set, set); lu_context_enter(&env.le_ctx); diff --git a/drivers/staging/lustre/lustre/ptlrpc/recover.c b/drivers/staging/lustre/lustre/ptlrpc/recover.c index e4d3f23e9f3a..5bbd23eebfa6 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/recover.c +++ b/drivers/staging/lustre/lustre/ptlrpc/recover.c @@ -347,7 +347,7 @@ int ptlrpc_recover_import(struct obd_import *imp, char *new_uuid, int async) if (!async) { struct l_wait_info lwi; - int secs = cfs_time_seconds(obd_timeout); + int secs = obd_timeout * HZ; CDEBUG(D_HA, "%s: recovery started, waiting %u seconds\n", obd2cli_tgt(imp->imp_obd), secs); diff --git a/drivers/staging/lustre/lustre/ptlrpc/service.c b/drivers/staging/lustre/lustre/ptlrpc/service.c index d688cb3ff157..4a8a591e0067 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/service.c +++ b/drivers/staging/lustre/lustre/ptlrpc/service.c @@ -2149,7 +2149,7 @@ static int ptlrpc_main(void *arg) * Wait for a timeout (unless something else * happens) before I try again */ - svcpt->scp_rqbd_timeout = cfs_time_seconds(1) / 10; + svcpt->scp_rqbd_timeout = HZ / 10; CDEBUG(D_RPCTRACE, "Posted buffers: %d\n", svcpt->scp_nrqbds_posted); } @@ -2588,7 +2588,7 @@ static void ptlrpc_wait_replies(struct ptlrpc_service_part *svcpt) { while (1) { int rc; - struct l_wait_info lwi = LWI_TIMEOUT(cfs_time_seconds(10), + struct l_wait_info lwi = LWI_TIMEOUT(10 * HZ, NULL, NULL); rc = l_wait_event(svcpt->scp_waitq, @@ -2660,8 +2660,8 @@ ptlrpc_service_unlink_rqbd(struct ptlrpc_service *svc) * of sluggish LNDs */ lwi = LWI_TIMEOUT_INTERVAL( - cfs_time_seconds(LONG_UNLINK), - cfs_time_seconds(1), NULL, NULL); + LONG_UNLINK * HZ, + HZ, NULL, NULL); rc = l_wait_event(svcpt->scp_waitq, svcpt->scp_nrqbds_posted == 0, &lwi); if (rc == -ETIMEDOUT) { From neilb at suse.com Mon Dec 18 07:18:00 2017 From: neilb at suse.com (NeilBrown) Date: Mon, 18 Dec 2017 18:18:00 +1100 Subject: [lustre-devel] [PATCH 05/16] staging: lustre: introduce and use l_wait_event_abortable() In-Reply-To: <151358127190.5099.12792810096274074963.stgit@noble> References: <151358127190.5099.12792810096274074963.stgit@noble> Message-ID: <151358147998.5099.13286478253284938255.stgit@noble> lustre sometimes wants to wait for an event, but abort if one of a specific list of signals arrives. This is a little bit like wait_event_killable(), except that the signals are identified a different way. So introduce l_wait_event_abortable() which provides this functionality. Having separate functions for separate needs is more in line with the pattern set by include/linux/wait.h, than having a single function which tries to include all possible needs. Also introduce l_wait_event_abortable_exclusive(). Note that l_wait_event() return -EINTR on a signal, while Linux wait_event functions return -ERESTARTSYS. l_wait_event_abortable_exclusive follows the Linux pattern. Signed-off-by: NeilBrown --- drivers/staging/lustre/lustre/include/lustre_lib.h | 24 ++++++++++++++++++++ drivers/staging/lustre/lustre/ldlm/ldlm_resource.c | 12 +++++----- drivers/staging/lustre/lustre/llite/llite_lib.c | 12 +++------- drivers/staging/lustre/lustre/obdclass/genops.c | 9 +++----- drivers/staging/lustre/lustre/obdclass/llog_obd.c | 5 ++-- drivers/staging/lustre/lustre/osc/osc_page.c | 6 ++--- drivers/staging/lustre/lustre/osc/osc_request.c | 6 ++--- 7 files changed, 43 insertions(+), 31 deletions(-) diff --git a/drivers/staging/lustre/lustre/include/lustre_lib.h b/drivers/staging/lustre/lustre/include/lustre_lib.h index fcf31c779e98..d64306291b47 100644 --- a/drivers/staging/lustre/lustre/include/lustre_lib.h +++ b/drivers/staging/lustre/lustre/include/lustre_lib.h @@ -425,4 +425,28 @@ do { \ __ret = __wait_event_noload_timeout(wq_head, condition, timeout);\ __ret; \ }) + +/* l_wait_event_abortable() is a bit like wait_event_killable() + * except there is a fix set of signals which will abort: + * LUSTRE_FATAL_SIGS + */ +#define l_wait_event_abortable(wq, condition) \ +({ \ + sigset_t __blocked; \ + int __ret = 0; \ + __blocked = cfs_block_sigsinv(LUSTRE_FATAL_SIGS); \ + __ret = wait_event_interruptible(wq, condition); \ + cfs_restore_sigs(__blocked); \ + __ret; \ +}) + +#define l_wait_event_abortable_exclusive(wq, condition) \ +({ \ + sigset_t __blocked; \ + int __ret = 0; \ + __blocked = cfs_block_sigsinv(LUSTRE_FATAL_SIGS); \ + __ret = wait_event_interruptible_exclusive(wq, condition); \ + cfs_restore_sigs(__blocked); \ + __ret; \ +}) #endif /* _LUSTRE_LIB_H */ diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_resource.c b/drivers/staging/lustre/lustre/ldlm/ldlm_resource.c index 2e66825c8f4b..30044357616d 100644 --- a/drivers/staging/lustre/lustre/ldlm/ldlm_resource.c +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_resource.c @@ -879,7 +879,6 @@ static int __ldlm_namespace_free(struct ldlm_namespace *ns, int force) ldlm_namespace_cleanup(ns, force ? LDLM_FL_LOCAL_ONLY : 0); if (atomic_read(&ns->ns_bref) > 0) { - struct l_wait_info lwi = LWI_INTR(LWI_ON_SIGNAL_NOOP, NULL); int rc; CDEBUG(D_DLMTRACE, @@ -887,11 +886,12 @@ static int __ldlm_namespace_free(struct ldlm_namespace *ns, int force) ldlm_ns_name(ns), atomic_read(&ns->ns_bref)); force_wait: if (force) - lwi = LWI_TIMEOUT(msecs_to_jiffies(obd_timeout * - MSEC_PER_SEC) / 4, NULL, NULL); - - rc = l_wait_event(ns->ns_waitq, - atomic_read(&ns->ns_bref) == 0, &lwi); + rc = wait_event_timeout(ns->ns_waitq, + atomic_read(&ns->ns_bref) == 0, + obd_timeout * HZ / 4) ? 0 : -ETIMEDOUT; + else + rc = l_wait_event_abortable(ns->ns_waitq, + atomic_read(&ns->ns_bref) == 0); /* Forced cleanups should be able to reclaim all references, * so it's safe to wait forever... we can't leak locks... diff --git a/drivers/staging/lustre/lustre/llite/llite_lib.c b/drivers/staging/lustre/lustre/llite/llite_lib.c index 0a9183f271f5..33dc15e9aebb 100644 --- a/drivers/staging/lustre/lustre/llite/llite_lib.c +++ b/drivers/staging/lustre/lustre/llite/llite_lib.c @@ -986,16 +986,12 @@ void ll_put_super(struct super_block *sb) } /* Wait for unstable pages to be committed to stable storage */ - if (!force) { - struct l_wait_info lwi = LWI_INTR(LWI_ON_SIGNAL_NOOP, NULL); - - rc = l_wait_event(sbi->ll_cache->ccc_unstable_waitq, - !atomic_long_read(&sbi->ll_cache->ccc_unstable_nr), - &lwi); - } + if (!force) + rc = l_wait_event_abortable(sbi->ll_cache->ccc_unstable_waitq, + !atomic_long_read(&sbi->ll_cache->ccc_unstable_nr)); ccc_count = atomic_long_read(&sbi->ll_cache->ccc_unstable_nr); - if (!force && rc != -EINTR) + if (!force && rc != -ERESTARTSYS) LASSERTF(!ccc_count, "count: %li\n", ccc_count); /* We need to set force before the lov_disconnect in diff --git a/drivers/staging/lustre/lustre/obdclass/genops.c b/drivers/staging/lustre/lustre/obdclass/genops.c index 78f0fa1dff45..2aa350214c63 100644 --- a/drivers/staging/lustre/lustre/obdclass/genops.c +++ b/drivers/staging/lustre/lustre/obdclass/genops.c @@ -1332,7 +1332,6 @@ static bool obd_request_slot_avail(struct client_obd *cli, int obd_get_request_slot(struct client_obd *cli) { struct obd_request_slot_waiter orsw; - struct l_wait_info lwi; int rc; spin_lock(&cli->cl_loi_list_lock); @@ -1347,11 +1346,9 @@ int obd_get_request_slot(struct client_obd *cli) orsw.orsw_signaled = false; spin_unlock(&cli->cl_loi_list_lock); - lwi = LWI_INTR(LWI_ON_SIGNAL_NOOP, NULL); - rc = l_wait_event(orsw.orsw_waitq, - obd_request_slot_avail(cli, &orsw) || - orsw.orsw_signaled, - &lwi); + rc = l_wait_event_abortable(orsw.orsw_waitq, + obd_request_slot_avail(cli, &orsw) || + orsw.orsw_signaled); /* * Here, we must take the lock to avoid the on-stack 'orsw' to be diff --git a/drivers/staging/lustre/lustre/obdclass/llog_obd.c b/drivers/staging/lustre/lustre/obdclass/llog_obd.c index 28bbaa2136ac..26aea114a29b 100644 --- a/drivers/staging/lustre/lustre/obdclass/llog_obd.c +++ b/drivers/staging/lustre/lustre/obdclass/llog_obd.c @@ -104,7 +104,6 @@ EXPORT_SYMBOL(__llog_ctxt_put); int llog_cleanup(const struct lu_env *env, struct llog_ctxt *ctxt) { - struct l_wait_info lwi = LWI_INTR(LWI_ON_SIGNAL_NOOP, NULL); struct obd_llog_group *olg; int rc, idx; @@ -129,8 +128,8 @@ int llog_cleanup(const struct lu_env *env, struct llog_ctxt *ctxt) CERROR("Error %d while cleaning up ctxt %p\n", rc, ctxt); - l_wait_event(olg->olg_waitq, - llog_group_ctxt_null(olg, idx), &lwi); + l_wait_event_abortable(olg->olg_waitq, + llog_group_ctxt_null(olg, idx)); return rc; } diff --git a/drivers/staging/lustre/lustre/osc/osc_page.c b/drivers/staging/lustre/lustre/osc/osc_page.c index 20094b6309f9..6fdd521feb21 100644 --- a/drivers/staging/lustre/lustre/osc/osc_page.c +++ b/drivers/staging/lustre/lustre/osc/osc_page.c @@ -759,7 +759,6 @@ static long osc_lru_reclaim(struct client_obd *cli, unsigned long npages) static int osc_lru_alloc(const struct lu_env *env, struct client_obd *cli, struct osc_page *opg) { - struct l_wait_info lwi = LWI_INTR(LWI_ON_SIGNAL_NOOP, NULL); struct osc_io *oio = osc_env_io(env); int rc = 0; @@ -782,9 +781,8 @@ static int osc_lru_alloc(const struct lu_env *env, struct client_obd *cli, cond_resched(); - rc = l_wait_event(osc_lru_waitq, - atomic_long_read(cli->cl_lru_left) > 0, - &lwi); + rc = l_wait_event_abortable(osc_lru_waitq, + atomic_long_read(cli->cl_lru_left) > 0); if (rc < 0) break; diff --git a/drivers/staging/lustre/lustre/osc/osc_request.c b/drivers/staging/lustre/lustre/osc/osc_request.c index 45b1ebf33363..074b5ce6284c 100644 --- a/drivers/staging/lustre/lustre/osc/osc_request.c +++ b/drivers/staging/lustre/lustre/osc/osc_request.c @@ -552,14 +552,12 @@ static int osc_destroy(const struct lu_env *env, struct obd_export *exp, req->rq_interpret_reply = osc_destroy_interpret; if (!osc_can_send_destroy(cli)) { - struct l_wait_info lwi = LWI_INTR(LWI_ON_SIGNAL_NOOP, NULL); - /* * Wait until the number of on-going destroy RPCs drops * under max_rpc_in_flight */ - l_wait_event_exclusive(cli->cl_destroy_waitq, - osc_can_send_destroy(cli), &lwi); + l_wait_event_abortable_exclusive(cli->cl_destroy_waitq, + osc_can_send_destroy(cli)); } /* Do not wait for response */ From neilb at suse.com Mon Dec 18 07:17:59 2017 From: neilb at suse.com (NeilBrown) Date: Mon, 18 Dec 2017 18:17:59 +1100 Subject: [lustre-devel] [PATCH 02/16] staging: lustre: replace simple cases of l_wait_event() with wait_event(). In-Reply-To: <151358127190.5099.12792810096274074963.stgit@noble> References: <151358127190.5099.12792810096274074963.stgit@noble> Message-ID: <151358147981.5099.13114335078693829049.stgit@noble> When the lwi arg is full of zeros, l_wait_event() behaves almost identically to the standard wait_event() interface, so use that instead. The only difference in behavior is that l_wait_event() blocks all signals and uses an TASK_INTERRUPTIBLE wait, while wait_event() does not block signals, but waits in state TASK_UNINTERRUPTIBLE. This means that processes blocked in wait_event() will contribute to the load average. This behavior is (arguably) more correct - in most cases. In some cases, the wait is in a service thread waiting for work to do. In these case we should wait TASK_NOLOAD order with TASK_UNINTERRUPTIBLE. To facilitate this, add a "wait_event_noload()" macro. This should eventually be moved into include/linux/wait.h. There is one case where wait_event_exclusive_noload() is needed. So we add a macro for that too. Signed-off-by: NeilBrown --- drivers/staging/lustre/lustre/include/lustre_lib.h | 47 ++++++++++++++++--- drivers/staging/lustre/lustre/ldlm/ldlm_lock.c | 4 -- drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c | 8 +-- drivers/staging/lustre/lustre/ldlm/ldlm_pool.c | 5 +- drivers/staging/lustre/lustre/llite/statahead.c | 50 ++++++++------------ drivers/staging/lustre/lustre/lov/lov_object.c | 6 +- drivers/staging/lustre/lustre/mgc/mgc_request.c | 4 -- drivers/staging/lustre/lustre/obdclass/cl_io.c | 6 +- drivers/staging/lustre/lustre/obdclass/genops.c | 15 ++---- drivers/staging/lustre/lustre/osc/osc_cache.c | 5 +- drivers/staging/lustre/lustre/osc/osc_object.c | 4 -- drivers/staging/lustre/lustre/ptlrpc/pinger.c | 10 ++-- drivers/staging/lustre/lustre/ptlrpc/sec_gc.c | 11 ++-- drivers/staging/lustre/lustre/ptlrpc/service.c | 13 ++--- 14 files changed, 93 insertions(+), 95 deletions(-) diff --git a/drivers/staging/lustre/lustre/include/lustre_lib.h b/drivers/staging/lustre/lustre/include/lustre_lib.h index ca1dce15337e..08bdd618ea7d 100644 --- a/drivers/staging/lustre/lustre/include/lustre_lib.h +++ b/drivers/staging/lustre/lustre/include/lustre_lib.h @@ -333,12 +333,6 @@ do { \ __ret; \ }) -#define l_wait_condition(wq, condition) \ -({ \ - struct l_wait_info lwi = { 0 }; \ - l_wait_event(wq, condition, &lwi); \ -}) - #define l_wait_condition_exclusive(wq, condition) \ ({ \ struct l_wait_info lwi = { 0 }; \ @@ -353,4 +347,45 @@ do { \ /** @} lib */ +#define __wait_event_noload(wq_head, condition) \ + (void)___wait_event(wq_head, condition, (TASK_UNINTERRUPTIBLE | TASK_NOLOAD), 0, 0, \ + schedule()) + +/** + * wait_event_noload - sleep, without registering load, until a condition gets true + * @wq_head: the waitqueue to wait on + * @condition: a C expression for the event to wait for + * + * The process is put to sleep (TASK_UNINTERRUPTIBLE|TASK_NOLOAD) until the + * @condition evaluates to true. The @condition is checked each time + * the waitqueue @wq_head is woken up. + * + * wake_up() has to be called after changing any variable that could + * change the result of the wait condition. + * + * This can be used instead of wait_event() when the event + * being waited for is does not imply load on the system, but + * when responding to signals is no appropriate, such as in + * a kernel service thread. + */ +#define wait_event_noload(wq_head, condition) \ +do { \ + might_sleep(); \ + if (condition) \ + break; \ + __wait_event_noload(wq_head, condition); \ +} while (0) + +/* + * Just like wait_event_noload(), except it sets exclusive flag + */ +#define wait_event_exclusive_noload(wq_head, condition) \ +do { \ + if (condition) \ + break; \ + (void)___wait_event(wq_head, condition, \ + (TASK_UNINTERRUPTIBLE | TASK_NOLOAD), 1, 0, \ + schedule()); \ +} while (0) + #endif /* _LUSTRE_LIB_H */ diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c b/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c index 7cbc6a06afec..975fabc73148 100644 --- a/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c @@ -1913,14 +1913,12 @@ void ldlm_cancel_callback(struct ldlm_lock *lock) ldlm_set_bl_done(lock); wake_up_all(&lock->l_waitq); } else if (!ldlm_is_bl_done(lock)) { - struct l_wait_info lwi = { 0 }; - /* * The lock is guaranteed to have been canceled once * returning from this function. */ unlock_res_and_lock(lock); - l_wait_event(lock->l_waitq, is_bl_done(lock), &lwi); + wait_event(lock->l_waitq, is_bl_done(lock)); lock_res_and_lock(lock); } } diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c b/drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c index 5f6e7c933b81..d9835418d340 100644 --- a/drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c @@ -833,17 +833,15 @@ static int ldlm_bl_thread_main(void *arg) /* cannot use bltd after this, it is only on caller's stack */ while (1) { - struct l_wait_info lwi = { 0 }; struct ldlm_bl_work_item *blwi = NULL; struct obd_export *exp = NULL; int rc; rc = ldlm_bl_get_work(blp, &blwi, &exp); if (!rc) - l_wait_event_exclusive(blp->blp_waitq, - ldlm_bl_get_work(blp, &blwi, - &exp), - &lwi); + wait_event_exclusive_noload(blp->blp_waitq, + ldlm_bl_get_work(blp, &blwi, + &exp)); atomic_inc(&blp->blp_busy_threads); if (ldlm_bl_thread_need_create(blp, blwi)) diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c b/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c index 8563bd32befa..d562f90cee97 100644 --- a/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c @@ -1031,7 +1031,6 @@ static int ldlm_pools_thread_main(void *arg) static int ldlm_pools_thread_start(void) { - struct l_wait_info lwi = { 0 }; struct task_struct *task; if (ldlm_pools_thread) @@ -1052,8 +1051,8 @@ static int ldlm_pools_thread_start(void) ldlm_pools_thread = NULL; return PTR_ERR(task); } - l_wait_event(ldlm_pools_thread->t_ctl_waitq, - thread_is_running(ldlm_pools_thread), &lwi); + wait_event(ldlm_pools_thread->t_ctl_waitq, + thread_is_running(ldlm_pools_thread)); return 0; } diff --git a/drivers/staging/lustre/lustre/llite/statahead.c b/drivers/staging/lustre/lustre/llite/statahead.c index 90c7324575e4..39040916a043 100644 --- a/drivers/staging/lustre/lustre/llite/statahead.c +++ b/drivers/staging/lustre/lustre/llite/statahead.c @@ -864,7 +864,6 @@ static int ll_agl_thread(void *arg) struct ll_sb_info *sbi = ll_i2sbi(dir); struct ll_statahead_info *sai; struct ptlrpc_thread *thread; - struct l_wait_info lwi = { 0 }; sai = ll_sai_get(dir); thread = &sai->sai_agl_thread; @@ -885,10 +884,9 @@ static int ll_agl_thread(void *arg) wake_up(&thread->t_ctl_waitq); while (1) { - l_wait_event(thread->t_ctl_waitq, - !list_empty(&sai->sai_agls) || - !thread_is_running(thread), - &lwi); + wait_event_noload(thread->t_ctl_waitq, + !list_empty(&sai->sai_agls) || + !thread_is_running(thread)); if (!thread_is_running(thread)) break; @@ -932,7 +930,6 @@ static int ll_agl_thread(void *arg) static void ll_start_agl(struct dentry *parent, struct ll_statahead_info *sai) { struct ptlrpc_thread *thread = &sai->sai_agl_thread; - struct l_wait_info lwi = { 0 }; struct ll_inode_info *plli; struct task_struct *task; @@ -948,9 +945,8 @@ static void ll_start_agl(struct dentry *parent, struct ll_statahead_info *sai) return; } - l_wait_event(thread->t_ctl_waitq, - thread_is_running(thread) || thread_is_stopped(thread), - &lwi); + wait_event(thread->t_ctl_waitq, + thread_is_running(thread) || thread_is_stopped(thread)); } /* statahead thread main function */ @@ -968,7 +964,6 @@ static int ll_statahead_thread(void *arg) int first = 0; int rc = 0; struct md_op_data *op_data; - struct l_wait_info lwi = { 0 }; sai = ll_sai_get(dir); sa_thread = &sai->sai_thread; @@ -1069,12 +1064,11 @@ static int ll_statahead_thread(void *arg) /* wait for spare statahead window */ do { - l_wait_event(sa_thread->t_ctl_waitq, - !sa_sent_full(sai) || - sa_has_callback(sai) || - !list_empty(&sai->sai_agls) || - !thread_is_running(sa_thread), - &lwi); + wait_event(sa_thread->t_ctl_waitq, + !sa_sent_full(sai) || + sa_has_callback(sai) || + !list_empty(&sai->sai_agls) || + !thread_is_running(sa_thread)); sa_handle_callback(sai); spin_lock(&lli->lli_agl_lock); @@ -1128,11 +1122,10 @@ static int ll_statahead_thread(void *arg) * for file release to stop me. */ while (thread_is_running(sa_thread)) { - l_wait_event(sa_thread->t_ctl_waitq, - sa_has_callback(sai) || - !agl_list_empty(sai) || - !thread_is_running(sa_thread), - &lwi); + wait_event(sa_thread->t_ctl_waitq, + sa_has_callback(sai) || + !agl_list_empty(sai) || + !thread_is_running(sa_thread)); sa_handle_callback(sai); } @@ -1145,9 +1138,8 @@ static int ll_statahead_thread(void *arg) CDEBUG(D_READA, "stop agl thread: sai %p pid %u\n", sai, (unsigned int)agl_thread->t_pid); - l_wait_event(agl_thread->t_ctl_waitq, - thread_is_stopped(agl_thread), - &lwi); + wait_event(agl_thread->t_ctl_waitq, + thread_is_stopped(agl_thread)); } else { /* Set agl_thread flags anyway. */ thread_set_flags(agl_thread, SVC_STOPPED); @@ -1159,8 +1151,8 @@ static int ll_statahead_thread(void *arg) */ while (sai->sai_sent != sai->sai_replied) { /* in case we're not woken up, timeout wait */ - lwi = LWI_TIMEOUT(msecs_to_jiffies(MSEC_PER_SEC >> 3), - NULL, NULL); + struct l_wait_info lwi = LWI_TIMEOUT(msecs_to_jiffies(MSEC_PER_SEC >> 3), + NULL, NULL); l_wait_event(sa_thread->t_ctl_waitq, sai->sai_sent == sai->sai_replied, &lwi); } @@ -1520,7 +1512,6 @@ static int start_statahead_thread(struct inode *dir, struct dentry *dentry) { struct ll_inode_info *lli = ll_i2info(dir); struct ll_statahead_info *sai = NULL; - struct l_wait_info lwi = { 0 }; struct ptlrpc_thread *thread; struct task_struct *task; struct dentry *parent = dentry->d_parent; @@ -1570,9 +1561,8 @@ static int start_statahead_thread(struct inode *dir, struct dentry *dentry) goto out; } - l_wait_event(thread->t_ctl_waitq, - thread_is_running(thread) || thread_is_stopped(thread), - &lwi); + wait_event(thread->t_ctl_waitq, + thread_is_running(thread) || thread_is_stopped(thread)); ll_sai_put(sai); /* diff --git a/drivers/staging/lustre/lustre/lov/lov_object.c b/drivers/staging/lustre/lustre/lov/lov_object.c index 897cf2cd4a24..aa82f2ed40ae 100644 --- a/drivers/staging/lustre/lustre/lov/lov_object.c +++ b/drivers/staging/lustre/lustre/lov/lov_object.c @@ -723,15 +723,13 @@ static void lov_conf_unlock(struct lov_object *lov) static int lov_layout_wait(const struct lu_env *env, struct lov_object *lov) { - struct l_wait_info lwi = { 0 }; - while (atomic_read(&lov->lo_active_ios) > 0) { CDEBUG(D_INODE, "file:" DFID " wait for active IO, now: %d.\n", PFID(lu_object_fid(lov2lu(lov))), atomic_read(&lov->lo_active_ios)); - l_wait_event(lov->lo_waitq, - atomic_read(&lov->lo_active_ios) == 0, &lwi); + wait_event(lov->lo_waitq, + atomic_read(&lov->lo_active_ios) == 0); } return 0; } diff --git a/drivers/staging/lustre/lustre/mgc/mgc_request.c b/drivers/staging/lustre/lustre/mgc/mgc_request.c index 79ff85feab64..81b101941eec 100644 --- a/drivers/staging/lustre/lustre/mgc/mgc_request.c +++ b/drivers/staging/lustre/lustre/mgc/mgc_request.c @@ -601,9 +601,7 @@ static int mgc_requeue_thread(void *data) config_log_put(cld_prev); /* Wait a bit to see if anyone else needs a requeue */ - lwi = (struct l_wait_info) { 0 }; - l_wait_event(rq_waitq, rq_state & (RQ_NOW | RQ_STOP), - &lwi); + wait_event(rq_waitq, rq_state & (RQ_NOW | RQ_STOP)); spin_lock(&config_list_lock); } diff --git a/drivers/staging/lustre/lustre/obdclass/cl_io.c b/drivers/staging/lustre/lustre/obdclass/cl_io.c index 6ec5218a18c1..a3fb2bbde70f 100644 --- a/drivers/staging/lustre/lustre/obdclass/cl_io.c +++ b/drivers/staging/lustre/lustre/obdclass/cl_io.c @@ -1110,10 +1110,8 @@ int cl_sync_io_wait(const struct lu_env *env, struct cl_sync_io *anchor, CERROR("IO failed: %d, still wait for %d remaining entries\n", rc, atomic_read(&anchor->csi_sync_nr)); - lwi = (struct l_wait_info) { 0 }; - (void)l_wait_event(anchor->csi_waitq, - atomic_read(&anchor->csi_sync_nr) == 0, - &lwi); + wait_event(anchor->csi_waitq, + atomic_read(&anchor->csi_sync_nr) == 0); } else { rc = anchor->csi_sync_rc; } diff --git a/drivers/staging/lustre/lustre/obdclass/genops.c b/drivers/staging/lustre/lustre/obdclass/genops.c index b1d6ba4a3190..78f0fa1dff45 100644 --- a/drivers/staging/lustre/lustre/obdclass/genops.c +++ b/drivers/staging/lustre/lustre/obdclass/genops.c @@ -1237,12 +1237,10 @@ static int obd_zombie_is_idle(void) */ void obd_zombie_barrier(void) { - struct l_wait_info lwi = { 0 }; - if (obd_zombie_pid == current_pid()) /* don't wait for myself */ return; - l_wait_event(obd_zombie_waitq, obd_zombie_is_idle(), &lwi); + wait_event(obd_zombie_waitq, obd_zombie_is_idle()); } EXPORT_SYMBOL(obd_zombie_barrier); @@ -1257,10 +1255,8 @@ static int obd_zombie_impexp_thread(void *unused) obd_zombie_pid = current_pid(); while (!test_bit(OBD_ZOMBIE_STOP, &obd_zombie_flags)) { - struct l_wait_info lwi = { 0 }; - - l_wait_event(obd_zombie_waitq, - !obd_zombie_impexp_check(NULL), &lwi); + wait_event_noload(obd_zombie_waitq, + !obd_zombie_impexp_check(NULL)); obd_zombie_impexp_cull(); /* @@ -1593,7 +1589,6 @@ static inline bool obd_mod_rpc_slot_avail(struct client_obd *cli, u16 obd_get_mod_rpc_slot(struct client_obd *cli, __u32 opc, struct lookup_intent *it) { - struct l_wait_info lwi = LWI_INTR(NULL, NULL); bool close_req = false; u16 i, max; @@ -1631,8 +1626,8 @@ u16 obd_get_mod_rpc_slot(struct client_obd *cli, __u32 opc, CDEBUG(D_RPCTRACE, "%s: sleeping for a modify RPC slot opc %u, max %hu\n", cli->cl_import->imp_obd->obd_name, opc, max); - l_wait_event(cli->cl_mod_rpcs_waitq, - obd_mod_rpc_slot_avail(cli, close_req), &lwi); + wait_event(cli->cl_mod_rpcs_waitq, + obd_mod_rpc_slot_avail(cli, close_req)); } while (true); } EXPORT_SYMBOL(obd_get_mod_rpc_slot); diff --git a/drivers/staging/lustre/lustre/osc/osc_cache.c b/drivers/staging/lustre/lustre/osc/osc_cache.c index 5767ac2a7d16..d58a25a2a5b4 100644 --- a/drivers/staging/lustre/lustre/osc/osc_cache.c +++ b/drivers/staging/lustre/lustre/osc/osc_cache.c @@ -964,9 +964,8 @@ static int osc_extent_wait(const struct lu_env *env, struct osc_extent *ext, "%s: wait ext to %u timedout, recovery in progress?\n", cli_name(osc_cli(obj)), state); - lwi = LWI_INTR(NULL, NULL); - rc = l_wait_event(ext->oe_waitq, extent_wait_cb(ext, state), - &lwi); + wait_event(ext->oe_waitq, extent_wait_cb(ext, state)); + rc = 0; } if (rc == 0 && ext->oe_rc < 0) rc = ext->oe_rc; diff --git a/drivers/staging/lustre/lustre/osc/osc_object.c b/drivers/staging/lustre/lustre/osc/osc_object.c index f82c87a77550..1de25496a7d9 100644 --- a/drivers/staging/lustre/lustre/osc/osc_object.c +++ b/drivers/staging/lustre/lustre/osc/osc_object.c @@ -454,12 +454,10 @@ struct lu_object *osc_object_alloc(const struct lu_env *env, int osc_object_invalidate(const struct lu_env *env, struct osc_object *osc) { - struct l_wait_info lwi = { 0 }; - CDEBUG(D_INODE, "Invalidate osc object: %p, # of active IOs: %d\n", osc, atomic_read(&osc->oo_nr_ios)); - l_wait_event(osc->oo_io_waitq, !atomic_read(&osc->oo_nr_ios), &lwi); + wait_event(osc->oo_io_waitq, !atomic_read(&osc->oo_nr_ios)); /* Discard all dirty pages of this object. */ osc_cache_truncate_start(env, osc, 0, NULL); diff --git a/drivers/staging/lustre/lustre/ptlrpc/pinger.c b/drivers/staging/lustre/lustre/ptlrpc/pinger.c index fe6b47bfe8be..4148a6661dcf 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/pinger.c +++ b/drivers/staging/lustre/lustre/ptlrpc/pinger.c @@ -291,7 +291,6 @@ static struct ptlrpc_thread pinger_thread; int ptlrpc_start_pinger(void) { - struct l_wait_info lwi = { 0 }; struct task_struct *task; int rc; @@ -310,8 +309,8 @@ int ptlrpc_start_pinger(void) CERROR("cannot start pinger thread: rc = %d\n", rc); return rc; } - l_wait_event(pinger_thread.t_ctl_waitq, - thread_is_running(&pinger_thread), &lwi); + wait_event(pinger_thread.t_ctl_waitq, + thread_is_running(&pinger_thread)); return 0; } @@ -320,7 +319,6 @@ static int ptlrpc_pinger_remove_timeouts(void); int ptlrpc_stop_pinger(void) { - struct l_wait_info lwi = { 0 }; int rc = 0; if (thread_is_init(&pinger_thread) || @@ -331,8 +329,8 @@ int ptlrpc_stop_pinger(void) thread_set_flags(&pinger_thread, SVC_STOPPING); wake_up(&pinger_thread.t_ctl_waitq); - l_wait_event(pinger_thread.t_ctl_waitq, - thread_is_stopped(&pinger_thread), &lwi); + wait_event(pinger_thread.t_ctl_waitq, + thread_is_stopped(&pinger_thread)); return rc; } diff --git a/drivers/staging/lustre/lustre/ptlrpc/sec_gc.c b/drivers/staging/lustre/lustre/ptlrpc/sec_gc.c index d85c8638c009..e4197a60d1e2 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/sec_gc.c +++ b/drivers/staging/lustre/lustre/ptlrpc/sec_gc.c @@ -197,7 +197,6 @@ static int sec_gc_main(void *arg) int sptlrpc_gc_init(void) { - struct l_wait_info lwi = { 0 }; struct task_struct *task; mutex_init(&sec_gc_mutex); @@ -214,18 +213,16 @@ int sptlrpc_gc_init(void) return PTR_ERR(task); } - l_wait_event(sec_gc_thread.t_ctl_waitq, - thread_is_running(&sec_gc_thread), &lwi); + wait_event(sec_gc_thread.t_ctl_waitq, + thread_is_running(&sec_gc_thread)); return 0; } void sptlrpc_gc_fini(void) { - struct l_wait_info lwi = { 0 }; - thread_set_flags(&sec_gc_thread, SVC_STOPPING); wake_up(&sec_gc_thread.t_ctl_waitq); - l_wait_event(sec_gc_thread.t_ctl_waitq, - thread_is_stopped(&sec_gc_thread), &lwi); + wait_event(sec_gc_thread.t_ctl_waitq, + thread_is_stopped(&sec_gc_thread)); } diff --git a/drivers/staging/lustre/lustre/ptlrpc/service.c b/drivers/staging/lustre/lustre/ptlrpc/service.c index 63be6e7273f3..d688cb3ff157 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/service.c +++ b/drivers/staging/lustre/lustre/ptlrpc/service.c @@ -2233,7 +2233,7 @@ static int ptlrpc_hr_main(void *arg) wake_up(&ptlrpc_hr.hr_waitq); while (!ptlrpc_hr.hr_stopping) { - l_wait_condition(hrt->hrt_waitq, hrt_dont_sleep(hrt, &replies)); + wait_event_noload(hrt->hrt_waitq, hrt_dont_sleep(hrt, &replies)); while (!list_empty(&replies)) { struct ptlrpc_reply_state *rs; @@ -2312,7 +2312,6 @@ static int ptlrpc_start_hr_threads(void) static void ptlrpc_svcpt_stop_threads(struct ptlrpc_service_part *svcpt) { - struct l_wait_info lwi = { 0 }; struct ptlrpc_thread *thread; LIST_HEAD(zombie); @@ -2341,8 +2340,8 @@ static void ptlrpc_svcpt_stop_threads(struct ptlrpc_service_part *svcpt) CDEBUG(D_INFO, "waiting for stopping-thread %s #%u\n", svcpt->scp_service->srv_thread_name, thread->t_id); - l_wait_event(thread->t_ctl_waitq, - thread_is_stopped(thread), &lwi); + wait_event(thread->t_ctl_waitq, + thread_is_stopped(thread)); spin_lock(&svcpt->scp_lock); } @@ -2403,7 +2402,6 @@ int ptlrpc_start_threads(struct ptlrpc_service *svc) int ptlrpc_start_thread(struct ptlrpc_service_part *svcpt, int wait) { - struct l_wait_info lwi = { 0 }; struct ptlrpc_thread *thread; struct ptlrpc_service *svc; struct task_struct *task; @@ -2499,9 +2497,8 @@ int ptlrpc_start_thread(struct ptlrpc_service_part *svcpt, int wait) if (!wait) return 0; - l_wait_event(thread->t_ctl_waitq, - thread_is_running(thread) || thread_is_stopped(thread), - &lwi); + wait_event(thread->t_ctl_waitq, + thread_is_running(thread) || thread_is_stopped(thread)); rc = thread_is_stopped(thread) ? thread->t_id : 0; return rc; From neilb at suse.com Mon Dec 18 07:18:00 2017 From: neilb at suse.com (NeilBrown) Date: Mon, 18 Dec 2017 18:18:00 +1100 Subject: [lustre-devel] [PATCH 08/16] staging: lustre: open code polling loop instead of using l_wait_event() In-Reply-To: <151358127190.5099.12792810096274074963.stgit@noble> References: <151358127190.5099.12792810096274074963.stgit@noble> Message-ID: <151358148008.5099.7316878897181140635.stgit@noble> Two places that LWI_TIMEOUT_INTERVAL() is used, the outcome is a simple polling loop that polls every second for some event (with a limit). So write a simple loop to make this more apparent. Signed-off-by: NeilBrown --- drivers/staging/lustre/lustre/llite/llite_lib.c | 11 +++++------ drivers/staging/lustre/lustre/lov/lov_request.c | 12 +++++------- 2 files changed, 10 insertions(+), 13 deletions(-) diff --git a/drivers/staging/lustre/lustre/llite/llite_lib.c b/drivers/staging/lustre/lustre/llite/llite_lib.c index 33dc15e9aebb..f6642fa30428 100644 --- a/drivers/staging/lustre/lustre/llite/llite_lib.c +++ b/drivers/staging/lustre/lustre/llite/llite_lib.c @@ -1984,8 +1984,7 @@ void ll_umount_begin(struct super_block *sb) struct ll_sb_info *sbi = ll_s2sbi(sb); struct obd_device *obd; struct obd_ioctl_data *ioc_data; - wait_queue_head_t waitq; - struct l_wait_info lwi; + int cnt = 0; CDEBUG(D_VFSTRACE, "VFS Op: superblock %p count %d active %d\n", sb, sb->s_count, atomic_read(&sb->s_active)); @@ -2021,10 +2020,10 @@ void ll_umount_begin(struct super_block *sb) * and then continue. For now, we just periodically checking for vfs * to decrement mnt_cnt and hope to finish it within 10sec. */ - init_waitqueue_head(&waitq); - lwi = LWI_TIMEOUT_INTERVAL(10 * HZ, - HZ, NULL, NULL); - l_wait_event(waitq, may_umount(sbi->ll_mnt.mnt), &lwi); + while (cnt < 10 && !may_umount(sbi->ll_mnt.mnt)) { + schedule_timeout_uninterruptible(HZ); + cnt ++; + } schedule(); } diff --git a/drivers/staging/lustre/lustre/lov/lov_request.c b/drivers/staging/lustre/lustre/lov/lov_request.c index fb3b7a7fa32a..c1e58fcc30b3 100644 --- a/drivers/staging/lustre/lustre/lov/lov_request.c +++ b/drivers/staging/lustre/lustre/lov/lov_request.c @@ -99,8 +99,7 @@ static int lov_check_set(struct lov_obd *lov, int idx) */ static int lov_check_and_wait_active(struct lov_obd *lov, int ost_idx) { - wait_queue_head_t waitq; - struct l_wait_info lwi; + int cnt = 0; struct lov_tgt_desc *tgt; int rc = 0; @@ -125,11 +124,10 @@ static int lov_check_and_wait_active(struct lov_obd *lov, int ost_idx) mutex_unlock(&lov->lov_lock); - init_waitqueue_head(&waitq); - lwi = LWI_TIMEOUT_INTERVAL(obd_timeout * HZ, - HZ, NULL, NULL); - - rc = l_wait_event(waitq, lov_check_set(lov, ost_idx), &lwi); + while (cnt < obd_timeout && !lov_check_set(lov, ost_idx)) { + schedule_timeout_uninterruptible(HZ); + cnt ++; + } if (tgt->ltd_active) return 1; From neilb at suse.com Mon Dec 18 07:18:00 2017 From: neilb at suse.com (NeilBrown) Date: Mon, 18 Dec 2017 18:18:00 +1100 Subject: [lustre-devel] [PATCH 06/16] staging: lustre: simplify l_wait_event when intr handler but no timeout. In-Reply-To: <151358127190.5099.12792810096274074963.stgit@noble> References: <151358127190.5099.12792810096274074963.stgit@noble> Message-ID: <151358148002.5099.9981162814357119900.stgit@noble> If l_wait_event() is given a function to be called on a signal, but no timeout or timeout handler, then the intr function is simply called at the end if the wait was aborted by a signal. So a simpler way to write the code (in the one place this case is used) it to open-code the body of the function after the wait_event, if -ERESTARTSYS was returned. Signed-off-by: NeilBrown --- drivers/staging/lustre/lustre/ldlm/ldlm_flock.c | 30 +++++------------------ 1 file changed, 7 insertions(+), 23 deletions(-) diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_flock.c b/drivers/staging/lustre/lustre/ldlm/ldlm_flock.c index 657ab95091a0..411b540b96d9 100644 --- a/drivers/staging/lustre/lustre/ldlm/ldlm_flock.c +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_flock.c @@ -310,24 +310,6 @@ static int ldlm_process_flock_lock(struct ldlm_lock *req) return LDLM_ITER_CONTINUE; } -struct ldlm_flock_wait_data { - struct ldlm_lock *fwd_lock; -}; - -static void -ldlm_flock_interrupted_wait(void *data) -{ - struct ldlm_lock *lock; - - lock = ((struct ldlm_flock_wait_data *)data)->fwd_lock; - - lock_res_and_lock(lock); - - /* client side - set flag to prevent lock from being put on LRU list */ - ldlm_set_cbpending(lock); - unlock_res_and_lock(lock); -} - /** * Flock completion callback function. * @@ -342,8 +324,6 @@ int ldlm_flock_completion_ast(struct ldlm_lock *lock, __u64 flags, void *data) { struct file_lock *getlk = lock->l_ast_data; - struct ldlm_flock_wait_data fwd; - struct l_wait_info lwi; int rc = 0; OBD_FAIL_TIMEOUT(OBD_FAIL_LDLM_CP_CB_WAIT2, 4); @@ -372,13 +352,17 @@ ldlm_flock_completion_ast(struct ldlm_lock *lock, __u64 flags, void *data) LDLM_DEBUG(lock, "client-side enqueue returned a blocked lock, sleeping"); - fwd.fwd_lock = lock; - lwi = LWI_TIMEOUT_INTR(0, NULL, ldlm_flock_interrupted_wait, &fwd); /* Go to sleep until the lock is granted. */ - rc = l_wait_event(lock->l_waitq, is_granted_or_cancelled(lock), &lwi); + rc = l_wait_event_abortable(lock->l_waitq, is_granted_or_cancelled(lock)); if (rc) { + lock_res_and_lock(lock); + + /* client side - set flag to prevent lock from being put on LRU list */ + ldlm_set_cbpending(lock); + unlock_res_and_lock(lock); + LDLM_DEBUG(lock, "client-side enqueue waking up: failed (%d)", rc); return rc; From neilb at suse.com Mon Dec 18 07:18:00 2017 From: neilb at suse.com (NeilBrown) Date: Mon, 18 Dec 2017 18:18:00 +1100 Subject: [lustre-devel] [PATCH 07/16] staging: lustre: simplify waiting in ldlm_completion_ast() In-Reply-To: <151358127190.5099.12792810096274074963.stgit@noble> References: <151358127190.5099.12792810096274074963.stgit@noble> Message-ID: <151358148005.5099.1737285068022718842.stgit@noble> If a signal-callback (lwi_on_signal) is set without lwi_allow_intr, as is the case in ldlm_completion_ast(), the behavior depends on the timeout set. If a timeout is set, then signals are ignored. If the timeout is reached, the timeout handler is called. If no timeout is set, then "fatal" signals are not ignored. If one arrives the callback is run, but as the callback is empty in this case, that is not relevant. So treat the two cases separately. In one, use l_wait_event_abortable(). In the other, use wait_event_timeout(), and if the timeout happens - perform the timeout-call function (opencoded). Signed-off-by: NeilBrown --- drivers/staging/lustre/lustre/ldlm/ldlm_request.c | 51 +++++++-------------- 1 file changed, 17 insertions(+), 34 deletions(-) diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_request.c b/drivers/staging/lustre/lustre/ldlm/ldlm_request.c index a244fa717134..05bec9fd629b 100644 --- a/drivers/staging/lustre/lustre/ldlm/ldlm_request.c +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_request.c @@ -72,15 +72,6 @@ MODULE_PARM_DESC(ldlm_enqueue_min, "lock enqueue timeout minimum"); /* in client side, whether the cached locks will be canceled before replay */ unsigned int ldlm_cancel_unused_locks_before_replay = 1; -static void interrupted_completion_wait(void *data) -{ -} - -struct lock_wait_data { - struct ldlm_lock *lwd_lock; - __u32 lwd_conn_cnt; -}; - struct ldlm_async_args { struct lustre_handle lock_handle; }; @@ -112,10 +103,8 @@ static int ldlm_request_bufsize(int count, int type) return sizeof(struct ldlm_request) + avail; } -static int ldlm_expired_completion_wait(void *data) +static int ldlm_expired_completion_wait(struct ldlm_lock *lock, struct obd_import *imp2) { - struct lock_wait_data *lwd = data; - struct ldlm_lock *lock = lwd->lwd_lock; struct obd_import *imp; struct obd_device *obd; @@ -140,7 +129,7 @@ static int ldlm_expired_completion_wait(void *data) obd = lock->l_conn_export->exp_obd; imp = obd->u.cli.cl_import; - ptlrpc_fail_import(imp, lwd->lwd_conn_cnt); + ptlrpc_fail_import(imp, imp2 ? imp2->imp_conn_cnt : 0); LDLM_ERROR(lock, "lock timed out (enqueued at %lld, %llds ago), entering recovery for %s@%s", (s64)lock->l_last_activity, @@ -251,10 +240,8 @@ EXPORT_SYMBOL(ldlm_completion_ast_async); int ldlm_completion_ast(struct ldlm_lock *lock, __u64 flags, void *data) { /* XXX ALLOCATE - 160 bytes */ - struct lock_wait_data lwd; struct obd_device *obd; struct obd_import *imp = NULL; - struct l_wait_info lwi; __u32 timeout; int rc = 0; @@ -281,32 +268,28 @@ int ldlm_completion_ast(struct ldlm_lock *lock, __u64 flags, void *data) timeout = ldlm_cp_timeout(lock); - lwd.lwd_lock = lock; lock->l_last_activity = ktime_get_real_seconds(); - if (ldlm_is_no_timeout(lock)) { - LDLM_DEBUG(lock, "waiting indefinitely because of NO_TIMEOUT"); - lwi = LWI_INTR(interrupted_completion_wait, &lwd); - } else { - lwi = LWI_TIMEOUT_INTR(timeout * HZ, - ldlm_expired_completion_wait, - interrupted_completion_wait, &lwd); - } - - if (imp) { - spin_lock(&imp->imp_lock); - lwd.lwd_conn_cnt = imp->imp_conn_cnt; - spin_unlock(&imp->imp_lock); - } - if (OBD_FAIL_CHECK_RESET(OBD_FAIL_LDLM_INTR_CP_AST, OBD_FAIL_LDLM_CP_BL_RACE | OBD_FAIL_ONCE)) { ldlm_set_fail_loc(lock); rc = -EINTR; } else { - /* Go to sleep until the lock is granted or cancelled. */ - rc = l_wait_event(lock->l_waitq, - is_granted_or_cancelled(lock), &lwi); + /* Go to sleep until the lock is granted or canceled. */ + if (ldlm_is_no_timeout(lock)) { + LDLM_DEBUG(lock, "waiting indefinitely because of NO_TIMEOUT"); + rc = l_wait_event_abortable(lock->l_waitq, + is_granted_or_cancelled(lock)); + } else { + rc = wait_event_timeout(lock->l_waitq, + is_granted_or_cancelled(lock), + timeout * HZ); + if (rc == 0) { + ldlm_expired_completion_wait(lock, imp); + rc = -ETIMEDOUT; + } else + rc = 0; + } } if (rc) { From neilb at suse.com Mon Dec 18 07:17:59 2017 From: neilb at suse.com (NeilBrown) Date: Mon, 18 Dec 2017 18:17:59 +1100 Subject: [lustre-devel] [PATCH 04/16] staging: lustre: use wait_event_timeout() where appropriate. In-Reply-To: <151358127190.5099.12792810096274074963.stgit@noble> References: <151358127190.5099.12792810096274074963.stgit@noble> Message-ID: <151358147993.5099.9887255633793256926.stgit@noble> When the lwi arg has a timeout, but no timeout callback function, l_wait_event() acts much the same as wait_event_timeout() - the wait is not interruptible and simply waits for the event or the timeouts. The most noticable difference is that the return value is -ETIMEDOUT or 0, rather than 0 or non-zero. Another difference is that the process waiting is included in the load-average. This is probably more correct. A final difference is that if the timeout is zero, l_wait_event() will not time out at all. In the one case where that is possible we need to conditionally use wait_event_noload(). So replace all such calls with wait_event_timeout(), being careful of the return value. In one case, there is no event, so use schedule_timeout_uninterruptible(). Note that the presence or absence of LWI_ON_SIGNAL_NOOP has no effect in these cases. Signed-off-by: NeilBrown --- drivers/staging/lustre/lustre/include/lustre_lib.h | 37 ++++++++++++++++++++ drivers/staging/lustre/lustre/ldlm/ldlm_lock.c | 10 ++--- drivers/staging/lustre/lustre/ldlm/ldlm_pool.c | 12 ++---- drivers/staging/lustre/lustre/llite/statahead.c | 14 +++----- drivers/staging/lustre/lustre/mdc/mdc_request.c | 5 +-- drivers/staging/lustre/lustre/mgc/mgc_request.c | 15 +++----- drivers/staging/lustre/lustre/obdclass/cl_io.c | 17 +++++---- drivers/staging/lustre/lustre/osc/osc_cache.c | 25 ++++++-------- drivers/staging/lustre/lustre/ptlrpc/events.c | 7 +--- drivers/staging/lustre/lustre/ptlrpc/import.c | 12 +++--- .../staging/lustre/lustre/ptlrpc/pack_generic.c | 9 ++--- drivers/staging/lustre/lustre/ptlrpc/pinger.c | 12 ++---- drivers/staging/lustre/lustre/ptlrpc/recover.c | 12 +++--- drivers/staging/lustre/lustre/ptlrpc/sec_gc.c | 10 ++--- drivers/staging/lustre/lustre/ptlrpc/service.c | 10 ++--- 15 files changed, 104 insertions(+), 103 deletions(-) diff --git a/drivers/staging/lustre/lustre/include/lustre_lib.h b/drivers/staging/lustre/lustre/include/lustre_lib.h index 08bdd618ea7d..fcf31c779e98 100644 --- a/drivers/staging/lustre/lustre/include/lustre_lib.h +++ b/drivers/staging/lustre/lustre/include/lustre_lib.h @@ -388,4 +388,41 @@ do { \ schedule()); \ } while (0) +#define __wait_event_noload_timeout(wq_head, condition, timeout) \ + ___wait_event(wq_head, ___wait_cond_timeout(condition), \ + (TASK_UNINTERRUPTIBLE | TASK_NOLOAD), 0, timeout, \ + __ret = schedule_timeout(__ret)) + +/** + * wait_event_noload_timeout - sleep until a condition gets true or a timeout elapses + * @wq_head: the waitqueue to wait on + * @condition: a C expression for the event to wait for + * @timeout: timeout, in jiffies + * + * The process is put to sleep (TASK_UNINTERRUPTIBLE | TASK_NOLOAD) until the + * @condition evaluates to true. The @condition is checked each time + * the waitqueue @wq_head is woken up. + * + * wake_up() has to be called after changing any variable that could + * change the result of the wait condition. + * + * This is suitable for service threads that are waiting for work to do + * where there is no implication that the event not being true yet implies + * any load on the system, and where it is not appropriate for the + * soft-lockup detector to warning if the wait is unusually long. + * + * Returns: + * 0 if the @condition evaluated to %false after the @timeout elapsed, + * 1 if the @condition evaluated to %true after the @timeout elapsed, + * or the remaining jiffies (at least 1) if the @condition evaluated + * to %true before the @timeout elapsed. + */ +#define wait_event_noload_timeout(wq_head, condition, timeout) \ +({ \ + long __ret = timeout; \ + might_sleep(); \ + if (!___wait_cond_timeout(condition)) \ + __ret = __wait_event_noload_timeout(wq_head, condition, timeout);\ + __ret; \ +}) #endif /* _LUSTRE_LIB_H */ diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c b/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c index 53500883f243..ba720a888da3 100644 --- a/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c @@ -1349,7 +1349,6 @@ enum ldlm_mode ldlm_lock_match(struct ldlm_namespace *ns, __u64 flags, if ((flags & LDLM_FL_LVB_READY) && !ldlm_is_lvb_ready(lock)) { __u64 wait_flags = LDLM_FL_LVB_READY | LDLM_FL_DESTROYED | LDLM_FL_FAIL_NOTIFIED; - struct l_wait_info lwi; if (lock->l_completion_ast) { int err = lock->l_completion_ast(lock, @@ -1366,13 +1365,10 @@ enum ldlm_mode ldlm_lock_match(struct ldlm_namespace *ns, __u64 flags, } } - lwi = LWI_TIMEOUT_INTR(obd_timeout * HZ, - NULL, LWI_ON_SIGNAL_NOOP, NULL); - /* XXX FIXME see comment on CAN_MATCH in lustre_dlm.h */ - l_wait_event(lock->l_waitq, - lock->l_flags & wait_flags, - &lwi); + wait_event_timeout(lock->l_waitq, + lock->l_flags & wait_flags, + obd_timeout * HZ); if (!ldlm_is_lvb_ready(lock)) { if (flags & LDLM_FL_TEST_LOCK) LDLM_LOCK_RELEASE(lock); diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c b/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c index eb1f3d45c68c..c42b65173407 100644 --- a/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c @@ -997,8 +997,6 @@ static int ldlm_pools_thread_main(void *arg) "ldlm_poold", current_pid()); while (1) { - struct l_wait_info lwi; - /* * Recal all pools on this tick. */ @@ -1008,12 +1006,10 @@ static int ldlm_pools_thread_main(void *arg) * Wait until the next check time, or until we're * stopped. */ - lwi = LWI_TIMEOUT(c_time * HZ, - NULL, NULL); - l_wait_event(thread->t_ctl_waitq, - thread_is_stopping(thread) || - thread_is_event(thread), - &lwi); + wait_event_noload_timeout(thread->t_ctl_waitq, + thread_is_stopping(thread) || + thread_is_event(thread), + c_time * HZ); if (thread_test_and_clear_flags(thread, SVC_STOPPING)) break; diff --git a/drivers/staging/lustre/lustre/llite/statahead.c b/drivers/staging/lustre/lustre/llite/statahead.c index 9f9897523374..aecad3e6e808 100644 --- a/drivers/staging/lustre/lustre/llite/statahead.c +++ b/drivers/staging/lustre/lustre/llite/statahead.c @@ -1151,10 +1151,9 @@ static int ll_statahead_thread(void *arg) */ while (sai->sai_sent != sai->sai_replied) { /* in case we're not woken up, timeout wait */ - struct l_wait_info lwi = LWI_TIMEOUT(msecs_to_jiffies(MSEC_PER_SEC >> 3), - NULL, NULL); - l_wait_event(sa_thread->t_ctl_waitq, - sai->sai_sent == sai->sai_replied, &lwi); + wait_event_timeout(sa_thread->t_ctl_waitq, + sai->sai_sent == sai->sai_replied, + HZ>>3); } /* release resources held by statahead RPCs */ @@ -1374,7 +1373,6 @@ static int revalidate_statahead_dentry(struct inode *dir, { struct ll_inode_info *lli = ll_i2info(dir); struct sa_entry *entry = NULL; - struct l_wait_info lwi = { 0 }; struct ll_dentry_data *ldd; int rc = 0; @@ -1424,10 +1422,8 @@ static int revalidate_statahead_dentry(struct inode *dir, spin_lock(&lli->lli_sa_lock); sai->sai_index_wait = entry->se_index; spin_unlock(&lli->lli_sa_lock); - lwi = LWI_TIMEOUT_INTR(30 * HZ, NULL, - LWI_ON_SIGNAL_NOOP, NULL); - rc = l_wait_event(sai->sai_waitq, sa_ready(entry), &lwi); - if (rc < 0) { + if (0 == wait_event_timeout(sai->sai_waitq, + sa_ready(entry), 30 * HZ)) { /* * entry may not be ready, so it may be used by inflight * statahead RPC, don't free it. diff --git a/drivers/staging/lustre/lustre/mdc/mdc_request.c b/drivers/staging/lustre/lustre/mdc/mdc_request.c index b12518ba5ae9..0409b4948c39 100644 --- a/drivers/staging/lustre/lustre/mdc/mdc_request.c +++ b/drivers/staging/lustre/lustre/mdc/mdc_request.c @@ -838,7 +838,6 @@ static int mdc_getpage(struct obd_export *exp, const struct lu_fid *fid, struct ptlrpc_bulk_desc *desc; struct ptlrpc_request *req; wait_queue_head_t waitq; - struct l_wait_info lwi; int resends = 0; int rc; int i; @@ -888,9 +887,7 @@ static int mdc_getpage(struct obd_export *exp, const struct lu_fid *fid, exp->exp_obd->obd_name, -EIO); return -EIO; } - lwi = LWI_TIMEOUT_INTR(resends * HZ, NULL, NULL, - NULL); - l_wait_event(waitq, 0, &lwi); + wait_event_timeout(waitq, 0, resends * HZ); goto restart_bulk; } diff --git a/drivers/staging/lustre/lustre/mgc/mgc_request.c b/drivers/staging/lustre/lustre/mgc/mgc_request.c index 4c2e20ff95d7..b42ec5b4a24c 100644 --- a/drivers/staging/lustre/lustre/mgc/mgc_request.c +++ b/drivers/staging/lustre/lustre/mgc/mgc_request.c @@ -535,7 +535,6 @@ static int mgc_requeue_thread(void *data) spin_lock(&config_list_lock); rq_state |= RQ_RUNNING; while (!(rq_state & RQ_STOP)) { - struct l_wait_info lwi; struct config_llog_data *cld, *cld_prev; int rand = prandom_u32_max(MGC_TIMEOUT_RAND_CENTISEC); int to; @@ -556,9 +555,9 @@ static int mgc_requeue_thread(void *data) to = msecs_to_jiffies(MGC_TIMEOUT_MIN_SECONDS * MSEC_PER_SEC); /* rand is centi-seconds */ to += msecs_to_jiffies(rand * MSEC_PER_SEC / 100); - lwi = LWI_TIMEOUT(to, NULL, NULL); - l_wait_event(rq_waitq, rq_state & (RQ_STOP | RQ_PRECLEANUP), - &lwi); + wait_event_noload_timeout(rq_waitq, + rq_state & (RQ_STOP | RQ_PRECLEANUP), + to); /* * iterate & processing through the list. for each cld, process @@ -1628,9 +1627,7 @@ int mgc_process_log(struct obd_device *mgc, struct config_llog_data *cld) if (rcl == -ESHUTDOWN && atomic_read(&mgc->u.cli.cl_mgc_refcount) > 0 && !retry) { - int secs = obd_timeout * HZ; struct obd_import *imp; - struct l_wait_info lwi; mutex_unlock(&cld->cld_lock); imp = class_exp2cliimp(mgc->u.cli.cl_mgc_mgsexp); @@ -1645,9 +1642,9 @@ int mgc_process_log(struct obd_device *mgc, struct config_llog_data *cld) */ ptlrpc_pinger_force(imp); - lwi = LWI_TIMEOUT(secs, NULL, NULL); - l_wait_event(imp->imp_recovery_waitq, - !mgc_import_in_recovery(imp), &lwi); + wait_event_timeout(imp->imp_recovery_waitq, + !mgc_import_in_recovery(imp), + obd_timeout * HZ); if (imp->imp_state == LUSTRE_IMP_FULL) { retry = true; diff --git a/drivers/staging/lustre/lustre/obdclass/cl_io.c b/drivers/staging/lustre/lustre/obdclass/cl_io.c index d8be01b9257f..5330962c1f66 100644 --- a/drivers/staging/lustre/lustre/obdclass/cl_io.c +++ b/drivers/staging/lustre/lustre/obdclass/cl_io.c @@ -1097,16 +1097,19 @@ EXPORT_SYMBOL(cl_sync_io_init); int cl_sync_io_wait(const struct lu_env *env, struct cl_sync_io *anchor, long timeout) { - struct l_wait_info lwi = LWI_TIMEOUT_INTR(timeout * HZ, - NULL, NULL, NULL); - int rc; + int rc = 1; LASSERT(timeout >= 0); - rc = l_wait_event(anchor->csi_waitq, - atomic_read(&anchor->csi_sync_nr) == 0, - &lwi); - if (rc < 0) { + if (timeout == 0) + wait_event_noload(anchor->csi_waitq, + atomic_read(&anchor->csi_sync_nr) == 0); + else + rc = wait_event_timeout(anchor->csi_waitq, + atomic_read(&anchor->csi_sync_nr) == 0, + timeout * HZ); + if (rc == 0) { + rc = -ETIMEDOUT; CERROR("IO failed: %d, still wait for %d remaining entries\n", rc, atomic_read(&anchor->csi_sync_nr)); diff --git a/drivers/staging/lustre/lustre/osc/osc_cache.c b/drivers/staging/lustre/lustre/osc/osc_cache.c index e6d99dc048ce..186cc1a0130a 100644 --- a/drivers/staging/lustre/lustre/osc/osc_cache.c +++ b/drivers/staging/lustre/lustre/osc/osc_cache.c @@ -934,8 +934,6 @@ static int osc_extent_wait(const struct lu_env *env, struct osc_extent *ext, enum osc_extent_state state) { struct osc_object *obj = ext->oe_obj; - struct l_wait_info lwi = LWI_TIMEOUT_INTR(600 * HZ, NULL, - LWI_ON_SIGNAL_NOOP, NULL); int rc = 0; osc_object_lock(obj); @@ -958,17 +956,19 @@ static int osc_extent_wait(const struct lu_env *env, struct osc_extent *ext, osc_extent_release(env, ext); /* wait for the extent until its state becomes @state */ - rc = l_wait_event(ext->oe_waitq, extent_wait_cb(ext, state), &lwi); - if (rc == -ETIMEDOUT) { + rc = wait_event_timeout(ext->oe_waitq, + extent_wait_cb(ext, state), 600 * HZ); + if (rc == 0) { OSC_EXTENT_DUMP(D_ERROR, ext, "%s: wait ext to %u timedout, recovery in progress?\n", cli_name(osc_cli(obj)), state); wait_event(ext->oe_waitq, extent_wait_cb(ext, state)); - rc = 0; } - if (rc == 0 && ext->oe_rc < 0) + if (ext->oe_rc < 0) rc = ext->oe_rc; + else + rc = 0; return rc; } @@ -1568,12 +1568,9 @@ static int osc_enter_cache(const struct lu_env *env, struct client_obd *cli, struct osc_object *osc = oap->oap_obj; struct lov_oinfo *loi = osc->oo_oinfo; struct osc_cache_waiter ocw; - struct l_wait_info lwi; + unsigned long timeout = (AT_OFF ? obd_timeout : at_max) * HZ; int rc = -EDQUOT; - lwi = LWI_TIMEOUT_INTR((AT_OFF ? obd_timeout : at_max) * HZ, - NULL, LWI_ON_SIGNAL_NOOP, NULL); - OSC_DUMP_GRANT(D_CACHE, cli, "need:%d\n", bytes); spin_lock(&cli->cl_loi_list_lock); @@ -1616,13 +1613,15 @@ static int osc_enter_cache(const struct lu_env *env, struct client_obd *cli, CDEBUG(D_CACHE, "%s: sleeping for cache space @ %p for %p\n", cli_name(cli), &ocw, oap); - rc = l_wait_event(ocw.ocw_waitq, ocw_granted(cli, &ocw), &lwi); + rc = wait_event_timeout(ocw.ocw_waitq, + ocw_granted(cli, &ocw), timeout); spin_lock(&cli->cl_loi_list_lock); - if (rc < 0) { - /* l_wait_event is interrupted by signal, or timed out */ + if (rc == 0) { + /* wait_event is interrupted by signal, or timed out */ list_del_init(&ocw.ocw_entry); + rc = -ETIMEDOUT; break; } LASSERT(list_empty(&ocw.ocw_entry)); diff --git a/drivers/staging/lustre/lustre/ptlrpc/events.c b/drivers/staging/lustre/lustre/ptlrpc/events.c index 71f7588570ef..130bacc2c891 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/events.c +++ b/drivers/staging/lustre/lustre/ptlrpc/events.c @@ -490,8 +490,6 @@ int ptlrpc_uuid_to_peer(struct obd_uuid *uuid, static void ptlrpc_ni_fini(void) { - wait_queue_head_t waitq; - struct l_wait_info lwi; int rc; int retries; @@ -515,10 +513,7 @@ static void ptlrpc_ni_fini(void) if (retries != 0) CWARN("Event queue still busy\n"); - /* Wait for a bit */ - init_waitqueue_head(&waitq); - lwi = LWI_TIMEOUT(2 * HZ, NULL, NULL); - l_wait_event(waitq, 0, &lwi); + schedule_timeout_uninterruptible(2 * HZ); break; } } diff --git a/drivers/staging/lustre/lustre/ptlrpc/import.c b/drivers/staging/lustre/lustre/ptlrpc/import.c index 0eba5f18bd3b..34b4075fac42 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/import.c +++ b/drivers/staging/lustre/lustre/ptlrpc/import.c @@ -430,21 +430,19 @@ void ptlrpc_fail_import(struct obd_import *imp, __u32 conn_cnt) int ptlrpc_reconnect_import(struct obd_import *imp) { - struct l_wait_info lwi; - int secs = obd_timeout * HZ; int rc; ptlrpc_pinger_force(imp); CDEBUG(D_HA, "%s: recovery started, waiting %u seconds\n", - obd2cli_tgt(imp->imp_obd), secs); + obd2cli_tgt(imp->imp_obd), obd_timeout); - lwi = LWI_TIMEOUT(secs, NULL, NULL); - rc = l_wait_event(imp->imp_recovery_waitq, - !ptlrpc_import_in_recovery(imp), &lwi); + rc = wait_event_timeout(imp->imp_recovery_waitq, + !ptlrpc_import_in_recovery(imp), + obd_timeout * HZ); CDEBUG(D_HA, "%s: recovery finished s:%s\n", obd2cli_tgt(imp->imp_obd), ptlrpc_import_state_name(imp->imp_state)); - return rc; + return rc == 0 ? -ETIMEDOUT : 0; } EXPORT_SYMBOL(ptlrpc_reconnect_import); diff --git a/drivers/staging/lustre/lustre/ptlrpc/pack_generic.c b/drivers/staging/lustre/lustre/ptlrpc/pack_generic.c index c060d6f5015a..6a1c3c041096 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/pack_generic.c +++ b/drivers/staging/lustre/lustre/ptlrpc/pack_generic.c @@ -260,17 +260,16 @@ lustre_get_emerg_rs(struct ptlrpc_service_part *svcpt) /* See if we have anything in a pool, and wait if nothing */ while (list_empty(&svcpt->scp_rep_idle)) { - struct l_wait_info lwi; int rc; spin_unlock(&svcpt->scp_rep_lock); /* If we cannot get anything for some long time, we better * bail out instead of waiting infinitely */ - lwi = LWI_TIMEOUT(10 * HZ, NULL, NULL); - rc = l_wait_event(svcpt->scp_rep_waitq, - !list_empty(&svcpt->scp_rep_idle), &lwi); - if (rc != 0) + rc = wait_event_timeout(svcpt->scp_rep_waitq, + !list_empty(&svcpt->scp_rep_idle), + 10 * HZ); + if (rc == 0) goto out; spin_lock(&svcpt->scp_rep_lock); } diff --git a/drivers/staging/lustre/lustre/ptlrpc/pinger.c b/drivers/staging/lustre/lustre/ptlrpc/pinger.c index da3afda72e14..ed919507a50a 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/pinger.c +++ b/drivers/staging/lustre/lustre/ptlrpc/pinger.c @@ -228,7 +228,6 @@ static int ptlrpc_pinger_main(void *arg) /* And now, loop forever, pinging as needed. */ while (1) { unsigned long this_ping = cfs_time_current(); - struct l_wait_info lwi; long time_to_next_wake; struct timeout_item *item; struct list_head *iter; @@ -266,13 +265,10 @@ static int ptlrpc_pinger_main(void *arg) cfs_time_add(this_ping, PING_INTERVAL * HZ)); if (time_to_next_wake > 0) { - lwi = LWI_TIMEOUT(max_t(long, time_to_next_wake, - HZ), - NULL, NULL); - l_wait_event(thread->t_ctl_waitq, - thread_is_stopping(thread) || - thread_is_event(thread), - &lwi); + wait_event_noload_timeout(thread->t_ctl_waitq, + thread_is_stopping(thread) || + thread_is_event(thread), + max_t(long, time_to_next_wake, HZ)); if (thread_test_and_clear_flags(thread, SVC_STOPPING)) break; /* woken after adding import to reset timer */ diff --git a/drivers/staging/lustre/lustre/ptlrpc/recover.c b/drivers/staging/lustre/lustre/ptlrpc/recover.c index 5bbd23eebfa6..c8a7fca6d906 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/recover.c +++ b/drivers/staging/lustre/lustre/ptlrpc/recover.c @@ -346,17 +346,15 @@ int ptlrpc_recover_import(struct obd_import *imp, char *new_uuid, int async) goto out; if (!async) { - struct l_wait_info lwi; - int secs = obd_timeout * HZ; - CDEBUG(D_HA, "%s: recovery started, waiting %u seconds\n", - obd2cli_tgt(imp->imp_obd), secs); + obd2cli_tgt(imp->imp_obd), obd_timeout); - lwi = LWI_TIMEOUT(secs, NULL, NULL); - rc = l_wait_event(imp->imp_recovery_waitq, - !ptlrpc_import_in_recovery(imp), &lwi); + rc = wait_event_timeout(imp->imp_recovery_waitq, + !ptlrpc_import_in_recovery(imp), + obd_timeout * HZ); CDEBUG(D_HA, "%s: recovery finished\n", obd2cli_tgt(imp->imp_obd)); + rc = rc? 0 : -ETIMEDOUT; } out: diff --git a/drivers/staging/lustre/lustre/ptlrpc/sec_gc.c b/drivers/staging/lustre/lustre/ptlrpc/sec_gc.c index e4197a60d1e2..a10890b8324a 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/sec_gc.c +++ b/drivers/staging/lustre/lustre/ptlrpc/sec_gc.c @@ -142,7 +142,6 @@ static void sec_do_gc(struct ptlrpc_sec *sec) static int sec_gc_main(void *arg) { struct ptlrpc_thread *thread = arg; - struct l_wait_info lwi; unshare_fs_struct(); @@ -179,12 +178,9 @@ static int sec_gc_main(void *arg) /* check ctx list again before sleep */ sec_process_ctx_list(); - - lwi = LWI_TIMEOUT(msecs_to_jiffies(SEC_GC_INTERVAL * MSEC_PER_SEC), - NULL, NULL); - l_wait_event(thread->t_ctl_waitq, - thread_is_stopping(thread), - &lwi); + wait_event_noload_timeout(thread->t_ctl_waitq, + thread_is_stopping(thread), + SEC_GC_INTERVAL * HZ); if (thread_test_and_clear_flags(thread, SVC_STOPPING)) break; diff --git a/drivers/staging/lustre/lustre/ptlrpc/service.c b/drivers/staging/lustre/lustre/ptlrpc/service.c index 4a8a591e0067..c568baf8c28f 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/service.c +++ b/drivers/staging/lustre/lustre/ptlrpc/service.c @@ -2588,13 +2588,11 @@ static void ptlrpc_wait_replies(struct ptlrpc_service_part *svcpt) { while (1) { int rc; - struct l_wait_info lwi = LWI_TIMEOUT(10 * HZ, - NULL, NULL); - rc = l_wait_event(svcpt->scp_waitq, - atomic_read(&svcpt->scp_nreps_difficult) == 0, - &lwi); - if (rc == 0) + rc = wait_event_timeout(svcpt->scp_waitq, + atomic_read(&svcpt->scp_nreps_difficult) == 0, + 10 * HZ); + if (rc > 0) break; CWARN("Unexpectedly long timeout %s %p\n", svcpt->scp_service->srv_name, svcpt->scp_service); From neilb at suse.com Mon Dec 18 07:18:00 2017 From: neilb at suse.com (NeilBrown) Date: Mon, 18 Dec 2017 18:18:00 +1100 Subject: [lustre-devel] [PATCH 10/16] staging: lustre: remove back_to_sleep() and use loops. In-Reply-To: <151358127190.5099.12792810096274074963.stgit@noble> References: <151358127190.5099.12792810096274074963.stgit@noble> Message-ID: <151358148016.5099.17721203202254142965.stgit@noble> When 'back_to_sleep()' is passed as the 'timeout' function, the effect is to wait indefinitely for the event, polling on the timeout in case we missed a wake_up. If LWI_ON_SIGNAL_NOOP is given, then also abort (at the timeout) if a "fatal" signal is pending. Make this more obvious is both places "back_to_sleep()" is used, adding l_fatal_signal_pending() to allow testing for "fatal" signals. Signed-off-by: NeilBrown --- drivers/staging/lustre/lustre/include/lustre_lib.h | 8 ++++---- drivers/staging/lustre/lustre/ptlrpc/import.c | 11 ++++++----- drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c | 8 ++++---- 3 files changed, 14 insertions(+), 13 deletions(-) diff --git a/drivers/staging/lustre/lustre/include/lustre_lib.h b/drivers/staging/lustre/lustre/include/lustre_lib.h index d64306291b47..d157ed35a0bf 100644 --- a/drivers/staging/lustre/lustre/include/lustre_lib.h +++ b/drivers/staging/lustre/lustre/include/lustre_lib.h @@ -140,10 +140,6 @@ void target_send_reply(struct ptlrpc_request *req, int rc, int fail_id); * XXX nikita: some ptlrpc daemon threads have races of that sort. * */ -static inline int back_to_sleep(void *arg) -{ - return 0; -} #define LWI_ON_SIGNAL_NOOP ((void (*)(void *))(-1)) @@ -200,6 +196,10 @@ struct l_wait_info { #define LUSTRE_FATAL_SIGS (sigmask(SIGKILL) | sigmask(SIGINT) | \ sigmask(SIGTERM) | sigmask(SIGQUIT) | \ sigmask(SIGALRM)) +static inline int l_fatal_signal_pending(struct task_struct *p) +{ + return signal_pending(p) && sigtestsetmask(&p->pending.signal, LUSTRE_FATAL_SIGS); +} /** * wait_queue_entry_t of Linux (version < 2.6.34) is a FIFO list for exclusively diff --git a/drivers/staging/lustre/lustre/ptlrpc/import.c b/drivers/staging/lustre/lustre/ptlrpc/import.c index b6cef8e97435..decaa9bccdc8 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/import.c +++ b/drivers/staging/lustre/lustre/ptlrpc/import.c @@ -1496,7 +1496,6 @@ int ptlrpc_disconnect_import(struct obd_import *imp, int noclose) } if (ptlrpc_import_in_recovery(imp)) { - struct l_wait_info lwi; long timeout; if (AT_OFF) { @@ -1510,10 +1509,12 @@ int ptlrpc_disconnect_import(struct obd_import *imp, int noclose) timeout = at_get(&imp->imp_at.iat_service_estimate[idx]) * HZ; } - lwi = LWI_TIMEOUT_INTR(cfs_timeout_cap(timeout), - back_to_sleep, LWI_ON_SIGNAL_NOOP, NULL); - rc = l_wait_event(imp->imp_recovery_waitq, - !ptlrpc_import_in_recovery(imp), &lwi); + while (wait_event_timeout(imp->imp_recovery_waitq, + !ptlrpc_import_in_recovery(imp), + cfs_timeout_cap(timeout)) == 0) + if (l_fatal_signal_pending(current)) + break; + /* else just keep waiting */ } spin_lock(&imp->imp_lock); diff --git a/drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c b/drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c index dad2f9290f70..0bdf1f54629b 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c +++ b/drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c @@ -230,12 +230,12 @@ void ptlrpcd_add_req(struct ptlrpc_request *req) spin_lock(&req->rq_lock); if (req->rq_invalid_rqset) { - struct l_wait_info lwi = LWI_TIMEOUT(5 * HZ, - back_to_sleep, NULL); - req->rq_invalid_rqset = 0; spin_unlock(&req->rq_lock); - l_wait_event(req->rq_set_waitq, !req->rq_set, &lwi); + while (wait_event_timeout(req->rq_set_waitq, + !req->rq_set, + 5 * HZ) == 0) + ; } else if (req->rq_set) { /* If we have a valid "rq_set", just reuse it to avoid double * linked. From neilb at suse.com Mon Dec 18 07:18:00 2017 From: neilb at suse.com (NeilBrown) Date: Mon, 18 Dec 2017 18:18:00 +1100 Subject: [lustre-devel] [PATCH 12/16] staging: lustre: use wait_event_timeout in ptlrpcd() In-Reply-To: <151358127190.5099.12792810096274074963.stgit@noble> References: <151358127190.5099.12792810096274074963.stgit@noble> Message-ID: <151358148023.5099.3845154685635781293.stgit@noble> We can replace l_wait_event() with wait_event_noload_timeout() here providing we call the timeout function when wait_event_timeout() returns zero. Signed-off-by: NeilBrown --- drivers/staging/lustre/lustre/ptlrpc/client.c | 12 ++++++++---- .../staging/lustre/lustre/ptlrpc/ptlrpc_internal.h | 2 +- drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c | 9 +++++---- 3 files changed, 14 insertions(+), 9 deletions(-) diff --git a/drivers/staging/lustre/lustre/ptlrpc/client.c b/drivers/staging/lustre/lustre/ptlrpc/client.c index 81b7a7046d82..3e6d22beb9f5 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/client.c +++ b/drivers/staging/lustre/lustre/ptlrpc/client.c @@ -2125,9 +2125,8 @@ int ptlrpc_expire_one_request(struct ptlrpc_request *req, int async_unlink) * Callback used when waiting on sets with l_wait_event. * Always returns 1. */ -int ptlrpc_expired_set(void *data) +int ptlrpc_expired_set(struct ptlrpc_request_set *set) { - struct ptlrpc_request_set *set = data; struct list_head *tmp; time64_t now = ktime_get_real_seconds(); @@ -2164,6 +2163,11 @@ int ptlrpc_expired_set(void *data) */ return 1; } +static int ptlrpc_expired_set_void(void *data) +{ + struct ptlrpc_request_set *set = data; + return ptlrpc_expired_set(set); +} /** * Sets rq_intr flag in \a req under spinlock. @@ -2286,7 +2290,7 @@ int ptlrpc_set_wait(struct ptlrpc_request_set *set) * so we allow interrupts during the timeout. */ lwi = LWI_TIMEOUT_INTR_ALL(HZ, - ptlrpc_expired_set, + ptlrpc_expired_set_void, ptlrpc_interrupted_set, set); else /* @@ -2295,7 +2299,7 @@ int ptlrpc_set_wait(struct ptlrpc_request_set *set) * complete, or an in-flight req times out. */ lwi = LWI_TIMEOUT((timeout ? timeout : 1) * HZ, - ptlrpc_expired_set, set); + ptlrpc_expired_set_void, set); rc = l_wait_event(set->set_waitq, ptlrpc_check_set(NULL, set), &lwi); diff --git a/drivers/staging/lustre/lustre/ptlrpc/ptlrpc_internal.h b/drivers/staging/lustre/lustre/ptlrpc/ptlrpc_internal.h index f9decbd1459d..6dd52d99d454 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/ptlrpc_internal.h +++ b/drivers/staging/lustre/lustre/ptlrpc/ptlrpc_internal.h @@ -68,7 +68,7 @@ void ptlrpc_request_cache_free(struct ptlrpc_request *req); void ptlrpc_init_xid(void); void ptlrpc_set_add_new_req(struct ptlrpcd_ctl *pc, struct ptlrpc_request *req); -int ptlrpc_expired_set(void *data); +int ptlrpc_expired_set(struct ptlrpc_request_set *set); int ptlrpc_set_next_timeout(struct ptlrpc_request_set *set); void ptlrpc_resend_req(struct ptlrpc_request *request); void ptlrpc_set_bulk_mbits(struct ptlrpc_request *req); diff --git a/drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c b/drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c index 0bdf1f54629b..b7df38e71946 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c +++ b/drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c @@ -434,16 +434,17 @@ static int ptlrpcd(void *arg) * new_req_list and ptlrpcd_check() moves them into the set. */ do { - struct l_wait_info lwi; int timeout; timeout = ptlrpc_set_next_timeout(set); - lwi = LWI_TIMEOUT((timeout ? timeout : 1) * HZ, - ptlrpc_expired_set, set); lu_context_enter(&env.le_ctx); lu_context_enter(env.le_ses); - l_wait_event(set->set_waitq, ptlrpcd_check(&env, pc), &lwi); + if (wait_event_noload_timeout(set->set_waitq, + ptlrpcd_check(&env, pc), + (timeout ? timeout : 1) * HZ) == 0) + ptlrpc_expired_set(set); + lu_context_exit(&env.le_ctx); lu_context_exit(env.le_ses); From neilb at suse.com Mon Dec 18 07:18:00 2017 From: neilb at suse.com (NeilBrown) Date: Mon, 18 Dec 2017 18:18:00 +1100 Subject: [lustre-devel] [PATCH 09/16] staging: lustre: simplify waiting in ptlrpc_invalidate_import() In-Reply-To: <151358127190.5099.12792810096274074963.stgit@noble> References: <151358127190.5099.12792810096274074963.stgit@noble> Message-ID: <151358148012.5099.9791328135740800489.stgit@noble> This wait current wakes up every second to re-test if imp_flight is zero. If we ensure wakeup is called whenever imp_flight is decremented to zero, we can just have a simple wait_event_timeout(). So add a wake_up_all to the one place it is missing, and simplify the wait_event. Signed-off-by: NeilBrown --- drivers/staging/lustre/lustre/ptlrpc/client.c | 3 ++- drivers/staging/lustre/lustre/ptlrpc/import.c | 21 ++++++++------------- 2 files changed, 10 insertions(+), 14 deletions(-) diff --git a/drivers/staging/lustre/lustre/ptlrpc/client.c b/drivers/staging/lustre/lustre/ptlrpc/client.c index 0ab13f8e5993..81b7a7046d82 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/client.c +++ b/drivers/staging/lustre/lustre/ptlrpc/client.c @@ -1588,7 +1588,8 @@ static int ptlrpc_send_new_req(struct ptlrpc_request *req) spin_lock(&imp->imp_lock); if (!list_empty(&req->rq_list)) { list_del_init(&req->rq_list); - atomic_dec(&req->rq_import->imp_inflight); + if (atomic_dec_and_test(&req->rq_import->imp_inflight)) + wake_up_all(&req->rq_import->imp_recovery_waitq); } spin_unlock(&imp->imp_lock); ptlrpc_rqphase_move(req, RQ_PHASE_NEW); diff --git a/drivers/staging/lustre/lustre/ptlrpc/import.c b/drivers/staging/lustre/lustre/ptlrpc/import.c index 34b4075fac42..b6cef8e97435 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/import.c +++ b/drivers/staging/lustre/lustre/ptlrpc/import.c @@ -265,7 +265,6 @@ void ptlrpc_invalidate_import(struct obd_import *imp) { struct list_head *tmp, *n; struct ptlrpc_request *req; - struct l_wait_info lwi; unsigned int timeout; int rc; @@ -306,19 +305,15 @@ void ptlrpc_invalidate_import(struct obd_import *imp) * callbacks. Cap it at obd_timeout -- these should all * have been locally cancelled by ptlrpc_abort_inflight. */ - lwi = LWI_TIMEOUT_INTERVAL( - cfs_timeout_cap(timeout * HZ), - (timeout > 1) ? HZ : - HZ / 2, - NULL, NULL); - rc = l_wait_event(imp->imp_recovery_waitq, - (atomic_read(&imp->imp_inflight) == 0), - &lwi); - if (rc) { + rc = wait_event_timeout(imp->imp_recovery_waitq, + atomic_read(&imp->imp_inflight) == 0, + obd_timeout * HZ); + + if (rc == 0) { const char *cli_tgt = obd2cli_tgt(imp->imp_obd); - CERROR("%s: rc = %d waiting for callback (%d != 0)\n", - cli_tgt, rc, + CERROR("%s: timeout waiting for callback (%d != 0)\n", + cli_tgt, atomic_read(&imp->imp_inflight)); spin_lock(&imp->imp_lock); @@ -365,7 +360,7 @@ void ptlrpc_invalidate_import(struct obd_import *imp) } spin_unlock(&imp->imp_lock); } - } while (rc != 0); + } while (rc == 0); /* * Let's additionally check that no new rpcs added to import in From neilb at suse.com Mon Dec 18 07:18:00 2017 From: neilb at suse.com (NeilBrown) Date: Mon, 18 Dec 2017 18:18:00 +1100 Subject: [lustre-devel] [PATCH 11/16] staging: lustre: make polling loop in ptlrpc_unregister_bulk more obvious In-Reply-To: <151358127190.5099.12792810096274074963.stgit@noble> References: <151358127190.5099.12792810096274074963.stgit@noble> Message-ID: <151358148020.5099.4047116902359752959.stgit@noble> This use of l_wait_event() is a polling loop that re-checks every second. Make this more obvious with a while loop and wait_event_timeout(). Signed-off-by: NeilBrown --- drivers/staging/lustre/lustre/ptlrpc/niobuf.c | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/drivers/staging/lustre/lustre/ptlrpc/niobuf.c b/drivers/staging/lustre/lustre/ptlrpc/niobuf.c index 0c2ded721c49..5606c8f01b5b 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/niobuf.c +++ b/drivers/staging/lustre/lustre/ptlrpc/niobuf.c @@ -229,7 +229,6 @@ int ptlrpc_unregister_bulk(struct ptlrpc_request *req, int async) { struct ptlrpc_bulk_desc *desc = req->rq_bulk; wait_queue_head_t *wq; - struct l_wait_info lwi; int rc; LASSERT(!in_interrupt()); /* might sleep */ @@ -246,7 +245,7 @@ int ptlrpc_unregister_bulk(struct ptlrpc_request *req, int async) /* the unlink ensures the callback happens ASAP and is the last * one. If it fails, it must be because completion just happened, - * but we must still l_wait_event() in this case to give liblustre + * but we must still wait_event() in this case to give liblustre * a chance to run client_bulk_callback() */ mdunlink_iterate_helper(desc->bd_mds, desc->bd_md_max_brw); @@ -270,15 +269,16 @@ int ptlrpc_unregister_bulk(struct ptlrpc_request *req, int async) /* Network access will complete in finite time but the HUGE * timeout lets us CWARN for visibility of sluggish LNDs */ - lwi = LWI_TIMEOUT_INTERVAL(LONG_UNLINK * HZ, - HZ, NULL, NULL); - rc = l_wait_event(*wq, !ptlrpc_client_bulk_active(req), &lwi); - if (rc == 0) { + int cnt = 0; + while (cnt < LONG_UNLINK && + (rc = wait_event_timeout(*wq, !ptlrpc_client_bulk_active(req), + HZ)) == 0) + cnt += 1; + if (rc > 0) { ptlrpc_rqphase_move(req, req->rq_next_phase); return 1; } - LASSERT(rc == -ETIMEDOUT); DEBUG_REQ(D_WARNING, req, "Unexpectedly long timeout: desc %p", desc); } From neilb at suse.com Mon Dec 18 07:18:00 2017 From: neilb at suse.com (NeilBrown) Date: Mon, 18 Dec 2017 18:18:00 +1100 Subject: [lustre-devel] [PATCH 14/16] staging: lustre: use explicit poll loop in ptlrpc_service_unlink_rqbd In-Reply-To: <151358127190.5099.12792810096274074963.stgit@noble> References: <151358127190.5099.12792810096274074963.stgit@noble> Message-ID: <151358148030.5099.6919803999088109150.stgit@noble> Rather an using l_wait_event(), use wait_event_timeout() with an explicit loop so it is easier to see what is happening. Signed-off-by: NeilBrown --- drivers/staging/lustre/lustre/ptlrpc/service.c | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/drivers/staging/lustre/lustre/ptlrpc/service.c b/drivers/staging/lustre/lustre/ptlrpc/service.c index c568baf8c28f..8c77693510a1 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/service.c +++ b/drivers/staging/lustre/lustre/ptlrpc/service.c @@ -2617,7 +2617,7 @@ ptlrpc_service_unlink_rqbd(struct ptlrpc_service *svc) { struct ptlrpc_service_part *svcpt; struct ptlrpc_request_buffer_desc *rqbd; - struct l_wait_info lwi; + int cnt; int rc; int i; @@ -2657,12 +2657,13 @@ ptlrpc_service_unlink_rqbd(struct ptlrpc_service *svc) * the HUGE timeout lets us CWARN for visibility * of sluggish LNDs */ - lwi = LWI_TIMEOUT_INTERVAL( - LONG_UNLINK * HZ, - HZ, NULL, NULL); - rc = l_wait_event(svcpt->scp_waitq, - svcpt->scp_nrqbds_posted == 0, &lwi); - if (rc == -ETIMEDOUT) { + cnt = 0; + while (cnt < LONG_UNLINK && + (rc = wait_event_timeout(svcpt->scp_waitq, + svcpt->scp_nrqbds_posted == 0, + HZ)) == 0) + cnt ++; + if (rc == 0) { CWARN("Service %s waiting for request buffers\n", svcpt->scp_service->srv_name); } From neilb at suse.com Mon Dec 18 07:18:00 2017 From: neilb at suse.com (NeilBrown) Date: Mon, 18 Dec 2017 18:18:00 +1100 Subject: [lustre-devel] [PATCH 15/16] staging: lustre: use explicit poll loop in ptlrpc_unregister_reply In-Reply-To: <151358127190.5099.12792810096274074963.stgit@noble> References: <151358127190.5099.12792810096274074963.stgit@noble> Message-ID: <151358148034.5099.17386685003288433122.stgit@noble> replace l_wait_event() with wait_event_timeout() and explicit loop. This approach is easier to understand. Signed-off-by: NeilBrown --- drivers/staging/lustre/lustre/ptlrpc/client.c | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/drivers/staging/lustre/lustre/ptlrpc/client.c b/drivers/staging/lustre/lustre/ptlrpc/client.c index 3e6d22beb9f5..bb8c9ab68f5f 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/client.c +++ b/drivers/staging/lustre/lustre/ptlrpc/client.c @@ -2500,7 +2500,6 @@ static int ptlrpc_unregister_reply(struct ptlrpc_request *request, int async) { int rc; wait_queue_head_t *wq; - struct l_wait_info lwi; /* Might sleep. */ LASSERT(!in_interrupt()); @@ -2543,16 +2542,17 @@ static int ptlrpc_unregister_reply(struct ptlrpc_request *request, int async) * Network access will complete in finite time but the HUGE * timeout lets us CWARN for visibility of sluggish NALs */ - lwi = LWI_TIMEOUT_INTERVAL(LONG_UNLINK * HZ, - HZ, NULL, NULL); - rc = l_wait_event(*wq, !ptlrpc_client_recv_or_unlink(request), - &lwi); - if (rc == 0) { + int cnt = 0; + while (cnt < LONG_UNLINK && + (rc = wait_event_timeout(*wq, + !ptlrpc_client_recv_or_unlink(request), + HZ)) == 0) + cnt += 1; + if (rc > 0) { ptlrpc_rqphase_move(request, request->rq_next_phase); return 1; } - LASSERT(rc == -ETIMEDOUT); DEBUG_REQ(D_WARNING, request, "Unexpectedly long timeout receiving_reply=%d req_ulinked=%d reply_unlinked=%d", request->rq_receiving_reply, From neilb at suse.com Mon Dec 18 07:18:00 2017 From: neilb at suse.com (NeilBrown) Date: Mon, 18 Dec 2017 18:18:00 +1100 Subject: [lustre-devel] [PATCH 16/16] staging: lustre: remove l_wait_event from ptlrpc_set_wait In-Reply-To: <151358127190.5099.12792810096274074963.stgit@noble> References: <151358127190.5099.12792810096274074963.stgit@noble> Message-ID: <151358148037.5099.1272439971724416652.stgit@noble> This is the last remaining use of l_wait_event(). It is the only use of LWI_TIMEOUT_INTR_ALL() which has a meaning that timeouts can be interrupted. Only interrupts by "fatal" signals are allowed, so introduce l_wait_event_abortable_timeout() to support this. Signed-off-by: NeilBrown --- drivers/staging/lustre/lustre/include/lustre_lib.h | 10 ++ drivers/staging/lustre/lustre/ptlrpc/client.c | 86 ++++++++------------ .../staging/lustre/lustre/ptlrpc/ptlrpc_internal.h | 2 3 files changed, 47 insertions(+), 51 deletions(-) diff --git a/drivers/staging/lustre/lustre/include/lustre_lib.h b/drivers/staging/lustre/lustre/include/lustre_lib.h index d157ed35a0bf..b23d3015b1c0 100644 --- a/drivers/staging/lustre/lustre/include/lustre_lib.h +++ b/drivers/staging/lustre/lustre/include/lustre_lib.h @@ -440,6 +440,16 @@ do { \ __ret; \ }) +#define l_wait_event_abortable_timeout(wq, condition, timeout) \ +({ \ + sigset_t __blocked; \ + int __ret = 0; \ + __blocked = cfs_block_sigsinv(LUSTRE_FATAL_SIGS); \ + __ret = wait_event_interruptible_timeout(wq, condition, timeout);\ + cfs_restore_sigs(__blocked); \ + __ret; \ +}) + #define l_wait_event_abortable_exclusive(wq, condition) \ ({ \ sigset_t __blocked; \ diff --git a/drivers/staging/lustre/lustre/ptlrpc/client.c b/drivers/staging/lustre/lustre/ptlrpc/client.c index bb8c9ab68f5f..95131a34a98f 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/client.c +++ b/drivers/staging/lustre/lustre/ptlrpc/client.c @@ -1774,7 +1774,7 @@ int ptlrpc_check_set(const struct lu_env *env, struct ptlrpc_request_set *set) } /* - * ptlrpc_set_wait->l_wait_event sets lwi_allow_intr + * ptlrpc_set_wait allow signal to abort the timeout * so it sets rq_intr regardless of individual rpc * timeouts. The synchronous IO waiting path sets * rq_intr irrespective of whether ptlrpcd @@ -2122,10 +2122,9 @@ int ptlrpc_expire_one_request(struct ptlrpc_request *req, int async_unlink) /** * Time out all uncompleted requests in request set pointed by \a data - * Callback used when waiting on sets with l_wait_event. - * Always returns 1. + * Called when wait_event_timeout times out. */ -int ptlrpc_expired_set(struct ptlrpc_request_set *set) +void ptlrpc_expired_set(struct ptlrpc_request_set *set) { struct list_head *tmp; time64_t now = ktime_get_real_seconds(); @@ -2155,18 +2154,6 @@ int ptlrpc_expired_set(struct ptlrpc_request_set *set) */ ptlrpc_expire_one_request(req, 1); } - - /* - * When waiting for a whole set, we always break out of the - * sleep so we can recalculate the timeout, or enable interrupts - * if everyone's timed out. - */ - return 1; -} -static int ptlrpc_expired_set_void(void *data) -{ - struct ptlrpc_request_set *set = data; - return ptlrpc_expired_set(set); } /** @@ -2182,11 +2169,10 @@ EXPORT_SYMBOL(ptlrpc_mark_interrupted); /** * Interrupts (sets interrupted flag) all uncompleted requests in - * a set \a data. Callback for l_wait_event for interruptible waits. + * a set \a data. Called when wait_event_timeout receives signal. */ -static void ptlrpc_interrupted_set(void *data) +static void ptlrpc_interrupted_set(struct ptlrpc_request_set *set) { - struct ptlrpc_request_set *set = data; struct list_head *tmp; CDEBUG(D_RPCTRACE, "INTERRUPTED SET %p\n", set); @@ -2256,7 +2242,6 @@ int ptlrpc_set_wait(struct ptlrpc_request_set *set) { struct list_head *tmp; struct ptlrpc_request *req; - struct l_wait_info lwi; int rc, timeout; if (set->set_producer) @@ -2282,46 +2267,47 @@ int ptlrpc_set_wait(struct ptlrpc_request_set *set) CDEBUG(D_RPCTRACE, "set %p going to sleep for %d seconds\n", set, timeout); - if (timeout == 0 && !signal_pending(current)) + if (timeout == 0 && !signal_pending(current)) { /* * No requests are in-flight (ether timed out * or delayed), so we can allow interrupts. * We still want to block for a limited time, * so we allow interrupts during the timeout. */ - lwi = LWI_TIMEOUT_INTR_ALL(HZ, - ptlrpc_expired_set_void, - ptlrpc_interrupted_set, set); - else + rc = l_wait_event_abortable_timeout(set->set_waitq, + ptlrpc_check_set(NULL, set), + HZ); + if (rc == 0) { + rc = -ETIMEDOUT; + ptlrpc_expired_set(set); + } else if (rc < 0) { + rc = -EINTR; + ptlrpc_interrupted_set(set); + } else + rc = 0; + } else { /* * At least one request is in flight, so no * interrupts are allowed. Wait until all * complete, or an in-flight req times out. */ - lwi = LWI_TIMEOUT((timeout ? timeout : 1) * HZ, - ptlrpc_expired_set_void, set); - - rc = l_wait_event(set->set_waitq, ptlrpc_check_set(NULL, set), &lwi); - - /* - * LU-769 - if we ignored the signal because it was already - * pending when we started, we need to handle it now or we risk - * it being ignored forever - */ - if (rc == -ETIMEDOUT && !lwi.lwi_allow_intr && - signal_pending(current)) { - sigset_t blocked_sigs = - cfs_block_sigsinv(LUSTRE_FATAL_SIGS); - - /* - * In fact we only interrupt for the "fatal" signals - * like SIGINT or SIGKILL. We still ignore less - * important signals since ptlrpc set is not easily - * reentrant from userspace again - */ - if (signal_pending(current)) - ptlrpc_interrupted_set(set); - cfs_restore_sigs(blocked_sigs); + rc = wait_event_timeout(set->set_waitq, + ptlrpc_check_set(NULL, set), + (timeout ? timeout : 1) * HZ); + if (rc == 0) { + ptlrpc_expired_set(set); + rc = -ETIMEDOUT; + /* + * LU-769 - if we ignored the signal + * because it was already pending when + * we started, we need to handle it + * now or we risk it being ignored + * forever + */ + if (l_fatal_signal_pending(current)) + ptlrpc_interrupted_set(set); + } else + rc = 0; } LASSERT(rc == 0 || rc == -EINTR || rc == -ETIMEDOUT); @@ -2528,7 +2514,7 @@ static int ptlrpc_unregister_reply(struct ptlrpc_request *request, int async) return 0; /* - * We have to l_wait_event() whatever the result, to give liblustre + * We have to wait_event_timeout() whatever the result, to give liblustre * a chance to run reply_in_callback(), and to make sure we've * unlinked before returning a req to the pool. */ diff --git a/drivers/staging/lustre/lustre/ptlrpc/ptlrpc_internal.h b/drivers/staging/lustre/lustre/ptlrpc/ptlrpc_internal.h index 6dd52d99d454..b7a8d7537a66 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/ptlrpc_internal.h +++ b/drivers/staging/lustre/lustre/ptlrpc/ptlrpc_internal.h @@ -68,7 +68,7 @@ void ptlrpc_request_cache_free(struct ptlrpc_request *req); void ptlrpc_init_xid(void); void ptlrpc_set_add_new_req(struct ptlrpcd_ctl *pc, struct ptlrpc_request *req); -int ptlrpc_expired_set(struct ptlrpc_request_set *set); +void ptlrpc_expired_set(struct ptlrpc_request_set *set); int ptlrpc_set_next_timeout(struct ptlrpc_request_set *set); void ptlrpc_resend_req(struct ptlrpc_request *request); void ptlrpc_set_bulk_mbits(struct ptlrpc_request *req); From neilb at suse.com Mon Dec 18 07:18:00 2017 From: neilb at suse.com (NeilBrown) Date: Mon, 18 Dec 2017 18:18:00 +1100 Subject: [lustre-devel] [PATCH 13/16] staging: lustre: improve waiting in sptlrpc_req_refresh_ctx In-Reply-To: <151358127190.5099.12792810096274074963.stgit@noble> References: <151358127190.5099.12792810096274074963.stgit@noble> Message-ID: <151358148027.5099.4578208141131727886.stgit@noble> Replace l_wait_event with wait_event_timeout() and call the handler function explicitly. This makes it more clear what is happening. Signed-off-by: NeilBrown --- drivers/staging/lustre/lustre/ptlrpc/sec.c | 23 +++++++++++++---------- 1 file changed, 13 insertions(+), 10 deletions(-) diff --git a/drivers/staging/lustre/lustre/ptlrpc/sec.c b/drivers/staging/lustre/lustre/ptlrpc/sec.c index 617e004d00f8..6292840f1fca 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/sec.c +++ b/drivers/staging/lustre/lustre/ptlrpc/sec.c @@ -554,9 +554,8 @@ int ctx_check_refresh(struct ptlrpc_cli_ctx *ctx) } static -int ctx_refresh_timeout(void *data) +int ctx_refresh_timeout(struct ptlrpc_request *req) { - struct ptlrpc_request *req = data; int rc; /* conn_cnt is needed in expire_one_request */ @@ -575,10 +574,8 @@ int ctx_refresh_timeout(void *data) } static -void ctx_refresh_interrupt(void *data) +void ctx_refresh_interrupt(struct ptlrpc_request *req) { - struct ptlrpc_request *req = data; - spin_lock(&req->rq_lock); req->rq_intr = 1; spin_unlock(&req->rq_lock); @@ -611,7 +608,6 @@ int sptlrpc_req_refresh_ctx(struct ptlrpc_request *req, long timeout) { struct ptlrpc_cli_ctx *ctx = req->rq_cli_ctx; struct ptlrpc_sec *sec; - struct l_wait_info lwi; int rc; LASSERT(ctx); @@ -743,10 +739,17 @@ int sptlrpc_req_refresh_ctx(struct ptlrpc_request *req, long timeout) req->rq_restart = 0; spin_unlock(&req->rq_lock); - lwi = LWI_TIMEOUT_INTR(msecs_to_jiffies(timeout * MSEC_PER_SEC), - ctx_refresh_timeout, ctx_refresh_interrupt, - req); - rc = l_wait_event(req->rq_reply_waitq, ctx_check_refresh(ctx), &lwi); + rc = wait_event_timeout(req->rq_reply_waitq, + ctx_check_refresh(ctx), + timeout * HZ); + if (rc > 0) + rc = 0; + else if (ctx_refresh_timeout(req)) + rc = -ETIMEDOUT; + else if (l_fatal_signal_pending(current)) { + rc = -EINTR; + ctx_refresh_interrupt(req); + } /* * following cases could lead us here: From paf at cray.com Mon Dec 18 17:48:53 2017 From: paf at cray.com (Patrick Farrell) Date: Mon, 18 Dec 2017 17:48:53 +0000 Subject: [lustre-devel] [PATCH 02/16] staging: lustre: replace simple cases of l_wait_event() with wait_event(). In-Reply-To: <151358147981.5099.13114335078693829049.stgit@noble> References: <151358127190.5099.12792810096274074963.stgit@noble> <151358147981.5099.13114335078693829049.stgit@noble> Message-ID: Ah, finally we¹ve got that NOLOAD flag! This will clear up several nasty bugs around ptrace and sigkill that come when waiting with signals blocked in TASK_INTERRUPTIBLE. I see it was added in 2015Š The joys of working with vendor kernels. Thanks for these, Neil. I¹ll try to take a careful look. Given the description of the commit that added TASK_NOLOAD, I¹m kind of shocked to see it has almost no users in the kernel yet. Just a few callers of schedule_timeout_idle, it looks like. Even so, why not put those macros in sched.h to begin with? - Patrick On 12/18/17, 1:17 AM, "lustre-devel on behalf of NeilBrown" wrote: >When the lwi arg is full of zeros, l_wait_event() behaves almost >identically to the standard wait_event() interface, so use that >instead. > >The only difference in behavior is that l_wait_event() blocks all >signals and uses an TASK_INTERRUPTIBLE wait, while wait_event() >does not block signals, but waits in state TASK_UNINTERRUPTIBLE. >This means that processes blocked in wait_event() will contribute >to the load average. This behavior is (arguably) more correct - in >most cases. > >In some cases, the wait is in a service thread waiting for work to >do. In these case we should wait TASK_NOLOAD order with >TASK_UNINTERRUPTIBLE. To facilitate this, add a "wait_event_noload()" >macro. This should eventually be moved into include/linux/wait.h. > >There is one case where wait_event_exclusive_noload() is needed. >So we add a macro for that too. > >Signed-off-by: NeilBrown >--- > drivers/staging/lustre/lustre/include/lustre_lib.h | 47 >++++++++++++++++--- > drivers/staging/lustre/lustre/ldlm/ldlm_lock.c | 4 -- > drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c | 8 +-- > drivers/staging/lustre/lustre/ldlm/ldlm_pool.c | 5 +- > drivers/staging/lustre/lustre/llite/statahead.c | 50 >++++++++------------ > drivers/staging/lustre/lustre/lov/lov_object.c | 6 +- > drivers/staging/lustre/lustre/mgc/mgc_request.c | 4 -- > drivers/staging/lustre/lustre/obdclass/cl_io.c | 6 +- > drivers/staging/lustre/lustre/obdclass/genops.c | 15 ++---- > drivers/staging/lustre/lustre/osc/osc_cache.c | 5 +- > drivers/staging/lustre/lustre/osc/osc_object.c | 4 -- > drivers/staging/lustre/lustre/ptlrpc/pinger.c | 10 ++-- > drivers/staging/lustre/lustre/ptlrpc/sec_gc.c | 11 ++-- > drivers/staging/lustre/lustre/ptlrpc/service.c | 13 ++--- > 14 files changed, 93 insertions(+), 95 deletions(-) > >diff --git a/drivers/staging/lustre/lustre/include/lustre_lib.h >b/drivers/staging/lustre/lustre/include/lustre_lib.h >index ca1dce15337e..08bdd618ea7d 100644 >--- a/drivers/staging/lustre/lustre/include/lustre_lib.h >+++ b/drivers/staging/lustre/lustre/include/lustre_lib.h >@@ -333,12 +333,6 @@ do { \ > __ret; \ > }) > >-#define l_wait_condition(wq, condition) \ >-({ \ >- struct l_wait_info lwi = { 0 }; \ >- l_wait_event(wq, condition, &lwi); \ >-}) >- > #define l_wait_condition_exclusive(wq, condition) \ > ({ \ > struct l_wait_info lwi = { 0 }; \ >@@ -353,4 +347,45 @@ do { \ > > /** @} lib */ > >+#define __wait_event_noload(wq_head, condition) \ >+ (void)___wait_event(wq_head, condition, (TASK_UNINTERRUPTIBLE | >TASK_NOLOAD), 0, 0, \ >+ schedule()) >+ >+/** >+ * wait_event_noload - sleep, without registering load, until a >condition gets true >+ * @wq_head: the waitqueue to wait on >+ * @condition: a C expression for the event to wait for >+ * >+ * The process is put to sleep (TASK_UNINTERRUPTIBLE|TASK_NOLOAD) until >the >+ * @condition evaluates to true. The @condition is checked each time >+ * the waitqueue @wq_head is woken up. >+ * >+ * wake_up() has to be called after changing any variable that could >+ * change the result of the wait condition. >+ * >+ * This can be used instead of wait_event() when the event >+ * being waited for is does not imply load on the system, but >+ * when responding to signals is no appropriate, such as in >+ * a kernel service thread. >+ */ >+#define wait_event_noload(wq_head, condition) \ >+do { \ >+ might_sleep(); \ >+ if (condition) \ >+ break; \ >+ __wait_event_noload(wq_head, condition); \ >+} while (0) >+ >+/* >+ * Just like wait_event_noload(), except it sets exclusive flag >+ */ >+#define wait_event_exclusive_noload(wq_head, condition) \ >+do { \ >+ if (condition) \ >+ break; \ >+ (void)___wait_event(wq_head, condition, \ >+ (TASK_UNINTERRUPTIBLE | TASK_NOLOAD), 1, 0, \ >+ schedule()); \ >+} while (0) >+ > #endif /* _LUSTRE_LIB_H */ >diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c >b/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c >index 7cbc6a06afec..975fabc73148 100644 >--- a/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c >+++ b/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c >@@ -1913,14 +1913,12 @@ void ldlm_cancel_callback(struct ldlm_lock *lock) > ldlm_set_bl_done(lock); > wake_up_all(&lock->l_waitq); > } else if (!ldlm_is_bl_done(lock)) { >- struct l_wait_info lwi = { 0 }; >- > /* > * The lock is guaranteed to have been canceled once > * returning from this function. > */ > unlock_res_and_lock(lock); >- l_wait_event(lock->l_waitq, is_bl_done(lock), &lwi); >+ wait_event(lock->l_waitq, is_bl_done(lock)); > lock_res_and_lock(lock); > } > } >diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c >b/drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c >index 5f6e7c933b81..d9835418d340 100644 >--- a/drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c >+++ b/drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c >@@ -833,17 +833,15 @@ static int ldlm_bl_thread_main(void *arg) > /* cannot use bltd after this, it is only on caller's stack */ > > while (1) { >- struct l_wait_info lwi = { 0 }; > struct ldlm_bl_work_item *blwi = NULL; > struct obd_export *exp = NULL; > int rc; > > rc = ldlm_bl_get_work(blp, &blwi, &exp); > if (!rc) >- l_wait_event_exclusive(blp->blp_waitq, >- ldlm_bl_get_work(blp, &blwi, >- &exp), >- &lwi); >+ wait_event_exclusive_noload(blp->blp_waitq, >+ ldlm_bl_get_work(blp, &blwi, >+ &exp)); > atomic_inc(&blp->blp_busy_threads); > > if (ldlm_bl_thread_need_create(blp, blwi)) >diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c >b/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c >index 8563bd32befa..d562f90cee97 100644 >--- a/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c >+++ b/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c >@@ -1031,7 +1031,6 @@ static int ldlm_pools_thread_main(void *arg) > > static int ldlm_pools_thread_start(void) > { >- struct l_wait_info lwi = { 0 }; > struct task_struct *task; > > if (ldlm_pools_thread) >@@ -1052,8 +1051,8 @@ static int ldlm_pools_thread_start(void) > ldlm_pools_thread = NULL; > return PTR_ERR(task); > } >- l_wait_event(ldlm_pools_thread->t_ctl_waitq, >- thread_is_running(ldlm_pools_thread), &lwi); >+ wait_event(ldlm_pools_thread->t_ctl_waitq, >+ thread_is_running(ldlm_pools_thread)); > return 0; > } > >diff --git a/drivers/staging/lustre/lustre/llite/statahead.c >b/drivers/staging/lustre/lustre/llite/statahead.c >index 90c7324575e4..39040916a043 100644 >--- a/drivers/staging/lustre/lustre/llite/statahead.c >+++ b/drivers/staging/lustre/lustre/llite/statahead.c >@@ -864,7 +864,6 @@ static int ll_agl_thread(void *arg) > struct ll_sb_info *sbi = ll_i2sbi(dir); > struct ll_statahead_info *sai; > struct ptlrpc_thread *thread; >- struct l_wait_info lwi = { 0 }; > > sai = ll_sai_get(dir); > thread = &sai->sai_agl_thread; >@@ -885,10 +884,9 @@ static int ll_agl_thread(void *arg) > wake_up(&thread->t_ctl_waitq); > > while (1) { >- l_wait_event(thread->t_ctl_waitq, >- !list_empty(&sai->sai_agls) || >- !thread_is_running(thread), >- &lwi); >+ wait_event_noload(thread->t_ctl_waitq, >+ !list_empty(&sai->sai_agls) || >+ !thread_is_running(thread)); > > if (!thread_is_running(thread)) > break; >@@ -932,7 +930,6 @@ static int ll_agl_thread(void *arg) > static void ll_start_agl(struct dentry *parent, struct ll_statahead_info >*sai) > { > struct ptlrpc_thread *thread = &sai->sai_agl_thread; >- struct l_wait_info lwi = { 0 }; > struct ll_inode_info *plli; > struct task_struct *task; > >@@ -948,9 +945,8 @@ static void ll_start_agl(struct dentry *parent, >struct ll_statahead_info *sai) > return; > } > >- l_wait_event(thread->t_ctl_waitq, >- thread_is_running(thread) || thread_is_stopped(thread), >- &lwi); >+ wait_event(thread->t_ctl_waitq, >+ thread_is_running(thread) || thread_is_stopped(thread)); > } > > /* statahead thread main function */ >@@ -968,7 +964,6 @@ static int ll_statahead_thread(void *arg) > int first = 0; > int rc = 0; > struct md_op_data *op_data; >- struct l_wait_info lwi = { 0 }; > > sai = ll_sai_get(dir); > sa_thread = &sai->sai_thread; >@@ -1069,12 +1064,11 @@ static int ll_statahead_thread(void *arg) > > /* wait for spare statahead window */ > do { >- l_wait_event(sa_thread->t_ctl_waitq, >- !sa_sent_full(sai) || >- sa_has_callback(sai) || >- !list_empty(&sai->sai_agls) || >- !thread_is_running(sa_thread), >- &lwi); >+ wait_event(sa_thread->t_ctl_waitq, >+ !sa_sent_full(sai) || >+ sa_has_callback(sai) || >+ !list_empty(&sai->sai_agls) || >+ !thread_is_running(sa_thread)); > sa_handle_callback(sai); > > spin_lock(&lli->lli_agl_lock); >@@ -1128,11 +1122,10 @@ static int ll_statahead_thread(void *arg) > * for file release to stop me. > */ > while (thread_is_running(sa_thread)) { >- l_wait_event(sa_thread->t_ctl_waitq, >- sa_has_callback(sai) || >- !agl_list_empty(sai) || >- !thread_is_running(sa_thread), >- &lwi); >+ wait_event(sa_thread->t_ctl_waitq, >+ sa_has_callback(sai) || >+ !agl_list_empty(sai) || >+ !thread_is_running(sa_thread)); > > sa_handle_callback(sai); > } >@@ -1145,9 +1138,8 @@ static int ll_statahead_thread(void *arg) > > CDEBUG(D_READA, "stop agl thread: sai %p pid %u\n", > sai, (unsigned int)agl_thread->t_pid); >- l_wait_event(agl_thread->t_ctl_waitq, >- thread_is_stopped(agl_thread), >- &lwi); >+ wait_event(agl_thread->t_ctl_waitq, >+ thread_is_stopped(agl_thread)); > } else { > /* Set agl_thread flags anyway. */ > thread_set_flags(agl_thread, SVC_STOPPED); >@@ -1159,8 +1151,8 @@ static int ll_statahead_thread(void *arg) > */ > while (sai->sai_sent != sai->sai_replied) { > /* in case we're not woken up, timeout wait */ >- lwi = LWI_TIMEOUT(msecs_to_jiffies(MSEC_PER_SEC >> 3), >- NULL, NULL); >+ struct l_wait_info lwi = LWI_TIMEOUT(msecs_to_jiffies(MSEC_PER_SEC >> >3), >+ NULL, NULL); > l_wait_event(sa_thread->t_ctl_waitq, > sai->sai_sent == sai->sai_replied, &lwi); > } >@@ -1520,7 +1512,6 @@ static int start_statahead_thread(struct inode >*dir, struct dentry *dentry) > { > struct ll_inode_info *lli = ll_i2info(dir); > struct ll_statahead_info *sai = NULL; >- struct l_wait_info lwi = { 0 }; > struct ptlrpc_thread *thread; > struct task_struct *task; > struct dentry *parent = dentry->d_parent; >@@ -1570,9 +1561,8 @@ static int start_statahead_thread(struct inode >*dir, struct dentry *dentry) > goto out; > } > >- l_wait_event(thread->t_ctl_waitq, >- thread_is_running(thread) || thread_is_stopped(thread), >- &lwi); >+ wait_event(thread->t_ctl_waitq, >+ thread_is_running(thread) || thread_is_stopped(thread)); > ll_sai_put(sai); > > /* >diff --git a/drivers/staging/lustre/lustre/lov/lov_object.c >b/drivers/staging/lustre/lustre/lov/lov_object.c >index 897cf2cd4a24..aa82f2ed40ae 100644 >--- a/drivers/staging/lustre/lustre/lov/lov_object.c >+++ b/drivers/staging/lustre/lustre/lov/lov_object.c >@@ -723,15 +723,13 @@ static void lov_conf_unlock(struct lov_object *lov) > > static int lov_layout_wait(const struct lu_env *env, struct lov_object >*lov) > { >- struct l_wait_info lwi = { 0 }; >- > while (atomic_read(&lov->lo_active_ios) > 0) { > CDEBUG(D_INODE, "file:" DFID " wait for active IO, now: %d.\n", > PFID(lu_object_fid(lov2lu(lov))), > atomic_read(&lov->lo_active_ios)); > >- l_wait_event(lov->lo_waitq, >- atomic_read(&lov->lo_active_ios) == 0, &lwi); >+ wait_event(lov->lo_waitq, >+ atomic_read(&lov->lo_active_ios) == 0); > } > return 0; > } >diff --git a/drivers/staging/lustre/lustre/mgc/mgc_request.c >b/drivers/staging/lustre/lustre/mgc/mgc_request.c >index 79ff85feab64..81b101941eec 100644 >--- a/drivers/staging/lustre/lustre/mgc/mgc_request.c >+++ b/drivers/staging/lustre/lustre/mgc/mgc_request.c >@@ -601,9 +601,7 @@ static int mgc_requeue_thread(void *data) > config_log_put(cld_prev); > > /* Wait a bit to see if anyone else needs a requeue */ >- lwi = (struct l_wait_info) { 0 }; >- l_wait_event(rq_waitq, rq_state & (RQ_NOW | RQ_STOP), >- &lwi); >+ wait_event(rq_waitq, rq_state & (RQ_NOW | RQ_STOP)); > spin_lock(&config_list_lock); > } > >diff --git a/drivers/staging/lustre/lustre/obdclass/cl_io.c >b/drivers/staging/lustre/lustre/obdclass/cl_io.c >index 6ec5218a18c1..a3fb2bbde70f 100644 >--- a/drivers/staging/lustre/lustre/obdclass/cl_io.c >+++ b/drivers/staging/lustre/lustre/obdclass/cl_io.c >@@ -1110,10 +1110,8 @@ int cl_sync_io_wait(const struct lu_env *env, >struct cl_sync_io *anchor, > CERROR("IO failed: %d, still wait for %d remaining entries\n", > rc, atomic_read(&anchor->csi_sync_nr)); > >- lwi = (struct l_wait_info) { 0 }; >- (void)l_wait_event(anchor->csi_waitq, >- atomic_read(&anchor->csi_sync_nr) == 0, >- &lwi); >+ wait_event(anchor->csi_waitq, >+ atomic_read(&anchor->csi_sync_nr) == 0); > } else { > rc = anchor->csi_sync_rc; > } >diff --git a/drivers/staging/lustre/lustre/obdclass/genops.c >b/drivers/staging/lustre/lustre/obdclass/genops.c >index b1d6ba4a3190..78f0fa1dff45 100644 >--- a/drivers/staging/lustre/lustre/obdclass/genops.c >+++ b/drivers/staging/lustre/lustre/obdclass/genops.c >@@ -1237,12 +1237,10 @@ static int obd_zombie_is_idle(void) > */ > void obd_zombie_barrier(void) > { >- struct l_wait_info lwi = { 0 }; >- > if (obd_zombie_pid == current_pid()) > /* don't wait for myself */ > return; >- l_wait_event(obd_zombie_waitq, obd_zombie_is_idle(), &lwi); >+ wait_event(obd_zombie_waitq, obd_zombie_is_idle()); > } > EXPORT_SYMBOL(obd_zombie_barrier); > >@@ -1257,10 +1255,8 @@ static int obd_zombie_impexp_thread(void *unused) > obd_zombie_pid = current_pid(); > > while (!test_bit(OBD_ZOMBIE_STOP, &obd_zombie_flags)) { >- struct l_wait_info lwi = { 0 }; >- >- l_wait_event(obd_zombie_waitq, >- !obd_zombie_impexp_check(NULL), &lwi); >+ wait_event_noload(obd_zombie_waitq, >+ !obd_zombie_impexp_check(NULL)); > obd_zombie_impexp_cull(); > > /* >@@ -1593,7 +1589,6 @@ static inline bool obd_mod_rpc_slot_avail(struct >client_obd *cli, > u16 obd_get_mod_rpc_slot(struct client_obd *cli, __u32 opc, > struct lookup_intent *it) > { >- struct l_wait_info lwi = LWI_INTR(NULL, NULL); > bool close_req = false; > u16 i, max; > >@@ -1631,8 +1626,8 @@ u16 obd_get_mod_rpc_slot(struct client_obd *cli, >__u32 opc, > CDEBUG(D_RPCTRACE, "%s: sleeping for a modify RPC slot opc %u, max >%hu\n", > cli->cl_import->imp_obd->obd_name, opc, max); > >- l_wait_event(cli->cl_mod_rpcs_waitq, >- obd_mod_rpc_slot_avail(cli, close_req), &lwi); >+ wait_event(cli->cl_mod_rpcs_waitq, >+ obd_mod_rpc_slot_avail(cli, close_req)); > } while (true); > } > EXPORT_SYMBOL(obd_get_mod_rpc_slot); >diff --git a/drivers/staging/lustre/lustre/osc/osc_cache.c >b/drivers/staging/lustre/lustre/osc/osc_cache.c >index 5767ac2a7d16..d58a25a2a5b4 100644 >--- a/drivers/staging/lustre/lustre/osc/osc_cache.c >+++ b/drivers/staging/lustre/lustre/osc/osc_cache.c >@@ -964,9 +964,8 @@ static int osc_extent_wait(const struct lu_env *env, >struct osc_extent *ext, > "%s: wait ext to %u timedout, recovery in progress?\n", > cli_name(osc_cli(obj)), state); > >- lwi = LWI_INTR(NULL, NULL); >- rc = l_wait_event(ext->oe_waitq, extent_wait_cb(ext, state), >- &lwi); >+ wait_event(ext->oe_waitq, extent_wait_cb(ext, state)); >+ rc = 0; > } > if (rc == 0 && ext->oe_rc < 0) > rc = ext->oe_rc; >diff --git a/drivers/staging/lustre/lustre/osc/osc_object.c >b/drivers/staging/lustre/lustre/osc/osc_object.c >index f82c87a77550..1de25496a7d9 100644 >--- a/drivers/staging/lustre/lustre/osc/osc_object.c >+++ b/drivers/staging/lustre/lustre/osc/osc_object.c >@@ -454,12 +454,10 @@ struct lu_object *osc_object_alloc(const struct >lu_env *env, > > int osc_object_invalidate(const struct lu_env *env, struct osc_object >*osc) > { >- struct l_wait_info lwi = { 0 }; >- > CDEBUG(D_INODE, "Invalidate osc object: %p, # of active IOs: %d\n", > osc, atomic_read(&osc->oo_nr_ios)); > >- l_wait_event(osc->oo_io_waitq, !atomic_read(&osc->oo_nr_ios), &lwi); >+ wait_event(osc->oo_io_waitq, !atomic_read(&osc->oo_nr_ios)); > > /* Discard all dirty pages of this object. */ > osc_cache_truncate_start(env, osc, 0, NULL); >diff --git a/drivers/staging/lustre/lustre/ptlrpc/pinger.c >b/drivers/staging/lustre/lustre/ptlrpc/pinger.c >index fe6b47bfe8be..4148a6661dcf 100644 >--- a/drivers/staging/lustre/lustre/ptlrpc/pinger.c >+++ b/drivers/staging/lustre/lustre/ptlrpc/pinger.c >@@ -291,7 +291,6 @@ static struct ptlrpc_thread pinger_thread; > > int ptlrpc_start_pinger(void) > { >- struct l_wait_info lwi = { 0 }; > struct task_struct *task; > int rc; > >@@ -310,8 +309,8 @@ int ptlrpc_start_pinger(void) > CERROR("cannot start pinger thread: rc = %d\n", rc); > return rc; > } >- l_wait_event(pinger_thread.t_ctl_waitq, >- thread_is_running(&pinger_thread), &lwi); >+ wait_event(pinger_thread.t_ctl_waitq, >+ thread_is_running(&pinger_thread)); > > return 0; > } >@@ -320,7 +319,6 @@ static int ptlrpc_pinger_remove_timeouts(void); > > int ptlrpc_stop_pinger(void) > { >- struct l_wait_info lwi = { 0 }; > int rc = 0; > > if (thread_is_init(&pinger_thread) || >@@ -331,8 +329,8 @@ int ptlrpc_stop_pinger(void) > thread_set_flags(&pinger_thread, SVC_STOPPING); > wake_up(&pinger_thread.t_ctl_waitq); > >- l_wait_event(pinger_thread.t_ctl_waitq, >- thread_is_stopped(&pinger_thread), &lwi); >+ wait_event(pinger_thread.t_ctl_waitq, >+ thread_is_stopped(&pinger_thread)); > > return rc; > } >diff --git a/drivers/staging/lustre/lustre/ptlrpc/sec_gc.c >b/drivers/staging/lustre/lustre/ptlrpc/sec_gc.c >index d85c8638c009..e4197a60d1e2 100644 >--- a/drivers/staging/lustre/lustre/ptlrpc/sec_gc.c >+++ b/drivers/staging/lustre/lustre/ptlrpc/sec_gc.c >@@ -197,7 +197,6 @@ static int sec_gc_main(void *arg) > > int sptlrpc_gc_init(void) > { >- struct l_wait_info lwi = { 0 }; > struct task_struct *task; > > mutex_init(&sec_gc_mutex); >@@ -214,18 +213,16 @@ int sptlrpc_gc_init(void) > return PTR_ERR(task); > } > >- l_wait_event(sec_gc_thread.t_ctl_waitq, >- thread_is_running(&sec_gc_thread), &lwi); >+ wait_event(sec_gc_thread.t_ctl_waitq, >+ thread_is_running(&sec_gc_thread)); > return 0; > } > > void sptlrpc_gc_fini(void) > { >- struct l_wait_info lwi = { 0 }; >- > thread_set_flags(&sec_gc_thread, SVC_STOPPING); > wake_up(&sec_gc_thread.t_ctl_waitq); > >- l_wait_event(sec_gc_thread.t_ctl_waitq, >- thread_is_stopped(&sec_gc_thread), &lwi); >+ wait_event(sec_gc_thread.t_ctl_waitq, >+ thread_is_stopped(&sec_gc_thread)); > } >diff --git a/drivers/staging/lustre/lustre/ptlrpc/service.c >b/drivers/staging/lustre/lustre/ptlrpc/service.c >index 63be6e7273f3..d688cb3ff157 100644 >--- a/drivers/staging/lustre/lustre/ptlrpc/service.c >+++ b/drivers/staging/lustre/lustre/ptlrpc/service.c >@@ -2233,7 +2233,7 @@ static int ptlrpc_hr_main(void *arg) > wake_up(&ptlrpc_hr.hr_waitq); > > while (!ptlrpc_hr.hr_stopping) { >- l_wait_condition(hrt->hrt_waitq, hrt_dont_sleep(hrt, &replies)); >+ wait_event_noload(hrt->hrt_waitq, hrt_dont_sleep(hrt, &replies)); > > while (!list_empty(&replies)) { > struct ptlrpc_reply_state *rs; >@@ -2312,7 +2312,6 @@ static int ptlrpc_start_hr_threads(void) > > static void ptlrpc_svcpt_stop_threads(struct ptlrpc_service_part *svcpt) > { >- struct l_wait_info lwi = { 0 }; > struct ptlrpc_thread *thread; > LIST_HEAD(zombie); > >@@ -2341,8 +2340,8 @@ static void ptlrpc_svcpt_stop_threads(struct >ptlrpc_service_part *svcpt) > > CDEBUG(D_INFO, "waiting for stopping-thread %s #%u\n", > svcpt->scp_service->srv_thread_name, thread->t_id); >- l_wait_event(thread->t_ctl_waitq, >- thread_is_stopped(thread), &lwi); >+ wait_event(thread->t_ctl_waitq, >+ thread_is_stopped(thread)); > > spin_lock(&svcpt->scp_lock); > } >@@ -2403,7 +2402,6 @@ int ptlrpc_start_threads(struct ptlrpc_service *svc) > > int ptlrpc_start_thread(struct ptlrpc_service_part *svcpt, int wait) > { >- struct l_wait_info lwi = { 0 }; > struct ptlrpc_thread *thread; > struct ptlrpc_service *svc; > struct task_struct *task; >@@ -2499,9 +2497,8 @@ int ptlrpc_start_thread(struct ptlrpc_service_part >*svcpt, int wait) > if (!wait) > return 0; > >- l_wait_event(thread->t_ctl_waitq, >- thread_is_running(thread) || thread_is_stopped(thread), >- &lwi); >+ wait_event(thread->t_ctl_waitq, >+ thread_is_running(thread) || thread_is_stopped(thread)); > > rc = thread_is_stopped(thread) ? thread->t_id : 0; > return rc; > > >_______________________________________________ >lustre-devel mailing list >lustre-devel at lists.lustre.org >http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org From paf at cray.com Mon Dec 18 18:03:30 2017 From: paf at cray.com (Patrick Farrell) Date: Mon, 18 Dec 2017 18:03:30 +0000 Subject: [lustre-devel] [PATCH 02/16] staging: lustre: replace simple cases of l_wait_event() with wait_event(). In-Reply-To: <151358147981.5099.13114335078693829049.stgit@noble> References: <151358127190.5099.12792810096274074963.stgit@noble> <151358147981.5099.13114335078693829049.stgit@noble> Message-ID: The wait calls in ll_statahead_thread are done in a service thread, and should probably *not* contribute to load. The one in osc_extent_wait is perhaps tough - It is called both from user threads & daemon threads depending on the situation. The effect of adding that to load average could be significant for some activities, even when no user threads are busy. Thoughts from other Lustre people would be welcome here. Similar issues for osc_object_invalidate. (If no one else speaks up, my vote is no contribution to load for those OSC waits.) Otherwise this one looks good... On 12/18/17, 1:17 AM, "lustre-devel on behalf of NeilBrown" wrote: >@@ -968,7 +964,6 @@ static int ll_statahead_thread(void *arg) > int first = 0; > int rc = 0; > struct md_op_data *op_data; >- struct l_wait_info lwi = { 0 }; > sai = ll_sai_get(dir); > sa_thread = &sai->sai_thread; >@@ -1069,12 +1064,11 @@ static int ll_statahead_thread(void *arg) > /* wait for spare statahead window */ > do { >- l_wait_event(sa_thread->t_ctl_waitq, >- !sa_sent_full(sai) || >- sa_has_callback(sai) || >- !list_empty(&sai->sai_agls) || >- !thread_is_running(sa_thread), >- &lwi); >+ wait_event(sa_thread->t_ctl_waitq, >+ !sa_sent_full(sai) || >+ sa_has_callback(sai) || >+ !list_empty(&sai->sai_agls) || >+ !thread_is_running(sa_thread)); > sa_handle_callback(sai); > spin_lock(&lli->lli_agl_lock); >@@ -1128,11 +1122,10 @@ static int ll_statahead_thread(void *arg) > * for file release to stop me. > */ > while (thread_is_running(sa_thread)) { >- l_wait_event(sa_thread->t_ctl_waitq, >- sa_has_callback(sai) || >- !agl_list_empty(sai) || >- !thread_is_running(sa_thread), >- &lwi); >+ wait_event(sa_thread->t_ctl_waitq, >+ sa_has_callback(sai) || >+ !agl_list_empty(sai) || >+ !thread_is_running(sa_thread)); > sa_handle_callback(sai); > } >@@ -1145,9 +1138,8 @@ static int ll_statahead_thread(void *arg) > CDEBUG(D_READA, "stop agl thread: sai %p pid %u\n", > sai, (unsigned int)agl_thread->t_pid); >- l_wait_event(agl_thread->t_ctl_waitq, >- thread_is_stopped(agl_thread), >- &lwi); >+ wait_event(agl_thread->t_ctl_waitq, >+ thread_is_stopped(agl_thread)); > } else { > /* Set agl_thread flags anyway. */ > thread_set_flags(agl_thread, SVC_STOPPED); >@@ -1159,8 +1151,8 @@ static int ll_statahead_thread(void *arg) > */ > while (sai->sai_sent != sai->sai_replied) { > /* in case we're not woken up, timeout wait */ >- lwi = LWI_TIMEOUT(msecs_to_jiffies(MSEC_PER_SEC >> 3), >- NULL, NULL); >+ struct l_wait_info lwi = LWI_TIMEOUT(msecs_to_jiffies(MSEC_PER_SEC >> >3), >+ NULL, NULL); > l_wait_event(sa_thread->t_ctl_waitq, > sai->sai_sent == sai->sai_replied, &lwi); > } From paf at cray.com Mon Dec 18 20:23:40 2017 From: paf at cray.com (Patrick Farrell) Date: Mon, 18 Dec 2017 20:23:40 +0000 Subject: [lustre-devel] [PATCH 04/16] staging: lustre: use wait_event_timeout() where appropriate. In-Reply-To: <151358147993.5099.9887255633793256926.stgit@noble> References: <151358127190.5099.12792810096274074963.stgit@noble> <151358147993.5099.9887255633793256926.stgit@noble> Message-ID: Same thing as the other one, ll_statahead_thread should not contribute to load. IMO, mgc_process_log should not contribute to load in the case where it¹s waiting for an import to recover, that¹s likely to be a pretty long wait and doesn¹t really represent load. It¹s waiting for network recovery, basically. The OSC functions get the same question as the other ones - they happen from user threads and from daemons. Curious again what Andreas and/or Oleg think. I think ptlrpc_reconnect_import should probably *not* contribute to load, it¹s waiting for network recovery. Same with ptlrpc_recover_import. On 12/18/17, 1:17 AM, "lustre-devel on behalf of NeilBrown" wrote: >When the lwi arg has a timeout, but no timeout >callback function, l_wait_event() acts much the same as >wait_event_timeout() - the wait is not interruptible and >simply waits for the event or the timeouts. > >The most noticable difference is that the return value is >-ETIMEDOUT or 0, rather than 0 or non-zero. > >Another difference is that the process waiting is included in >the load-average. This is probably more correct. > >A final difference is that if the timeout is zero, l_wait_event() >will not time out at all. In the one case where that is possible >we need to conditionally use wait_event_noload(). > >So replace all such calls with wait_event_timeout(), being >careful of the return value. > >In one case, there is no event, so use >schedule_timeout_uninterruptible(). > > >Note that the presence or absence of LWI_ON_SIGNAL_NOOP >has no effect in these cases. > >Signed-off-by: NeilBrown >--- > drivers/staging/lustre/lustre/include/lustre_lib.h | 37 >++++++++++++++++++++ > drivers/staging/lustre/lustre/ldlm/ldlm_lock.c | 10 ++--- > drivers/staging/lustre/lustre/ldlm/ldlm_pool.c | 12 ++---- > drivers/staging/lustre/lustre/llite/statahead.c | 14 +++----- > drivers/staging/lustre/lustre/mdc/mdc_request.c | 5 +-- > drivers/staging/lustre/lustre/mgc/mgc_request.c | 15 +++----- > drivers/staging/lustre/lustre/obdclass/cl_io.c | 17 +++++---- > drivers/staging/lustre/lustre/osc/osc_cache.c | 25 ++++++-------- > drivers/staging/lustre/lustre/ptlrpc/events.c | 7 +--- > drivers/staging/lustre/lustre/ptlrpc/import.c | 12 +++--- > .../staging/lustre/lustre/ptlrpc/pack_generic.c | 9 ++--- > drivers/staging/lustre/lustre/ptlrpc/pinger.c | 12 ++---- > drivers/staging/lustre/lustre/ptlrpc/recover.c | 12 +++--- > drivers/staging/lustre/lustre/ptlrpc/sec_gc.c | 10 ++--- > drivers/staging/lustre/lustre/ptlrpc/service.c | 10 ++--- > 15 files changed, 104 insertions(+), 103 deletions(-) > >diff --git a/drivers/staging/lustre/lustre/include/lustre_lib.h >b/drivers/staging/lustre/lustre/include/lustre_lib.h >index 08bdd618ea7d..fcf31c779e98 100644 >--- a/drivers/staging/lustre/lustre/include/lustre_lib.h >+++ b/drivers/staging/lustre/lustre/include/lustre_lib.h >@@ -388,4 +388,41 @@ do { \ > schedule()); \ > } while (0) > >+#define __wait_event_noload_timeout(wq_head, condition, timeout) \ >+ ___wait_event(wq_head, ___wait_cond_timeout(condition), \ >+ (TASK_UNINTERRUPTIBLE | TASK_NOLOAD), 0, timeout, \ >+ __ret = schedule_timeout(__ret)) >+ >+/** >+ * wait_event_noload_timeout - sleep until a condition gets true or a >timeout elapses >+ * @wq_head: the waitqueue to wait on >+ * @condition: a C expression for the event to wait for >+ * @timeout: timeout, in jiffies >+ * >+ * The process is put to sleep (TASK_UNINTERRUPTIBLE | TASK_NOLOAD) >until the >+ * @condition evaluates to true. The @condition is checked each time >+ * the waitqueue @wq_head is woken up. >+ * >+ * wake_up() has to be called after changing any variable that could >+ * change the result of the wait condition. >+ * >+ * This is suitable for service threads that are waiting for work to do >+ * where there is no implication that the event not being true yet >implies >+ * any load on the system, and where it is not appropriate for the >+ * soft-lockup detector to warning if the wait is unusually long. >+ * >+ * Returns: >+ * 0 if the @condition evaluated to %false after the @timeout elapsed, >+ * 1 if the @condition evaluated to %true after the @timeout elapsed, >+ * or the remaining jiffies (at least 1) if the @condition evaluated >+ * to %true before the @timeout elapsed. >+ */ >+#define wait_event_noload_timeout(wq_head, condition, timeout) \ >+({ \ >+ long __ret = timeout; \ >+ might_sleep(); \ >+ if (!___wait_cond_timeout(condition)) \ >+ __ret = __wait_event_noload_timeout(wq_head, condition, timeout);\ >+ __ret; \ >+}) > #endif /* _LUSTRE_LIB_H */ >diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c >b/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c >index 53500883f243..ba720a888da3 100644 >--- a/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c >+++ b/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c >@@ -1349,7 +1349,6 @@ enum ldlm_mode ldlm_lock_match(struct >ldlm_namespace *ns, __u64 flags, > if ((flags & LDLM_FL_LVB_READY) && !ldlm_is_lvb_ready(lock)) { > __u64 wait_flags = LDLM_FL_LVB_READY | > LDLM_FL_DESTROYED | LDLM_FL_FAIL_NOTIFIED; >- struct l_wait_info lwi; > > if (lock->l_completion_ast) { > int err = lock->l_completion_ast(lock, >@@ -1366,13 +1365,10 @@ enum ldlm_mode ldlm_lock_match(struct >ldlm_namespace *ns, __u64 flags, > } > } > >- lwi = LWI_TIMEOUT_INTR(obd_timeout * HZ, >- NULL, LWI_ON_SIGNAL_NOOP, NULL); >- > /* XXX FIXME see comment on CAN_MATCH in lustre_dlm.h */ >- l_wait_event(lock->l_waitq, >- lock->l_flags & wait_flags, >- &lwi); >+ wait_event_timeout(lock->l_waitq, >+ lock->l_flags & wait_flags, >+ obd_timeout * HZ); > if (!ldlm_is_lvb_ready(lock)) { > if (flags & LDLM_FL_TEST_LOCK) > LDLM_LOCK_RELEASE(lock); >diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c >b/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c >index eb1f3d45c68c..c42b65173407 100644 >--- a/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c >+++ b/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c >@@ -997,8 +997,6 @@ static int ldlm_pools_thread_main(void *arg) > "ldlm_poold", current_pid()); > > while (1) { >- struct l_wait_info lwi; >- > /* > * Recal all pools on this tick. > */ >@@ -1008,12 +1006,10 @@ static int ldlm_pools_thread_main(void *arg) > * Wait until the next check time, or until we're > * stopped. > */ >- lwi = LWI_TIMEOUT(c_time * HZ, >- NULL, NULL); >- l_wait_event(thread->t_ctl_waitq, >- thread_is_stopping(thread) || >- thread_is_event(thread), >- &lwi); >+ wait_event_noload_timeout(thread->t_ctl_waitq, >+ thread_is_stopping(thread) || >+ thread_is_event(thread), >+ c_time * HZ); > > if (thread_test_and_clear_flags(thread, SVC_STOPPING)) > break; >diff --git a/drivers/staging/lustre/lustre/llite/statahead.c >b/drivers/staging/lustre/lustre/llite/statahead.c >index 9f9897523374..aecad3e6e808 100644 >--- a/drivers/staging/lustre/lustre/llite/statahead.c >+++ b/drivers/staging/lustre/lustre/llite/statahead.c >@@ -1151,10 +1151,9 @@ static int ll_statahead_thread(void *arg) > */ > while (sai->sai_sent != sai->sai_replied) { > /* in case we're not woken up, timeout wait */ >- struct l_wait_info lwi = LWI_TIMEOUT(msecs_to_jiffies(MSEC_PER_SEC >> >3), >- NULL, NULL); >- l_wait_event(sa_thread->t_ctl_waitq, >- sai->sai_sent == sai->sai_replied, &lwi); >+ wait_event_timeout(sa_thread->t_ctl_waitq, >+ sai->sai_sent == sai->sai_replied, >+ HZ>>3); > } > > /* release resources held by statahead RPCs */ >@@ -1374,7 +1373,6 @@ static int revalidate_statahead_dentry(struct inode >*dir, > { > struct ll_inode_info *lli = ll_i2info(dir); > struct sa_entry *entry = NULL; >- struct l_wait_info lwi = { 0 }; > struct ll_dentry_data *ldd; > int rc = 0; > >@@ -1424,10 +1422,8 @@ static int revalidate_statahead_dentry(struct >inode *dir, > spin_lock(&lli->lli_sa_lock); > sai->sai_index_wait = entry->se_index; > spin_unlock(&lli->lli_sa_lock); >- lwi = LWI_TIMEOUT_INTR(30 * HZ, NULL, >- LWI_ON_SIGNAL_NOOP, NULL); >- rc = l_wait_event(sai->sai_waitq, sa_ready(entry), &lwi); >- if (rc < 0) { >+ if (0 == wait_event_timeout(sai->sai_waitq, >+ sa_ready(entry), 30 * HZ)) { > /* > * entry may not be ready, so it may be used by inflight > * statahead RPC, don't free it. >diff --git a/drivers/staging/lustre/lustre/mdc/mdc_request.c >b/drivers/staging/lustre/lustre/mdc/mdc_request.c >index b12518ba5ae9..0409b4948c39 100644 >--- a/drivers/staging/lustre/lustre/mdc/mdc_request.c >+++ b/drivers/staging/lustre/lustre/mdc/mdc_request.c >@@ -838,7 +838,6 @@ static int mdc_getpage(struct obd_export *exp, const >struct lu_fid *fid, > struct ptlrpc_bulk_desc *desc; > struct ptlrpc_request *req; > wait_queue_head_t waitq; >- struct l_wait_info lwi; > int resends = 0; > int rc; > int i; >@@ -888,9 +887,7 @@ static int mdc_getpage(struct obd_export *exp, const >struct lu_fid *fid, > exp->exp_obd->obd_name, -EIO); > return -EIO; > } >- lwi = LWI_TIMEOUT_INTR(resends * HZ, NULL, NULL, >- NULL); >- l_wait_event(waitq, 0, &lwi); >+ wait_event_timeout(waitq, 0, resends * HZ); > > goto restart_bulk; > } >diff --git a/drivers/staging/lustre/lustre/mgc/mgc_request.c >b/drivers/staging/lustre/lustre/mgc/mgc_request.c >index 4c2e20ff95d7..b42ec5b4a24c 100644 >--- a/drivers/staging/lustre/lustre/mgc/mgc_request.c >+++ b/drivers/staging/lustre/lustre/mgc/mgc_request.c >@@ -535,7 +535,6 @@ static int mgc_requeue_thread(void *data) > spin_lock(&config_list_lock); > rq_state |= RQ_RUNNING; > while (!(rq_state & RQ_STOP)) { >- struct l_wait_info lwi; > struct config_llog_data *cld, *cld_prev; > int rand = prandom_u32_max(MGC_TIMEOUT_RAND_CENTISEC); > int to; >@@ -556,9 +555,9 @@ static int mgc_requeue_thread(void *data) > to = msecs_to_jiffies(MGC_TIMEOUT_MIN_SECONDS * MSEC_PER_SEC); > /* rand is centi-seconds */ > to += msecs_to_jiffies(rand * MSEC_PER_SEC / 100); >- lwi = LWI_TIMEOUT(to, NULL, NULL); >- l_wait_event(rq_waitq, rq_state & (RQ_STOP | RQ_PRECLEANUP), >- &lwi); >+ wait_event_noload_timeout(rq_waitq, >+ rq_state & (RQ_STOP | RQ_PRECLEANUP), >+ to); > > /* > * iterate & processing through the list. for each cld, process >@@ -1628,9 +1627,7 @@ int mgc_process_log(struct obd_device *mgc, struct >config_llog_data *cld) > > if (rcl == -ESHUTDOWN && > atomic_read(&mgc->u.cli.cl_mgc_refcount) > 0 && !retry) { >- int secs = obd_timeout * HZ; > struct obd_import *imp; >- struct l_wait_info lwi; > > mutex_unlock(&cld->cld_lock); > imp = class_exp2cliimp(mgc->u.cli.cl_mgc_mgsexp); >@@ -1645,9 +1642,9 @@ int mgc_process_log(struct obd_device *mgc, struct >config_llog_data *cld) > */ > ptlrpc_pinger_force(imp); > >- lwi = LWI_TIMEOUT(secs, NULL, NULL); >- l_wait_event(imp->imp_recovery_waitq, >- !mgc_import_in_recovery(imp), &lwi); >+ wait_event_timeout(imp->imp_recovery_waitq, >+ !mgc_import_in_recovery(imp), >+ obd_timeout * HZ); > > if (imp->imp_state == LUSTRE_IMP_FULL) { > retry = true; >diff --git a/drivers/staging/lustre/lustre/obdclass/cl_io.c >b/drivers/staging/lustre/lustre/obdclass/cl_io.c >index d8be01b9257f..5330962c1f66 100644 >--- a/drivers/staging/lustre/lustre/obdclass/cl_io.c >+++ b/drivers/staging/lustre/lustre/obdclass/cl_io.c >@@ -1097,16 +1097,19 @@ EXPORT_SYMBOL(cl_sync_io_init); > int cl_sync_io_wait(const struct lu_env *env, struct cl_sync_io *anchor, > long timeout) > { >- struct l_wait_info lwi = LWI_TIMEOUT_INTR(timeout * HZ, >- NULL, NULL, NULL); >- int rc; >+ int rc = 1; > > LASSERT(timeout >= 0); > >- rc = l_wait_event(anchor->csi_waitq, >- atomic_read(&anchor->csi_sync_nr) == 0, >- &lwi); >- if (rc < 0) { >+ if (timeout == 0) >+ wait_event_noload(anchor->csi_waitq, >+ atomic_read(&anchor->csi_sync_nr) == 0); >+ else >+ rc = wait_event_timeout(anchor->csi_waitq, >+ atomic_read(&anchor->csi_sync_nr) == 0, >+ timeout * HZ); >+ if (rc == 0) { >+ rc = -ETIMEDOUT; > CERROR("IO failed: %d, still wait for %d remaining entries\n", > rc, atomic_read(&anchor->csi_sync_nr)); > >diff --git a/drivers/staging/lustre/lustre/osc/osc_cache.c >b/drivers/staging/lustre/lustre/osc/osc_cache.c >index e6d99dc048ce..186cc1a0130a 100644 >--- a/drivers/staging/lustre/lustre/osc/osc_cache.c >+++ b/drivers/staging/lustre/lustre/osc/osc_cache.c >@@ -934,8 +934,6 @@ static int osc_extent_wait(const struct lu_env *env, >struct osc_extent *ext, > enum osc_extent_state state) > { > struct osc_object *obj = ext->oe_obj; >- struct l_wait_info lwi = LWI_TIMEOUT_INTR(600 * HZ, NULL, >- LWI_ON_SIGNAL_NOOP, NULL); > int rc = 0; > > osc_object_lock(obj); >@@ -958,17 +956,19 @@ static int osc_extent_wait(const struct lu_env >*env, struct osc_extent *ext, > osc_extent_release(env, ext); > > /* wait for the extent until its state becomes @state */ >- rc = l_wait_event(ext->oe_waitq, extent_wait_cb(ext, state), &lwi); >- if (rc == -ETIMEDOUT) { >+ rc = wait_event_timeout(ext->oe_waitq, >+ extent_wait_cb(ext, state), 600 * HZ); >+ if (rc == 0) { > OSC_EXTENT_DUMP(D_ERROR, ext, > "%s: wait ext to %u timedout, recovery in progress?\n", > cli_name(osc_cli(obj)), state); > > wait_event(ext->oe_waitq, extent_wait_cb(ext, state)); >- rc = 0; > } >- if (rc == 0 && ext->oe_rc < 0) >+ if (ext->oe_rc < 0) > rc = ext->oe_rc; >+ else >+ rc = 0; > return rc; > } > >@@ -1568,12 +1568,9 @@ static int osc_enter_cache(const struct lu_env >*env, struct client_obd *cli, > struct osc_object *osc = oap->oap_obj; > struct lov_oinfo *loi = osc->oo_oinfo; > struct osc_cache_waiter ocw; >- struct l_wait_info lwi; >+ unsigned long timeout = (AT_OFF ? obd_timeout : at_max) * HZ; > int rc = -EDQUOT; > >- lwi = LWI_TIMEOUT_INTR((AT_OFF ? obd_timeout : at_max) * HZ, >- NULL, LWI_ON_SIGNAL_NOOP, NULL); >- > OSC_DUMP_GRANT(D_CACHE, cli, "need:%d\n", bytes); > > spin_lock(&cli->cl_loi_list_lock); >@@ -1616,13 +1613,15 @@ static int osc_enter_cache(const struct lu_env >*env, struct client_obd *cli, > CDEBUG(D_CACHE, "%s: sleeping for cache space @ %p for %p\n", > cli_name(cli), &ocw, oap); > >- rc = l_wait_event(ocw.ocw_waitq, ocw_granted(cli, &ocw), &lwi); >+ rc = wait_event_timeout(ocw.ocw_waitq, >+ ocw_granted(cli, &ocw), timeout); > > spin_lock(&cli->cl_loi_list_lock); > >- if (rc < 0) { >- /* l_wait_event is interrupted by signal, or timed out */ >+ if (rc == 0) { >+ /* wait_event is interrupted by signal, or timed out */ > list_del_init(&ocw.ocw_entry); >+ rc = -ETIMEDOUT; > break; > } > LASSERT(list_empty(&ocw.ocw_entry)); >diff --git a/drivers/staging/lustre/lustre/ptlrpc/events.c >b/drivers/staging/lustre/lustre/ptlrpc/events.c >index 71f7588570ef..130bacc2c891 100644 >--- a/drivers/staging/lustre/lustre/ptlrpc/events.c >+++ b/drivers/staging/lustre/lustre/ptlrpc/events.c >@@ -490,8 +490,6 @@ int ptlrpc_uuid_to_peer(struct obd_uuid *uuid, > > static void ptlrpc_ni_fini(void) > { >- wait_queue_head_t waitq; >- struct l_wait_info lwi; > int rc; > int retries; > >@@ -515,10 +513,7 @@ static void ptlrpc_ni_fini(void) > if (retries != 0) > CWARN("Event queue still busy\n"); > >- /* Wait for a bit */ >- init_waitqueue_head(&waitq); >- lwi = LWI_TIMEOUT(2 * HZ, NULL, NULL); >- l_wait_event(waitq, 0, &lwi); >+ schedule_timeout_uninterruptible(2 * HZ); > break; > } > } >diff --git a/drivers/staging/lustre/lustre/ptlrpc/import.c >b/drivers/staging/lustre/lustre/ptlrpc/import.c >index 0eba5f18bd3b..34b4075fac42 100644 >--- a/drivers/staging/lustre/lustre/ptlrpc/import.c >+++ b/drivers/staging/lustre/lustre/ptlrpc/import.c >@@ -430,21 +430,19 @@ void ptlrpc_fail_import(struct obd_import *imp, >__u32 conn_cnt) > > int ptlrpc_reconnect_import(struct obd_import *imp) > { >- struct l_wait_info lwi; >- int secs = obd_timeout * HZ; > int rc; > > ptlrpc_pinger_force(imp); > > CDEBUG(D_HA, "%s: recovery started, waiting %u seconds\n", >- obd2cli_tgt(imp->imp_obd), secs); >+ obd2cli_tgt(imp->imp_obd), obd_timeout); > >- lwi = LWI_TIMEOUT(secs, NULL, NULL); >- rc = l_wait_event(imp->imp_recovery_waitq, >- !ptlrpc_import_in_recovery(imp), &lwi); >+ rc = wait_event_timeout(imp->imp_recovery_waitq, >+ !ptlrpc_import_in_recovery(imp), >+ obd_timeout * HZ); > CDEBUG(D_HA, "%s: recovery finished s:%s\n", obd2cli_tgt(imp->imp_obd), > ptlrpc_import_state_name(imp->imp_state)); >- return rc; >+ return rc == 0 ? -ETIMEDOUT : 0; > } > EXPORT_SYMBOL(ptlrpc_reconnect_import); > >diff --git a/drivers/staging/lustre/lustre/ptlrpc/pack_generic.c >b/drivers/staging/lustre/lustre/ptlrpc/pack_generic.c >index c060d6f5015a..6a1c3c041096 100644 >--- a/drivers/staging/lustre/lustre/ptlrpc/pack_generic.c >+++ b/drivers/staging/lustre/lustre/ptlrpc/pack_generic.c >@@ -260,17 +260,16 @@ lustre_get_emerg_rs(struct ptlrpc_service_part >*svcpt) > > /* See if we have anything in a pool, and wait if nothing */ > while (list_empty(&svcpt->scp_rep_idle)) { >- struct l_wait_info lwi; > int rc; > > spin_unlock(&svcpt->scp_rep_lock); > /* If we cannot get anything for some long time, we better > * bail out instead of waiting infinitely > */ >- lwi = LWI_TIMEOUT(10 * HZ, NULL, NULL); >- rc = l_wait_event(svcpt->scp_rep_waitq, >- !list_empty(&svcpt->scp_rep_idle), &lwi); >- if (rc != 0) >+ rc = wait_event_timeout(svcpt->scp_rep_waitq, >+ !list_empty(&svcpt->scp_rep_idle), >+ 10 * HZ); >+ if (rc == 0) > goto out; > spin_lock(&svcpt->scp_rep_lock); > } >diff --git a/drivers/staging/lustre/lustre/ptlrpc/pinger.c >b/drivers/staging/lustre/lustre/ptlrpc/pinger.c >index da3afda72e14..ed919507a50a 100644 >--- a/drivers/staging/lustre/lustre/ptlrpc/pinger.c >+++ b/drivers/staging/lustre/lustre/ptlrpc/pinger.c >@@ -228,7 +228,6 @@ static int ptlrpc_pinger_main(void *arg) > /* And now, loop forever, pinging as needed. */ > while (1) { > unsigned long this_ping = cfs_time_current(); >- struct l_wait_info lwi; > long time_to_next_wake; > struct timeout_item *item; > struct list_head *iter; >@@ -266,13 +265,10 @@ static int ptlrpc_pinger_main(void *arg) > cfs_time_add(this_ping, > PING_INTERVAL * HZ)); > if (time_to_next_wake > 0) { >- lwi = LWI_TIMEOUT(max_t(long, time_to_next_wake, >- HZ), >- NULL, NULL); >- l_wait_event(thread->t_ctl_waitq, >- thread_is_stopping(thread) || >- thread_is_event(thread), >- &lwi); >+ wait_event_noload_timeout(thread->t_ctl_waitq, >+ thread_is_stopping(thread) || >+ thread_is_event(thread), >+ max_t(long, time_to_next_wake, HZ)); > if (thread_test_and_clear_flags(thread, SVC_STOPPING)) > break; > /* woken after adding import to reset timer */ >diff --git a/drivers/staging/lustre/lustre/ptlrpc/recover.c >b/drivers/staging/lustre/lustre/ptlrpc/recover.c >index 5bbd23eebfa6..c8a7fca6d906 100644 >--- a/drivers/staging/lustre/lustre/ptlrpc/recover.c >+++ b/drivers/staging/lustre/lustre/ptlrpc/recover.c >@@ -346,17 +346,15 @@ int ptlrpc_recover_import(struct obd_import *imp, >char *new_uuid, int async) > goto out; > > if (!async) { >- struct l_wait_info lwi; >- int secs = obd_timeout * HZ; >- > CDEBUG(D_HA, "%s: recovery started, waiting %u seconds\n", >- obd2cli_tgt(imp->imp_obd), secs); >+ obd2cli_tgt(imp->imp_obd), obd_timeout); > >- lwi = LWI_TIMEOUT(secs, NULL, NULL); >- rc = l_wait_event(imp->imp_recovery_waitq, >- !ptlrpc_import_in_recovery(imp), &lwi); >+ rc = wait_event_timeout(imp->imp_recovery_waitq, >+ !ptlrpc_import_in_recovery(imp), >+ obd_timeout * HZ); > CDEBUG(D_HA, "%s: recovery finished\n", > obd2cli_tgt(imp->imp_obd)); >+ rc = rc? 0 : -ETIMEDOUT; > } > > out: >diff --git a/drivers/staging/lustre/lustre/ptlrpc/sec_gc.c >b/drivers/staging/lustre/lustre/ptlrpc/sec_gc.c >index e4197a60d1e2..a10890b8324a 100644 >--- a/drivers/staging/lustre/lustre/ptlrpc/sec_gc.c >+++ b/drivers/staging/lustre/lustre/ptlrpc/sec_gc.c >@@ -142,7 +142,6 @@ static void sec_do_gc(struct ptlrpc_sec *sec) > static int sec_gc_main(void *arg) > { > struct ptlrpc_thread *thread = arg; >- struct l_wait_info lwi; > > unshare_fs_struct(); > >@@ -179,12 +178,9 @@ static int sec_gc_main(void *arg) > > /* check ctx list again before sleep */ > sec_process_ctx_list(); >- >- lwi = LWI_TIMEOUT(msecs_to_jiffies(SEC_GC_INTERVAL * MSEC_PER_SEC), >- NULL, NULL); >- l_wait_event(thread->t_ctl_waitq, >- thread_is_stopping(thread), >- &lwi); >+ wait_event_noload_timeout(thread->t_ctl_waitq, >+ thread_is_stopping(thread), >+ SEC_GC_INTERVAL * HZ); > > if (thread_test_and_clear_flags(thread, SVC_STOPPING)) > break; >diff --git a/drivers/staging/lustre/lustre/ptlrpc/service.c >b/drivers/staging/lustre/lustre/ptlrpc/service.c >index 4a8a591e0067..c568baf8c28f 100644 >--- a/drivers/staging/lustre/lustre/ptlrpc/service.c >+++ b/drivers/staging/lustre/lustre/ptlrpc/service.c >@@ -2588,13 +2588,11 @@ static void ptlrpc_wait_replies(struct >ptlrpc_service_part *svcpt) > { > while (1) { > int rc; >- struct l_wait_info lwi = LWI_TIMEOUT(10 * HZ, >- NULL, NULL); > >- rc = l_wait_event(svcpt->scp_waitq, >- atomic_read(&svcpt->scp_nreps_difficult) == 0, >- &lwi); >- if (rc == 0) >+ rc = wait_event_timeout(svcpt->scp_waitq, >+ atomic_read(&svcpt->scp_nreps_difficult) == 0, >+ 10 * HZ); >+ if (rc > 0) > break; > CWARN("Unexpectedly long timeout %s %p\n", > svcpt->scp_service->srv_name, svcpt->scp_service); > > >_______________________________________________ >lustre-devel mailing list >lustre-devel at lists.lustre.org >http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org From paf at cray.com Mon Dec 18 20:55:10 2017 From: paf at cray.com (Patrick Farrell) Date: Mon, 18 Dec 2017 20:55:10 +0000 Subject: [lustre-devel] [PATCH 08/16] staging: lustre: open code polling loop instead of using l_wait_event() In-Reply-To: <151358148008.5099.7316878897181140635.stgit@noble> References: <151358127190.5099.12792810096274074963.stgit@noble> <151358148008.5099.7316878897181140635.stgit@noble> Message-ID: The lov_check_and_wait_active wait is usually (always?) going to be asynchronous from userspace and probably shouldn¹t contribute to load. So I guess that means schedule_timeout_idle. On 12/18/17, 1:18 AM, "lustre-devel on behalf of NeilBrown" wrote: >Two places that LWI_TIMEOUT_INTERVAL() is used, the outcome is a >simple polling loop that polls every second for some event (with a >limit). > >So write a simple loop to make this more apparent. > >Signed-off-by: NeilBrown >--- > drivers/staging/lustre/lustre/llite/llite_lib.c | 11 +++++------ > drivers/staging/lustre/lustre/lov/lov_request.c | 12 +++++------- > 2 files changed, 10 insertions(+), 13 deletions(-) > >diff --git a/drivers/staging/lustre/lustre/llite/llite_lib.c >b/drivers/staging/lustre/lustre/llite/llite_lib.c >index 33dc15e9aebb..f6642fa30428 100644 >--- a/drivers/staging/lustre/lustre/llite/llite_lib.c >+++ b/drivers/staging/lustre/lustre/llite/llite_lib.c >@@ -1984,8 +1984,7 @@ void ll_umount_begin(struct super_block *sb) > struct ll_sb_info *sbi = ll_s2sbi(sb); > struct obd_device *obd; > struct obd_ioctl_data *ioc_data; >- wait_queue_head_t waitq; >- struct l_wait_info lwi; >+ int cnt = 0; > > CDEBUG(D_VFSTRACE, "VFS Op: superblock %p count %d active %d\n", sb, > sb->s_count, atomic_read(&sb->s_active)); >@@ -2021,10 +2020,10 @@ void ll_umount_begin(struct super_block *sb) > * and then continue. For now, we just periodically checking for vfs > * to decrement mnt_cnt and hope to finish it within 10sec. > */ >- init_waitqueue_head(&waitq); >- lwi = LWI_TIMEOUT_INTERVAL(10 * HZ, >- HZ, NULL, NULL); >- l_wait_event(waitq, may_umount(sbi->ll_mnt.mnt), &lwi); >+ while (cnt < 10 && !may_umount(sbi->ll_mnt.mnt)) { >+ schedule_timeout_uninterruptible(HZ); >+ cnt ++; >+ } > > schedule(); > } >diff --git a/drivers/staging/lustre/lustre/lov/lov_request.c >b/drivers/staging/lustre/lustre/lov/lov_request.c >index fb3b7a7fa32a..c1e58fcc30b3 100644 >--- a/drivers/staging/lustre/lustre/lov/lov_request.c >+++ b/drivers/staging/lustre/lustre/lov/lov_request.c >@@ -99,8 +99,7 @@ static int lov_check_set(struct lov_obd *lov, int idx) > */ > static int lov_check_and_wait_active(struct lov_obd *lov, int ost_idx) > { >- wait_queue_head_t waitq; >- struct l_wait_info lwi; >+ int cnt = 0; > struct lov_tgt_desc *tgt; > int rc = 0; > >@@ -125,11 +124,10 @@ static int lov_check_and_wait_active(struct lov_obd >*lov, int ost_idx) > > mutex_unlock(&lov->lov_lock); > >- init_waitqueue_head(&waitq); >- lwi = LWI_TIMEOUT_INTERVAL(obd_timeout * HZ, >- HZ, NULL, NULL); >- >- rc = l_wait_event(waitq, lov_check_set(lov, ost_idx), &lwi); >+ while (cnt < obd_timeout && !lov_check_set(lov, ost_idx)) { >+ schedule_timeout_uninterruptible(HZ); >+ cnt ++; >+ } > if (tgt->ltd_active) > return 1; > > > >_______________________________________________ >lustre-devel mailing list >lustre-devel at lists.lustre.org >http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org From paf at cray.com Mon Dec 18 21:03:54 2017 From: paf at cray.com (Patrick Farrell) Date: Mon, 18 Dec 2017 21:03:54 +0000 Subject: [lustre-devel] [PATCH 11/16] staging: lustre: make polling loop in ptlrpc_unregister_bulk more obvious In-Reply-To: <151358148020.5099.4047116902359752959.stgit@noble> References: <151358127190.5099.12792810096274074963.stgit@noble> <151358148020.5099.4047116902359752959.stgit@noble> Message-ID: This probably shouldn¹t contribute to load, it¹s often (mostly?) run out of the ptlrpcd daemons. - Patrick On 12/18/17, 1:18 AM, "lustre-devel on behalf of NeilBrown" wrote: >This use of l_wait_event() is a polling loop that re-checks >every second. Make this more obvious with a while loop >and wait_event_timeout(). > >Signed-off-by: NeilBrown >--- > drivers/staging/lustre/lustre/ptlrpc/niobuf.c | 14 +++++++------- > 1 file changed, 7 insertions(+), 7 deletions(-) > >diff --git a/drivers/staging/lustre/lustre/ptlrpc/niobuf.c >b/drivers/staging/lustre/lustre/ptlrpc/niobuf.c >index 0c2ded721c49..5606c8f01b5b 100644 >--- a/drivers/staging/lustre/lustre/ptlrpc/niobuf.c >+++ b/drivers/staging/lustre/lustre/ptlrpc/niobuf.c >@@ -229,7 +229,6 @@ int ptlrpc_unregister_bulk(struct ptlrpc_request >*req, int async) > { > struct ptlrpc_bulk_desc *desc = req->rq_bulk; > wait_queue_head_t *wq; >- struct l_wait_info lwi; > int rc; > > LASSERT(!in_interrupt()); /* might sleep */ >@@ -246,7 +245,7 @@ int ptlrpc_unregister_bulk(struct ptlrpc_request >*req, int async) > > /* the unlink ensures the callback happens ASAP and is the last > * one. If it fails, it must be because completion just happened, >- * but we must still l_wait_event() in this case to give liblustre >+ * but we must still wait_event() in this case to give liblustre > * a chance to run client_bulk_callback() > */ > mdunlink_iterate_helper(desc->bd_mds, desc->bd_md_max_brw); >@@ -270,15 +269,16 @@ int ptlrpc_unregister_bulk(struct ptlrpc_request >*req, int async) > /* Network access will complete in finite time but the HUGE > * timeout lets us CWARN for visibility of sluggish LNDs > */ >- lwi = LWI_TIMEOUT_INTERVAL(LONG_UNLINK * HZ, >- HZ, NULL, NULL); >- rc = l_wait_event(*wq, !ptlrpc_client_bulk_active(req), &lwi); >- if (rc == 0) { >+ int cnt = 0; >+ while (cnt < LONG_UNLINK && >+ (rc = wait_event_timeout(*wq, !ptlrpc_client_bulk_active(req), >+ HZ)) == 0) >+ cnt += 1; >+ if (rc > 0) { > ptlrpc_rqphase_move(req, req->rq_next_phase); > return 1; > } > >- LASSERT(rc == -ETIMEDOUT); > DEBUG_REQ(D_WARNING, req, "Unexpectedly long timeout: desc %p", > desc); > } > > >_______________________________________________ >lustre-devel mailing list >lustre-devel at lists.lustre.org >http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org From paf at cray.com Mon Dec 18 21:09:18 2017 From: paf at cray.com (Patrick Farrell) Date: Mon, 18 Dec 2017 21:09:18 +0000 Subject: [lustre-devel] [PATCH 15/16] staging: lustre: use explicit poll loop in ptlrpc_unregister_reply In-Reply-To: <151358148034.5099.17386685003288433122.stgit@noble> References: <151358127190.5099.12792810096274074963.stgit@noble> <151358148034.5099.17386685003288433122.stgit@noble> Message-ID: This should not contribute to load, since it¹s called out of the ptlrpcd daemons. On 12/18/17, 1:18 AM, "lustre-devel on behalf of NeilBrown" wrote: >replace l_wait_event() with wait_event_timeout() and explicit >loop. This approach is easier to understand. > >Signed-off-by: NeilBrown >--- > drivers/staging/lustre/lustre/ptlrpc/client.c | 14 +++++++------- > 1 file changed, 7 insertions(+), 7 deletions(-) > >diff --git a/drivers/staging/lustre/lustre/ptlrpc/client.c >b/drivers/staging/lustre/lustre/ptlrpc/client.c >index 3e6d22beb9f5..bb8c9ab68f5f 100644 >--- a/drivers/staging/lustre/lustre/ptlrpc/client.c >+++ b/drivers/staging/lustre/lustre/ptlrpc/client.c >@@ -2500,7 +2500,6 @@ static int ptlrpc_unregister_reply(struct >ptlrpc_request *request, int async) > { > int rc; > wait_queue_head_t *wq; >- struct l_wait_info lwi; > > /* Might sleep. */ > LASSERT(!in_interrupt()); >@@ -2543,16 +2542,17 @@ static int ptlrpc_unregister_reply(struct >ptlrpc_request *request, int async) > * Network access will complete in finite time but the HUGE > * timeout lets us CWARN for visibility of sluggish NALs > */ >- lwi = LWI_TIMEOUT_INTERVAL(LONG_UNLINK * HZ, >- HZ, NULL, NULL); >- rc = l_wait_event(*wq, !ptlrpc_client_recv_or_unlink(request), >- &lwi); >- if (rc == 0) { >+ int cnt = 0; >+ while (cnt < LONG_UNLINK && >+ (rc = wait_event_timeout(*wq, >+ !ptlrpc_client_recv_or_unlink(request), >+ HZ)) == 0) >+ cnt += 1; >+ if (rc > 0) { > ptlrpc_rqphase_move(request, request->rq_next_phase); > return 1; > } > >- LASSERT(rc == -ETIMEDOUT); > DEBUG_REQ(D_WARNING, request, > "Unexpectedly long timeout receiving_reply=%d req_ulinked=%d >reply_unlinked=%d", > request->rq_receiving_reply, > > >_______________________________________________ >lustre-devel mailing list >lustre-devel at lists.lustre.org >http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org From neilb at suse.com Mon Dec 18 21:37:19 2017 From: neilb at suse.com (NeilBrown) Date: Tue, 19 Dec 2017 08:37:19 +1100 Subject: [lustre-devel] [PATCH 02/16] staging: lustre: replace simple cases of l_wait_event() with wait_event(). In-Reply-To: References: <151358127190.5099.12792810096274074963.stgit@noble> <151358147981.5099.13114335078693829049.stgit@noble> Message-ID: <87vah3lnu8.fsf@notabene.neil.brown.name> On Mon, Dec 18 2017, Patrick Farrell wrote: > Ah, finally we¹ve got that NOLOAD flag! This will clear up several nasty > bugs around ptrace and sigkill that come when waiting with signals blocked > in TASK_INTERRUPTIBLE. I see it was added in 2015Š The joys of working > with vendor kernels. I does take a while for these things to filter through. Maybe I should examine all places that call flush_signals() and see which ones really want to use TASK_IDLE (which is what I should have used, instead of TASK_UNINTERRUPTIBLE | TASK_NOLOAD). > > Thanks for these, Neil. I¹ll try to take a careful look. > > Given the description of the commit that added TASK_NOLOAD, I¹m kind of > shocked to see it has almost no users in the kernel yet. Just a few > callers of schedule_timeout_idle, it looks like. Even so, why not put > those macros in sched.h to begin with? Always hard to know how best to handle patch that affect multiple subsystems. Once I figure out what to do with l_wait_event_exclusive_head(), I'll probably send any patches related to that together with the wait_noload changes... Though maybe I need to use swait_event_idle(). Time to learn about "simple wait queues" I guess. Thanks, NeilBrown > > - Patrick > > On 12/18/17, 1:17 AM, "lustre-devel on behalf of NeilBrown" > wrote: > >>When the lwi arg is full of zeros, l_wait_event() behaves almost >>identically to the standard wait_event() interface, so use that >>instead. >> >>The only difference in behavior is that l_wait_event() blocks all >>signals and uses an TASK_INTERRUPTIBLE wait, while wait_event() >>does not block signals, but waits in state TASK_UNINTERRUPTIBLE. >>This means that processes blocked in wait_event() will contribute >>to the load average. This behavior is (arguably) more correct - in >>most cases. >> >>In some cases, the wait is in a service thread waiting for work to >>do. In these case we should wait TASK_NOLOAD order with >>TASK_UNINTERRUPTIBLE. To facilitate this, add a "wait_event_noload()" >>macro. This should eventually be moved into include/linux/wait.h. >> >>There is one case where wait_event_exclusive_noload() is needed. >>So we add a macro for that too. >> >>Signed-off-by: NeilBrown >>--- >> drivers/staging/lustre/lustre/include/lustre_lib.h | 47 >>++++++++++++++++--- >> drivers/staging/lustre/lustre/ldlm/ldlm_lock.c | 4 -- >> drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c | 8 +-- >> drivers/staging/lustre/lustre/ldlm/ldlm_pool.c | 5 +- >> drivers/staging/lustre/lustre/llite/statahead.c | 50 >>++++++++------------ >> drivers/staging/lustre/lustre/lov/lov_object.c | 6 +- >> drivers/staging/lustre/lustre/mgc/mgc_request.c | 4 -- >> drivers/staging/lustre/lustre/obdclass/cl_io.c | 6 +- >> drivers/staging/lustre/lustre/obdclass/genops.c | 15 ++---- >> drivers/staging/lustre/lustre/osc/osc_cache.c | 5 +- >> drivers/staging/lustre/lustre/osc/osc_object.c | 4 -- >> drivers/staging/lustre/lustre/ptlrpc/pinger.c | 10 ++-- >> drivers/staging/lustre/lustre/ptlrpc/sec_gc.c | 11 ++-- >> drivers/staging/lustre/lustre/ptlrpc/service.c | 13 ++--- >> 14 files changed, 93 insertions(+), 95 deletions(-) >> >>diff --git a/drivers/staging/lustre/lustre/include/lustre_lib.h >>b/drivers/staging/lustre/lustre/include/lustre_lib.h >>index ca1dce15337e..08bdd618ea7d 100644 >>--- a/drivers/staging/lustre/lustre/include/lustre_lib.h >>+++ b/drivers/staging/lustre/lustre/include/lustre_lib.h >>@@ -333,12 +333,6 @@ do { \ >> __ret; \ >> }) >> >>-#define l_wait_condition(wq, condition) \ >>-({ \ >>- struct l_wait_info lwi = { 0 }; \ >>- l_wait_event(wq, condition, &lwi); \ >>-}) >>- >> #define l_wait_condition_exclusive(wq, condition) \ >> ({ \ >> struct l_wait_info lwi = { 0 }; \ >>@@ -353,4 +347,45 @@ do { \ >> >> /** @} lib */ >> >>+#define __wait_event_noload(wq_head, condition) \ >>+ (void)___wait_event(wq_head, condition, (TASK_UNINTERRUPTIBLE | >>TASK_NOLOAD), 0, 0, \ >>+ schedule()) >>+ >>+/** >>+ * wait_event_noload - sleep, without registering load, until a >>condition gets true >>+ * @wq_head: the waitqueue to wait on >>+ * @condition: a C expression for the event to wait for >>+ * >>+ * The process is put to sleep (TASK_UNINTERRUPTIBLE|TASK_NOLOAD) until >>the >>+ * @condition evaluates to true. The @condition is checked each time >>+ * the waitqueue @wq_head is woken up. >>+ * >>+ * wake_up() has to be called after changing any variable that could >>+ * change the result of the wait condition. >>+ * >>+ * This can be used instead of wait_event() when the event >>+ * being waited for is does not imply load on the system, but >>+ * when responding to signals is no appropriate, such as in >>+ * a kernel service thread. >>+ */ >>+#define wait_event_noload(wq_head, condition) \ >>+do { \ >>+ might_sleep(); \ >>+ if (condition) \ >>+ break; \ >>+ __wait_event_noload(wq_head, condition); \ >>+} while (0) >>+ >>+/* >>+ * Just like wait_event_noload(), except it sets exclusive flag >>+ */ >>+#define wait_event_exclusive_noload(wq_head, condition) \ >>+do { \ >>+ if (condition) \ >>+ break; \ >>+ (void)___wait_event(wq_head, condition, \ >>+ (TASK_UNINTERRUPTIBLE | TASK_NOLOAD), 1, 0, \ >>+ schedule()); \ >>+} while (0) >>+ >> #endif /* _LUSTRE_LIB_H */ >>diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c >>b/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c >>index 7cbc6a06afec..975fabc73148 100644 >>--- a/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c >>+++ b/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c >>@@ -1913,14 +1913,12 @@ void ldlm_cancel_callback(struct ldlm_lock *lock) >> ldlm_set_bl_done(lock); >> wake_up_all(&lock->l_waitq); >> } else if (!ldlm_is_bl_done(lock)) { >>- struct l_wait_info lwi = { 0 }; >>- >> /* >> * The lock is guaranteed to have been canceled once >> * returning from this function. >> */ >> unlock_res_and_lock(lock); >>- l_wait_event(lock->l_waitq, is_bl_done(lock), &lwi); >>+ wait_event(lock->l_waitq, is_bl_done(lock)); >> lock_res_and_lock(lock); >> } >> } >>diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c >>b/drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c >>index 5f6e7c933b81..d9835418d340 100644 >>--- a/drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c >>+++ b/drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c >>@@ -833,17 +833,15 @@ static int ldlm_bl_thread_main(void *arg) >> /* cannot use bltd after this, it is only on caller's stack */ >> >> while (1) { >>- struct l_wait_info lwi = { 0 }; >> struct ldlm_bl_work_item *blwi = NULL; >> struct obd_export *exp = NULL; >> int rc; >> >> rc = ldlm_bl_get_work(blp, &blwi, &exp); >> if (!rc) >>- l_wait_event_exclusive(blp->blp_waitq, >>- ldlm_bl_get_work(blp, &blwi, >>- &exp), >>- &lwi); >>+ wait_event_exclusive_noload(blp->blp_waitq, >>+ ldlm_bl_get_work(blp, &blwi, >>+ &exp)); >> atomic_inc(&blp->blp_busy_threads); >> >> if (ldlm_bl_thread_need_create(blp, blwi)) >>diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c >>b/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c >>index 8563bd32befa..d562f90cee97 100644 >>--- a/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c >>+++ b/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c >>@@ -1031,7 +1031,6 @@ static int ldlm_pools_thread_main(void *arg) >> >> static int ldlm_pools_thread_start(void) >> { >>- struct l_wait_info lwi = { 0 }; >> struct task_struct *task; >> >> if (ldlm_pools_thread) >>@@ -1052,8 +1051,8 @@ static int ldlm_pools_thread_start(void) >> ldlm_pools_thread = NULL; >> return PTR_ERR(task); >> } >>- l_wait_event(ldlm_pools_thread->t_ctl_waitq, >>- thread_is_running(ldlm_pools_thread), &lwi); >>+ wait_event(ldlm_pools_thread->t_ctl_waitq, >>+ thread_is_running(ldlm_pools_thread)); >> return 0; >> } >> >>diff --git a/drivers/staging/lustre/lustre/llite/statahead.c >>b/drivers/staging/lustre/lustre/llite/statahead.c >>index 90c7324575e4..39040916a043 100644 >>--- a/drivers/staging/lustre/lustre/llite/statahead.c >>+++ b/drivers/staging/lustre/lustre/llite/statahead.c >>@@ -864,7 +864,6 @@ static int ll_agl_thread(void *arg) >> struct ll_sb_info *sbi = ll_i2sbi(dir); >> struct ll_statahead_info *sai; >> struct ptlrpc_thread *thread; >>- struct l_wait_info lwi = { 0 }; >> >> sai = ll_sai_get(dir); >> thread = &sai->sai_agl_thread; >>@@ -885,10 +884,9 @@ static int ll_agl_thread(void *arg) >> wake_up(&thread->t_ctl_waitq); >> >> while (1) { >>- l_wait_event(thread->t_ctl_waitq, >>- !list_empty(&sai->sai_agls) || >>- !thread_is_running(thread), >>- &lwi); >>+ wait_event_noload(thread->t_ctl_waitq, >>+ !list_empty(&sai->sai_agls) || >>+ !thread_is_running(thread)); >> >> if (!thread_is_running(thread)) >> break; >>@@ -932,7 +930,6 @@ static int ll_agl_thread(void *arg) >> static void ll_start_agl(struct dentry *parent, struct ll_statahead_info >>*sai) >> { >> struct ptlrpc_thread *thread = &sai->sai_agl_thread; >>- struct l_wait_info lwi = { 0 }; >> struct ll_inode_info *plli; >> struct task_struct *task; >> >>@@ -948,9 +945,8 @@ static void ll_start_agl(struct dentry *parent, >>struct ll_statahead_info *sai) >> return; >> } >> >>- l_wait_event(thread->t_ctl_waitq, >>- thread_is_running(thread) || thread_is_stopped(thread), >>- &lwi); >>+ wait_event(thread->t_ctl_waitq, >>+ thread_is_running(thread) || thread_is_stopped(thread)); >> } >> >> /* statahead thread main function */ >>@@ -968,7 +964,6 @@ static int ll_statahead_thread(void *arg) >> int first = 0; >> int rc = 0; >> struct md_op_data *op_data; >>- struct l_wait_info lwi = { 0 }; >> >> sai = ll_sai_get(dir); >> sa_thread = &sai->sai_thread; >>@@ -1069,12 +1064,11 @@ static int ll_statahead_thread(void *arg) >> >> /* wait for spare statahead window */ >> do { >>- l_wait_event(sa_thread->t_ctl_waitq, >>- !sa_sent_full(sai) || >>- sa_has_callback(sai) || >>- !list_empty(&sai->sai_agls) || >>- !thread_is_running(sa_thread), >>- &lwi); >>+ wait_event(sa_thread->t_ctl_waitq, >>+ !sa_sent_full(sai) || >>+ sa_has_callback(sai) || >>+ !list_empty(&sai->sai_agls) || >>+ !thread_is_running(sa_thread)); >> sa_handle_callback(sai); >> >> spin_lock(&lli->lli_agl_lock); >>@@ -1128,11 +1122,10 @@ static int ll_statahead_thread(void *arg) >> * for file release to stop me. >> */ >> while (thread_is_running(sa_thread)) { >>- l_wait_event(sa_thread->t_ctl_waitq, >>- sa_has_callback(sai) || >>- !agl_list_empty(sai) || >>- !thread_is_running(sa_thread), >>- &lwi); >>+ wait_event(sa_thread->t_ctl_waitq, >>+ sa_has_callback(sai) || >>+ !agl_list_empty(sai) || >>+ !thread_is_running(sa_thread)); >> >> sa_handle_callback(sai); >> } >>@@ -1145,9 +1138,8 @@ static int ll_statahead_thread(void *arg) >> >> CDEBUG(D_READA, "stop agl thread: sai %p pid %u\n", >> sai, (unsigned int)agl_thread->t_pid); >>- l_wait_event(agl_thread->t_ctl_waitq, >>- thread_is_stopped(agl_thread), >>- &lwi); >>+ wait_event(agl_thread->t_ctl_waitq, >>+ thread_is_stopped(agl_thread)); >> } else { >> /* Set agl_thread flags anyway. */ >> thread_set_flags(agl_thread, SVC_STOPPED); >>@@ -1159,8 +1151,8 @@ static int ll_statahead_thread(void *arg) >> */ >> while (sai->sai_sent != sai->sai_replied) { >> /* in case we're not woken up, timeout wait */ >>- lwi = LWI_TIMEOUT(msecs_to_jiffies(MSEC_PER_SEC >> 3), >>- NULL, NULL); >>+ struct l_wait_info lwi = LWI_TIMEOUT(msecs_to_jiffies(MSEC_PER_SEC >> >>3), >>+ NULL, NULL); >> l_wait_event(sa_thread->t_ctl_waitq, >> sai->sai_sent == sai->sai_replied, &lwi); >> } >>@@ -1520,7 +1512,6 @@ static int start_statahead_thread(struct inode >>*dir, struct dentry *dentry) >> { >> struct ll_inode_info *lli = ll_i2info(dir); >> struct ll_statahead_info *sai = NULL; >>- struct l_wait_info lwi = { 0 }; >> struct ptlrpc_thread *thread; >> struct task_struct *task; >> struct dentry *parent = dentry->d_parent; >>@@ -1570,9 +1561,8 @@ static int start_statahead_thread(struct inode >>*dir, struct dentry *dentry) >> goto out; >> } >> >>- l_wait_event(thread->t_ctl_waitq, >>- thread_is_running(thread) || thread_is_stopped(thread), >>- &lwi); >>+ wait_event(thread->t_ctl_waitq, >>+ thread_is_running(thread) || thread_is_stopped(thread)); >> ll_sai_put(sai); >> >> /* >>diff --git a/drivers/staging/lustre/lustre/lov/lov_object.c >>b/drivers/staging/lustre/lustre/lov/lov_object.c >>index 897cf2cd4a24..aa82f2ed40ae 100644 >>--- a/drivers/staging/lustre/lustre/lov/lov_object.c >>+++ b/drivers/staging/lustre/lustre/lov/lov_object.c >>@@ -723,15 +723,13 @@ static void lov_conf_unlock(struct lov_object *lov) >> >> static int lov_layout_wait(const struct lu_env *env, struct lov_object >>*lov) >> { >>- struct l_wait_info lwi = { 0 }; >>- >> while (atomic_read(&lov->lo_active_ios) > 0) { >> CDEBUG(D_INODE, "file:" DFID " wait for active IO, now: %d.\n", >> PFID(lu_object_fid(lov2lu(lov))), >> atomic_read(&lov->lo_active_ios)); >> >>- l_wait_event(lov->lo_waitq, >>- atomic_read(&lov->lo_active_ios) == 0, &lwi); >>+ wait_event(lov->lo_waitq, >>+ atomic_read(&lov->lo_active_ios) == 0); >> } >> return 0; >> } >>diff --git a/drivers/staging/lustre/lustre/mgc/mgc_request.c >>b/drivers/staging/lustre/lustre/mgc/mgc_request.c >>index 79ff85feab64..81b101941eec 100644 >>--- a/drivers/staging/lustre/lustre/mgc/mgc_request.c >>+++ b/drivers/staging/lustre/lustre/mgc/mgc_request.c >>@@ -601,9 +601,7 @@ static int mgc_requeue_thread(void *data) >> config_log_put(cld_prev); >> >> /* Wait a bit to see if anyone else needs a requeue */ >>- lwi = (struct l_wait_info) { 0 }; >>- l_wait_event(rq_waitq, rq_state & (RQ_NOW | RQ_STOP), >>- &lwi); >>+ wait_event(rq_waitq, rq_state & (RQ_NOW | RQ_STOP)); >> spin_lock(&config_list_lock); >> } >> >>diff --git a/drivers/staging/lustre/lustre/obdclass/cl_io.c >>b/drivers/staging/lustre/lustre/obdclass/cl_io.c >>index 6ec5218a18c1..a3fb2bbde70f 100644 >>--- a/drivers/staging/lustre/lustre/obdclass/cl_io.c >>+++ b/drivers/staging/lustre/lustre/obdclass/cl_io.c >>@@ -1110,10 +1110,8 @@ int cl_sync_io_wait(const struct lu_env *env, >>struct cl_sync_io *anchor, >> CERROR("IO failed: %d, still wait for %d remaining entries\n", >> rc, atomic_read(&anchor->csi_sync_nr)); >> >>- lwi = (struct l_wait_info) { 0 }; >>- (void)l_wait_event(anchor->csi_waitq, >>- atomic_read(&anchor->csi_sync_nr) == 0, >>- &lwi); >>+ wait_event(anchor->csi_waitq, >>+ atomic_read(&anchor->csi_sync_nr) == 0); >> } else { >> rc = anchor->csi_sync_rc; >> } >>diff --git a/drivers/staging/lustre/lustre/obdclass/genops.c >>b/drivers/staging/lustre/lustre/obdclass/genops.c >>index b1d6ba4a3190..78f0fa1dff45 100644 >>--- a/drivers/staging/lustre/lustre/obdclass/genops.c >>+++ b/drivers/staging/lustre/lustre/obdclass/genops.c >>@@ -1237,12 +1237,10 @@ static int obd_zombie_is_idle(void) >> */ >> void obd_zombie_barrier(void) >> { >>- struct l_wait_info lwi = { 0 }; >>- >> if (obd_zombie_pid == current_pid()) >> /* don't wait for myself */ >> return; >>- l_wait_event(obd_zombie_waitq, obd_zombie_is_idle(), &lwi); >>+ wait_event(obd_zombie_waitq, obd_zombie_is_idle()); >> } >> EXPORT_SYMBOL(obd_zombie_barrier); >> >>@@ -1257,10 +1255,8 @@ static int obd_zombie_impexp_thread(void *unused) >> obd_zombie_pid = current_pid(); >> >> while (!test_bit(OBD_ZOMBIE_STOP, &obd_zombie_flags)) { >>- struct l_wait_info lwi = { 0 }; >>- >>- l_wait_event(obd_zombie_waitq, >>- !obd_zombie_impexp_check(NULL), &lwi); >>+ wait_event_noload(obd_zombie_waitq, >>+ !obd_zombie_impexp_check(NULL)); >> obd_zombie_impexp_cull(); >> >> /* >>@@ -1593,7 +1589,6 @@ static inline bool obd_mod_rpc_slot_avail(struct >>client_obd *cli, >> u16 obd_get_mod_rpc_slot(struct client_obd *cli, __u32 opc, >> struct lookup_intent *it) >> { >>- struct l_wait_info lwi = LWI_INTR(NULL, NULL); >> bool close_req = false; >> u16 i, max; >> >>@@ -1631,8 +1626,8 @@ u16 obd_get_mod_rpc_slot(struct client_obd *cli, >>__u32 opc, >> CDEBUG(D_RPCTRACE, "%s: sleeping for a modify RPC slot opc %u, max >>%hu\n", >> cli->cl_import->imp_obd->obd_name, opc, max); >> >>- l_wait_event(cli->cl_mod_rpcs_waitq, >>- obd_mod_rpc_slot_avail(cli, close_req), &lwi); >>+ wait_event(cli->cl_mod_rpcs_waitq, >>+ obd_mod_rpc_slot_avail(cli, close_req)); >> } while (true); >> } >> EXPORT_SYMBOL(obd_get_mod_rpc_slot); >>diff --git a/drivers/staging/lustre/lustre/osc/osc_cache.c >>b/drivers/staging/lustre/lustre/osc/osc_cache.c >>index 5767ac2a7d16..d58a25a2a5b4 100644 >>--- a/drivers/staging/lustre/lustre/osc/osc_cache.c >>+++ b/drivers/staging/lustre/lustre/osc/osc_cache.c >>@@ -964,9 +964,8 @@ static int osc_extent_wait(const struct lu_env *env, >>struct osc_extent *ext, >> "%s: wait ext to %u timedout, recovery in progress?\n", >> cli_name(osc_cli(obj)), state); >> >>- lwi = LWI_INTR(NULL, NULL); >>- rc = l_wait_event(ext->oe_waitq, extent_wait_cb(ext, state), >>- &lwi); >>+ wait_event(ext->oe_waitq, extent_wait_cb(ext, state)); >>+ rc = 0; >> } >> if (rc == 0 && ext->oe_rc < 0) >> rc = ext->oe_rc; >>diff --git a/drivers/staging/lustre/lustre/osc/osc_object.c >>b/drivers/staging/lustre/lustre/osc/osc_object.c >>index f82c87a77550..1de25496a7d9 100644 >>--- a/drivers/staging/lustre/lustre/osc/osc_object.c >>+++ b/drivers/staging/lustre/lustre/osc/osc_object.c >>@@ -454,12 +454,10 @@ struct lu_object *osc_object_alloc(const struct >>lu_env *env, >> >> int osc_object_invalidate(const struct lu_env *env, struct osc_object >>*osc) >> { >>- struct l_wait_info lwi = { 0 }; >>- >> CDEBUG(D_INODE, "Invalidate osc object: %p, # of active IOs: %d\n", >> osc, atomic_read(&osc->oo_nr_ios)); >> >>- l_wait_event(osc->oo_io_waitq, !atomic_read(&osc->oo_nr_ios), &lwi); >>+ wait_event(osc->oo_io_waitq, !atomic_read(&osc->oo_nr_ios)); >> >> /* Discard all dirty pages of this object. */ >> osc_cache_truncate_start(env, osc, 0, NULL); >>diff --git a/drivers/staging/lustre/lustre/ptlrpc/pinger.c >>b/drivers/staging/lustre/lustre/ptlrpc/pinger.c >>index fe6b47bfe8be..4148a6661dcf 100644 >>--- a/drivers/staging/lustre/lustre/ptlrpc/pinger.c >>+++ b/drivers/staging/lustre/lustre/ptlrpc/pinger.c >>@@ -291,7 +291,6 @@ static struct ptlrpc_thread pinger_thread; >> >> int ptlrpc_start_pinger(void) >> { >>- struct l_wait_info lwi = { 0 }; >> struct task_struct *task; >> int rc; >> >>@@ -310,8 +309,8 @@ int ptlrpc_start_pinger(void) >> CERROR("cannot start pinger thread: rc = %d\n", rc); >> return rc; >> } >>- l_wait_event(pinger_thread.t_ctl_waitq, >>- thread_is_running(&pinger_thread), &lwi); >>+ wait_event(pinger_thread.t_ctl_waitq, >>+ thread_is_running(&pinger_thread)); >> >> return 0; >> } >>@@ -320,7 +319,6 @@ static int ptlrpc_pinger_remove_timeouts(void); >> >> int ptlrpc_stop_pinger(void) >> { >>- struct l_wait_info lwi = { 0 }; >> int rc = 0; >> >> if (thread_is_init(&pinger_thread) || >>@@ -331,8 +329,8 @@ int ptlrpc_stop_pinger(void) >> thread_set_flags(&pinger_thread, SVC_STOPPING); >> wake_up(&pinger_thread.t_ctl_waitq); >> >>- l_wait_event(pinger_thread.t_ctl_waitq, >>- thread_is_stopped(&pinger_thread), &lwi); >>+ wait_event(pinger_thread.t_ctl_waitq, >>+ thread_is_stopped(&pinger_thread)); >> >> return rc; >> } >>diff --git a/drivers/staging/lustre/lustre/ptlrpc/sec_gc.c >>b/drivers/staging/lustre/lustre/ptlrpc/sec_gc.c >>index d85c8638c009..e4197a60d1e2 100644 >>--- a/drivers/staging/lustre/lustre/ptlrpc/sec_gc.c >>+++ b/drivers/staging/lustre/lustre/ptlrpc/sec_gc.c >>@@ -197,7 +197,6 @@ static int sec_gc_main(void *arg) >> >> int sptlrpc_gc_init(void) >> { >>- struct l_wait_info lwi = { 0 }; >> struct task_struct *task; >> >> mutex_init(&sec_gc_mutex); >>@@ -214,18 +213,16 @@ int sptlrpc_gc_init(void) >> return PTR_ERR(task); >> } >> >>- l_wait_event(sec_gc_thread.t_ctl_waitq, >>- thread_is_running(&sec_gc_thread), &lwi); >>+ wait_event(sec_gc_thread.t_ctl_waitq, >>+ thread_is_running(&sec_gc_thread)); >> return 0; >> } >> >> void sptlrpc_gc_fini(void) >> { >>- struct l_wait_info lwi = { 0 }; >>- >> thread_set_flags(&sec_gc_thread, SVC_STOPPING); >> wake_up(&sec_gc_thread.t_ctl_waitq); >> >>- l_wait_event(sec_gc_thread.t_ctl_waitq, >>- thread_is_stopped(&sec_gc_thread), &lwi); >>+ wait_event(sec_gc_thread.t_ctl_waitq, >>+ thread_is_stopped(&sec_gc_thread)); >> } >>diff --git a/drivers/staging/lustre/lustre/ptlrpc/service.c >>b/drivers/staging/lustre/lustre/ptlrpc/service.c >>index 63be6e7273f3..d688cb3ff157 100644 >>--- a/drivers/staging/lustre/lustre/ptlrpc/service.c >>+++ b/drivers/staging/lustre/lustre/ptlrpc/service.c >>@@ -2233,7 +2233,7 @@ static int ptlrpc_hr_main(void *arg) >> wake_up(&ptlrpc_hr.hr_waitq); >> >> while (!ptlrpc_hr.hr_stopping) { >>- l_wait_condition(hrt->hrt_waitq, hrt_dont_sleep(hrt, &replies)); >>+ wait_event_noload(hrt->hrt_waitq, hrt_dont_sleep(hrt, &replies)); >> >> while (!list_empty(&replies)) { >> struct ptlrpc_reply_state *rs; >>@@ -2312,7 +2312,6 @@ static int ptlrpc_start_hr_threads(void) >> >> static void ptlrpc_svcpt_stop_threads(struct ptlrpc_service_part *svcpt) >> { >>- struct l_wait_info lwi = { 0 }; >> struct ptlrpc_thread *thread; >> LIST_HEAD(zombie); >> >>@@ -2341,8 +2340,8 @@ static void ptlrpc_svcpt_stop_threads(struct >>ptlrpc_service_part *svcpt) >> >> CDEBUG(D_INFO, "waiting for stopping-thread %s #%u\n", >> svcpt->scp_service->srv_thread_name, thread->t_id); >>- l_wait_event(thread->t_ctl_waitq, >>- thread_is_stopped(thread), &lwi); >>+ wait_event(thread->t_ctl_waitq, >>+ thread_is_stopped(thread)); >> >> spin_lock(&svcpt->scp_lock); >> } >>@@ -2403,7 +2402,6 @@ int ptlrpc_start_threads(struct ptlrpc_service *svc) >> >> int ptlrpc_start_thread(struct ptlrpc_service_part *svcpt, int wait) >> { >>- struct l_wait_info lwi = { 0 }; >> struct ptlrpc_thread *thread; >> struct ptlrpc_service *svc; >> struct task_struct *task; >>@@ -2499,9 +2497,8 @@ int ptlrpc_start_thread(struct ptlrpc_service_part >>*svcpt, int wait) >> if (!wait) >> return 0; >> >>- l_wait_event(thread->t_ctl_waitq, >>- thread_is_running(thread) || thread_is_stopped(thread), >>- &lwi); >>+ wait_event(thread->t_ctl_waitq, >>+ thread_is_running(thread) || thread_is_stopped(thread)); >> >> rc = thread_is_stopped(thread) ? thread->t_id : 0; >> return rc; >> >> >>_______________________________________________ >>lustre-devel mailing list >>lustre-devel at lists.lustre.org >>http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 832 bytes Desc: not available URL: From neilb at suse.com Mon Dec 18 23:01:47 2017 From: neilb at suse.com (NeilBrown) Date: Tue, 19 Dec 2017 10:01:47 +1100 Subject: [lustre-devel] [PATCH SET 6: 0/2] staging: lustre: two miscellaneous patches Message-ID: <151363798156.11899.768707943431676479.stgit@noble> I thought I had mode one-off patches but this, but it seems there are only two ready to go now. The first removes another local interface in favour of a more general interface, the second removing a warning (which actually say "BUG:"). Thanks, NeilBrown --- NeilBrown (2): staging: lustre: use strim instead of cfs_trimwhite. staging: lustre: disable preempt while sampling processor id. .../lustre/include/linux/libcfs/libcfs_string.h | 1 - drivers/staging/lustre/lnet/libcfs/libcfs_string.c | 20 ------------------- .../staging/lustre/lnet/libcfs/linux/linux-cpu.c | 21 ++++++++++---------- drivers/staging/lustre/lnet/lnet/config.c | 10 +++++----- drivers/staging/lustre/lnet/lnet/router_proc.c | 2 +- 5 files changed, 17 insertions(+), 37 deletions(-) -- Signature From neilb at suse.com Mon Dec 18 23:01:47 2017 From: neilb at suse.com (NeilBrown) Date: Tue, 19 Dec 2017 10:01:47 +1100 Subject: [lustre-devel] [PATCH 1/2] staging: lustre: use strim instead of cfs_trimwhite. In-Reply-To: <151363798156.11899.768707943431676479.stgit@noble> References: <151363798156.11899.768707943431676479.stgit@noble> Message-ID: <151363810769.11899.15503930546886979038.stgit@noble> Linux lib provides identical functionality to cfs_trimwhite, so discard that code and use the standard. Signed-off-by: NeilBrown --- .../lustre/include/linux/libcfs/libcfs_string.h | 1 - drivers/staging/lustre/lnet/libcfs/libcfs_string.c | 20 -------------------- .../staging/lustre/lnet/libcfs/linux/linux-cpu.c | 8 ++++---- drivers/staging/lustre/lnet/lnet/config.c | 10 +++++----- drivers/staging/lustre/lnet/lnet/router_proc.c | 2 +- 5 files changed, 10 insertions(+), 31 deletions(-) diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_string.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_string.h index c1375733ff31..66463477074a 100644 --- a/drivers/staging/lustre/include/linux/libcfs/libcfs_string.h +++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_string.h @@ -73,7 +73,6 @@ struct cfs_expr_list { struct list_head el_exprs; }; -char *cfs_trimwhite(char *str); int cfs_gettok(struct cfs_lstr *next, char delim, struct cfs_lstr *res); int cfs_str2num_check(char *str, int nob, unsigned int *num, unsigned int min, unsigned int max); diff --git a/drivers/staging/lustre/lnet/libcfs/libcfs_string.c b/drivers/staging/lustre/lnet/libcfs/libcfs_string.c index b1d8faa3f7aa..442889a3d729 100644 --- a/drivers/staging/lustre/lnet/libcfs/libcfs_string.c +++ b/drivers/staging/lustre/lnet/libcfs/libcfs_string.c @@ -137,26 +137,6 @@ char *cfs_firststr(char *str, size_t size) } EXPORT_SYMBOL(cfs_firststr); -char * -cfs_trimwhite(char *str) -{ - char *end; - - while (isspace(*str)) - str++; - - end = str + strlen(str); - while (end > str) { - if (!isspace(end[-1])) - break; - end--; - } - - *end = 0; - return str; -} -EXPORT_SYMBOL(cfs_trimwhite); - /** * Extracts tokens from strings. * diff --git a/drivers/staging/lustre/lnet/libcfs/linux/linux-cpu.c b/drivers/staging/lustre/lnet/libcfs/linux/linux-cpu.c index e9156bf05ed4..d30650f8dcb4 100644 --- a/drivers/staging/lustre/lnet/libcfs/linux/linux-cpu.c +++ b/drivers/staging/lustre/lnet/libcfs/linux/linux-cpu.c @@ -818,7 +818,7 @@ cfs_cpt_table_create_pattern(char *pattern) int c; int i; - str = cfs_trimwhite(pattern); + str = strim(pattern); if (*str == 'n' || *str == 'N') { pattern = str + 1; if (*pattern != '\0') { @@ -870,7 +870,7 @@ cfs_cpt_table_create_pattern(char *pattern) high = node ? MAX_NUMNODES - 1 : nr_cpu_ids - 1; - for (str = cfs_trimwhite(pattern), c = 0;; c++) { + for (str = strim(pattern), c = 0;; c++) { struct cfs_range_expr *range; struct cfs_expr_list *el; char *bracket = strchr(str, '['); @@ -905,7 +905,7 @@ cfs_cpt_table_create_pattern(char *pattern) goto failed; } - str = cfs_trimwhite(str + n); + str = strim(str + n); if (str != bracket) { CERROR("Invalid pattern %s\n", str); goto failed; @@ -945,7 +945,7 @@ cfs_cpt_table_create_pattern(char *pattern) goto failed; } - str = cfs_trimwhite(bracket + 1); + str = strim(bracket + 1); } return cptab; diff --git a/drivers/staging/lustre/lnet/lnet/config.c b/drivers/staging/lustre/lnet/lnet/config.c index fd53c74766a7..44eeca63f458 100644 --- a/drivers/staging/lustre/lnet/lnet/config.c +++ b/drivers/staging/lustre/lnet/lnet/config.c @@ -269,7 +269,7 @@ lnet_parse_networks(struct list_head *nilist, char *networks) if (comma) *comma++ = 0; - net = libcfs_str2net(cfs_trimwhite(str)); + net = libcfs_str2net(strim(str)); if (net == LNET_NIDNET(LNET_NID_ANY)) { LCONSOLE_ERROR_MSG(0x113, @@ -292,7 +292,7 @@ lnet_parse_networks(struct list_head *nilist, char *networks) } *bracket = 0; - net = libcfs_str2net(cfs_trimwhite(str)); + net = libcfs_str2net(strim(str)); if (net == LNET_NIDNET(LNET_NID_ANY)) { tmp = str; goto failed_syntax; @@ -322,7 +322,7 @@ lnet_parse_networks(struct list_head *nilist, char *networks) if (comma) *comma++ = 0; - iface = cfs_trimwhite(iface); + iface = strim(iface); if (!*iface) { tmp = iface; goto failed_syntax; @@ -356,7 +356,7 @@ lnet_parse_networks(struct list_head *nilist, char *networks) comma = strchr(bracket + 1, ','); if (comma) { *comma = 0; - str = cfs_trimwhite(str); + str = strim(str); if (*str) { tmp = str; goto failed_syntax; @@ -365,7 +365,7 @@ lnet_parse_networks(struct list_head *nilist, char *networks) continue; } - str = cfs_trimwhite(str); + str = strim(str); if (*str) { tmp = str; goto failed_syntax; diff --git a/drivers/staging/lustre/lnet/lnet/router_proc.c b/drivers/staging/lustre/lnet/lnet/router_proc.c index 8d6d6b4d6619..1a71ffebc889 100644 --- a/drivers/staging/lustre/lnet/lnet/router_proc.c +++ b/drivers/staging/lustre/lnet/lnet/router_proc.c @@ -829,7 +829,7 @@ static int __proc_lnet_portal_rotor(void *data, int write, if (rc < 0) goto out; - tmp = cfs_trimwhite(buf); + tmp = strim(buf); rc = -EINVAL; lnet_res_lock(0); From neilb at suse.com Mon Dec 18 23:01:47 2017 From: neilb at suse.com (NeilBrown) Date: Tue, 19 Dec 2017 10:01:47 +1100 Subject: [lustre-devel] [PATCH 2/2] staging: lustre: disable preempt while sampling processor id. In-Reply-To: <151363798156.11899.768707943431676479.stgit@noble> References: <151363798156.11899.768707943431676479.stgit@noble> Message-ID: <151363810776.11899.1696679196545013564.stgit@noble> Calling smp_processor_id() without disabling preemption triggers a warning (if CONFIG_DEBUG_PREEMPT). I think the result of cfs_cpt_current() is only used as a hint for load balancing, rather than as a precise and stable indicator of the current CPU. So it doesn't need to be called with preemption disabled. So disable preemption inside cfs_cpt_current() to silence the warning. Signed-off-by: NeilBrown --- .../staging/lustre/lnet/libcfs/linux/linux-cpu.c | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/drivers/staging/lustre/lnet/libcfs/linux/linux-cpu.c b/drivers/staging/lustre/lnet/libcfs/linux/linux-cpu.c index d30650f8dcb4..ca8518b8a3e0 100644 --- a/drivers/staging/lustre/lnet/libcfs/linux/linux-cpu.c +++ b/drivers/staging/lustre/lnet/libcfs/linux/linux-cpu.c @@ -517,19 +517,20 @@ EXPORT_SYMBOL(cfs_cpt_spread_node); int cfs_cpt_current(struct cfs_cpt_table *cptab, int remap) { - int cpu = smp_processor_id(); - int cpt = cptab->ctb_cpu2cpt[cpu]; + int cpu; + int cpt; - if (cpt < 0) { - if (!remap) - return cpt; + preempt_disable(); + cpu = smp_processor_id(); + cpt = cptab->ctb_cpu2cpt[cpu]; + if (cpt < 0 && remap) { /* don't return negative value for safety of upper layer, * instead we shadow the unknown cpu to a valid partition ID */ cpt = cpu % cptab->ctb_nparts; } - + preempt_enable(); return cpt; } EXPORT_SYMBOL(cfs_cpt_current); From andreas.dilger at intel.com Tue Dec 19 00:52:06 2017 From: andreas.dilger at intel.com (Dilger, Andreas) Date: Tue, 19 Dec 2017 00:52:06 +0000 Subject: [lustre-devel] [PATCH SERIES 4: 0/4] staging: lustre: use standard prng In-Reply-To: <151356119565.24290.5570928075046338898.stgit@noble> References: <151356119565.24290.5570928075046338898.stgit@noble> Message-ID: <47156CF4-EC7A-40EB-B7B3-46D2E7B879D0@intel.com> On Dec 17, 2017, at 18:41, NeilBrown wrote: > > Lustre has its own internal PRNG code. > This adds nothing of value to the Linux standard prng code, > so switch over to using the standard interfaces. > This adds a few callers to add_device_randomness(), which > helps everyone, and removes unnecessary code. Neil, Thanks for the patches. I'll run them through our testing system, but they look good at first glance. An interesting anecdote as this code is removed... When it was first added, we were running Lustre on a single-threaded runtime environment without any local storage, interrupts, local clock, or h/w RNG (Catamount, on the ASCI Red Storm https://en.wikipedia.org/wiki/Red_Storm_(computing) supercomputer) and since there were thousands of nodes booting up and mounting Lustre, there were often some with identical random number states/seeds after boot, so we had to fold in the only unique state that we had on each node - the network address. That system is long gone, and it is good to clean up this code in a more portable manner. Cheers, Andreas > --- > > NeilBrown (4): > staging: lustre: replace cfs_rand() with prandom_u32_max() > staging: lustre: replace cfs_srand() calls with add_device_randomness(). > staging: lustre: replace cfs_get_random_bytes calls with get_random_byte() > staging: lustre: libcfs: remove prng > > > .../staging/lustre/include/linux/libcfs/libcfs.h | 10 - > drivers/staging/lustre/lnet/libcfs/Makefile | 2 > drivers/staging/lustre/lnet/libcfs/fail.c | 2 > drivers/staging/lustre/lnet/libcfs/prng.c | 137 -------------------- > drivers/staging/lustre/lnet/lnet/net_fault.c | 38 +++--- > drivers/staging/lustre/lnet/lnet/router.c | 19 +-- > drivers/staging/lustre/lustre/include/obd_class.h | 2 > drivers/staging/lustre/lustre/llite/super25.c | 17 +- > drivers/staging/lustre/lustre/mgc/mgc_request.c | 4 - > .../lustre/lustre/obdclass/lustre_handles.c | 9 - > drivers/staging/lustre/lustre/ptlrpc/client.c | 2 > 11 files changed, 42 insertions(+), 200 deletions(-) > delete mode 100644 drivers/staging/lustre/lnet/libcfs/prng.c > > -- > Signature > > _______________________________________________ > lustre-devel mailing list > lustre-devel at lists.lustre.org > http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Intel Corporation From andreas.dilger at intel.com Tue Dec 19 10:37:23 2017 From: andreas.dilger at intel.com (Dilger, Andreas) Date: Tue, 19 Dec 2017 10:37:23 +0000 Subject: [lustre-devel] [PATCH 02/16] staging: lustre: replace simple cases of l_wait_event() with wait_event(). In-Reply-To: References: <151358127190.5099.12792810096274074963.stgit@noble> <151358147981.5099.13114335078693829049.stgit@noble> Message-ID: <9A64B5D7-C979-4400-9AFA-9C2AFC486129@intel.com> On Dec 18, 2017, at 11:03, Patrick Farrell wrote: > > The wait calls in ll_statahead_thread are done in a service thread, and > should probably *not* contribute to load. > > The one in osc_extent_wait is perhaps tough - It is called both from user > threads & daemon threads depending on the situation. The effect of adding > that to load average could be significant for some activities, even when > no user threads are busy. Thoughts from other Lustre people would be > welcome here. The main reasons we started using l_wait_event() were: - it is used by the request handling threads, and wait_event() caused the load average to always be == number of service threads, which was wrong if those threads were idle waiting for requests to arrive. That is mostly a server problem, but a couple of request handlers are on the client also (DLM lock cancellation threads, etc.) that shouldn't contribute to load. It looks like there is a better solution for this today with TASK_IDLE. - we want the userspace threads to be interruptible if the server is not functional, but the client should at least get a chance to complete the RPC if the server is just busy. Since Lustre needs to work in systems with 10,000+ clients pounding a server, the server response time is not necessarily fast. The l_wait_event() behavior is that it blocks signals until the RPC timeout, which will normally succeed, but after the timeout the signals are unblocked and the user thread can be interrupted if the user wants, but it will keep waiting for the RPC to finish if not. This is half-way between NFS soft and hard mounts. I don't think there is an equivalent wait_event_* for that behaviour. Cheers, Andreas > Similar issues for osc_object_invalidate. > > (If no one else speaks up, my vote is no contribution to load for those > OSC waits.) > > Otherwise this one looks good... > > On 12/18/17, 1:17 AM, "lustre-devel on behalf of NeilBrown" > wrote: > >> @@ -968,7 +964,6 @@ static int ll_statahead_thread(void *arg) >> int first = 0; >> int rc = 0; >> struct md_op_data *op_data; >> - struct l_wait_info lwi = { 0 }; >> sai = ll_sai_get(dir); >> sa_thread = &sai->sai_thread; >> @@ -1069,12 +1064,11 @@ static int ll_statahead_thread(void *arg) >> /* wait for spare statahead window */ >> do { >> - l_wait_event(sa_thread->t_ctl_waitq, >> - !sa_sent_full(sai) || >> - sa_has_callback(sai) || >> - !list_empty(&sai->sai_agls) || >> - !thread_is_running(sa_thread), >> - &lwi); >> + wait_event(sa_thread->t_ctl_waitq, >> + !sa_sent_full(sai) || >> + sa_has_callback(sai) || >> + !list_empty(&sai->sai_agls) || >> + !thread_is_running(sa_thread)); >> sa_handle_callback(sai); >> spin_lock(&lli->lli_agl_lock); >> @@ -1128,11 +1122,10 @@ static int ll_statahead_thread(void *arg) >> * for file release to stop me. >> */ >> while (thread_is_running(sa_thread)) { >> - l_wait_event(sa_thread->t_ctl_waitq, >> - sa_has_callback(sai) || >> - !agl_list_empty(sai) || >> - !thread_is_running(sa_thread), >> - &lwi); >> + wait_event(sa_thread->t_ctl_waitq, >> + sa_has_callback(sai) || >> + !agl_list_empty(sai) || >> + !thread_is_running(sa_thread)); >> sa_handle_callback(sai); >> } >> @@ -1145,9 +1138,8 @@ static int ll_statahead_thread(void *arg) >> CDEBUG(D_READA, "stop agl thread: sai %p pid %u\n", >> sai, (unsigned int)agl_thread->t_pid); >> - l_wait_event(agl_thread->t_ctl_waitq, >> - thread_is_stopped(agl_thread), >> - &lwi); >> + wait_event(agl_thread->t_ctl_waitq, >> + thread_is_stopped(agl_thread)); >> } else { >> /* Set agl_thread flags anyway. */ >> thread_set_flags(agl_thread, SVC_STOPPED); >> @@ -1159,8 +1151,8 @@ static int ll_statahead_thread(void *arg) >> */ >> while (sai->sai_sent != sai->sai_replied) { >> /* in case we're not woken up, timeout wait */ >> - lwi = LWI_TIMEOUT(msecs_to_jiffies(MSEC_PER_SEC >> 3), >> - NULL, NULL); >> + struct l_wait_info lwi = LWI_TIMEOUT(msecs_to_jiffies(MSEC_PER_SEC >> >> 3), >> + NULL, NULL); >> l_wait_event(sa_thread->t_ctl_waitq, >> sai->sai_sent == sai->sai_replied, &lwi); >> } > Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Intel Corporation From romeusmeister at gmail.com Tue Dec 19 17:56:54 2017 From: romeusmeister at gmail.com (Roman Storozhenko) Date: Tue, 19 Dec 2017 20:56:54 +0300 Subject: [lustre-devel] [PATCH v4] staging: lustre: Replace 'uint32_t' with 'u32' and 'uint64_t' with 'u64' Message-ID: <20171219175654.GA13794@home> There are two reasons for replacing 'uint32_t' with 'u32' and 'uint64_t' with 'u64': 1) As Linus Torvalds have said we should use kernel types: http://lkml.iu.edu/hypermail//linux/kernel/1506.0/00160.html 2) There are only few places in the lustre codebase that use such types. In the most cases it uses 'u32' and 'u64'. Signed-off-by: Roman Storozhenko --- In the third version of that patch Dan Carpenter pointed out that the patch body should be self-explanatory: " > There are two reasons for that: What I'm asking is there are two reasons for what? Where is the first part of that paragraph? " In the second version of this patch I forgot to add subsystem and driver. As Greg K-H mentioned: "Please fix up the subject to have the subsystem and driver name in it: Subject: [PATCH] staging: lustre: ..." In the first version of this patch I replaced 'uint32_t' with '__u32' and 'uint64_t' with '__u64'. I was suggested to fix that by Greg K-H: "The __ types are only needed for when you cross the user/kernel boundry. Otherwise just use the "normal" types of u32 and u64. Do the changes you made here all cross that boundry? If not, please fix this up." I asked lustre community whether those code used only in the kernel space and Andreas Dilger said: "These headers are for kernel code only, so should use the "u32" and similar types, rather than the "__u32" that are used for user-kernel structures." So I have replaced my first patch version with this one. drivers/staging/lustre/lustre/include/lustre_sec.h | 4 ++-- drivers/staging/lustre/lustre/llite/vvp_dev.c | 2 +- drivers/staging/lustre/lustre/lov/lov_internal.h | 12 ++++++------ drivers/staging/lustre/lustre/osc/osc_internal.h | 6 +++--- 4 files changed, 12 insertions(+), 12 deletions(-) diff --git a/drivers/staging/lustre/lustre/include/lustre_sec.h b/drivers/staging/lustre/lustre/include/lustre_sec.h index a40f706..64b6fd4 100644 --- a/drivers/staging/lustre/lustre/include/lustre_sec.h +++ b/drivers/staging/lustre/lustre/include/lustre_sec.h @@ -341,8 +341,8 @@ void sptlrpc_conf_client_adapt(struct obd_device *obd); #define SPTLRPC_MAX_PAYLOAD (1024) struct vfs_cred { - uint32_t vc_uid; - uint32_t vc_gid; + u32 vc_uid; + u32 vc_gid; }; struct ptlrpc_ctx_ops { diff --git a/drivers/staging/lustre/lustre/llite/vvp_dev.c b/drivers/staging/lustre/lustre/llite/vvp_dev.c index 8ccc8b7..987c03b 100644 --- a/drivers/staging/lustre/lustre/llite/vvp_dev.c +++ b/drivers/staging/lustre/lustre/llite/vvp_dev.c @@ -384,7 +384,7 @@ int cl_sb_fini(struct super_block *sb) struct vvp_pgcache_id { unsigned int vpi_bucket; unsigned int vpi_depth; - uint32_t vpi_index; + u32 vpi_index; unsigned int vpi_curdep; struct lu_object_header *vpi_obj; diff --git a/drivers/staging/lustre/lustre/lov/lov_internal.h b/drivers/staging/lustre/lustre/lov/lov_internal.h index ae28ddf..a56d71c 100644 --- a/drivers/staging/lustre/lustre/lov/lov_internal.h +++ b/drivers/staging/lustre/lustre/lov/lov_internal.h @@ -115,19 +115,19 @@ static inline const struct lsm_operations *lsm_op_find(int magic) */ #if BITS_PER_LONG == 64 # define lov_do_div64(n, base) ({ \ - uint64_t __base = (base); \ - uint64_t __rem; \ - __rem = ((uint64_t)(n)) % __base; \ - (n) = ((uint64_t)(n)) / __base; \ + u64 __base = (base); \ + u64 __rem; \ + __rem = ((u64)(n)) % __base; \ + (n) = ((u64)(n)) / __base; \ __rem; \ }) #elif BITS_PER_LONG == 32 # define lov_do_div64(n, base) ({ \ - uint64_t __rem; \ + u64 __rem; \ if ((sizeof(base) > 4) && (((base) & 0xffffffff00000000ULL) != 0)) { \ int __remainder; \ LASSERTF(!((base) & (LOV_MIN_STRIPE_SIZE - 1)), "64 bit lov " \ - "division %llu / %llu\n", (n), (uint64_t)(base)); \ + "division %llu / %llu\n", (n), (u64)(base)); \ __remainder = (n) & (LOV_MIN_STRIPE_SIZE - 1); \ (n) >>= LOV_MIN_STRIPE_BITS; \ __rem = do_div(n, (base) >> LOV_MIN_STRIPE_BITS); \ diff --git a/drivers/staging/lustre/lustre/osc/osc_internal.h b/drivers/staging/lustre/lustre/osc/osc_internal.h index feda61b..32db150 100644 --- a/drivers/staging/lustre/lustre/osc/osc_internal.h +++ b/drivers/staging/lustre/lustre/osc/osc_internal.h @@ -168,9 +168,9 @@ struct osc_device { /* Write stats is actually protected by client_obd's lock. */ struct osc_stats { - uint64_t os_lockless_writes; /* by bytes */ - uint64_t os_lockless_reads; /* by bytes */ - uint64_t os_lockless_truncates; /* by times */ + u64 os_lockless_writes; /* by bytes */ + u64 os_lockless_reads; /* by bytes */ + u64 os_lockless_truncates; /* by times */ } od_stats; /* configuration item(s) */ -- 2.7.4 From neilb at suse.com Tue Dec 19 20:49:34 2017 From: neilb at suse.com (NeilBrown) Date: Wed, 20 Dec 2017 07:49:34 +1100 Subject: [lustre-devel] [PATCH 02/16] staging: lustre: replace simple cases of l_wait_event() with wait_event(). In-Reply-To: <9A64B5D7-C979-4400-9AFA-9C2AFC486129@intel.com> References: <151358127190.5099.12792810096274074963.stgit@noble> <151358147981.5099.13114335078693829049.stgit@noble> <9A64B5D7-C979-4400-9AFA-9C2AFC486129@intel.com> Message-ID: <87efnql9y9.fsf@notabene.neil.brown.name> On Tue, Dec 19 2017, Dilger, Andreas wrote: > On Dec 18, 2017, at 11:03, Patrick Farrell wrote: >> >> The wait calls in ll_statahead_thread are done in a service thread, and >> should probably *not* contribute to load. >> >> The one in osc_extent_wait is perhaps tough - It is called both from user >> threads & daemon threads depending on the situation. The effect of adding >> that to load average could be significant for some activities, even when >> no user threads are busy. Thoughts from other Lustre people would be >> welcome here. > > The main reasons we started using l_wait_event() were: > - it is used by the request handling threads, and wait_event() caused the > load average to always be == number of service threads, which was > wrong if those threads were idle waiting for requests to arrive. > That is mostly a server problem, but a couple of request handlers are > on the client also (DLM lock cancellation threads, etc.) that shouldn't > contribute to load. It looks like there is a better solution for this > today with TASK_IDLE. > - we want the userspace threads to be interruptible if the server is not > functional, but the client should at least get a chance to complete the > RPC if the server is just busy. Since Lustre needs to work in systems > with 10,000+ clients pounding a server, the server response time is not > necessarily fast. The l_wait_event() behavior is that it blocks signals > until the RPC timeout, which will normally succeed, but after the timeout > the signals are unblocked and the user thread can be interrupted if the > user wants, but it will keep waiting for the RPC to finish if not. This > is half-way between NFS soft and hard mounts. I don't think there is an > equivalent wait_event_* for that behaviour. Thanks for the historical background, it can often help to understand why the code is the way it is! Thanks particularly for that second point. I missed it in the code, and skimmed over the explanatory comment too quickly (I'm afraid I don't trust comments very much). I would implement this two-stage wait by first calling swait_event_idle_timeout(), then if that times out, swait_event_interruptible() rather than trying to combine then both into one super-macro. At least, that is my thought before I try to write the code - maybe I'll change my mind. Anyway, it is clear now that this l_wait_event() series needs to be rewritten now that I have a better understanding. Thanks, NeilBrown > > Cheers, Andreas > >> Similar issues for osc_object_invalidate. >> >> (If no one else speaks up, my vote is no contribution to load for those >> OSC waits.) >> >> Otherwise this one looks good... >> >> On 12/18/17, 1:17 AM, "lustre-devel on behalf of NeilBrown" >> wrote: >> >>> @@ -968,7 +964,6 @@ static int ll_statahead_thread(void *arg) >>> int first = 0; >>> int rc = 0; >>> struct md_op_data *op_data; >>> - struct l_wait_info lwi = { 0 }; >>> sai = ll_sai_get(dir); >>> sa_thread = &sai->sai_thread; >>> @@ -1069,12 +1064,11 @@ static int ll_statahead_thread(void *arg) >>> /* wait for spare statahead window */ >>> do { >>> - l_wait_event(sa_thread->t_ctl_waitq, >>> - !sa_sent_full(sai) || >>> - sa_has_callback(sai) || >>> - !list_empty(&sai->sai_agls) || >>> - !thread_is_running(sa_thread), >>> - &lwi); >>> + wait_event(sa_thread->t_ctl_waitq, >>> + !sa_sent_full(sai) || >>> + sa_has_callback(sai) || >>> + !list_empty(&sai->sai_agls) || >>> + !thread_is_running(sa_thread)); >>> sa_handle_callback(sai); >>> spin_lock(&lli->lli_agl_lock); >>> @@ -1128,11 +1122,10 @@ static int ll_statahead_thread(void *arg) >>> * for file release to stop me. >>> */ >>> while (thread_is_running(sa_thread)) { >>> - l_wait_event(sa_thread->t_ctl_waitq, >>> - sa_has_callback(sai) || >>> - !agl_list_empty(sai) || >>> - !thread_is_running(sa_thread), >>> - &lwi); >>> + wait_event(sa_thread->t_ctl_waitq, >>> + sa_has_callback(sai) || >>> + !agl_list_empty(sai) || >>> + !thread_is_running(sa_thread)); >>> sa_handle_callback(sai); >>> } >>> @@ -1145,9 +1138,8 @@ static int ll_statahead_thread(void *arg) >>> CDEBUG(D_READA, "stop agl thread: sai %p pid %u\n", >>> sai, (unsigned int)agl_thread->t_pid); >>> - l_wait_event(agl_thread->t_ctl_waitq, >>> - thread_is_stopped(agl_thread), >>> - &lwi); >>> + wait_event(agl_thread->t_ctl_waitq, >>> + thread_is_stopped(agl_thread)); >>> } else { >>> /* Set agl_thread flags anyway. */ >>> thread_set_flags(agl_thread, SVC_STOPPED); >>> @@ -1159,8 +1151,8 @@ static int ll_statahead_thread(void *arg) >>> */ >>> while (sai->sai_sent != sai->sai_replied) { >>> /* in case we're not woken up, timeout wait */ >>> - lwi = LWI_TIMEOUT(msecs_to_jiffies(MSEC_PER_SEC >> 3), >>> - NULL, NULL); >>> + struct l_wait_info lwi = LWI_TIMEOUT(msecs_to_jiffies(MSEC_PER_SEC >> >>> 3), >>> + NULL, NULL); >>> l_wait_event(sa_thread->t_ctl_waitq, >>> sai->sai_sent == sai->sai_replied, &lwi); >>> } >> > > Cheers, Andreas > -- > Andreas Dilger > Lustre Principal Architect > Intel Corporation -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 832 bytes Desc: not available URL: From andreas.dilger at intel.com Tue Dec 19 23:37:31 2017 From: andreas.dilger at intel.com (Dilger, Andreas) Date: Tue, 19 Dec 2017 23:37:31 +0000 Subject: [lustre-devel] [PATCH 2/2] staging: lustre: disable preempt while sampling processor id. In-Reply-To: <151363810776.11899.1696679196545013564.stgit@noble> References: <151363798156.11899.768707943431676479.stgit@noble> <151363810776.11899.1696679196545013564.stgit@noble> Message-ID: <124DA88F-854D-41B0-9A3C-84AE37D4DD24@intel.com> On Dec 18, 2017, at 16:01, NeilBrown wrote: > > Calling smp_processor_id() without disabling preemption > triggers a warning (if CONFIG_DEBUG_PREEMPT). > I think the result of cfs_cpt_current() is only used as a hint for > load balancing, rather than as a precise and stable indicator of > the current CPU. So it doesn't need to be called with > preemption disabled. > > So disable preemption inside cfs_cpt_current() to silence the warning. > > Signed-off-by: NeilBrown Reviewed-by: Andreas Dilger > --- > .../staging/lustre/lnet/libcfs/linux/linux-cpu.c | 13 +++++++------ > 1 file changed, 7 insertions(+), 6 deletions(-) > > diff --git a/drivers/staging/lustre/lnet/libcfs/linux/linux-cpu.c b/drivers/staging/lustre/lnet/libcfs/linux/linux-cpu.c > index d30650f8dcb4..ca8518b8a3e0 100644 > --- a/drivers/staging/lustre/lnet/libcfs/linux/linux-cpu.c > +++ b/drivers/staging/lustre/lnet/libcfs/linux/linux-cpu.c > @@ -517,19 +517,20 @@ EXPORT_SYMBOL(cfs_cpt_spread_node); > int > cfs_cpt_current(struct cfs_cpt_table *cptab, int remap) > { > - int cpu = smp_processor_id(); > - int cpt = cptab->ctb_cpu2cpt[cpu]; > + int cpu; > + int cpt; > > - if (cpt < 0) { > - if (!remap) > - return cpt; > + preempt_disable(); > + cpu = smp_processor_id(); > + cpt = cptab->ctb_cpu2cpt[cpu]; > > + if (cpt < 0 && remap) { > /* don't return negative value for safety of upper layer, > * instead we shadow the unknown cpu to a valid partition ID > */ > cpt = cpu % cptab->ctb_nparts; > } > - > + preempt_enable(); > return cpt; > } > EXPORT_SYMBOL(cfs_cpt_current); > > Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Intel Corporation From andreas.dilger at intel.com Tue Dec 19 23:38:08 2017 From: andreas.dilger at intel.com (Dilger, Andreas) Date: Tue, 19 Dec 2017 23:38:08 +0000 Subject: [lustre-devel] [PATCH 1/2] staging: lustre: use strim instead of cfs_trimwhite. In-Reply-To: <151363810769.11899.15503930546886979038.stgit@noble> References: <151363798156.11899.768707943431676479.stgit@noble> <151363810769.11899.15503930546886979038.stgit@noble> Message-ID: <17ADFDA6-8E70-4019-B749-B095F56F3DC0@intel.com> On Dec 18, 2017, at 16:01, NeilBrown wrote: > > Linux lib provides identical functionality to cfs_trimwhite, > so discard that code and use the standard. > > Signed-off-by: NeilBrown Reviewed-by: Andreas Dilger > --- > .../lustre/include/linux/libcfs/libcfs_string.h | 1 - > drivers/staging/lustre/lnet/libcfs/libcfs_string.c | 20 -------------------- > .../staging/lustre/lnet/libcfs/linux/linux-cpu.c | 8 ++++---- > drivers/staging/lustre/lnet/lnet/config.c | 10 +++++----- > drivers/staging/lustre/lnet/lnet/router_proc.c | 2 +- > 5 files changed, 10 insertions(+), 31 deletions(-) > > diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_string.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_string.h > index c1375733ff31..66463477074a 100644 > --- a/drivers/staging/lustre/include/linux/libcfs/libcfs_string.h > +++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_string.h > @@ -73,7 +73,6 @@ struct cfs_expr_list { > struct list_head el_exprs; > }; > > -char *cfs_trimwhite(char *str); > int cfs_gettok(struct cfs_lstr *next, char delim, struct cfs_lstr *res); > int cfs_str2num_check(char *str, int nob, unsigned int *num, > unsigned int min, unsigned int max); > diff --git a/drivers/staging/lustre/lnet/libcfs/libcfs_string.c b/drivers/staging/lustre/lnet/libcfs/libcfs_string.c > index b1d8faa3f7aa..442889a3d729 100644 > --- a/drivers/staging/lustre/lnet/libcfs/libcfs_string.c > +++ b/drivers/staging/lustre/lnet/libcfs/libcfs_string.c > @@ -137,26 +137,6 @@ char *cfs_firststr(char *str, size_t size) > } > EXPORT_SYMBOL(cfs_firststr); > > -char * > -cfs_trimwhite(char *str) > -{ > - char *end; > - > - while (isspace(*str)) > - str++; > - > - end = str + strlen(str); > - while (end > str) { > - if (!isspace(end[-1])) > - break; > - end--; > - } > - > - *end = 0; > - return str; > -} > -EXPORT_SYMBOL(cfs_trimwhite); > - > /** > * Extracts tokens from strings. > * > diff --git a/drivers/staging/lustre/lnet/libcfs/linux/linux-cpu.c b/drivers/staging/lustre/lnet/libcfs/linux/linux-cpu.c > index e9156bf05ed4..d30650f8dcb4 100644 > --- a/drivers/staging/lustre/lnet/libcfs/linux/linux-cpu.c > +++ b/drivers/staging/lustre/lnet/libcfs/linux/linux-cpu.c > @@ -818,7 +818,7 @@ cfs_cpt_table_create_pattern(char *pattern) > int c; > int i; > > - str = cfs_trimwhite(pattern); > + str = strim(pattern); > if (*str == 'n' || *str == 'N') { > pattern = str + 1; > if (*pattern != '\0') { > @@ -870,7 +870,7 @@ cfs_cpt_table_create_pattern(char *pattern) > > high = node ? MAX_NUMNODES - 1 : nr_cpu_ids - 1; > > - for (str = cfs_trimwhite(pattern), c = 0;; c++) { > + for (str = strim(pattern), c = 0;; c++) { > struct cfs_range_expr *range; > struct cfs_expr_list *el; > char *bracket = strchr(str, '['); > @@ -905,7 +905,7 @@ cfs_cpt_table_create_pattern(char *pattern) > goto failed; > } > > - str = cfs_trimwhite(str + n); > + str = strim(str + n); > if (str != bracket) { > CERROR("Invalid pattern %s\n", str); > goto failed; > @@ -945,7 +945,7 @@ cfs_cpt_table_create_pattern(char *pattern) > goto failed; > } > > - str = cfs_trimwhite(bracket + 1); > + str = strim(bracket + 1); > } > > return cptab; > diff --git a/drivers/staging/lustre/lnet/lnet/config.c b/drivers/staging/lustre/lnet/lnet/config.c > index fd53c74766a7..44eeca63f458 100644 > --- a/drivers/staging/lustre/lnet/lnet/config.c > +++ b/drivers/staging/lustre/lnet/lnet/config.c > @@ -269,7 +269,7 @@ lnet_parse_networks(struct list_head *nilist, char *networks) > > if (comma) > *comma++ = 0; > - net = libcfs_str2net(cfs_trimwhite(str)); > + net = libcfs_str2net(strim(str)); > > if (net == LNET_NIDNET(LNET_NID_ANY)) { > LCONSOLE_ERROR_MSG(0x113, > @@ -292,7 +292,7 @@ lnet_parse_networks(struct list_head *nilist, char *networks) > } > > *bracket = 0; > - net = libcfs_str2net(cfs_trimwhite(str)); > + net = libcfs_str2net(strim(str)); > if (net == LNET_NIDNET(LNET_NID_ANY)) { > tmp = str; > goto failed_syntax; > @@ -322,7 +322,7 @@ lnet_parse_networks(struct list_head *nilist, char *networks) > if (comma) > *comma++ = 0; > > - iface = cfs_trimwhite(iface); > + iface = strim(iface); > if (!*iface) { > tmp = iface; > goto failed_syntax; > @@ -356,7 +356,7 @@ lnet_parse_networks(struct list_head *nilist, char *networks) > comma = strchr(bracket + 1, ','); > if (comma) { > *comma = 0; > - str = cfs_trimwhite(str); > + str = strim(str); > if (*str) { > tmp = str; > goto failed_syntax; > @@ -365,7 +365,7 @@ lnet_parse_networks(struct list_head *nilist, char *networks) > continue; > } > > - str = cfs_trimwhite(str); > + str = strim(str); > if (*str) { > tmp = str; > goto failed_syntax; > diff --git a/drivers/staging/lustre/lnet/lnet/router_proc.c b/drivers/staging/lustre/lnet/lnet/router_proc.c > index 8d6d6b4d6619..1a71ffebc889 100644 > --- a/drivers/staging/lustre/lnet/lnet/router_proc.c > +++ b/drivers/staging/lustre/lnet/lnet/router_proc.c > @@ -829,7 +829,7 @@ static int __proc_lnet_portal_rotor(void *data, int write, > if (rc < 0) > goto out; > > - tmp = cfs_trimwhite(buf); > + tmp = strim(buf); > > rc = -EINVAL; > lnet_res_lock(0); > > Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Intel Corporation From lkp at intel.com Wed Dec 20 04:12:24 2017 From: lkp at intel.com (kbuild test robot) Date: Wed, 20 Dec 2017 12:12:24 +0800 Subject: [lustre-devel] [PATCH 03/15] staging: lustre: replace simple cases of LIBCFS_ALLOC with kzalloc. In-Reply-To: <151355799030.6200.8100283223734974485.stgit@noble> References: <151355799030.6200.8100283223734974485.stgit@noble> Message-ID: <201712201243.tbG54o5k%fengguang.wu@intel.com> Hi NeilBrown, Thank you for the patch! Yet something to improve: [auto build test ERROR on staging/staging-testing] [also build test ERROR on next-20171219] [cannot apply to v4.15-rc4] [if your patch is applied to the wrong git tree, please drop us a note to help improve the system] url: https://github.com/0day-ci/linux/commits/NeilBrown/staging-lustre-convert-most-LIBCFS-ALLOC-to-k-malloc/20171220-113029 config: i386-randconfig-x009-201751 (attached as .config) compiler: gcc-7 (Debian 7.2.0-12) 7.2.1 20171025 reproduce: # save the attached .config to linux build tree make ARCH=i386 All errors (new ones prefixed by >>): drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c: In function 'kiblnd_dev_failover': >> drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c:2395:2: error: 'kdev' undeclared (first use in this function); did you mean 'hdev'? kdev = kzalloc(sizeof(*hdev), GFP_NOFS); ^~~~ hdev drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c:2395:2: note: each undeclared identifier is reported only once for each function it appears in vim +2395 drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c 2329 2330 int kiblnd_dev_failover(struct kib_dev *dev) 2331 { 2332 LIST_HEAD(zombie_tpo); 2333 LIST_HEAD(zombie_ppo); 2334 LIST_HEAD(zombie_fpo); 2335 struct rdma_cm_id *cmid = NULL; 2336 struct kib_hca_dev *hdev = NULL; 2337 struct ib_pd *pd; 2338 struct kib_net *net; 2339 struct sockaddr_in addr; 2340 unsigned long flags; 2341 int rc = 0; 2342 int i; 2343 2344 LASSERT(*kiblnd_tunables.kib_dev_failover > 1 || 2345 dev->ibd_can_failover || !dev->ibd_hdev); 2346 2347 rc = kiblnd_dev_need_failover(dev); 2348 if (rc <= 0) 2349 goto out; 2350 2351 if (dev->ibd_hdev && 2352 dev->ibd_hdev->ibh_cmid) { 2353 /* 2354 * XXX it's not good to close old listener at here, 2355 * because we can fail to create new listener. 2356 * But we have to close it now, otherwise rdma_bind_addr 2357 * will return EADDRINUSE... How crap! 2358 */ 2359 write_lock_irqsave(&kiblnd_data.kib_global_lock, flags); 2360 2361 cmid = dev->ibd_hdev->ibh_cmid; 2362 /* 2363 * make next schedule of kiblnd_dev_need_failover() 2364 * return 1 for me 2365 */ 2366 dev->ibd_hdev->ibh_cmid = NULL; 2367 write_unlock_irqrestore(&kiblnd_data.kib_global_lock, flags); 2368 2369 rdma_destroy_id(cmid); 2370 } 2371 2372 cmid = kiblnd_rdma_create_id(kiblnd_cm_callback, dev, RDMA_PS_TCP, 2373 IB_QPT_RC); 2374 if (IS_ERR(cmid)) { 2375 rc = PTR_ERR(cmid); 2376 CERROR("Failed to create cmid for failover: %d\n", rc); 2377 goto out; 2378 } 2379 2380 memset(&addr, 0, sizeof(addr)); 2381 addr.sin_family = AF_INET; 2382 addr.sin_addr.s_addr = htonl(dev->ibd_ifip); 2383 addr.sin_port = htons(*kiblnd_tunables.kib_service); 2384 2385 /* Bind to failover device or port */ 2386 rc = rdma_bind_addr(cmid, (struct sockaddr *)&addr); 2387 if (rc || !cmid->device) { 2388 CERROR("Failed to bind %s:%pI4h to device(%p): %d\n", 2389 dev->ibd_ifname, &dev->ibd_ifip, 2390 cmid->device, rc); 2391 rdma_destroy_id(cmid); 2392 goto out; 2393 } 2394 > 2395 kdev = kzalloc(sizeof(*hdev), GFP_NOFS); 2396 if (!hdev) { 2397 CERROR("Failed to allocate kib_hca_dev\n"); 2398 rdma_destroy_id(cmid); 2399 rc = -ENOMEM; 2400 goto out; 2401 } 2402 2403 atomic_set(&hdev->ibh_ref, 1); 2404 hdev->ibh_dev = dev; 2405 hdev->ibh_cmid = cmid; 2406 hdev->ibh_ibdev = cmid->device; 2407 2408 pd = ib_alloc_pd(cmid->device, 0); 2409 if (IS_ERR(pd)) { 2410 rc = PTR_ERR(pd); 2411 CERROR("Can't allocate PD: %d\n", rc); 2412 goto out; 2413 } 2414 2415 hdev->ibh_pd = pd; 2416 2417 rc = rdma_listen(cmid, 0); 2418 if (rc) { 2419 CERROR("Can't start new listener: %d\n", rc); 2420 goto out; 2421 } 2422 2423 rc = kiblnd_hdev_get_attr(hdev); 2424 if (rc) { 2425 CERROR("Can't get device attributes: %d\n", rc); 2426 goto out; 2427 } 2428 2429 write_lock_irqsave(&kiblnd_data.kib_global_lock, flags); 2430 2431 swap(dev->ibd_hdev, hdev); /* take over the refcount */ 2432 2433 list_for_each_entry(net, &dev->ibd_nets, ibn_list) { 2434 cfs_cpt_for_each(i, lnet_cpt_table()) { 2435 kiblnd_fail_poolset(&net->ibn_tx_ps[i]->tps_poolset, 2436 &zombie_tpo); 2437 2438 if (net->ibn_fmr_ps) 2439 kiblnd_fail_fmr_poolset(net->ibn_fmr_ps[i], 2440 &zombie_fpo); 2441 } 2442 } 2443 2444 write_unlock_irqrestore(&kiblnd_data.kib_global_lock, flags); 2445 out: 2446 if (!list_empty(&zombie_tpo)) 2447 kiblnd_destroy_pool_list(&zombie_tpo); 2448 if (!list_empty(&zombie_ppo)) 2449 kiblnd_destroy_pool_list(&zombie_ppo); 2450 if (!list_empty(&zombie_fpo)) 2451 kiblnd_destroy_fmr_pool_list(&zombie_fpo); 2452 if (hdev) 2453 kiblnd_hdev_decref(hdev); 2454 2455 if (rc) 2456 dev->ibd_failed_failover++; 2457 else 2458 dev->ibd_failed_failover = 0; 2459 2460 return rc; 2461 } 2462 --- 0-DAY kernel test infrastructure Open Source Technology Center https://lists.01.org/pipermail/kbuild-all Intel Corporation -------------- next part -------------- A non-text attachment was scrubbed... Name: .config.gz Type: application/gzip Size: 30413 bytes Desc: not available URL: From lkp at intel.com Wed Dec 20 07:14:30 2017 From: lkp at intel.com (kbuild test robot) Date: Wed, 20 Dec 2017 15:14:30 +0800 Subject: [lustre-devel] [PATCH 03/15] staging: lustre: replace simple cases of LIBCFS_ALLOC with kzalloc. In-Reply-To: <151355799030.6200.8100283223734974485.stgit@noble> References: <151355799030.6200.8100283223734974485.stgit@noble> Message-ID: <201712201404.5ovmUtOH%fengguang.wu@intel.com> Hi NeilBrown, Thank you for the patch! Yet something to improve: [auto build test ERROR on staging/staging-testing] [also build test ERROR on next-20171220] [cannot apply to v4.15-rc4] [if your patch is applied to the wrong git tree, please drop us a note to help improve the system] url: https://github.com/0day-ci/linux/commits/NeilBrown/staging-lustre-convert-most-LIBCFS-ALLOC-to-k-malloc/20171220-113029 config: x86_64-randconfig-r0-12200451 (attached as .config) compiler: gcc-6 (Debian 6.4.0-9) 6.4.0 20171026 reproduce: # save the attached .config to linux build tree make ARCH=x86_64 All errors (new ones prefixed by >>): drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c: In function 'kiblnd_dev_failover': >> drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c:2395:2: error: 'kdev' undeclared (first use in this function) kdev = kzalloc(sizeof(*hdev), GFP_NOFS); ^~~~ drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c:2395:2: note: each undeclared identifier is reported only once for each function it appears in vim +/kdev +2395 drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c 2329 2330 int kiblnd_dev_failover(struct kib_dev *dev) 2331 { 2332 LIST_HEAD(zombie_tpo); 2333 LIST_HEAD(zombie_ppo); 2334 LIST_HEAD(zombie_fpo); 2335 struct rdma_cm_id *cmid = NULL; 2336 struct kib_hca_dev *hdev = NULL; 2337 struct ib_pd *pd; 2338 struct kib_net *net; 2339 struct sockaddr_in addr; 2340 unsigned long flags; 2341 int rc = 0; 2342 int i; 2343 2344 LASSERT(*kiblnd_tunables.kib_dev_failover > 1 || 2345 dev->ibd_can_failover || !dev->ibd_hdev); 2346 2347 rc = kiblnd_dev_need_failover(dev); 2348 if (rc <= 0) 2349 goto out; 2350 2351 if (dev->ibd_hdev && 2352 dev->ibd_hdev->ibh_cmid) { 2353 /* 2354 * XXX it's not good to close old listener at here, 2355 * because we can fail to create new listener. 2356 * But we have to close it now, otherwise rdma_bind_addr 2357 * will return EADDRINUSE... How crap! 2358 */ 2359 write_lock_irqsave(&kiblnd_data.kib_global_lock, flags); 2360 2361 cmid = dev->ibd_hdev->ibh_cmid; 2362 /* 2363 * make next schedule of kiblnd_dev_need_failover() 2364 * return 1 for me 2365 */ 2366 dev->ibd_hdev->ibh_cmid = NULL; 2367 write_unlock_irqrestore(&kiblnd_data.kib_global_lock, flags); 2368 2369 rdma_destroy_id(cmid); 2370 } 2371 2372 cmid = kiblnd_rdma_create_id(kiblnd_cm_callback, dev, RDMA_PS_TCP, 2373 IB_QPT_RC); 2374 if (IS_ERR(cmid)) { 2375 rc = PTR_ERR(cmid); 2376 CERROR("Failed to create cmid for failover: %d\n", rc); 2377 goto out; 2378 } 2379 2380 memset(&addr, 0, sizeof(addr)); 2381 addr.sin_family = AF_INET; 2382 addr.sin_addr.s_addr = htonl(dev->ibd_ifip); 2383 addr.sin_port = htons(*kiblnd_tunables.kib_service); 2384 2385 /* Bind to failover device or port */ 2386 rc = rdma_bind_addr(cmid, (struct sockaddr *)&addr); 2387 if (rc || !cmid->device) { 2388 CERROR("Failed to bind %s:%pI4h to device(%p): %d\n", 2389 dev->ibd_ifname, &dev->ibd_ifip, 2390 cmid->device, rc); 2391 rdma_destroy_id(cmid); 2392 goto out; 2393 } 2394 > 2395 kdev = kzalloc(sizeof(*hdev), GFP_NOFS); 2396 if (!hdev) { 2397 CERROR("Failed to allocate kib_hca_dev\n"); 2398 rdma_destroy_id(cmid); 2399 rc = -ENOMEM; 2400 goto out; 2401 } 2402 2403 atomic_set(&hdev->ibh_ref, 1); 2404 hdev->ibh_dev = dev; 2405 hdev->ibh_cmid = cmid; 2406 hdev->ibh_ibdev = cmid->device; 2407 2408 pd = ib_alloc_pd(cmid->device, 0); 2409 if (IS_ERR(pd)) { 2410 rc = PTR_ERR(pd); 2411 CERROR("Can't allocate PD: %d\n", rc); 2412 goto out; 2413 } 2414 2415 hdev->ibh_pd = pd; 2416 2417 rc = rdma_listen(cmid, 0); 2418 if (rc) { 2419 CERROR("Can't start new listener: %d\n", rc); 2420 goto out; 2421 } 2422 2423 rc = kiblnd_hdev_get_attr(hdev); 2424 if (rc) { 2425 CERROR("Can't get device attributes: %d\n", rc); 2426 goto out; 2427 } 2428 2429 write_lock_irqsave(&kiblnd_data.kib_global_lock, flags); 2430 2431 swap(dev->ibd_hdev, hdev); /* take over the refcount */ 2432 2433 list_for_each_entry(net, &dev->ibd_nets, ibn_list) { 2434 cfs_cpt_for_each(i, lnet_cpt_table()) { 2435 kiblnd_fail_poolset(&net->ibn_tx_ps[i]->tps_poolset, 2436 &zombie_tpo); 2437 2438 if (net->ibn_fmr_ps) 2439 kiblnd_fail_fmr_poolset(net->ibn_fmr_ps[i], 2440 &zombie_fpo); 2441 } 2442 } 2443 2444 write_unlock_irqrestore(&kiblnd_data.kib_global_lock, flags); 2445 out: 2446 if (!list_empty(&zombie_tpo)) 2447 kiblnd_destroy_pool_list(&zombie_tpo); 2448 if (!list_empty(&zombie_ppo)) 2449 kiblnd_destroy_pool_list(&zombie_ppo); 2450 if (!list_empty(&zombie_fpo)) 2451 kiblnd_destroy_fmr_pool_list(&zombie_fpo); 2452 if (hdev) 2453 kiblnd_hdev_decref(hdev); 2454 2455 if (rc) 2456 dev->ibd_failed_failover++; 2457 else 2458 dev->ibd_failed_failover = 0; 2459 2460 return rc; 2461 } 2462 --- 0-DAY kernel test infrastructure Open Source Technology Center https://lists.01.org/pipermail/kbuild-all Intel Corporation -------------- next part -------------- A non-text attachment was scrubbed... Name: .config.gz Type: application/gzip Size: 30875 bytes Desc: not available URL: From lkp at intel.com Thu Dec 21 09:40:52 2017 From: lkp at intel.com (kbuild test robot) Date: Thu, 21 Dec 2017 17:40:52 +0800 Subject: [lustre-devel] [PATCH 03/15] staging: lustre: replace simple cases of LIBCFS_ALLOC with kzalloc. In-Reply-To: <151355799030.6200.8100283223734974485.stgit@noble> References: <151355799030.6200.8100283223734974485.stgit@noble> Message-ID: <201712211744.mY2ILpU6%fengguang.wu@intel.com> Hi NeilBrown, Thank you for the patch! Perhaps something to improve: [auto build test WARNING on staging/staging-testing] [also build test WARNING on next-20171221] [cannot apply to v4.15-rc4] [if your patch is applied to the wrong git tree, please drop us a note to help improve the system] url: https://github.com/0day-ci/linux/commits/NeilBrown/staging-lustre-convert-most-LIBCFS-ALLOC-to-k-malloc/20171220-113029 coccinelle warnings: (new ones prefixed by >>) >> drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c:886:2-7: WARNING: NULL check before freeing functions like kfree, debugfs_remove, debugfs_remove_recursive or usb_free_urb is not needed. Maybe consider reorganizing relevant code to avoid passing NULL values. Please review and possibly fold the followup patch. --- 0-DAY kernel test infrastructure Open Source Technology Center https://lists.01.org/pipermail/kbuild-all Intel Corporation From lkp at intel.com Thu Dec 21 09:40:53 2017 From: lkp at intel.com (kbuild test robot) Date: Thu, 21 Dec 2017 17:40:53 +0800 Subject: [lustre-devel] [PATCH] staging: lustre: fix ifnullfree.cocci warnings In-Reply-To: <151355799030.6200.8100283223734974485.stgit@noble> References: <151355799030.6200.8100283223734974485.stgit@noble> Message-ID: <20171221094052.GA9993@lkp-wsx02> From: Fengguang Wu drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c:886:2-7: WARNING: NULL check before freeing functions like kfree, debugfs_remove, debugfs_remove_recursive or usb_free_urb is not needed. Maybe consider reorganizing relevant code to avoid passing NULL values. NULL check before some freeing functions is not needed. Based on checkpatch warning "kfree(NULL) is safe this check is probably not required" and kfreeaddr.cocci by Julia Lawall. Generated by: scripts/coccinelle/free/ifnullfree.cocci Fixes: 862af72835c1 ("staging: lustre: replace simple cases of LIBCFS_ALLOC with kzalloc.") CC: NeilBrown Signed-off-by: Fengguang Wu --- o2iblnd.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) --- a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c +++ b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c @@ -882,8 +882,7 @@ void kiblnd_destroy_conn(struct kib_conn IBLND_RX_MSGS(conn) * sizeof(struct kib_rx)); } - if (conn->ibc_connvars) - kfree(conn->ibc_connvars); + kfree(conn->ibc_connvars); if (conn->ibc_hdev) kiblnd_hdev_decref(conn->ibc_hdev); From neilb at suse.com Fri Dec 22 03:11:03 2017 From: neilb at suse.com (NeilBrown) Date: Fri, 22 Dec 2017 14:11:03 +1100 Subject: [lustre-devel] [PATCH 0/2] Add new wait_event macros to support lustre Message-ID: <151391203805.23157.8332034037737592042.stgit@noble> Lustre has its own l_wait_event() macro to support waiting. Some uses can be converted to current wait_event macros, but others cannot. Particularly it needs to be able to wait in TASK_IDLE state, and needs exclusive waiters to be placed at the head of the queue. These two patches add required functionalty to wait.h so that lustre can be fully moved away from l_wait_event() Thanks, NeilBrown --- NeilBrown (2): sched/wait: add wait_event_idle() functions. sched/wait: add wait_event_idle_exclusive_lifo() include/linux/wait.h | 170 +++++++++++++++++++++++++++++++++++++++++++++++++- kernel/sched/wait.c | 3 + 2 files changed, 167 insertions(+), 6 deletions(-) -- Signature From neilb at suse.com Fri Dec 22 03:11:04 2017 From: neilb at suse.com (NeilBrown) Date: Fri, 22 Dec 2017 14:11:04 +1100 Subject: [lustre-devel] [PATCH 2/2] sched/wait: add wait_event_idle_exclusive_lifo() In-Reply-To: <151391203805.23157.8332034037737592042.stgit@noble> References: <151391203805.23157.8332034037737592042.stgit@noble> Message-ID: <151391226407.23157.14719462856574834182.stgit@noble> wait_event_*_exclusive() adds new waiters to the end of the quest, while non-exclusive wait_event adds to the head. This ensures that a wake_up will wake all non-exclusive waiters and at most one exclusive wait, but it means that exclusive waiters are woken in a FIFO order, so the task woken is the one least likely to have data in the CPU cache. When simple interaction with non-exclusive waiters is not important, and when choosing a cache-hot task is, the new wait_event_idle_exclusive_lifo() and wait_event_idle_exclusive_lifo_timeout() can be used. To implement these we introduce a new WQ_FLAG_LIFO which causes prepare_to_wait_event() to add to the head of the queue. This will be used to allow lustre's l_wait_event() to be replaced with more standard wait.h macros. Signed-off-by: NeilBrown --- include/linux/wait.h | 95 +++++++++++++++++++++++++++++++++++++++++++++++--- kernel/sched/wait.c | 3 +- 2 files changed, 91 insertions(+), 7 deletions(-) diff --git a/include/linux/wait.h b/include/linux/wait.h index 3aea0780c9d0..49cb393c53d5 100644 --- a/include/linux/wait.h +++ b/include/linux/wait.h @@ -20,6 +20,9 @@ int default_wake_function(struct wait_queue_entry *wq_entry, unsigned mode, int #define WQ_FLAG_EXCLUSIVE 0x01 #define WQ_FLAG_WOKEN 0x02 #define WQ_FLAG_BOOKMARK 0x04 +#define WQ_FLAG_LIFO 0x08 /* used with WQ_FLAG_EXCLUSIVE to force + * LIFO scheduling in prepare_to_wait_event(). + */ /* * A single wait-queue entry structure: @@ -247,7 +250,7 @@ extern void init_wait_entry(struct wait_queue_entry *wq_entry, int flags); struct wait_queue_entry __wq_entry; \ long __ret = ret; /* explicit shadow */ \ \ - init_wait_entry(&__wq_entry, exclusive ? WQ_FLAG_EXCLUSIVE : 0); \ + init_wait_entry(&__wq_entry, exclusive); \ for (;;) { \ long __int = prepare_to_wait_event(&wq_head, &__wq_entry, state);\ \ @@ -381,7 +384,8 @@ do { \ }) #define __wait_event_exclusive_cmd(wq_head, condition, cmd1, cmd2) \ - (void)___wait_event(wq_head, condition, TASK_UNINTERRUPTIBLE, 1, 0, \ + (void)___wait_event(wq_head, condition, TASK_UNINTERRUPTIBLE, \ + WQ_FLAG_EXCLUSIVE, 0, \ cmd1; schedule(); cmd2) /* * Just like wait_event_cmd(), except it sets exclusive flag @@ -558,7 +562,7 @@ do { \ }) #define __wait_event_interruptible_exclusive(wq, condition) \ - ___wait_event(wq, condition, TASK_INTERRUPTIBLE, 1, 0, \ + ___wait_event(wq, condition, TASK_INTERRUPTIBLE, WQ_FLAG_EXCLUSIVE, 0, \ schedule()) #define wait_event_interruptible_exclusive(wq, condition) \ @@ -571,7 +575,7 @@ do { \ }) #define __wait_event_killable_exclusive(wq, condition) \ - ___wait_event(wq, condition, TASK_KILLABLE, 1, 0, \ + ___wait_event(wq, condition, TASK_KILLABLE, WQ_FLAG_EXCLUSIVE, 0, \ schedule()) #define wait_event_killable_exclusive(wq, condition) \ @@ -585,7 +589,7 @@ do { \ #define __wait_event_freezable_exclusive(wq, condition) \ - ___wait_event(wq, condition, TASK_INTERRUPTIBLE, 1, 0, \ + ___wait_event(wq, condition, TASK_INTERRUPTIBLE, WQ_FLAG_EXCLUSIVE, 0, \ schedule(); try_to_freeze()) #define wait_event_freezable_exclusive(wq, condition) \ @@ -638,9 +642,88 @@ do { \ do { \ might_sleep(); \ if (!(condition)) \ - ___wait_event(wq_head, condition, TASK_IDLE, 1, 0, schedule()); \ + ___wait_event(wq_head, condition, TASK_IDLE, WQ_FLAG_EXCLUSIVE, \ + 0, schedule()); \ } while (0) +/** + * wait_event_idle_exclusive_lifo - wait for a condition without contributing to system load + * @wq_head: the waitqueue to wait on + * @condition: a C expression for the event to wait for + * + * The process is put to sleep (TASK_IDLE) until the + * @condition evaluates to true. + * The @condition is checked each time the waitqueue @wq_head is woken up. + * + * The process is put on the wait queue with an WQ_FLAG_EXCLUSIVE flag + * set thus when other process waits process on the list if this + * process is awaken further processes are not considered. + * + * Contrary to the usual practice with exclusive wait, this call adds + * the task to the head of the queue so that tasks are woken in a + * LIFO (rather than FIFO) order. This means that if both exclusive and + * non-exclusive waiter are waiting on the same queue, the non-exclusive + * waiters may *not* be woken on the next wakeup event. The benefit + * of using LIFO waits is that when multiple worker threads are + * available, the one with the warmest cache will preferentially + * be woken. + * + * wake_up() has to be called after changing any variable that could + * change the result of the wait condition. + * + */ +#define wait_event_idle_exclusive_lifo(wq_head, condition) \ +do { \ + might_sleep(); \ + if (!(condition)) \ + ___wait_event(wq_head, condition, TASK_IDLE, \ + WQ_FLAG_EXCLUSIVE | WQ_FLAG_LIFO, \ + 0, schedule()); \ +} while (0) + +/** + * wait_event_idle_exclusive_lifo_timeout - wait for a condition with timeout, without contributing to system load + * @wq_head: the waitqueue to wait on + * @condition: a C expression for the event to wait for + * @timeout: timeout, in jiffies + * + * The process is put to sleep (TASK_IDLE) until the + * @condition evaluates to true. + * The @condition is checked each time the waitqueue @wq_head is woken up. + * + * The process is put on the wait queue with an WQ_FLAG_EXCLUSIVE flag + * set thus when other process waits process on the list if this + * process is awaken further processes are not considered. + * + * Contrary to the usual practice with exclusive wait, this call adds + * the task to the head of the queue so that tasks are woken in a + * LIFO (rather than FIFO) order. This means that if both exclusive and + * non-exclusive waiter are waiting on the same queue, the non-exclusive + * waiters may *not* be woken on the next wakeup event. The benefit + * of using LIFO waits is that when multiple worker threads are + * available, the one with the warmest cache will preferentially + * be woken. + * + * wake_up() has to be called after changing any variable that could + * change the result of the wait condition. + * + * Returns: + * 0 if the @condition evaluated to %false after the @timeout elapsed, + * 1 if the @condition evaluated to %true after the @timeout elapsed, + * or the remaining jiffies (at least 1) if the @condition evaluated + * to %true before the @timeout elapsed. + */ +#define wait_event_idle_exclusive_lifo_timeout(wq_head, condition, timeout) \ +({ \ + long __ret = timeout; \ + might_sleep(); \ + if (!___wait_cond_timeout(condition)) \ + __ret = ___wait_event(wq_head, ___wait_cond_timeout(condition), TASK_IDLE, \ + WQ_FLAG_EXCLUSIVE | WQ_FLAG_LIFO, \ + timeout, __ret = schedule_timeout(__ret)); \ + __ret; \ +}) + #define __wait_event_idle_timeout(wq_head, condition, timeout) \ ___wait_event(wq_head, ___wait_cond_timeout(condition), \ TASK_IDLE, 0, timeout, \ diff --git a/kernel/sched/wait.c b/kernel/sched/wait.c index 929ecb7d6b78..a92f368acbb0 100644 --- a/kernel/sched/wait.c +++ b/kernel/sched/wait.c @@ -285,7 +285,8 @@ long prepare_to_wait_event(struct wait_queue_head *wq_head, struct wait_queue_en ret = -ERESTARTSYS; } else { if (list_empty(&wq_entry->entry)) { - if (wq_entry->flags & WQ_FLAG_EXCLUSIVE) + if ((wq_entry->flags & (WQ_FLAG_EXCLUSIVE | WQ_FLAG_LIFO)) == + WQ_FLAG_EXCLUSIVE) __add_wait_queue_entry_tail(wq_head, wq_entry); else __add_wait_queue(wq_head, wq_entry); From neilb at suse.com Fri Dec 22 03:11:04 2017 From: neilb at suse.com (NeilBrown) Date: Fri, 22 Dec 2017 14:11:04 +1100 Subject: [lustre-devel] [PATCH 1/2] sched/wait: add wait_event_idle() functions. In-Reply-To: <151391203805.23157.8332034037737592042.stgit@noble> References: <151391203805.23157.8332034037737592042.stgit@noble> Message-ID: <151391226402.23157.5260361528373791240.stgit@noble> The new TASK_IDLE state (TASK_UNINTERRUPTIBLE | __TASK_NOLOAD) is not much used. One way to make it easier to use is to add wait_event*() family functions that make use of it. This patch adds: wait_event_idle() wait_event_idle_timeout() wait_event_idle_exclusive() This set were chosen because lustre needs them before it can discard its own l_wait_event() macro. Signed-off-by: NeilBrown --- include/linux/wait.h | 77 ++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 77 insertions(+) diff --git a/include/linux/wait.h b/include/linux/wait.h index 158715445ffb..3aea0780c9d0 100644 --- a/include/linux/wait.h +++ b/include/linux/wait.h @@ -597,6 +597,83 @@ do { \ __ret; \ }) +/** + * wait_event_idle - wait for a condition with contributing to system load + * @wq_head: the waitqueue to wait on + * @condition: a C expression for the event to wait for + * + * The process is put to sleep (TASK_IDLE) until the + * @condition evaluates to true. + * The @condition is checked each time the waitqueue @wq_head is woken up. + * + * wake_up() has to be called after changing any variable that could + * change the result of the wait condition. + * + */ +#define wait_event_idle(wq_head, condition) \ +do { \ + might_sleep(); \ + if (!(condition)) \ + ___wait_event(wq_head, condition, TASK_IDLE, 0, 0, schedule()); \ +} while (0) + +/** + * wait_event_idle_exclusive - wait for a condition with contributing to system load + * @wq_head: the waitqueue to wait on + * @condition: a C expression for the event to wait for + * + * The process is put to sleep (TASK_IDLE) until the + * @condition evaluates to true. + * The @condition is checked each time the waitqueue @wq_head is woken up. + * + * The process is put on the wait queue with an WQ_FLAG_EXCLUSIVE flag + * set thus when other process waits process on the list if this + * process is awaken further processes are not considered. + * + * wake_up() has to be called after changing any variable that could + * change the result of the wait condition. + * + */ +#define wait_event_idle_exclusive(wq_head, condition) \ +do { \ + might_sleep(); \ + if (!(condition)) \ + ___wait_event(wq_head, condition, TASK_IDLE, 1, 0, schedule()); \ +} while (0) + +#define __wait_event_idle_timeout(wq_head, condition, timeout) \ + ___wait_event(wq_head, ___wait_cond_timeout(condition), \ + TASK_IDLE, 0, timeout, \ + __ret = schedule_timeout(__ret)) + +/** + * wait_event_idle_timeout - sleep without load until a condition gets true or a timeout elapses + * @wq_head: the waitqueue to wait on + * @condition: a C expression for the event to wait for + * @timeout: timeout, in jiffies + * + * The process is put to sleep (TASK_IDLE) until the + * @condition evaluates to true. The @condition is checked each time + * the waitqueue @wq_head is woken up. + * + * wake_up() has to be called after changing any variable that could + * change the result of the wait condition. + * + * Returns: + * 0 if the @condition evaluated to %false after the @timeout elapsed, + * 1 if the @condition evaluated to %true after the @timeout elapsed, + * or the remaining jiffies (at least 1) if the @condition evaluated + * to %true before the @timeout elapsed. + */ +#define wait_event_idle_timeout(wq_head, condition, timeout) \ +({ \ + long __ret = timeout; \ + might_sleep(); \ + if (!___wait_cond_timeout(condition)) \ + __ret = __wait_event_timeout(wq_head, condition, timeout); \ + __ret; \ +}) + extern int do_wait_intr(wait_queue_head_t *, wait_queue_entry_t *); extern int do_wait_intr_irq(wait_queue_head_t *, wait_queue_entry_t *); From andreas.dilger at intel.com Mon Dec 25 06:58:04 2017 From: andreas.dilger at intel.com (Dilger, Andreas) Date: Mon, 25 Dec 2017 06:58:04 +0000 Subject: [lustre-devel] [PATCH v4] staging: lustre: Replace 'uint32_t' with 'u32' and 'uint64_t' with 'u64' In-Reply-To: <20171219175654.GA13794@home> References: <20171219175654.GA13794@home> Message-ID: <42FB61E1-3C0B-44C8-999E-7FE17F55D821@intel.com> On Dec 19, 2017, at 10:56, Roman Storozhenko wrote: > > There are two reasons for replacing 'uint32_t' with 'u32' > and 'uint64_t' with 'u64': > > 1) As Linus Torvalds have said we should use kernel types: > http://lkml.iu.edu/hypermail//linux/kernel/1506.0/00160.html > > 2) There are only few places in the lustre codebase that use such types. > In the most cases it uses 'u32' and 'u64'. > > Signed-off-by: Roman Storozhenko Reviewed-by: Andreas Dilger > > --- > In the third version of that patch Dan Carpenter pointed out that the patch > body should be self-explanatory: > " >> There are two reasons for that: > What I'm asking is there are two reasons for what? Where is the first > part of that paragraph? > " > > In the second version of this patch I forgot to add subsystem and > driver. As Greg K-H mentioned: > "Please fix up the subject to have the subsystem and driver name in it: > Subject: [PATCH] staging: lustre: ..." > > In the first version of this patch I replaced 'uint32_t' with '__u32' and > 'uint64_t' with '__u64'. I was suggested to fix that by Greg K-H: > > "The __ types are only needed for when you cross the user/kernel boundry. > Otherwise just use the "normal" types of u32 and u64. > > Do the changes you made here all cross that boundry? If not, please fix > this up." > > I asked lustre community whether those code used only in the kernel > space and Andreas Dilger said: > > "These headers are for kernel code only, so should use the "u32" and > similar > types, rather than the "__u32" that are used for user-kernel > structures." > > So I have replaced my first patch version with this one. > > drivers/staging/lustre/lustre/include/lustre_sec.h | 4 ++-- > drivers/staging/lustre/lustre/llite/vvp_dev.c | 2 +- > drivers/staging/lustre/lustre/lov/lov_internal.h | 12 ++++++------ > drivers/staging/lustre/lustre/osc/osc_internal.h | 6 +++--- > 4 files changed, 12 insertions(+), 12 deletions(-) > > diff --git a/drivers/staging/lustre/lustre/include/lustre_sec.h b/drivers/staging/lustre/lustre/include/lustre_sec.h > index a40f706..64b6fd4 100644 > --- a/drivers/staging/lustre/lustre/include/lustre_sec.h > +++ b/drivers/staging/lustre/lustre/include/lustre_sec.h > @@ -341,8 +341,8 @@ void sptlrpc_conf_client_adapt(struct obd_device *obd); > #define SPTLRPC_MAX_PAYLOAD (1024) > > struct vfs_cred { > - uint32_t vc_uid; > - uint32_t vc_gid; > + u32 vc_uid; > + u32 vc_gid; > }; > > struct ptlrpc_ctx_ops { > diff --git a/drivers/staging/lustre/lustre/llite/vvp_dev.c b/drivers/staging/lustre/lustre/llite/vvp_dev.c > index 8ccc8b7..987c03b 100644 > --- a/drivers/staging/lustre/lustre/llite/vvp_dev.c > +++ b/drivers/staging/lustre/lustre/llite/vvp_dev.c > @@ -384,7 +384,7 @@ int cl_sb_fini(struct super_block *sb) > struct vvp_pgcache_id { > unsigned int vpi_bucket; > unsigned int vpi_depth; > - uint32_t vpi_index; > + u32 vpi_index; > > unsigned int vpi_curdep; > struct lu_object_header *vpi_obj; > diff --git a/drivers/staging/lustre/lustre/lov/lov_internal.h b/drivers/staging/lustre/lustre/lov/lov_internal.h > index ae28ddf..a56d71c 100644 > --- a/drivers/staging/lustre/lustre/lov/lov_internal.h > +++ b/drivers/staging/lustre/lustre/lov/lov_internal.h > @@ -115,19 +115,19 @@ static inline const struct lsm_operations *lsm_op_find(int magic) > */ > #if BITS_PER_LONG == 64 > # define lov_do_div64(n, base) ({ \ > - uint64_t __base = (base); \ > - uint64_t __rem; \ > - __rem = ((uint64_t)(n)) % __base; \ > - (n) = ((uint64_t)(n)) / __base; \ > + u64 __base = (base); \ > + u64 __rem; \ > + __rem = ((u64)(n)) % __base; \ > + (n) = ((u64)(n)) / __base; \ > __rem; \ > }) > #elif BITS_PER_LONG == 32 > # define lov_do_div64(n, base) ({ \ > - uint64_t __rem; \ > + u64 __rem; \ > if ((sizeof(base) > 4) && (((base) & 0xffffffff00000000ULL) != 0)) { \ > int __remainder; \ > LASSERTF(!((base) & (LOV_MIN_STRIPE_SIZE - 1)), "64 bit lov " \ > - "division %llu / %llu\n", (n), (uint64_t)(base)); \ > + "division %llu / %llu\n", (n), (u64)(base)); \ > __remainder = (n) & (LOV_MIN_STRIPE_SIZE - 1); \ > (n) >>= LOV_MIN_STRIPE_BITS; \ > __rem = do_div(n, (base) >> LOV_MIN_STRIPE_BITS); \ > diff --git a/drivers/staging/lustre/lustre/osc/osc_internal.h b/drivers/staging/lustre/lustre/osc/osc_internal.h > index feda61b..32db150 100644 > --- a/drivers/staging/lustre/lustre/osc/osc_internal.h > +++ b/drivers/staging/lustre/lustre/osc/osc_internal.h > @@ -168,9 +168,9 @@ struct osc_device { > > /* Write stats is actually protected by client_obd's lock. */ > struct osc_stats { > - uint64_t os_lockless_writes; /* by bytes */ > - uint64_t os_lockless_reads; /* by bytes */ > - uint64_t os_lockless_truncates; /* by times */ > + u64 os_lockless_writes; /* by bytes */ > + u64 os_lockless_reads; /* by bytes */ > + u64 os_lockless_truncates; /* by times */ > } od_stats; > > /* configuration item(s) */ > -- > 2.7.4 > > _______________________________________________ > lustre-devel mailing list > lustre-devel at lists.lustre.org > http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Intel Corporation