From sven.dziadek at gmx.de Sun Feb 4 11:24:05 2018 From: sven.dziadek at gmx.de (Sven Dziadek) Date: Sun, 4 Feb 2018 12:24:05 +0100 Subject: [lustre-devel] [PATCH] staging: lustre: llite: replace variable length array In-Reply-To: References: <20180127214228.26986-1-sven.dziadek@gmx.de> Message-ID: <58963c7f-139c-1156-0179-13decf0bcddf@gmx.de> On 01/30/2018 03:04 AM, Dilger, Andreas wrote: > On Jan 27, 2018, at 14:42, Sven Dziadek wrote: >> >> The functionality of the removed variable length array is already >> implemented by the function xattr_full_name in fs/xattr.c >> >> This fixes the sparse warning: >> warning: Variable length array is used. >> >> Signed-off-by: Sven Dziadek >> --- >> drivers/staging/lustre/lustre/llite/xattr.c | 12 ++++-------- >> 1 file changed, 4 insertions(+), 8 deletions(-) >> >> diff --git a/drivers/staging/lustre/lustre/llite/xattr.c b/drivers/staging/lustre/lustre/llite/xattr.c >> index 532384c91447..4fd28213c6a1 100644 >> --- a/drivers/staging/lustre/lustre/llite/xattr.c >> +++ b/drivers/staging/lustre/lustre/llite/xattr.c >> @@ -87,7 +87,6 @@ ll_xattr_set_common(const struct xattr_handler *handler, >> const char *name, const void *value, size_t size, >> int flags) >> { >> - char fullname[strlen(handler->prefix) + strlen(name) + 1]; >> struct ll_sb_info *sbi = ll_i2sbi(inode); >> struct ptlrpc_request *req = NULL; >> const char *pv = value; >> @@ -141,9 +140,8 @@ ll_xattr_set_common(const struct xattr_handler *handler, >> return -EPERM; >> } >> >> - sprintf(fullname, "%s%s\n", handler->prefix, name); >> - rc = md_setxattr(sbi->ll_md_exp, ll_inode2fid(inode), >> - valid, fullname, pv, size, 0, flags, >> + rc = md_setxattr(sbi->ll_md_exp, ll_inode2fid(inode), valid, >> + xattr_full_name(handler, name), pv, size, 0, flags, >> ll_i2suppgid(inode), &req); > > Hi Sven, > thanks for the patch. > > Looking at the details of "xattr_full_name()", this seems quite risky. This > is essentially returning the pointer _before_ "name" on the assumption that > it contains the full "prefix.name" string. IMHO, that is not necessarily a > safe assumption to make several layers down in the code. Thanks for your reply. And yeah, it feels strange, right. But it really looks like this is how the name and the prefix is handled in the xattr code. It seems this helper function has been introduced with commit e409de992e3ea (which also removes some fullname pointer) and indeed, fs/xattr.c splits the prefix and the name in xattr_resolve_name. So I still think the patch should be correct..? > James, I thought you had a patch for this to use kasprintf() instead of the > on-stack "fullname" declaration? Sorry for replying that late, I though James would maybe know if the above assumption is correct.. >> if (rc) { >> if (rc == -EOPNOTSUPP && handler->flags == XATTR_USER_T) { >> @@ -364,7 +362,6 @@ static int ll_xattr_get_common(const struct xattr_handler *handler, >> struct dentry *dentry, struct inode *inode, >> const char *name, void *buffer, size_t size) >> { >> - char fullname[strlen(handler->prefix) + strlen(name) + 1]; >> struct ll_sb_info *sbi = ll_i2sbi(inode); >> #ifdef CONFIG_FS_POSIX_ACL >> struct ll_inode_info *lli = ll_i2info(inode); >> @@ -411,9 +408,8 @@ static int ll_xattr_get_common(const struct xattr_handler *handler, >> if (handler->flags == XATTR_ACL_DEFAULT_T && !S_ISDIR(inode->i_mode)) >> return -ENODATA; >> #endif >> - sprintf(fullname, "%s%s\n", handler->prefix, name); >> - return ll_xattr_list(inode, fullname, handler->flags, buffer, size, >> - OBD_MD_FLXATTR); >> + return ll_xattr_list(inode, xattr_full_name(handler, name), >> + handler->flags, buffer, size, OBD_MD_FLXATTR); >> } >> >> static ssize_t ll_getxattr_lov(struct inode *inode, void *buf, size_t buf_size) >> -- >> 2.11.0 >> From neilb at suse.com Wed Feb 7 21:31:54 2018 From: neilb at suse.com (NeilBrown) Date: Thu, 08 Feb 2018 08:31:54 +1100 Subject: [lustre-devel] testing lustre In-Reply-To: References: <151538168618.23920.8261096424342988792.stgit@noble> <20180108145943.GA5761@kroah.com> Message-ID: <87po5gmqph.fsf@notabene.neil.brown.name> Hi, I've set up a little virtual cluster to run the sanity tests. One problem I have is that the test harness expects that after it creates a filesystem, the label (reported by e2label) will spontaneously change from e.g. "lustre:MDT-0000" to "lustre-MDT-0000" and that doesn't happen for me. If I go and change the label manually while the test is waiting, then the test continues and appears to run correctly. Can anyone tell me what this name change in meant to indicate, when where in the code it is expected to happen? Thanks, NeilBrown -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 832 bytes Desc: not available URL: From artem.blagodarenko at gmail.com Thu Feb 8 09:42:39 2018 From: artem.blagodarenko at gmail.com (=?utf-8?B?0JHQu9Cw0LPQvtC00LDRgNC10L3QutC+INCQ0YDRgtGR0Lw=?=) Date: Thu, 8 Feb 2018 12:42:39 +0300 Subject: [lustre-devel] testing lustre In-Reply-To: <87po5gmqph.fsf@notabene.neil.brown.name> References: <151538168618.23920.8261096424342988792.stgit@noble> <20180108145943.GA5761@kroah.com> <87po5gmqph.fsf@notabene.neil.brown.name> Message-ID: <94546355-FD93-4B50-AC98-4AB38EDC9BF0@gmail.com> Hello NeilBrown, Symbol “:” is added to name for targets which have never been registered. tunefs and mount use this symbol to set LDD_F_VIRGIN flag in ldd. Here is code from lustre/utils/mkfs_lustre.c (executed in tunefs) /* svname of the form lustre:OST1234 means never registered */ ret = strlen(ldd->ldd_svname); if (ldd->ldd_svname[ret - 8] == ':') { ldd->ldd_svname[ret - 8] = '-'; ldd->ldd_flags |= LDD_F_VIRGIN; And lustre/utils/mount_lustre.c /* svname of the form lustre:OST1234 means never registered */ rc = strlen(ldd->ldd_svname); if (strcmp(ldd->ldd_svname, "MGS") != 0) { if (rc < 8) { fprintf(stderr, "%s: invalid name '%s'\n", progname, ldd->ldd_svname); return EINVAL; } else if (ldd->ldd_svname[rc - 8] == ':') { ldd->ldd_svname[rc - 8] = '-'; ldd->ldd_flags |= LDD_F_VIRGIN; So I believe the test expects freshly created target. Best regards, Artem Blagodarenko. > On 8 Feb 2018, at 00:31, NeilBrown wrote: > > > Hi, > I've set up a little virtual cluster to run the sanity tests. > One problem I have is that the test harness expects that after it > creates a filesystem, the label (reported by e2label) will > spontaneously change from e.g. "lustre:MDT-0000" to "lustre-MDT-0000" > and that doesn't happen for me. > If I go and change the label manually while the test is waiting, > then the test continues and appears to run correctly. > > Can anyone tell me what this name change in meant to indicate, when > where in the code it is expected to happen? > > Thanks, > NeilBrown > _______________________________________________ > lustre-devel mailing list > lustre-devel at lists.lustre.org > http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org From neilb at suse.com Fri Feb 9 01:03:09 2018 From: neilb at suse.com (NeilBrown) Date: Fri, 09 Feb 2018 12:03:09 +1100 Subject: [lustre-devel] testing lustre In-Reply-To: <94546355-FD93-4B50-AC98-4AB38EDC9BF0@gmail.com> References: <151538168618.23920.8261096424342988792.stgit@noble> <20180108145943.GA5761@kroah.com> <87po5gmqph.fsf@notabene.neil.brown.name> <94546355-FD93-4B50-AC98-4AB38EDC9BF0@gmail.com> Message-ID: <87tvurkm9e.fsf@notabene.neil.brown.name> On Thu, Feb 08 2018, Благодаренко Артём wrote: > Hello NeilBrown, > > Symbol “:” is added to name for targets which have never been registered. tunefs and mount use this symbol to > set LDD_F_VIRGIN flag in ldd. > > Here is code from lustre/utils/mkfs_lustre.c (executed in tunefs) > > /* svname of the form lustre:OST1234 means never registered */ > ret = strlen(ldd->ldd_svname); > if (ldd->ldd_svname[ret - 8] == ':') { > ldd->ldd_svname[ret - 8] = '-'; > ldd->ldd_flags |= LDD_F_VIRGIN; > > And > > lustre/utils/mount_lustre.c > > /* svname of the form lustre:OST1234 means never registered */ > rc = strlen(ldd->ldd_svname); > if (strcmp(ldd->ldd_svname, "MGS") != 0) { > if (rc < 8) { > fprintf(stderr, "%s: invalid name '%s'\n", > progname, ldd->ldd_svname); > return EINVAL; > } else if (ldd->ldd_svname[rc - 8] == ':') { > ldd->ldd_svname[rc - 8] = '-'; > ldd->ldd_flags |= LDD_F_VIRGIN; > > So I believe the test expects freshly created target. Thanks for the reply. This is helpful, but there is still something missing in my understanding. At what stage does a filesystem transition from "VIRGIN" to "not VIRGIN"?? When does the ":" get replaced by a "-" in the on-disk volume name? What, exactly, does it mean to be "registered" ? I cannot find anywhere in the code that would write out a new label, or that would call e2label or tune2fs to do it. Thanks, NeilBrown -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 832 bytes Desc: not available URL: From neilb at suse.com Fri Feb 9 01:39:18 2018 From: neilb at suse.com (NeilBrown) Date: Fri, 09 Feb 2018 12:39:18 +1100 Subject: [lustre-devel] [PATCH 41/80] staging: lustre: lmv: separate master object with master stripe In-Reply-To: <1471378773-24590-42-git-send-email-jsimmons@infradead.org> References: <1471378773-24590-1-git-send-email-jsimmons@infradead.org> <1471378773-24590-42-git-send-email-jsimmons@infradead.org> Message-ID: <87r2pvkkl5.fsf@notabene.neil.brown.name> On Tue, Aug 16 2016, James Simmons wrote: > > +static inline bool > +lsm_md_eq(const struct lmv_stripe_md *lsm1, const struct lmv_stripe_md *lsm2) > +{ > + int idx; > + > + if (lsm1->lsm_md_magic != lsm2->lsm_md_magic || > + lsm1->lsm_md_stripe_count != lsm2->lsm_md_stripe_count || > + lsm1->lsm_md_master_mdt_index != lsm2->lsm_md_master_mdt_index || > + lsm1->lsm_md_hash_type != lsm2->lsm_md_hash_type || > + lsm1->lsm_md_layout_version != lsm2->lsm_md_layout_version || > + !strcmp(lsm1->lsm_md_pool_name, lsm2->lsm_md_pool_name)) > + return false; Hi James and all, This patch (8f18c8a48b736c2f in linux) is different from the corresponding patch in lustre-release (60e07b972114df). In that patch, the last clause in the 'if' condition is + strcmp(lsm1->lsm_md_pool_name, + lsm2->lsm_md_pool_name) != 0) Whoever converted it to "!strcmp()" inverted the condition. This is a perfect example of why I absolutely *loathe* the "!strcmp()" construct!! This causes many tests in the 'sanity' test suite to return -ENOMEM (that had me puzzled for a while!!). This seems to suggest that no-one has been testing the mainline linux lustre. It also seems to suggest that there is a good chance that there are other bugs that have crept in while no-one has really been caring. Given that the sanity test suite doesn't complete for me, but just hangs (in test_27z I think), that seems particularly likely. So my real question - to anyone interested in lustre for mainline linux - is: can we actually trust this code at all? I'm seriously tempted to suggest that we just rm -r drivers/staging/lustre drivers/staging is great for letting the community work on code that has been "thrown over the wall" and is not openly developed elsewhere, but that is not the case for lustre. lustre has (or seems to have) an open development process. Having on-going development happen both there and in drivers/staging seems a waste of resources. Might it make sense to instead start cleaning up the code in lustre-release so as to make it meet the upstream kernel standards. Then when the time is right, the kernel code can be moved *out* of lustre-release and *in* to linux. Then development can continue in Linux (just like it does with other Linux filesystems). An added bonus of this is that there is an obvious path to getting server support in mainline Linux. The current situation of client-only support seems weird given how interdependent the two are. What do others think? Is there any chance that the current lustre in Linux will ever be more than a poor second-cousin to the external lustre-release. If there isn't, should we just discard it now and move on? Thanks, NeilBrown -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 832 bytes Desc: not available URL: From oleg.drokin at intel.com Fri Feb 9 02:01:35 2018 From: oleg.drokin at intel.com (Oleg Drokin) Date: Thu, 8 Feb 2018 21:01:35 -0500 Subject: [lustre-devel] [PATCH 41/80] staging: lustre: lmv: separate master object with master stripe In-Reply-To: <87r2pvkkl5.fsf@notabene.neil.brown.name> References: <1471378773-24590-1-git-send-email-jsimmons@infradead.org> <1471378773-24590-42-git-send-email-jsimmons@infradead.org> <87r2pvkkl5.fsf@notabene.neil.brown.name> Message-ID: <09CE6CEC-52BC-4B29-B609-40DE68A64A33@intel.com> > On Feb 8, 2018, at 8:39 PM, NeilBrown wrote: > > On Tue, Aug 16 2016, James Simmons wrote: my that’s an old patch > >> >> +static inline bool >> +lsm_md_eq(const struct lmv_stripe_md *lsm1, const struct lmv_stripe_md *lsm2) >> +{ >> + int idx; >> + >> + if (lsm1->lsm_md_magic != lsm2->lsm_md_magic || >> + lsm1->lsm_md_stripe_count != lsm2->lsm_md_stripe_count || >> + lsm1->lsm_md_master_mdt_index != lsm2->lsm_md_master_mdt_index || >> + lsm1->lsm_md_hash_type != lsm2->lsm_md_hash_type || >> + lsm1->lsm_md_layout_version != lsm2->lsm_md_layout_version || >> + !strcmp(lsm1->lsm_md_pool_name, lsm2->lsm_md_pool_name)) >> + return false; > > Hi James and all, > This patch (8f18c8a48b736c2f in linux) is different from the > corresponding patch in lustre-release (60e07b972114df). > > In that patch, the last clause in the 'if' condition is > > + strcmp(lsm1->lsm_md_pool_name, > + lsm2->lsm_md_pool_name) != 0) > > Whoever converted it to "!strcmp()" inverted the condition. This is a > perfect example of why I absolutely *loathe* the "!strcmp()" construct!! > > This causes many tests in the 'sanity' test suite to return > -ENOMEM (that had me puzzled for a while!!). huh? I am not seeing anything of the sort and I was running sanity all the time until a recent pause (but going to resume). > This seems to suggest that no-one has been testing the mainline linux > lustre. > It also seems to suggest that there is a good chance that there > are other bugs that have crept in while no-one has really been caring. > Given that the sanity test suite doesn't complete for me, but just > hangs (in test_27z I think), that seems particularly likely. Works for me, here’s a run from earlier today on 4.15.0: == sanity test 27z: check SEQ/OID on the MDT and OST filesystems ===================================== 16:43:58 (1518126238) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169548 s, 61.8 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.02782 s, 75.4 MB/s check file /mnt/lustre/d27z.sanity/f27z.sanity-1 FID seq 0x200000401, oid 0x4640 ver 0x0 LOV seq 0x200000401, oid 0x4640, count: 1 want: stripe:0 ost:0 oid:314/0x13a seq:0 Stopping /mnt/lustre-ost1 (opts:) on centos6-17 pdsh at fedora1: centos6-17: ssh exited with exit code 1 pdsh at fedora1: centos6-17: ssh exited with exit code 1 pdsh at fedora1: centos6-17: ssh exited with exit code 1 Starting ost1: -o loop /tmp/lustre-ost1 /mnt/lustre-ost1 Failed to initialize ZFS library: 256 h2tcp: deprecated, use h2nettype instead centos6-17.localnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super all -lnet -lnd -pinger 16 pdsh at fedora1: centos6-17: ssh exited with exit code 1 pdsh at fedora1: centos6-17: ssh exited with exit code 1 Started lustre-OST0000 /mnt/lustre-ost1/O/0/d26/314: parent=[0x200000401:0x4640:0x0] stripe=0 stripe_size=0 stripe_count=0 check file /mnt/lustre/d27z.sanity/f27z.sanity-2 FID seq 0x200000401, oid 0x4642 ver 0x0 LOV seq 0x200000401, oid 0x4642, count: 2 want: stripe:0 ost:1 oid:1187/0x4a3 seq:0 Stopping /mnt/lustre-ost2 (opts:) on centos6-17 pdsh at fedora1: centos6-17: ssh exited with exit code 1 pdsh at fedora1: centos6-17: ssh exited with exit code 1 pdsh at fedora1: centos6-17: ssh exited with exit code 1 Starting ost2: -o loop /tmp/lustre-ost2 /mnt/lustre-ost2 Failed to initialize ZFS library: 256 h2tcp: deprecated, use h2nettype instead centos6-17.localnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super all -lnet -lnd -pinger 16 pdsh at fedora1: centos6-17: ssh exited with exit code 1 pdsh at fedora1: centos6-17: ssh exited with exit code 1 Started lustre-OST0001 /mnt/lustre-ost2/O/0/d3/1187: parent=[0x200000401:0x4642:0x0] stripe=0 stripe_size=0 stripe_count=0 want: stripe:1 ost:0 oid:315/0x13b seq:0 got: objid=0 seq=0 parent=[0x200000401:0x4642:0x0] stripe=1 Resetting fail_loc on all nodes...done. 16:44:32 (1518126272) waiting for centos6-16 network 5 secs ... 16:44:32 (1518126272) network interface is UP 16:44:33 (1518126273) waiting for centos6-17 network 5 secs ... 16:44:33 (1518126273) network interface is UP > So my real question - to anyone interested in lustre for mainline linux > - is: can we actually trust this code at all? Absolutely. Seems that you just stumbled upon a corner case that was not being hit by people that do the testing, so you have something unique about your setup, I guess. > I'm seriously tempted to suggest that we just > rm -r drivers/staging/lustre > > drivers/staging is great for letting the community work on code that has > been "thrown over the wall" and is not openly developed elsewhere, but > that is not the case for lustre. lustre has (or seems to have) an open > development process. Having on-going development happen both there and > in drivers/staging seems a waste of resources. It is a bit of a waste of resources, but there are some other things here. E.g. we cannot have any APIs with no users in the kernel. Also some people like to have in-kernel modules coming with their distros (there were some users that used staging client on ubuntu as their setup). Instead the plan was to clean up the staging client into acceptable state, move it out of staging, bring in all the missing features and then drop the client (more or less) from the lustre-release. > Might it make sense to instead start cleaning up the code in > lustre-release so as to make it meet the upstream kernel standards. > Then when the time is right, the kernel code can be moved *out* of > lustre-release and *in* to linux. Then development can continue in > Linux (just like it does with other Linux filesystems). While we can be cleaning lustre in lustre-release, there are some things we cannot do as easily, e.g. decoupling Lustre client from the server. Also it would not attract any reviews from all the janitor or (more importantly) Al Viro and other people with a sharp eyes. > An added bonus of this is that there is an obvious path to getting > server support in mainline Linux. The current situation of client-only > support seems weird given how interdependent the two are. Given the pushback Lustre client was given I have no hope Lustre server will get into mainline in my lifetime. > What do others think? Is there any chance that the current lustre in > Linux will ever be more than a poor second-cousin to the external > lustre-release. If there isn't, should we just discard it now and move > on? I think many useful cleanups and fixes came from the staging tree at the very least. The biggest problem with it all is that we are in staging tree so we cannot bring it to parity much. And we are in staging tree because there’s a whole bunch of “cleanups” requested that take a lot of effort (in both implementing them and then in finding other ways of achieving things that were done in old ways before). I understand that beggars cannot be choosers and while there are people that are grandfathered with their atrocities in current kernel tree, we must adhere to the shining standards first before having our chance, but the standards are not easy to adhere to in an established sizeable codebase. Realistically speaking I suspect if we drop Lustre from staging, it’s unlikely there would remain any steam behind the cleanup efforts at all. Bye, Oleg From neilb at suse.com Fri Feb 9 03:10:44 2018 From: neilb at suse.com (NeilBrown) Date: Fri, 09 Feb 2018 14:10:44 +1100 Subject: [lustre-devel] [PATCH 41/80] staging: lustre: lmv: separate master object with master stripe In-Reply-To: <09CE6CEC-52BC-4B29-B609-40DE68A64A33@intel.com> References: <1471378773-24590-1-git-send-email-jsimmons@infradead.org> <1471378773-24590-42-git-send-email-jsimmons@infradead.org> <87r2pvkkl5.fsf@notabene.neil.brown.name> <09CE6CEC-52BC-4B29-B609-40DE68A64A33@intel.com> Message-ID: <87o9kylux7.fsf@notabene.neil.brown.name> On Thu, Feb 08 2018, Oleg Drokin wrote: >> On Feb 8, 2018, at 8:39 PM, NeilBrown wrote: >> >> On Tue, Aug 16 2016, James Simmons wrote: > > my that’s an old patch > >> ... >> >> Whoever converted it to "!strcmp()" inverted the condition. This is a >> perfect example of why I absolutely *loathe* the "!strcmp()" construct!! >> >> This causes many tests in the 'sanity' test suite to return >> -ENOMEM (that had me puzzled for a while!!). > > huh? I am not seeing anything of the sort and I was running sanity > all the time until a recent pause (but going to resume). That does surprised me - I reproduce it every time. I have two VMs running a SLE12-SP2 kernel with patches from lustre-release applied. These are servers. They have 2 3G virtual disks each. I have two over VMs running current mainline. These are clients. I guess your 'recent pause' included between v4.15-rc1 (8e55b6fd0660) and v4.15-rc6 (a93639090a27) - a full month when lustre wouldn't work at all :-( > >> This seems to suggest that no-one has been testing the mainline linux >> lustre. >> It also seems to suggest that there is a good chance that there >> are other bugs that have crept in while no-one has really been caring. >> Given that the sanity test suite doesn't complete for me, but just >> hangs (in test_27z I think), that seems particularly likely. > > Works for me, here’s a run from earlier today on 4.15.0: Well that's encouraging .. I haven't looked into this one yet - I'm not even sure where to start. > >> So my real question - to anyone interested in lustre for mainline linux >> - is: can we actually trust this code at all? > > Absolutely. Seems that you just stumbled upon a corner case that was not > being hit by people that do the testing, so you have something unique about > your setup, I guess. > >> I'm seriously tempted to suggest that we just >> rm -r drivers/staging/lustre >> >> drivers/staging is great for letting the community work on code that has >> been "thrown over the wall" and is not openly developed elsewhere, but >> that is not the case for lustre. lustre has (or seems to have) an open >> development process. Having on-going development happen both there and >> in drivers/staging seems a waste of resources. > > It is a bit of a waste of resources, but there are some other things here. > E.g. we cannot have any APIs with no users in the kernel. > Also some people like to have in-kernel modules coming with their distros > (there were some users that used staging client on ubuntu as their > setup). > > Instead the plan was to clean up the staging client into acceptable state, > move it out of staging, bring in all the missing features and then > drop the client (more or less) from the lustre-release. That sounds like a great plan. Any idea why it didn't happen? It seems there is a lot of upstream work mixed in with the clean up, and I don't think that really helps anyone. Is it at all realistic that the client might be removed from lustre-release? That might be a good goal to work towards. > >> Might it make sense to instead start cleaning up the code in >> lustre-release so as to make it meet the upstream kernel standards. >> Then when the time is right, the kernel code can be moved *out* of >> lustre-release and *in* to linux. Then development can continue in >> Linux (just like it does with other Linux filesystems). > > While we can be cleaning lustre in lustre-release, there are some things > we cannot do as easily, e.g. decoupling Lustre client from the server. > Also it would not attract any reviews from all the janitor or > (more importantly) Al Viro and other people with a sharp eyes. > >> An added bonus of this is that there is an obvious path to getting >> server support in mainline Linux. The current situation of client-only >> support seems weird given how interdependent the two are. > > Given the pushback Lustre client was given I have no hope Lustre server > will get into mainline in my lifetime. Even if it is horrible it would be nice to have it in staging... I guess the changes required to ext4 prohibit that... I don't suppose it can be made to work with mainline ext4 in a reduced-functionality-and-performance way?? I think it would be a lot easier to motivate forward progress if there were a credible end goal of everything being in mainline. > >> What do others think? Is there any chance that the current lustre in >> Linux will ever be more than a poor second-cousin to the external >> lustre-release. If there isn't, should we just discard it now and move >> on? > > > I think many useful cleanups and fixes came from the staging tree at > the very least. > The biggest problem with it all is that we are in staging tree so > we cannot bring it to parity much. And we are in staging tree because > there’s a whole bunch of “cleanups” requested that take a lot of effort > (in both implementing them and then in finding other ways of achieving > things that were done in old ways before). Do you have a list of requested cleanups? I would find that to be useful. > I understand that beggars cannot be choosers and while there are people > that are grandfathered with their atrocities in current kernel tree, > we must adhere to the shining standards first before having our chance, > but the standards are not easy to adhere to in an established sizeable > codebase. > > Realistically speaking I suspect if we drop Lustre from staging, > it’s unlikely there would remain any steam behind the cleanup efforts > at all. Thanks for your thoughts, NeilBrown -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 832 bytes Desc: not available URL: From oleg.drokin at intel.com Fri Feb 9 03:50:32 2018 From: oleg.drokin at intel.com (Oleg Drokin) Date: Thu, 8 Feb 2018 22:50:32 -0500 Subject: [lustre-devel] [PATCH 41/80] staging: lustre: lmv: separate master object with master stripe In-Reply-To: <87o9kylux7.fsf@notabene.neil.brown.name> References: <1471378773-24590-1-git-send-email-jsimmons@infradead.org> <1471378773-24590-42-git-send-email-jsimmons@infradead.org> <87r2pvkkl5.fsf@notabene.neil.brown.name> <09CE6CEC-52BC-4B29-B609-40DE68A64A33@intel.com> <87o9kylux7.fsf@notabene.neil.brown.name> Message-ID: > On Feb 8, 2018, at 10:10 PM, NeilBrown wrote: > > On Thu, Feb 08 2018, Oleg Drokin wrote: > >>> On Feb 8, 2018, at 8:39 PM, NeilBrown wrote: >>> >>> On Tue, Aug 16 2016, James Simmons wrote: >> >> my that’s an old patch >> >>> > ... >>> >>> Whoever converted it to "!strcmp()" inverted the condition. This is a >>> perfect example of why I absolutely *loathe* the "!strcmp()" construct!! >>> >>> This causes many tests in the 'sanity' test suite to return >>> -ENOMEM (that had me puzzled for a while!!). >> >> huh? I am not seeing anything of the sort and I was running sanity >> all the time until a recent pause (but going to resume). > > That does surprised me - I reproduce it every time. > I have two VMs running a SLE12-SP2 kernel with patches from > lustre-release applied. These are servers. They have 2 3G virtual disks > each. > I have two over VMs running current mainline. These are clients. > > I guess your 'recent pause' included between v4.15-rc1 (8e55b6fd0660) > and v4.15-rc6 (a93639090a27) - a full month when lustre wouldn't work at > all :-( More than that, but I am pretty sure James Simmons is running tests all the time too (he has a different config, I only have tcp). >>> This seems to suggest that no-one has been testing the mainline linux >>> lustre. >>> It also seems to suggest that there is a good chance that there >>> are other bugs that have crept in while no-one has really been caring. >>> Given that the sanity test suite doesn't complete for me, but just >>> hangs (in test_27z I think), that seems particularly likely. >> >> Works for me, here’s a run from earlier today on 4.15.0: > > Well that's encouraging .. I haven't looked into this one yet - I'm not > even sure where to start. m… debug logs for example (greatly neutered in staging tree, but still useful)? try lctl dk and see what’s in there. >> Instead the plan was to clean up the staging client into acceptable state, >> move it out of staging, bring in all the missing features and then >> drop the client (more or less) from the lustre-release. > > That sounds like a great plan. Any idea why it didn't happen? Because meeting open-ended demands is hard and certain demands sound like “throw away your X and rewrite it from scratch" (e.g. everything IB-related). Certain things that sound useless (like the debug subsystem in Lustre) is very useful when you have a 10k nodes in a cluster and need to selectively pull stuff from a run to debug a complicated cross-node interaction. I asked NFS people how do they do it and they don’t have anything that scales and usually involves reducing the problem to a much smaller set of nodes first. > It seems there is a lot of upstream work mixed in with the clean up, and > I don't think that really helps anyone. I don’t understand what you mean here. > Is it at all realistic that the client might be removed from > lustre-release? That might be a good goal to work towards. Assuming we can bring the whole functionality over - sure. Of course there’d still be some separate development place and we would need to create patches (new features?) for like SuSE and other distros and for testing of server features, I guess, but that could just that - a side branch somewhere I hope. It’s not that we are super glad to chase every kernel vendors put out, of course it would be much easier if the kernels already included a very functional Lustre client. >>> Might it make sense to instead start cleaning up the code in >>> lustre-release so as to make it meet the upstream kernel standards. >>> Then when the time is right, the kernel code can be moved *out* of >>> lustre-release and *in* to linux. Then development can continue in >>> Linux (just like it does with other Linux filesystems). >> >> While we can be cleaning lustre in lustre-release, there are some things >> we cannot do as easily, e.g. decoupling Lustre client from the server. >> Also it would not attract any reviews from all the janitor or >> (more importantly) Al Viro and other people with a sharp eyes. >> >>> An added bonus of this is that there is an obvious path to getting >>> server support in mainline Linux. The current situation of client-only >>> support seems weird given how interdependent the two are. >> >> Given the pushback Lustre client was given I have no hope Lustre server >> will get into mainline in my lifetime. > > Even if it is horrible it would be nice to have it in staging... I guess > the changes required to ext4 prohibit that... I don't suppose it can be > made to work with mainline ext4 in a reduced-functionality-and-performance > way?? We support unpatched ZFS as a server too! ;) (and if somebody invests the time into it, there was some half-baked btrfs backend too I think). That said nobody here believes in any success of pushing Lustre server into mainline. It would just be easier to push the whole server into userspace (And there was a project like this in the past, now abandoned because it was mostly targeting Solaris anyway). > I think it would be a lot easier to motivate forward progress if there > were a credible end goal of everything being in mainline. > >> >>> What do others think? Is there any chance that the current lustre in >>> Linux will ever be more than a poor second-cousin to the external >>> lustre-release. If there isn't, should we just discard it now and move >>> on? >> >> >> I think many useful cleanups and fixes came from the staging tree at >> the very least. >> The biggest problem with it all is that we are in staging tree so >> we cannot bring it to parity much. And we are in staging tree because >> there’s a whole bunch of “cleanups” requested that take a lot of effort >> (in both implementing them and then in finding other ways of achieving >> things that were done in old ways before). > > Do you have a list of requested cleanups? I would find that to be > useful. As Greg would tell you, “if you don’t know what needs to be done, let’s just remove the whole thing from staging now”. I assume you saw drivers/staging/lustre/TODO already, it’s only partially done. We had a bunch of other requests from various people ranging from wholesale removal of various parts to making sure there’s no checkpatch warnings (Turned out rather hard to do, even though we greatly pared the numbers). I have some patches to make Lustre a lot more monolithic too. People want us to remove our indirections hell so the code is more readable (I have some patches that need to be freshened up some that help here a bit, but the work is huge.) Other requests come out as some of the prior ones get completed due to “you need o finish current level of cleanups so that we can see what other cleanups are needed, the current code is too bad to see everything” pretty much. Bye, Oleg From artem.blagodarenko at gmail.com Fri Feb 9 08:51:52 2018 From: artem.blagodarenko at gmail.com (=?utf-8?B?0JHQu9Cw0LPQvtC00LDRgNC10L3QutC+INCQ0YDRgtGR0Lw=?=) Date: Fri, 9 Feb 2018 11:51:52 +0300 Subject: [lustre-devel] testing lustre In-Reply-To: <87tvurkm9e.fsf@notabene.neil.brown.name> References: <151538168618.23920.8261096424342988792.stgit@noble> <20180108145943.GA5761@kroah.com> <87po5gmqph.fsf@notabene.neil.brown.name> <94546355-FD93-4B50-AC98-4AB38EDC9BF0@gmail.com> <87tvurkm9e.fsf@notabene.neil.brown.name> Message-ID: <3C5A4767-EBDC-42C0-9A46-46C8356C5067@gmail.com> Hello NeilBrown, > On 9 Feb 2018, at 04:03, NeilBrown wrote: > > On Thu, Feb 08 2018, Благодаренко Артём wrote: > >> Hello NeilBrown, >> >> Symbol “:” is added to name for targets which have never been registered. tunefs and mount use this symbol to >> set LDD_F_VIRGIN flag in ldd. >> >> Here is code from lustre/utils/mkfs_lustre.c (executed in tunefs) >> >> /* svname of the form lustre:OST1234 means never registered */ >> ret = strlen(ldd->ldd_svname); >> if (ldd->ldd_svname[ret - 8] == ':') { >> ldd->ldd_svname[ret - 8] = '-'; >> ldd->ldd_flags |= LDD_F_VIRGIN; >> >> And >> >> lustre/utils/mount_lustre.c >> >> /* svname of the form lustre:OST1234 means never registered */ >> rc = strlen(ldd->ldd_svname); >> if (strcmp(ldd->ldd_svname, "MGS") != 0) { >> if (rc < 8) { >> fprintf(stderr, "%s: invalid name '%s'\n", >> progname, ldd->ldd_svname); >> return EINVAL; >> } else if (ldd->ldd_svname[rc - 8] == ':') { >> ldd->ldd_svname[rc - 8] = '-'; >> ldd->ldd_flags |= LDD_F_VIRGIN; >> >> So I believe the test expects freshly created target. > > Thanks for the reply. This is helpful, but there is still something > missing in my understanding. > > At what stage does a filesystem transition from "VIRGIN" to "not > VIRGIN”?? To exclude misunderstanding let’s clarify - this is stage of target, not filesystem. At moment then target is formatted it is VIRGIN. Then it first time mounted it becomes not VIRGIN. > When does the ":" get replaced by a "-" in the on-disk volume name? > What, exactly, does it mean to be "registered” ? At moment target is mounting it sends request to mgs and becomes registered. This is requirement for successful registration. After such successful first mount “:” replaced to “-“ in the on-disk volume name. > I cannot find anywhere in the code that would write out a new label, or > that would call e2label or tune2fs to do it. mount utility (mount_lustre.c) Main-> parse_ldd() -> ldd() read ldd. In and change : to - in memory mount() - mount target, register it on mgs label_lustre () write new label with “-“ on disk > > Thanks, > NeilBrown Best regards, Artem Blagodarenko From neilb at suse.com Fri Feb 9 23:52:41 2018 From: neilb at suse.com (NeilBrown) Date: Sat, 10 Feb 2018 10:52:41 +1100 Subject: [lustre-devel] testing lustre In-Reply-To: <3C5A4767-EBDC-42C0-9A46-46C8356C5067@gmail.com> References: <151538168618.23920.8261096424342988792.stgit@noble> <20180108145943.GA5761@kroah.com> <87po5gmqph.fsf@notabene.neil.brown.name> <94546355-FD93-4B50-AC98-4AB38EDC9BF0@gmail.com> <87tvurkm9e.fsf@notabene.neil.brown.name> <3C5A4767-EBDC-42C0-9A46-46C8356C5067@gmail.com> Message-ID: <87r2ptpvp2.fsf@notabene.neil.brown.name> On Fri, Feb 09 2018, Благодаренко Артём wrote: > Hello NeilBrown, > > >> On 9 Feb 2018, at 04:03, NeilBrown wrote: >> >> On Thu, Feb 08 2018, Благодаренко Артём wrote: >> >>> Hello NeilBrown, >>> >>> Symbol “:” is added to name for targets which have never been registered. tunefs and mount use this symbol to >>> set LDD_F_VIRGIN flag in ldd. >>> >>> Here is code from lustre/utils/mkfs_lustre.c (executed in tunefs) >>> >>> /* svname of the form lustre:OST1234 means never registered */ >>> ret = strlen(ldd->ldd_svname); >>> if (ldd->ldd_svname[ret - 8] == ':') { >>> ldd->ldd_svname[ret - 8] = '-'; >>> ldd->ldd_flags |= LDD_F_VIRGIN; >>> >>> And >>> >>> lustre/utils/mount_lustre.c >>> >>> /* svname of the form lustre:OST1234 means never registered */ >>> rc = strlen(ldd->ldd_svname); >>> if (strcmp(ldd->ldd_svname, "MGS") != 0) { >>> if (rc < 8) { >>> fprintf(stderr, "%s: invalid name '%s'\n", >>> progname, ldd->ldd_svname); >>> return EINVAL; >>> } else if (ldd->ldd_svname[rc - 8] == ':') { >>> ldd->ldd_svname[rc - 8] = '-'; >>> ldd->ldd_flags |= LDD_F_VIRGIN; >>> >>> So I believe the test expects freshly created target. >> >> Thanks for the reply. This is helpful, but there is still something >> missing in my understanding. >> >> At what stage does a filesystem transition from "VIRGIN" to "not >> VIRGIN”?? > > To exclude misunderstanding let’s clarify - this is stage of target, not filesystem. > At moment then target is formatted it is VIRGIN. Then it first time mounted it becomes not VIRGIN. > >> When does the ":" get replaced by a "-" in the on-disk volume name? >> What, exactly, does it mean to be "registered” ? > > At moment target is mounting it sends request to mgs and becomes registered. > This is requirement for successful registration. After such successful first mount “:” replaced to “-“ in the on-disk volume name. > >> I cannot find anywhere in the code that would write out a new label, or >> that would call e2label or tune2fs to do it. > > mount utility (mount_lustre.c) > Main-> > parse_ldd() -> ldd() read ldd. In and change : to - in memory > mount() - mount target, register it on mgs > label_lustre () write new label with “-“ on disk Ahhh.. found it. Thanks for lot for the details!! And and I can see in an strace that this is being called - mount.lustre forks and runs "tune2fs -f -L ....". But still the volume name doesn't change. Maybe I have a buggy tune2fs. I note that tune2fs says "recovering journal" or something like that. I suspect we don't want that to happen when the filesystem is mounted. Maybe that is related to the failure to change the label. I will dig in more deeply. thanks a lot, NeilBrown > >> >> Thanks, >> NeilBrown > > Best regards, > Artem Blagodarenko -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 832 bytes Desc: not available URL: From oleg.drokin at intel.com Sat Feb 10 03:51:02 2018 From: oleg.drokin at intel.com (Oleg Drokin) Date: Fri, 9 Feb 2018 22:51:02 -0500 Subject: [lustre-devel] testing lustre In-Reply-To: <87r2ptpvp2.fsf@notabene.neil.brown.name> References: <151538168618.23920.8261096424342988792.stgit@noble> <20180108145943.GA5761@kroah.com> <87po5gmqph.fsf@notabene.neil.brown.name> <94546355-FD93-4B50-AC98-4AB38EDC9BF0@gmail.com> <87tvurkm9e.fsf@notabene.neil.brown.name> <3C5A4767-EBDC-42C0-9A46-46C8356C5067@gmail.com> <87r2ptpvp2.fsf@notabene.neil.brown.name> Message-ID: <649030A5-48A9-4C67-A911-28FBC5F183E1@intel.com> > On Feb 9, 2018, at 6:52 PM, NeilBrown wrote: > > On Fri, Feb 09 2018, Благодаренко Артём wrote: >> >>> I cannot find anywhere in the code that would write out a new label, or >>> that would call e2label or tune2fs to do it. >> >> mount utility (mount_lustre.c) >> Main-> >> parse_ldd() -> ldd() read ldd. In and change : to - in memory >> mount() - mount target, register it on mgs >> label_lustre () write new label with “-“ on disk > > Ahhh.. found it. Thanks for lot for the details!! > > And and I can see in an strace that this is being called - mount.lustre > forks and runs "tune2fs -f -L ....". > But still the volume name doesn't change. > Maybe I have a buggy tune2fs. > > I note that tune2fs says "recovering journal" or something like that. > I suspect we don't want that to happen when the filesystem is mounted. > Maybe that is related to the failure to change the label. > I will dig in more deeply. This sounds very familiar, but I cannot quite find the ticket for it. Anyway, I do wonder if you use our patched e2fsprogs and if not, if you could give it a try first. Bye, Oleg From jsimmons at infradead.org Sat Feb 10 20:57:11 2018 From: jsimmons at infradead.org (James Simmons) Date: Sat, 10 Feb 2018 20:57:11 +0000 (GMT) Subject: [lustre-devel] [PATCH 41/80] staging: lustre: lmv: separate master object with master stripe In-Reply-To: References: <1471378773-24590-1-git-send-email-jsimmons@infradead.org> <1471378773-24590-42-git-send-email-jsimmons@infradead.org> <87r2pvkkl5.fsf@notabene.neil.brown.name> <09CE6CEC-52BC-4B29-B609-40DE68A64A33@intel.com> <87o9kylux7.fsf@notabene.neil.brown.name> Message-ID: > > On Feb 8, 2018, at 10:10 PM, NeilBrown wrote: > > > > On Thu, Feb 08 2018, Oleg Drokin wrote: > > > >>> On Feb 8, 2018, at 8:39 PM, NeilBrown wrote: > >>> > >>> On Tue, Aug 16 2016, James Simmons wrote: > >> > >> my that’s an old patch > >> > >>> > > ... > >>> > >>> Whoever converted it to "!strcmp()" inverted the condition. This is a > >>> perfect example of why I absolutely *loathe* the "!strcmp()" construct!! > >>> > >>> This causes many tests in the 'sanity' test suite to return > >>> -ENOMEM (that had me puzzled for a while!!). > >> > >> huh? I am not seeing anything of the sort and I was running sanity > >> all the time until a recent pause (but going to resume). > > > > That does surprised me - I reproduce it every time. > > I have two VMs running a SLE12-SP2 kernel with patches from > > lustre-release applied. These are servers. They have 2 3G virtual disks > > each. > > I have two over VMs running current mainline. These are clients. > > > > I guess your 'recent pause' included between v4.15-rc1 (8e55b6fd0660) > > and v4.15-rc6 (a93639090a27) - a full month when lustre wouldn't work at > > all :-( > > More than that, but I am pretty sure James Simmons is running tests all the time too > (he has a different config, I only have tcp). Yes I have been testing and haven't encountered this problem. Let me try the fix you pointed out. > > Do you have a list of requested cleanups? I would find that to be > > useful. > > As Greg would tell you, “if you don’t know what needs to be done, > let’s just remove the whole thing from staging now”. > > I assume you saw drivers/staging/lustre/TODO already, it’s only partially done. Actually the complete list is at : https://jira.hpdd.intel.com/browse/LU-9679 I need to move that to our TODO list. Sorry I have been short on cycles. From jsimmons at infradead.org Sat Feb 10 22:14:19 2018 From: jsimmons at infradead.org (James Simmons) Date: Sat, 10 Feb 2018 22:14:19 +0000 (GMT) Subject: [lustre-devel] [PATCH 41/80] staging: lustre: lmv: separate master object with master stripe In-Reply-To: <87r2pvkkl5.fsf@notabene.neil.brown.name> References: <1471378773-24590-1-git-send-email-jsimmons@infradead.org> <1471378773-24590-42-git-send-email-jsimmons@infradead.org> <87r2pvkkl5.fsf@notabene.neil.brown.name> Message-ID: > > +static inline bool > > +lsm_md_eq(const struct lmv_stripe_md *lsm1, const struct lmv_stripe_md *lsm2) > > +{ > > + int idx; > > + > > + if (lsm1->lsm_md_magic != lsm2->lsm_md_magic || > > + lsm1->lsm_md_stripe_count != lsm2->lsm_md_stripe_count || > > + lsm1->lsm_md_master_mdt_index != lsm2->lsm_md_master_mdt_index || > > + lsm1->lsm_md_hash_type != lsm2->lsm_md_hash_type || > > + lsm1->lsm_md_layout_version != lsm2->lsm_md_layout_version || > > + !strcmp(lsm1->lsm_md_pool_name, lsm2->lsm_md_pool_name)) > > + return false; > > Hi James and all, > This patch (8f18c8a48b736c2f in linux) is different from the > corresponding patch in lustre-release (60e07b972114df). > > In that patch, the last clause in the 'if' condition is > > + strcmp(lsm1->lsm_md_pool_name, > + lsm2->lsm_md_pool_name) != 0) > > Whoever converted it to "!strcmp()" inverted the condition. This is a > perfect example of why I absolutely *loathe* the "!strcmp()" construct!! > > This causes many tests in the 'sanity' test suite to return > -ENOMEM (that had me puzzled for a while!!). > This seems to suggest that no-one has been testing the mainline linux > lustre. > It also seems to suggest that there is a good chance that there > are other bugs that have crept in while no-one has really been caring. > Given that the sanity test suite doesn't complete for me, but just > hangs (in test_27z I think), that seems particularly likely. > > > So my real question - to anyone interested in lustre for mainline linux > - is: can we actually trust this code at all? > I'm seriously tempted to suggest that we just > rm -r drivers/staging/lustre > > drivers/staging is great for letting the community work on code that has > been "thrown over the wall" and is not openly developed elsewhere, but > that is not the case for lustre. lustre has (or seems to have) an open > development process. Having on-going development happen both there and > in drivers/staging seems a waste of resources. > > Might it make sense to instead start cleaning up the code in > lustre-release so as to make it meet the upstream kernel standards. > Then when the time is right, the kernel code can be moved *out* of > lustre-release and *in* to linux. Then development can continue in > Linux (just like it does with other Linux filesystems). > > An added bonus of this is that there is an obvious path to getting > server support in mainline Linux. The current situation of client-only > support seems weird given how interdependent the two are. > > What do others think? Is there any chance that the current lustre in > Linux will ever be more than a poor second-cousin to the external > lustre-release. If there isn't, should we just discard it now and move > on? If you think that the OpenSFS/Intel branch (lustre-release) is the land of milk and honey you are very wrong. Take for example the UAPI header cleanup I push to the linux client several months ago. That work took 5 years to complete. I had to complete that work in the Intel branch since it impacted our tools. This isn't the only example. I worked along side Intel for increasing striping of a file to more then the 160 stripe limit Lustre use to have. That work took 3 years to complete. If the patch is more than one line it will normally take 1 to 2 months to land. It is common to have patches 6 months or more in age. This is one of the major reasons I'm involved in the upstream client work. If lustre remains a tiny under manned community it is doomed to remain a niche file system. For years I have tried to recruit new developers to help out and even gave talks at lustre conferences on internals. That effort was meet with little success. This is not the case with the linux lustre client. We do have people contributing including you. So the reality is that if we removed the lustre client it would be at least 3+ years before the code would be ready to merged back in. It would be another 3+ years before it left staging. Many cleanups in the linux client which impact many lines of code have not been ported to the Intel branch. It would take forever to get those in. Honestly I gave up some time ago for those types of cleanups. The cleanups done in the upstream client would have to be redone. What we really need is to expand the community. Recently a lot of work has gone into supporting Ubuntu for our utilities. I hope this helps to get Canonical involved with the upstream lustre client. The upstream client is not as bad as you think. A year ago no one in their right mind would touch the upstream client but their are actually sites using it today. Its not perfect but it is usable and it is improving all the time. Yes we have quite a few bugs to squash that show up in our test suite but the barrier to leaving staging is much much smaller than it used to be. Once the number of bugs reported in test suite becomes reasonable we can start auto testing patches posted here. The ultimate goal is that as more people join in the linux client effort and it becomes a full member of the broader linux open source community that we can leave the Intel lustre-release branch in the dust. I believe the future is much closer than you think. From jsimmons at infradead.org Sat Feb 10 23:07:20 2018 From: jsimmons at infradead.org (James Simmons) Date: Sat, 10 Feb 2018 23:07:20 +0000 (GMT) Subject: [lustre-devel] testing lustre In-Reply-To: <87po5gmqph.fsf@notabene.neil.brown.name> References: <151538168618.23920.8261096424342988792.stgit@noble> <20180108145943.GA5761@kroah.com> <87po5gmqph.fsf@notabene.neil.brown.name> Message-ID: > Hi, > I've set up a little virtual cluster to run the sanity tests. > One problem I have is that the test harness expects that after it > creates a filesystem, the label (reported by e2label) will > spontaneously change from e.g. "lustre:MDT-0000" to "lustre-MDT-0000" > and that doesn't happen for me. > If I go and change the label manually while the test is waiting, > then the test continues and appears to run correctly. > > Can anyone tell me what this name change in meant to indicate, when > where in the code it is expected to happen? > > Thanks, > NeilBrown Hi Neil Since I do regular testing of the upstream client I'm familiar with the strange oddities you have to deal with to make it work. Now in Lustre 2.10 the LNet layer changed in ways that broke using newer LNet tools with older lustre versions. You could avoid using lnetctl for network bring up but lctl net up doesn't always work with the new LNet stack. So the way I deal with it is to build a lustre 2.9 client which is what the upstream client is close too. At the same time I use the lustre 2.10 test suite with a patch that allows me to run the test suite in patchless server mode. That patchless server patch would take some effort to port to lustre 2.9 and I don't plan on having the upstream client stay at lustre 2.9 forever. You can read details about this at https://jira.hppd.intel.com/browse/lustre/LU-684 Also configure --disable-tests --disable-iokit doesn't work for lustre 2.9 so you have to build everything. All you need is lustre-client-2.9.0-*.rpm in the image. Next grab my lustre 2.10 test suite with the LU-684 patch from http://www.infradead.org/~jsimmons/tests-2.10.tgz The configuration files are in tests/cfg/*. The default conf files that comes with lustre are ncli.sh and local.sh. The file ncli.sh sources local.sh. Now for the tar ball I posted I have clients.sh and server.sh. A diff of ncli to client and a diff of local to server.sh will give you a idea of what is needed to change for testing. If you need any help just reply to this email. Note you might have better luck with lustre 2.10 rpms instead of lustre 2.9 with LNet setup. That is not the case for me. If you need anything just ping me. From jsimmons at infradead.org Sun Feb 11 21:38:01 2018 From: jsimmons at infradead.org (James Simmons) Date: Sun, 11 Feb 2018 21:38:01 +0000 (GMT) Subject: [lustre-devel] [PATCH] staging: lustre: llite: replace variable length array In-Reply-To: <58963c7f-139c-1156-0179-13decf0bcddf@gmx.de> References: <20180127214228.26986-1-sven.dziadek@gmx.de> <58963c7f-139c-1156-0179-13decf0bcddf@gmx.de> Message-ID: > On 01/30/2018 03:04 AM, Dilger, Andreas wrote: > > On Jan 27, 2018, at 14:42, Sven Dziadek wrote: > >> > >> The functionality of the removed variable length array is already > >> implemented by the function xattr_full_name in fs/xattr.c > >> > >> This fixes the sparse warning: > >> warning: Variable length array is used. > >> > >> Signed-off-by: Sven Dziadek > >> --- > >> drivers/staging/lustre/lustre/llite/xattr.c | 12 ++++-------- > >> 1 file changed, 4 insertions(+), 8 deletions(-) > >> > >> diff --git a/drivers/staging/lustre/lustre/llite/xattr.c b/drivers/staging/lustre/lustre/llite/xattr.c > >> index 532384c91447..4fd28213c6a1 100644 > >> --- a/drivers/staging/lustre/lustre/llite/xattr.c > >> +++ b/drivers/staging/lustre/lustre/llite/xattr.c > >> @@ -87,7 +87,6 @@ ll_xattr_set_common(const struct xattr_handler *handler, > >> const char *name, const void *value, size_t size, > >> int flags) > >> { > >> - char fullname[strlen(handler->prefix) + strlen(name) + 1]; > >> struct ll_sb_info *sbi = ll_i2sbi(inode); > >> struct ptlrpc_request *req = NULL; > >> const char *pv = value; > >> @@ -141,9 +140,8 @@ ll_xattr_set_common(const struct xattr_handler *handler, > >> return -EPERM; > >> } > >> > >> - sprintf(fullname, "%s%s\n", handler->prefix, name); > >> - rc = md_setxattr(sbi->ll_md_exp, ll_inode2fid(inode), > >> - valid, fullname, pv, size, 0, flags, > >> + rc = md_setxattr(sbi->ll_md_exp, ll_inode2fid(inode), valid, > >> + xattr_full_name(handler, name), pv, size, 0, flags, > >> ll_i2suppgid(inode), &req); > > > > Hi Sven, > > thanks for the patch. > > > > Looking at the details of "xattr_full_name()", this seems quite risky. This > > is essentially returning the pointer _before_ "name" on the assumption that > > it contains the full "prefix.name" string. IMHO, that is not necessarily a > > safe assumption to make several layers down in the code. > Thanks for your reply. And yeah, it feels strange, right. > > But it really looks like this is how the name and the prefix is handled > in the xattr code. > It seems this helper function has been introduced with commit > e409de992e3ea (which also removes some fullname pointer) and indeed, > fs/xattr.c splits the prefix and the name in xattr_resolve_name. > > So I still think the patch should be correct..? > > > James, I thought you had a patch for this to use kasprintf() instead of the > > on-stack "fullname" declaration? > Sorry for replying that late, I though James would maybe know if the > above assumption is correct.. Yes I do have a patch for this. I pushed them several months ago but they got missed. It happens. I will the patches again very soon. > >> if (rc) { > >> if (rc == -EOPNOTSUPP && handler->flags == XATTR_USER_T) { > >> @@ -364,7 +362,6 @@ static int ll_xattr_get_common(const struct xattr_handler *handler, > >> struct dentry *dentry, struct inode *inode, > >> const char *name, void *buffer, size_t size) > >> { > >> - char fullname[strlen(handler->prefix) + strlen(name) + 1]; > >> struct ll_sb_info *sbi = ll_i2sbi(inode); > >> #ifdef CONFIG_FS_POSIX_ACL > >> struct ll_inode_info *lli = ll_i2info(inode); > >> @@ -411,9 +408,8 @@ static int ll_xattr_get_common(const struct xattr_handler *handler, > >> if (handler->flags == XATTR_ACL_DEFAULT_T && !S_ISDIR(inode->i_mode)) > >> return -ENODATA; > >> #endif > >> - sprintf(fullname, "%s%s\n", handler->prefix, name); > >> - return ll_xattr_list(inode, fullname, handler->flags, buffer, size, > >> - OBD_MD_FLXATTR); > >> + return ll_xattr_list(inode, xattr_full_name(handler, name), > >> + handler->flags, buffer, size, OBD_MD_FLXATTR); > >> } > >> > >> static ssize_t ll_getxattr_lov(struct inode *inode, void *buf, size_t buf_size) > >> -- > >> 2.11.0 > >> > > From jsimmons at infradead.org Sun Feb 11 21:39:05 2018 From: jsimmons at infradead.org (James Simmons) Date: Sun, 11 Feb 2018 21:39:05 +0000 (GMT) Subject: [lustre-devel] [PATCH] staging: lustre: lnet: return of an error code should be negative In-Reply-To: <1517117095-12643-1-git-send-email-pundirsumit11@gmail.com> References: <1517117095-12643-1-git-send-email-pundirsumit11@gmail.com> Message-ID: > Return value of error codes should typically be negative. > Issue reported by checkpatch.pl > Reviewed-by: James Simmons > Signed-off-by: Sumit Pundir > --- > drivers/staging/lustre/lnet/selftest/framework.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/drivers/staging/lustre/lnet/selftest/framework.c b/drivers/staging/lustre/lnet/selftest/framework.c > index c7697f6..0ca1e3a 100644 > --- a/drivers/staging/lustre/lnet/selftest/framework.c > +++ b/drivers/staging/lustre/lnet/selftest/framework.c > @@ -187,7 +187,7 @@ sfw_del_session_timer(void) > return 0; > } > > - return EBUSY; /* racing with sfw_session_expired() */ > + return -EBUSY; /* racing with sfw_session_expired() */ > } > > static void > -- > 2.7.4 > > From jsimmons at infradead.org Sun Feb 11 22:42:14 2018 From: jsimmons at infradead.org (James Simmons) Date: Sun, 11 Feb 2018 22:42:14 +0000 (GMT) Subject: [lustre-devel] Proper Lustre Ubuntu/Debian support Message-ID: Hello Sorry I didn't get back to you earlier about our libtool issues. The good news is Stibor found the source of the problems. Also recently thanks to Schroeder we now have proper server side support on Ubuntu16 systems. We are now at the point that the final bits broken are the actual packaging. So I'm hoping to reach out to you to help out with this to figure out what is the proper way to support Ubuntu. Or at least point us to the who could lend a helping hand. Also recently greater interest in the lustre linux client has been happening. Would people from Ubuntu be interested in kernel development to help lustre get out of staging? From jsimmons at infradead.org Sun Feb 11 23:00:08 2018 From: jsimmons at infradead.org (James Simmons) Date: Sun, 11 Feb 2018 18:00:08 -0500 Subject: [lustre-devel] [PATCH] staging: lustre: update the TODO list Message-ID: <1518390008-17876-1-git-send-email-jsimmons@infradead.org> As more people become involved with the progression of the lustre client it needs to more clear what needs to be done to leave staging. Update the TODO list with the various bugs and changes to accomplish this. Some are simple bugs and others are far more complex task that will change many lines of code. Some even cover updating the user land utilities to meet the kernel requirements. Several bugs have already been addressed and just need to be pushed to the staging tree. Signed-off-by: James Simmons --- drivers/staging/lustre/TODO | 310 ++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 300 insertions(+), 10 deletions(-) diff --git a/drivers/staging/lustre/TODO b/drivers/staging/lustre/TODO index f194417..94446487 100644 --- a/drivers/staging/lustre/TODO +++ b/drivers/staging/lustre/TODO @@ -1,12 +1,302 @@ -* Possible remaining coding style fix. -* Remove deadcode. -* Separate client/server functionality. Functions only used by server can be - removed from client. -* Clean up libcfs layer. Ideally we can remove include/linux/libcfs entirely. -* Clean up CLIO layer. Lustre client readahead/writeback control needs to better - suit kernel providings. -* Add documents in Documentation. -* Other minor misc cleanups... +Currently all the work directed toward the lustre upstream client is tracked +at the following link: + +https://jira.hpdd.intel.com/browse/LU-9679 + +Under this ticket you will see the following work items that need to be +addressed: + +****************************************************************************** +* libcfs cleanup +* +* https://jira.hpdd.intel.com/browse/LU-9859 +* +* Track all the cleanups and simplification of the libcfs module. Remove +* functions the kernel provides. Possible intergrate some of the functionality +* into the kernel proper. +* +****************************************************************************** + +https://jira.hpdd.intel.com/browse/LU-100086 + +LNET_MINOR conflicts with USERIO_MINOR + +------------------------------------------------------------------------------ + +https://jira.hpdd.intel.com/browse/LU-8130 + +Fix and simplify libcfs hash handling + +------------------------------------------------------------------------------ + +https://jira.hpdd.intel.com/browse/LU-8703 + +The current way we handle SMP is wrong. Platforms like ARM and KNL can have +core and NUMA setups with things like NUMA nodes with no cores. We need to +handle such cases. This work also greatly simplified the lustre SMP code. + +------------------------------------------------------------------------------ + +https://jira.hpdd.intel.com/browse/LU-9019 + +Replace libcfs time API with standard kernel APIs. Also migrate away from +jiffies. We found jiffies can vary on nodes which can lead to corner cases +that can break the file system due to nodes having inconsistent behavior. +So move to time64_t and ktime_t as much as possible. + +****************************************************************************** +* Proper IB support for ko2iblnd +****************************************************************************** +https://jira.hpdd.intel.com/browse/LU-9179 + +Poor performance for the ko2iblnd driver. This is related to many of the +patches below that are missing from the linux client. +------------------------------------------------------------------------------ + +https://jira.hpdd.intel.com/browse/LU-9886 + +Crash in upstream kiblnd_handle_early_rxs() +------------------------------------------------------------------------------ + +https://jira.hpdd.intel.com/browse/LU-10394 / LU-10526 / LU-10089 + +Default to default to using MEM_REG +------------------------------------------------------------------------------ + +https://jira.hpdd.intel.com/browse/LU-10459 + +throttle tx based on queue depth +------------------------------------------------------------------------------ + +https://jira.hpdd.intel.com/browse/LU-9943 + +correct WR fast reg accounting +------------------------------------------------------------------------------ + +https://jira.hpdd.intel.com/browse/LU-10291 + +remove concurrent_sends tunable +------------------------------------------------------------------------------ + +https://jira.hpdd.intel.com/browse/LU-10213 + +calculate qp max_send_wrs properly +------------------------------------------------------------------------------ + +https://jira.hpdd.intel.com/browse/LU-9810 + +use less CQ entries for each connection +------------------------------------------------------------------------------ + +https://jira.hpdd.intel.com/browse/LU-10129 / LU-9180 + +rework map_on_demand behavior +------------------------------------------------------------------------------ + +https://jira.hpdd.intel.com/browse/LU-10129 + +query device capabilities +------------------------------------------------------------------------------ + +https://jira.hpdd.intel.com/browse/LU-10015 + +fix race at kiblnd_connect_peer +------------------------------------------------------------------------------ + +https://jira.hpdd.intel.com/browse/LU-9983 + +allow for discontiguous fragments +------------------------------------------------------------------------------ + +https://jira.hpdd.intel.com/browse/LU-9500 + +Don't Page Align remote_addr with FastReg +------------------------------------------------------------------------------ + +https://jira.hpdd.intel.com/browse/LU-9448 + +handle empty CPTs +------------------------------------------------------------------------------ + +https://jira.hpdd.intel.com/browse/LU-9507 + +Don't Assert On Reconnect with MultiQP +------------------------------------------------------------------------------ + +https://jira.hpdd.intel.com/browse/LU-9472 + +Fix FastReg map/unmap for MLX5 +------------------------------------------------------------------------------ + +https://jira.hpdd.intel.com/browse/LU-9425 + +Turn on 2 sges by default +------------------------------------------------------------------------------ + +https://jira.hpdd.intel.com/browse/LU-8943 + +Enable Multiple OPA Endpoints between Nodes +------------------------------------------------------------------------------ + +https://jira.hpdd.intel.com/browse/LU-5718 + +multiple sges for work request +------------------------------------------------------------------------------ + +https://jira.hpdd.intel.com/browse/LU-9094 + +kill timedout txs from ibp_tx_queue +------------------------------------------------------------------------------ + +https://jira.hpdd.intel.com/browse/LU-9094 + +reconnect peer for REJ_INVALID_SERVICE_ID +------------------------------------------------------------------------------ + +https://jira.hpdd.intel.com/browse/LU-8752 + +Stop MLX5 triggering a dump_cqe +------------------------------------------------------------------------------ + +https://jira.hpdd.intel.com/browse/LU-8874 + +Move ko2iblnd to latest RDMA changes +------------------------------------------------------------------------------ + +https://jira.hpdd.intel.com/browse/LU-8875 / LU-8874 + +Change to new RDMA done callback mechanism + +------------------------------------------------------------------------------ + +https://jira.hpdd.intel.com/browse/LU-9164 / LU-8874 + +Incorporate RDMA map/unamp API's into ko2iblnd + +****************************************************************************** +* sysfs/debugfs fixes +* +* https://jira.hpdd.intel.com/browse/LU-8066 +* +* The original migration to sysfs was done in haste without properly working +* utilities to test the changes. This covers the work to restore the proper +* behavior. Huge project to make this right. +* +****************************************************************************** + +https://jira.hpdd.intel.com/browse/LU-9431 + +The function class_process_proc_param was used for our mass updates of proc +tunables. It didn't work with sysfs and it was just ugly so it was removed. +In the process the ability to mass update thousands of clients was lost. This +work restores this in a sane way. + +------------------------------------------------------------------------------ +https://jira.hpdd.intel.com/browse/LU-9091 + +One the major request of users is the ability to pass in parameters into a +sysfs file in various different units. For example we can set max_pages_per_rpc +but this can vary on platforms due to different platform sizes. So you can +set this like max_pages_per_rpc=16MiB. The original code to handle this written +before the string helpers were created so the code doesn't follow that format +but it would be easy to move to. Currently the string helpers does the reverse +of what we need, changing bytes to string. We need to change a string to bytes. + +****************************************************************************** +* Proper user land to kernel space interface for Lustre +* +* https://jira.hpdd.intel.com/browse/LU-9680 +* +****************************************************************************** + +https://jira.hpdd.intel.com/browse/LU-8915 + +Don't use linux list structure as user land arguments for lnet selftest. +This code is pretty poor quality and really needs to be reworked. + +------------------------------------------------------------------------------ + +https://jira.hpdd.intel.com/browse/LU-8834 + +The lustre ioctl LL_IOC_FUTIMES_3 is very generic. Need to either work with +other file systems with similar functionality and make a common syscall +interface or rework our server code to automagically do it for us. + +------------------------------------------------------------------------------ + +https://jira.hpdd.intel.com/browse/LU-6202 + +Cleanup up ioctl handling. We have many obsolete ioctls. Also the way we do +ioctls can be changed over to netlink. This also has the benefit of working +better with HPC systems that do IO forwarding. Such systems don't like ioctls +very well. + +------------------------------------------------------------------------------ + +https://jira.hpdd.intel.com/browse/LU-9667 + +More cleanups by making our utilities use sysfs instead of ioctls for LNet. +Also it has been requested to move the remaining ioctls to the netlink API. + +****************************************************************************** +* Misc +****************************************************************************** + +------------------------------------------------------------------------------ +https://jira.hpdd.intel.com/browse/LU-9855 + +Clean up obdclass preprocessor code. One of the major eye sores is the various +pointer redirections and macros used by the obdclass. This makes the code very +difficult to understand. It was requested by the Al Viro to clean this up before +we leave staging. + +------------------------------------------------------------------------------ + +https://jira.hpdd.intel.com/browse/LU-9633 + +Migrate to sphinx kernel-doc style comments. Add documents in Documentation. + +------------------------------------------------------------------------------ + +https://jira.hpdd.intel.com/browse/LU-6142 + +Possible remaining coding style fix. Remove deadcode. Enforce kernel code +style. Other minor misc cleanups... + +------------------------------------------------------------------------------ + +https://jira.hpdd.intel.com/browse/LU-8837 + +Separate client/server functionality. Functions only used by server can be +removed from client. Most of this has been done but we need a inspect of the +code to make sure. + +------------------------------------------------------------------------------ + +https://jira.hpdd.intel.com/browse/LU-8964 + +Lustre client readahead/writeback control needs to better suit kernel providings. +Currently its being explored. We could end up replacing the CLIO read ahead +abstract with the kernel proper version. + +------------------------------------------------------------------------------ + +https://jira.hpdd.intel.com/browse/LU-9862 + +Patch that landed for LU-7890 leads to static checker errors +------------------------------------------------------------------------------ + +https://jira.hpdd.intel.com/browse/LU-9868 + +dcache/namei fixes for lustre +------------------------------------------------------------------------------ + +https://jira.hpdd.intel.com/browse/LU-10467 + +use standard linux wait_events macros work by Neil Brown + +------------------------------------------------------------------------------ Please send any patches to Greg Kroah-Hartman , Andreas Dilger -, and Oleg Drokin . +, James Simmons and +Oleg Drokin . -- 1.8.3.1 From joe at perches.com Sun Feb 11 23:16:01 2018 From: joe at perches.com (Joe Perches) Date: Sun, 11 Feb 2018 15:16:01 -0800 Subject: [lustre-devel] [PATCH] staging: lustre: update the TODO list In-Reply-To: <1518390008-17876-1-git-send-email-jsimmons@infradead.org> References: <1518390008-17876-1-git-send-email-jsimmons@infradead.org> Message-ID: <1518390961.4139.1.camel@perches.com> On Sun, 2018-02-11 at 18:00 -0500, James Simmons wrote: > As more people become involved with the progression of the lustre > client it needs to more clear what needs to be done to leave > staging. Update the TODO list with the various bugs and changes > to accomplish this. Some are simple bugs and others are far more > complex task that will change many lines of code. Some even cover > updating the user land utilities to meet the kernel requirements. > Several bugs have already been addressed and just need to be > pushed to the staging tree. Nice list. Are or should those entries that have been addressed also in this new list? From jsimmons at infradead.org Sun Feb 11 23:17:03 2018 From: jsimmons at infradead.org (James Simmons) Date: Sun, 11 Feb 2018 23:17:03 +0000 (GMT) Subject: [lustre-devel] Lustre upstream client TODO list Message-ID: So I sent a patch upstream that laid out what most needs to be done for the linux lustre client to leave staging. I placed the new text here for ease of read so you don't have to go searching for it. Feed back is welcomed. Hoepfully posting it will make it clear what needs to be done. Currently all the work directed toward the lustre upstream client is tracked at the following link: https://jira.hpdd.intel.com/browse/LU-9679 Under this ticket you will see the following work items that need to be addressed: ****************************************************************************** * libcfs cleanup * * https://jira.hpdd.intel.com/browse/LU-9859 * * Track all the cleanups and simplification of the libcfs module. Remove * functions the kernel provides. Possible intergrate some of the functionality * into the kernel proper. * ****************************************************************************** https://jira.hpdd.intel.com/browse/LU-100086 LNET_MINOR conflicts with USERIO_MINOR ------------------------------------------------------------------------------ https://jira.hpdd.intel.com/browse/LU-8130 Fix and simplify libcfs hash handling ------------------------------------------------------------------------------ https://jira.hpdd.intel.com/browse/LU-8703 The current way we handle SMP is wrong. Platforms like ARM and KNL can have core and NUMA setups with things like NUMA nodes with no cores. We need to handle such cases. This work also greatly simplified the lustre SMP code. ------------------------------------------------------------------------------ https://jira.hpdd.intel.com/browse/LU-9019 Replace libcfs time API with standard kernel APIs. Also migrate away from jiffies. We found jiffies can vary on nodes which can lead to corner cases that can break the file system due to nodes having inconsistent behavior. So move to time64_t and ktime_t as much as possible. ****************************************************************************** * Proper IB support for ko2iblnd ****************************************************************************** https://jira.hpdd.intel.com/browse/LU-9179 Poor performance for the ko2iblnd driver. This is related to many of the patches below that are missing from the linux client. ------------------------------------------------------------------------------ https://jira.hpdd.intel.com/browse/LU-9886 Crash in upstream kiblnd_handle_early_rxs() ------------------------------------------------------------------------------ https://jira.hpdd.intel.com/browse/LU-10394 / LU-10526 / LU-10089 Default to default to using MEM_REG ------------------------------------------------------------------------------ https://jira.hpdd.intel.com/browse/LU-10459 throttle tx based on queue depth ------------------------------------------------------------------------------ https://jira.hpdd.intel.com/browse/LU-9943 correct WR fast reg accounting ------------------------------------------------------------------------------ https://jira.hpdd.intel.com/browse/LU-10291 remove concurrent_sends tunable ------------------------------------------------------------------------------ https://jira.hpdd.intel.com/browse/LU-10213 calculate qp max_send_wrs properly ------------------------------------------------------------------------------ https://jira.hpdd.intel.com/browse/LU-9810 use less CQ entries for each connection ------------------------------------------------------------------------------ https://jira.hpdd.intel.com/browse/LU-10129 / LU-9180 rework map_on_demand behavior ------------------------------------------------------------------------------ https://jira.hpdd.intel.com/browse/LU-10129 query device capabilities ------------------------------------------------------------------------------ https://jira.hpdd.intel.com/browse/LU-10015 fix race at kiblnd_connect_peer ------------------------------------------------------------------------------ https://jira.hpdd.intel.com/browse/LU-9983 allow for discontiguous fragments ------------------------------------------------------------------------------ https://jira.hpdd.intel.com/browse/LU-9500 Don't Page Align remote_addr with FastReg ------------------------------------------------------------------------------ https://jira.hpdd.intel.com/browse/LU-9448 handle empty CPTs ------------------------------------------------------------------------------ https://jira.hpdd.intel.com/browse/LU-9507 Don't Assert On Reconnect with MultiQP ------------------------------------------------------------------------------ https://jira.hpdd.intel.com/browse/LU-9472 Fix FastReg map/unmap for MLX5 ------------------------------------------------------------------------------ https://jira.hpdd.intel.com/browse/LU-9425 Turn on 2 sges by default ------------------------------------------------------------------------------ https://jira.hpdd.intel.com/browse/LU-8943 Enable Multiple OPA Endpoints between Nodes ------------------------------------------------------------------------------ https://jira.hpdd.intel.com/browse/LU-5718 multiple sges for work request ------------------------------------------------------------------------------ https://jira.hpdd.intel.com/browse/LU-9094 kill timedout txs from ibp_tx_queue ------------------------------------------------------------------------------ https://jira.hpdd.intel.com/browse/LU-9094 reconnect peer for REJ_INVALID_SERVICE_ID ------------------------------------------------------------------------------ https://jira.hpdd.intel.com/browse/LU-8752 Stop MLX5 triggering a dump_cqe ------------------------------------------------------------------------------ https://jira.hpdd.intel.com/browse/LU-8874 Move ko2iblnd to latest RDMA changes ------------------------------------------------------------------------------ https://jira.hpdd.intel.com/browse/LU-8875 / LU-8874 Change to new RDMA done callback mechanism ------------------------------------------------------------------------------ https://jira.hpdd.intel.com/browse/LU-9164 / LU-8874 Incorporate RDMA map/unamp API's into ko2iblnd ****************************************************************************** * sysfs/debugfs fixes * * https://jira.hpdd.intel.com/browse/LU-8066 * * The original migration to sysfs was done in haste without properly working * utilities to test the changes. This covers the work to restore the proper * behavior. Huge project to make this right. * ****************************************************************************** https://jira.hpdd.intel.com/browse/LU-9431 The function class_process_proc_param was used for our mass updates of proc tunables. It didn't work with sysfs and it was just ugly so it was removed. In the process the ability to mass update thousands of clients was lost. This work restores this in a sane way. ------------------------------------------------------------------------------ https://jira.hpdd.intel.com/browse/LU-9091 One the major request of users is the ability to pass in parameters into a sysfs file in various different units. For example we can set max_pages_per_rpc but this can vary on platforms due to different platform sizes. So you can set this like max_pages_per_rpc=16MiB. The original code to handle this written before the string helpers were created so the code doesn't follow that format but it would be easy to move to. Currently the string helpers does the reverse of what we need, changing bytes to string. We need to change a string to bytes. ****************************************************************************** * Proper user land to kernel space interface for Lustre * * https://jira.hpdd.intel.com/browse/LU-9680 * ****************************************************************************** https://jira.hpdd.intel.com/browse/LU-8915 Don't use linux list structure as user land arguments for lnet selftest. This code is pretty poor quality and really needs to be reworked. ------------------------------------------------------------------------------ https://jira.hpdd.intel.com/browse/LU-8834 The lustre ioctl LL_IOC_FUTIMES_3 is very generic. Need to either work with other file systems with similar functionality and make a common syscall interface or rework our server code to automagically do it for us. ------------------------------------------------------------------------------ https://jira.hpdd.intel.com/browse/LU-6202 Cleanup up ioctl handling. We have many obsolete ioctls. Also the way we do ioctls can be changed over to netlink. This also has the benefit of working better with HPC systems that do IO forwarding. Such systems don't like ioctls very well. ------------------------------------------------------------------------------ https://jira.hpdd.intel.com/browse/LU-9667 More cleanups by making our utilities use sysfs instead of ioctls for LNet. Also it has been requested to move the remaining ioctls to the netlink API. ****************************************************************************** * Misc ****************************************************************************** ------------------------------------------------------------------------------ https://jira.hpdd.intel.com/browse/LU-9855 Clean up obdclass preprocessor code. One of the major eye sores is the various pointer redirections and macros used by the obdclass. This makes the code very difficult to understand. It was requested by the Al Viro to clean this up before we leave staging. ------------------------------------------------------------------------------ https://jira.hpdd.intel.com/browse/LU-9633 Migrate to sphinx kernel-doc style comments. Add documents in Documentation. ------------------------------------------------------------------------------ https://jira.hpdd.intel.com/browse/LU-6142 Possible remaining coding style fix. Remove deadcode. Enforce kernel code style. Other minor misc cleanups... ------------------------------------------------------------------------------ https://jira.hpdd.intel.com/browse/LU-8837 Separate client/server functionality. Functions only used by server can be removed from client. Most of this has been done but we need a inspect of the code to make sure. ------------------------------------------------------------------------------ https://jira.hpdd.intel.com/browse/LU-8964 Lustre client readahead/writeback control needs to better suit kernel providings. Currently its being explored. We could end up replacing the CLIO read ahead abstract with the kernel proper version. ------------------------------------------------------------------------------ https://jira.hpdd.intel.com/browse/LU-9862 Patch that landed for LU-7890 leads to static checker errors ------------------------------------------------------------------------------ https://jira.hpdd.intel.com/browse/LU-9868 dcache/namei fixes for lustre ------------------------------------------------------------------------------ https://jira.hpdd.intel.com/browse/LU-10467 use standard linux wait_events macros work by Neil Brown ------------------------------------------------------------------------------ From neilb at suse.com Sun Feb 11 23:44:23 2018 From: neilb at suse.com (NeilBrown) Date: Mon, 12 Feb 2018 10:44:23 +1100 Subject: [lustre-devel] [PATCH 41/80] staging: lustre: lmv: separate master object with master stripe In-Reply-To: References: <1471378773-24590-1-git-send-email-jsimmons@infradead.org> <1471378773-24590-42-git-send-email-jsimmons@infradead.org> <87r2pvkkl5.fsf@notabene.neil.brown.name> <09CE6CEC-52BC-4B29-B609-40DE68A64A33@intel.com> <87o9kylux7.fsf@notabene.neil.brown.name> Message-ID: <87h8qnrt0o.fsf@notabene.neil.brown.name> On Thu, Feb 08 2018, Oleg Drokin wrote: >> On Feb 8, 2018, at 10:10 PM, NeilBrown wrote: >> >> On Thu, Feb 08 2018, Oleg Drokin wrote: >> >>>> On Feb 8, 2018, at 8:39 PM, NeilBrown wrote: >>>> >>>> On Tue, Aug 16 2016, James Simmons wrote: >>> >>> my that’s an old patch >>> >>>> >> ... >>>> >>>> Whoever converted it to "!strcmp()" inverted the condition. This is a >>>> perfect example of why I absolutely *loathe* the "!strcmp()" construct!! >>>> >>>> This causes many tests in the 'sanity' test suite to return >>>> -ENOMEM (that had me puzzled for a while!!). >>> >>> huh? I am not seeing anything of the sort and I was running sanity >>> all the time until a recent pause (but going to resume). >> >> That does surprised me - I reproduce it every time. >> I have two VMs running a SLE12-SP2 kernel with patches from >> lustre-release applied. These are servers. They have 2 3G virtual disks >> each. >> I have two over VMs running current mainline. These are clients. >> >> I guess your 'recent pause' included between v4.15-rc1 (8e55b6fd0660) >> and v4.15-rc6 (a93639090a27) - a full month when lustre wouldn't work at >> all :-( > > More than that, but I am pretty sure James Simmons is running tests all the time too > (he has a different config, I only have tcp). > >>>> This seems to suggest that no-one has been testing the mainline linux >>>> lustre. >>>> It also seems to suggest that there is a good chance that there >>>> are other bugs that have crept in while no-one has really been caring. >>>> Given that the sanity test suite doesn't complete for me, but just >>>> hangs (in test_27z I think), that seems particularly likely. >>> >>> Works for me, here’s a run from earlier today on 4.15.0: >> >> Well that's encouraging .. I haven't looked into this one yet - I'm not >> even sure where to start. > > m… debug logs for example (greatly neutered in staging tree, but still useful)? > try lctl dk and see what’s in there. Debug logs seem to tell me that some message is being sent to a server and a reply is being received, but that request we are waiting on doesn't make progress. I plan to dig in and learn more about how lustre rpc works so I have a better changes of interpreted those debug logs. > >>> Instead the plan was to clean up the staging client into acceptable state, >>> move it out of staging, bring in all the missing features and then >>> drop the client (more or less) from the lustre-release. >> >> That sounds like a great plan. Any idea why it didn't happen? > > Because meeting open-ended demands is hard and certain demands sound like > “throw away your X and rewrite it from scratch" (e.g. everything IB-related). My narrow perspective on IB - from when rdma support was added to the NFS server - is that it is broken by design and impossible to do "right". So different people could easily have different ideas on how to make the best of a bad lot. I might try to have a look. > > Certain things that sound useless (like the debug subsystem in Lustre) > is very useful when you have a 10k nodes in a cluster and need to selectively > pull stuff from a run to debug a complicated cross-node interaction. > I asked NFS people how do they do it and they don’t have anything that scales > and usually involves reducing the problem to a much smaller set of nodes first. the "rpcdebug" stuff that Linux/nfs has is sometimes useful, but some parts are changing to tracepoints and some parts have remained, which is a little confusing. The fact that lustre tracing seems to *always* log everything so that if something goes wrong you can extract that last few meg(?) of logs seems really useful. I discovered - thanks to James - https://jira.hpdd.intel.com/browse/LU-8980 Add tracepoint support to Lustre which is "closed", but I cannot find any trace of tracepoints in drivers/staging or in lustre-release. Maybe I'm confused. I suspect tracepoints is a good way to go. > >> It seems there is a lot of upstream work mixed in with the clean up, and >> I don't think that really helps anyone. > > I don’t understand what you mean here. Just that I thought that the main point of drivers/staging is to get the code into a mergable state, and if feature addition happens at the same time, then priorities get blurred and goals don't get reached. > >> Is it at all realistic that the client might be removed from >> lustre-release? That might be a good goal to work towards. > > Assuming we can bring the whole functionality over - sure. > > Of course there’d still be some separate development place and we would > need to create patches (new features?) for like SuSE and other distros > and for testing of server features, I guess, but that could just that - > a side branch somewhere I hope. Of course - code doesn't go upstream until it is ready. Lots of development happens elsewhere. Of course distros like SUSE would generally rather ship code that was "ready" and so like to see it upstream. There is usually room for negotiation. > > It’s not that we are super glad to chase every kernel vendors put out, > of course it would be much easier if the kernels already included > a very functional Lustre client. > >>>> Might it make sense to instead start cleaning up the code in >>>> lustre-release so as to make it meet the upstream kernel standards. >>>> Then when the time is right, the kernel code can be moved *out* of >>>> lustre-release and *in* to linux. Then development can continue in >>>> Linux (just like it does with other Linux filesystems). >>> >>> While we can be cleaning lustre in lustre-release, there are some things >>> we cannot do as easily, e.g. decoupling Lustre client from the server. >>> Also it would not attract any reviews from all the janitor or >>> (more importantly) Al Viro and other people with a sharp eyes. >>> >>>> An added bonus of this is that there is an obvious path to getting >>>> server support in mainline Linux. The current situation of client-only >>>> support seems weird given how interdependent the two are. >>> >>> Given the pushback Lustre client was given I have no hope Lustre server >>> will get into mainline in my lifetime. >> >> Even if it is horrible it would be nice to have it in staging... I guess >> the changes required to ext4 prohibit that... I don't suppose it can be >> made to work with mainline ext4 in a reduced-functionality-and-performance >> way?? > > We support unpatched ZFS as a server too! ;) So that that mean you would expect lustre-server to work with unpatched ext4? In that case I won't give up hope of seeing the server in mainline in my lifetime. Client first though. > (and if somebody invests the time into it, there was some half-baked btrfs > backend too I think). > That said nobody here believes in any success of pushing Lustre server into > mainline. > It would just be easier to push the whole server into userspace (And there > was a project like this in the past, now abandoned because it was mostly > targeting Solaris anyway). > >> I think it would be a lot easier to motivate forward progress if there >> were a credible end goal of everything being in mainline. >> >>> >>>> What do others think? Is there any chance that the current lustre in >>>> Linux will ever be more than a poor second-cousin to the external >>>> lustre-release. If there isn't, should we just discard it now and move >>>> on? >>> >>> >>> I think many useful cleanups and fixes came from the staging tree at >>> the very least. >>> The biggest problem with it all is that we are in staging tree so >>> we cannot bring it to parity much. And we are in staging tree because >>> there’s a whole bunch of “cleanups” requested that take a lot of effort >>> (in both implementing them and then in finding other ways of achieving >>> things that were done in old ways before). >> >> Do you have a list of requested cleanups? I would find that to be >> useful. > > As Greg would tell you, “if you don’t know what needs to be done, > let’s just remove the whole thing from staging now”. Of course, but I don't expect that I will see the same things that others see. And if people have gone to the trouble to provide feedback, it seems polite to record that feed back for all to see. > > I assume you saw drivers/staging/lustre/TODO already, it’s only partially done. Yes - it isn't very detailed though. Maybe I'll flesh it out with some of the things you have said. > > We had a bunch of other requests from various people ranging from wholesale > removal of various parts to making sure there’s no checkpatch warnings > (Turned out rather hard to do, even though we greatly pared the > numbers). checkpatch is a useful guide, but an awful master. % find drivers/staging/lustre/ -name '*.[ch]' | while read a; do ./scripts/checkpatch.pl --max-line-length=10000 --no-summary -f $a done|grep '^ERROR' | sort | uniq -c 17 ERROR: Macros with complex values should be enclosed in parentheses 2 ERROR: Macros with multiple statements should be enclosed in a do - while loop 12 ERROR: No #include in ...include/uapi/... should use a uapi/ path prefix 1 ERROR: space required before the open brace '{' 8 ERROR: that open brace { should be on the previous line 1 ERROR: trailing statements should be on next line 1 ERROR: trailing whitespace Thanks isn't too bad - obviously nearly there with checkpatch. Lots more warnings - some might be interesting. wholesale removal - like the prng, the workqueues, and the ll_wait_event() macro? I can do that :-) > > I have some patches to make Lustre a lot more monolithic too. Yes, it annoys me that I cannot build without modules. I took some steps towards fixing that and went off down a rabbit hole.. Should be fairly easy. > People want us to remove our indirections hell so the code is more readable > (I have some patches that need to be freshened up some that help here a bit, > but the work is huge.) But indirections solve all problems :-) > > Other requests come out as some of the prior ones get completed due to > “you need o finish current level of cleanups so that we can see what other > cleanups are needed, the current code is too bad to see everything” pretty much. Thanks a lot for your helpful reply. NeilBrown -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 832 bytes Desc: not available URL: From neilb at suse.com Sun Feb 11 23:50:04 2018 From: neilb at suse.com (NeilBrown) Date: Mon, 12 Feb 2018 10:50:04 +1100 Subject: [lustre-devel] [PATCH 41/80] staging: lustre: lmv: separate master object with master stripe In-Reply-To: References: <1471378773-24590-1-git-send-email-jsimmons@infradead.org> <1471378773-24590-42-git-send-email-jsimmons@infradead.org> <87r2pvkkl5.fsf@notabene.neil.brown.name> <09CE6CEC-52BC-4B29-B609-40DE68A64A33@intel.com> <87o9kylux7.fsf@notabene.neil.brown.name> Message-ID: <87eflrrsr7.fsf@notabene.neil.brown.name> On Sat, Feb 10 2018, James Simmons wrote: >> > On Feb 8, 2018, at 10:10 PM, NeilBrown wrote: >> > >> > On Thu, Feb 08 2018, Oleg Drokin wrote: >> > >> >>> On Feb 8, 2018, at 8:39 PM, NeilBrown wrote: >> >>> >> >>> On Tue, Aug 16 2016, James Simmons wrote: >> >> >> >> my that’s an old patch >> >> >> >>> >> > ... >> >>> >> >>> Whoever converted it to "!strcmp()" inverted the condition. This is a >> >>> perfect example of why I absolutely *loathe* the "!strcmp()" construct!! >> >>> >> >>> This causes many tests in the 'sanity' test suite to return >> >>> -ENOMEM (that had me puzzled for a while!!). >> >> >> >> huh? I am not seeing anything of the sort and I was running sanity >> >> all the time until a recent pause (but going to resume). >> > >> > That does surprised me - I reproduce it every time. >> > I have two VMs running a SLE12-SP2 kernel with patches from >> > lustre-release applied. These are servers. They have 2 3G virtual disks >> > each. >> > I have two over VMs running current mainline. These are clients. >> > >> > I guess your 'recent pause' included between v4.15-rc1 (8e55b6fd0660) >> > and v4.15-rc6 (a93639090a27) - a full month when lustre wouldn't work at >> > all :-( >> >> More than that, but I am pretty sure James Simmons is running tests all the time too >> (he has a different config, I only have tcp). > > Yes I have been testing and haven't encountered this problem. Let me try > the fix you pointed out. Yeah, I guess I over reacted a bit in suggesting that no-one can have been testing - sorry about that. It seemed really strange though as the bug was so easy for me to hit. Maybe - as you suggest in another email - it is due to some client/server incompatibility. I guess it is unavoidable with an fs like lustre to have incompatible protocol changes. Is there any mechanism for detecting the version of other peers in the cluster and refusing to run if versions are incompatible? If you haven't hit the problem in testing, I suspect you aren't touching that code path at all. Maybe put a BUG() call in there to see :-) > >> > Do you have a list of requested cleanups? I would find that to be >> > useful. >> >> As Greg would tell you, “if you don’t know what needs to be done, >> let’s just remove the whole thing from staging now”. >> >> I assume you saw drivers/staging/lustre/TODO already, it’s only partially done. > > Actually the complete list is at : > > https://jira.hpdd.intel.com/browse/LU-9679 > > I need to move that to our TODO list. Sorry I have been short on cycles. Just adding that link to TODO would be a great start. I might do that when I next send some patches. Thanks, NeilBrown -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 832 bytes Desc: not available URL: From neilb at suse.com Sun Feb 11 23:54:51 2018 From: neilb at suse.com (NeilBrown) Date: Mon, 12 Feb 2018 10:54:51 +1100 Subject: [lustre-devel] Lustre upstream client TODO list In-Reply-To: References: Message-ID: <87bmgvrsj8.fsf@notabene.neil.brown.name> On Sun, Feb 11 2018, James Simmons wrote: > So I sent a patch upstream that laid out what most needs to be done for > the linux lustre client to leave staging. I placed the new text here for > ease of read so you don't have to go searching for it. Feed back is > welcomed. Hoepfully posting it will make it clear what needs to be done. Thanks so much for putting this together and pushing it out. I really appreciated it and hope to show that appreciation with patches :-) NeilBrown > > Currently all the work directed toward the lustre upstream client is tracked > at the following link: > > https://jira.hpdd.intel.com/browse/LU-9679 > > Under this ticket you will see the following work items that need to be > addressed: > > ****************************************************************************** > * libcfs cleanup > * > * https://jira.hpdd.intel.com/browse/LU-9859 > * > * Track all the cleanups and simplification of the libcfs module. Remove > * functions the kernel provides. Possible intergrate some of the functionality > * into the kernel proper. > * > ****************************************************************************** > > https://jira.hpdd.intel.com/browse/LU-100086 > > LNET_MINOR conflicts with USERIO_MINOR > > ------------------------------------------------------------------------------ > > https://jira.hpdd.intel.com/browse/LU-8130 > > Fix and simplify libcfs hash handling > > ------------------------------------------------------------------------------ > > https://jira.hpdd.intel.com/browse/LU-8703 > > The current way we handle SMP is wrong. Platforms like ARM and KNL can have > core and NUMA setups with things like NUMA nodes with no cores. We need to > handle such cases. This work also greatly simplified the lustre SMP code. > > ------------------------------------------------------------------------------ > > https://jira.hpdd.intel.com/browse/LU-9019 > > Replace libcfs time API with standard kernel APIs. Also migrate away from > jiffies. We found jiffies can vary on nodes which can lead to corner cases > that can break the file system due to nodes having inconsistent behavior. > So move to time64_t and ktime_t as much as possible. > > ****************************************************************************** > * Proper IB support for ko2iblnd > ****************************************************************************** > https://jira.hpdd.intel.com/browse/LU-9179 > > Poor performance for the ko2iblnd driver. This is related to many of the > patches below that are missing from the linux client. > ------------------------------------------------------------------------------ > > https://jira.hpdd.intel.com/browse/LU-9886 > > Crash in upstream kiblnd_handle_early_rxs() > ------------------------------------------------------------------------------ > > https://jira.hpdd.intel.com/browse/LU-10394 / LU-10526 / LU-10089 > > Default to default to using MEM_REG > ------------------------------------------------------------------------------ > > https://jira.hpdd.intel.com/browse/LU-10459 > > throttle tx based on queue depth > ------------------------------------------------------------------------------ > > https://jira.hpdd.intel.com/browse/LU-9943 > > correct WR fast reg accounting > ------------------------------------------------------------------------------ > > https://jira.hpdd.intel.com/browse/LU-10291 > > remove concurrent_sends tunable > ------------------------------------------------------------------------------ > > https://jira.hpdd.intel.com/browse/LU-10213 > > calculate qp max_send_wrs properly > ------------------------------------------------------------------------------ > > https://jira.hpdd.intel.com/browse/LU-9810 > > use less CQ entries for each connection > ------------------------------------------------------------------------------ > > https://jira.hpdd.intel.com/browse/LU-10129 / LU-9180 > > rework map_on_demand behavior > ------------------------------------------------------------------------------ > > https://jira.hpdd.intel.com/browse/LU-10129 > > query device capabilities > ------------------------------------------------------------------------------ > > https://jira.hpdd.intel.com/browse/LU-10015 > > fix race at kiblnd_connect_peer > ------------------------------------------------------------------------------ > > https://jira.hpdd.intel.com/browse/LU-9983 > > allow for discontiguous fragments > ------------------------------------------------------------------------------ > > https://jira.hpdd.intel.com/browse/LU-9500 > > Don't Page Align remote_addr with FastReg > ------------------------------------------------------------------------------ > > https://jira.hpdd.intel.com/browse/LU-9448 > > handle empty CPTs > ------------------------------------------------------------------------------ > > https://jira.hpdd.intel.com/browse/LU-9507 > > Don't Assert On Reconnect with MultiQP > ------------------------------------------------------------------------------ > > https://jira.hpdd.intel.com/browse/LU-9472 > > Fix FastReg map/unmap for MLX5 > ------------------------------------------------------------------------------ > > https://jira.hpdd.intel.com/browse/LU-9425 > > Turn on 2 sges by default > ------------------------------------------------------------------------------ > > https://jira.hpdd.intel.com/browse/LU-8943 > > Enable Multiple OPA Endpoints between Nodes > ------------------------------------------------------------------------------ > > https://jira.hpdd.intel.com/browse/LU-5718 > > multiple sges for work request > ------------------------------------------------------------------------------ > > https://jira.hpdd.intel.com/browse/LU-9094 > > kill timedout txs from ibp_tx_queue > ------------------------------------------------------------------------------ > > https://jira.hpdd.intel.com/browse/LU-9094 > > reconnect peer for REJ_INVALID_SERVICE_ID > ------------------------------------------------------------------------------ > > https://jira.hpdd.intel.com/browse/LU-8752 > > Stop MLX5 triggering a dump_cqe > ------------------------------------------------------------------------------ > > https://jira.hpdd.intel.com/browse/LU-8874 > > Move ko2iblnd to latest RDMA changes > ------------------------------------------------------------------------------ > > https://jira.hpdd.intel.com/browse/LU-8875 / LU-8874 > > Change to new RDMA done callback mechanism > > ------------------------------------------------------------------------------ > > https://jira.hpdd.intel.com/browse/LU-9164 / LU-8874 > > Incorporate RDMA map/unamp API's into ko2iblnd > > ****************************************************************************** > * sysfs/debugfs fixes > * > * https://jira.hpdd.intel.com/browse/LU-8066 > * > * The original migration to sysfs was done in haste without properly working > * utilities to test the changes. This covers the work to restore the proper > * behavior. Huge project to make this right. > * > ****************************************************************************** > > https://jira.hpdd.intel.com/browse/LU-9431 > > The function class_process_proc_param was used for our mass updates of proc > tunables. It didn't work with sysfs and it was just ugly so it was removed. > In the process the ability to mass update thousands of clients was lost. This > work restores this in a sane way. > > ------------------------------------------------------------------------------ > https://jira.hpdd.intel.com/browse/LU-9091 > > One the major request of users is the ability to pass in parameters into a > sysfs file in various different units. For example we can set max_pages_per_rpc > but this can vary on platforms due to different platform sizes. So you can > set this like max_pages_per_rpc=16MiB. The original code to handle this written > before the string helpers were created so the code doesn't follow that format > but it would be easy to move to. Currently the string helpers does the reverse > of what we need, changing bytes to string. We need to change a string to bytes. > > ****************************************************************************** > * Proper user land to kernel space interface for Lustre > * > * https://jira.hpdd.intel.com/browse/LU-9680 > * > ****************************************************************************** > > https://jira.hpdd.intel.com/browse/LU-8915 > > Don't use linux list structure as user land arguments for lnet selftest. > This code is pretty poor quality and really needs to be reworked. > > ------------------------------------------------------------------------------ > > https://jira.hpdd.intel.com/browse/LU-8834 > > The lustre ioctl LL_IOC_FUTIMES_3 is very generic. Need to either work with > other file systems with similar functionality and make a common syscall > interface or rework our server code to automagically do it for us. > > ------------------------------------------------------------------------------ > > https://jira.hpdd.intel.com/browse/LU-6202 > > Cleanup up ioctl handling. We have many obsolete ioctls. Also the way we do > ioctls can be changed over to netlink. This also has the benefit of working > better with HPC systems that do IO forwarding. Such systems don't like ioctls > very well. > > ------------------------------------------------------------------------------ > > https://jira.hpdd.intel.com/browse/LU-9667 > > More cleanups by making our utilities use sysfs instead of ioctls for LNet. > Also it has been requested to move the remaining ioctls to the netlink API. > > ****************************************************************************** > * Misc > ****************************************************************************** > > ------------------------------------------------------------------------------ > https://jira.hpdd.intel.com/browse/LU-9855 > > Clean up obdclass preprocessor code. One of the major eye sores is the various > pointer redirections and macros used by the obdclass. This makes the code very > difficult to understand. It was requested by the Al Viro to clean this up before > we leave staging. > > ------------------------------------------------------------------------------ > > https://jira.hpdd.intel.com/browse/LU-9633 > > Migrate to sphinx kernel-doc style comments. Add documents in Documentation. > > ------------------------------------------------------------------------------ > > https://jira.hpdd.intel.com/browse/LU-6142 > > Possible remaining coding style fix. Remove deadcode. Enforce kernel code > style. Other minor misc cleanups... > > ------------------------------------------------------------------------------ > > https://jira.hpdd.intel.com/browse/LU-8837 > > Separate client/server functionality. Functions only used by server can be > removed from client. Most of this has been done but we need a inspect of the > code to make sure. > > ------------------------------------------------------------------------------ > > https://jira.hpdd.intel.com/browse/LU-8964 > > Lustre client readahead/writeback control needs to better suit kernel providings. > Currently its being explored. We could end up replacing the CLIO read ahead > abstract with the kernel proper version. > > ------------------------------------------------------------------------------ > > https://jira.hpdd.intel.com/browse/LU-9862 > > Patch that landed for LU-7890 leads to static checker errors > ------------------------------------------------------------------------------ > > https://jira.hpdd.intel.com/browse/LU-9868 > > dcache/namei fixes for lustre > ------------------------------------------------------------------------------ > > https://jira.hpdd.intel.com/browse/LU-10467 > > use standard linux wait_events macros work by Neil Brown > > ------------------------------------------------------------------------------ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 832 bytes Desc: not available URL: From oleg.drokin at intel.com Mon Feb 12 00:06:30 2018 From: oleg.drokin at intel.com (Oleg Drokin) Date: Sun, 11 Feb 2018 19:06:30 -0500 Subject: [lustre-devel] [PATCH 41/80] staging: lustre: lmv: separate master object with master stripe In-Reply-To: <87eflrrsr7.fsf@notabene.neil.brown.name> References: <1471378773-24590-1-git-send-email-jsimmons@infradead.org> <1471378773-24590-42-git-send-email-jsimmons@infradead.org> <87r2pvkkl5.fsf@notabene.neil.brown.name> <09CE6CEC-52BC-4B29-B609-40DE68A64A33@intel.com> <87o9kylux7.fsf@notabene.neil.brown.name> <87eflrrsr7.fsf@notabene.neil.brown.name> Message-ID: <3E2C6545-B1F2-46AD-8539-A3F65DBE5EF9@intel.com> > On Feb 11, 2018, at 6:50 PM, NeilBrown wrote: > > Maybe - as you suggest in another email - it is due to some > client/server incompatibility. I guess it is unavoidable with an fs > like lustre to have incompatible protocol changes. Is there any > mechanism for detecting the version of other peers in the cluster and > refusing to run if versions are incompatible? Yes, client and server exchange “feature bits” at connect time and only use the subset of features that both can understand. Bye, Oleg From oleg.drokin at intel.com Mon Feb 12 00:52:26 2018 From: oleg.drokin at intel.com (Oleg Drokin) Date: Sun, 11 Feb 2018 19:52:26 -0500 Subject: [lustre-devel] [PATCH 41/80] staging: lustre: lmv: separate master object with master stripe In-Reply-To: <87h8qnrt0o.fsf@notabene.neil.brown.name> References: <1471378773-24590-1-git-send-email-jsimmons@infradead.org> <1471378773-24590-42-git-send-email-jsimmons@infradead.org> <87r2pvkkl5.fsf@notabene.neil.brown.name> <09CE6CEC-52BC-4B29-B609-40DE68A64A33@intel.com> <87o9kylux7.fsf@notabene.neil.brown.name> <87h8qnrt0o.fsf@notabene.neil.brown.name> Message-ID: <5928AEAA-EDD6-4A6E-A9BA-1FA5974C1BD9@intel.com> > On Feb 11, 2018, at 6:44 PM, NeilBrown wrote: > > On Thu, Feb 08 2018, Oleg Drokin wrote: >> >> Certain things that sound useless (like the debug subsystem in Lustre) >> is very useful when you have a 10k nodes in a cluster and need to selectively >> pull stuff from a run to debug a complicated cross-node interaction. >> I asked NFS people how do they do it and they don’t have anything that scales >> and usually involves reducing the problem to a much smaller set of nodes first. > > the "rpcdebug" stuff that Linux/nfs has is sometimes useful, but some parts > are changing to tracepoints and some parts have remained, which is a > little confusing. > > The fact that lustre tracing seems to *always* log everything so that if > something goes wrong you can extract that last few meg(?) of logs seems > really useful. Not really. Lustre also has a bitmask for logs (since otherwise all those prints are pretty cpu taxing), but what makes those logs better is: the size is unlimited, not constrained by dmesg buffer size. You can capture those logs from a crashdump (something I really wish somebody would implement for tracepoint buffers, but alas, I have not found anything for this yet - we have a crash plugin to extract lustre debug logs from a kernel crashdump). >>> >>> Even if it is horrible it would be nice to have it in staging... I guess >>> the changes required to ext4 prohibit that... I don't suppose it can be >>> made to work with mainline ext4 in a reduced-functionality-and-performance >>> way?? >> >> We support unpatched ZFS as a server too! ;) > > So that that mean you would expect lustre-server to work with unpatched > ext4? In that case I won't give up hope of seeing the server in mainline > in my lifetime. Client first though. While unpatched ext4 might in theory be possible, currently it does not export everything we need from the transaction/fs control perspective. Bye, Oleg From neilb at suse.com Mon Feb 12 01:05:44 2018 From: neilb at suse.com (NeilBrown) Date: Mon, 12 Feb 2018 12:05:44 +1100 Subject: [lustre-devel] testing lustre In-Reply-To: <649030A5-48A9-4C67-A911-28FBC5F183E1@intel.com> References: <151538168618.23920.8261096424342988792.stgit@noble> <20180108145943.GA5761@kroah.com> <87po5gmqph.fsf@notabene.neil.brown.name> <94546355-FD93-4B50-AC98-4AB38EDC9BF0@gmail.com> <87tvurkm9e.fsf@notabene.neil.brown.name> <3C5A4767-EBDC-42C0-9A46-46C8356C5067@gmail.com> <87r2ptpvp2.fsf@notabene.neil.brown.name> <649030A5-48A9-4C67-A911-28FBC5F183E1@intel.com> Message-ID: <874lmnrp93.fsf@notabene.neil.brown.name> On Fri, Feb 09 2018, Oleg Drokin wrote: >> On Feb 9, 2018, at 6:52 PM, NeilBrown wrote: >> >> On Fri, Feb 09 2018, Благодаренко Артём wrote: >>> >>>> I cannot find anywhere in the code that would write out a new label, or >>>> that would call e2label or tune2fs to do it. >>> >>> mount utility (mount_lustre.c) >>> Main-> >>> parse_ldd() -> ldd() read ldd. In and change : to - in memory >>> mount() - mount target, register it on mgs >>> label_lustre () write new label with “-“ on disk >> >> Ahhh.. found it. Thanks for lot for the details!! >> >> And and I can see in an strace that this is being called - mount.lustre >> forks and runs "tune2fs -f -L ....". >> But still the volume name doesn't change. >> Maybe I have a buggy tune2fs. >> >> I note that tune2fs says "recovering journal" or something like that. >> I suspect we don't want that to happen when the filesystem is mounted. >> Maybe that is related to the failure to change the label. >> I will dig in more deeply. > > This sounds very familiar, but I cannot quite find the ticket for it. > > Anyway, I do wonder if you use our patched e2fsprogs and if not, > if you could give it a try first. No, I haven't been using the patched e2fsprogs ... looks .... Oh, that branched from mainline in May 2015 with nothing new merged in since. One of the things that hasn't been merged in is Commit: c864b9eb5001 ("tune2fs: recover the journal") so lustre-tunefs doesn't try to recover the journal. Upstream does and breaks lustre. This is easily fixed with diff --git a/misc/tune2fs.c b/misc/tune2fs.c index c33fb9d80b10..703e55b6b972 100644 --- a/misc/tune2fs.c +++ b/misc/tune2fs.c @@ -3337,7 +3337,7 @@ _("Warning: The journal is dirty. You may wish to replay the journal like:\n\n" } #else /* Recover the journal if possible. */ - if ((open_flag & EXT2_FLAG_RW) && !(mount_flags & EXT2_MF_MOUNTED) && + if ((open_flag & EXT2_FLAG_RW) && !(mount_flags & (EXT2_MF_BUSY | EXT2_MF_MOUNTED)) && ext2fs_has_feature_journal_needs_recovery(fs->super)) { errcode_t err; which I think is a sensible patch in any case. I'll send it upstream. Thanks, NeilBrown -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 832 bytes Desc: not available URL: From paf at cray.com Mon Feb 12 01:15:09 2018 From: paf at cray.com (Patrick Farrell) Date: Mon, 12 Feb 2018 01:15:09 +0000 Subject: [lustre-devel] Lustre upstream client TODO list In-Reply-To: <87bmgvrsj8.fsf@notabene.neil.brown.name> References: , <87bmgvrsj8.fsf@notabene.neil.brown.name> Message-ID: Neil, Apologies if you've answered this elsewhere, but what's the genesis of your current (extremely welcome) interest in Lustre? Some commitment by SUSE? Regards, - Patrick ________________________________ From: lustre-devel on behalf of NeilBrown Sent: Sunday, February 11, 2018 5:54:51 PM To: James Simmons; Lustre Development List Cc: Oleg Drokin Subject: Re: [lustre-devel] Lustre upstream client TODO list On Sun, Feb 11 2018, James Simmons wrote: > So I sent a patch upstream that laid out what most needs to be done for > the linux lustre client to leave staging. I placed the new text here for > ease of read so you don't have to go searching for it. Feed back is > welcomed. Hoepfully posting it will make it clear what needs to be done. Thanks so much for putting this together and pushing it out. I really appreciated it and hope to show that appreciation with patches :-) NeilBrown > > Currently all the work directed toward the lustre upstream client is tracked > at the following link: > > https://jira.hpdd.intel.com/browse/LU-9679 > > Under this ticket you will see the following work items that need to be > addressed: > > ****************************************************************************** > * libcfs cleanup > * > * https://jira.hpdd.intel.com/browse/LU-9859 > * > * Track all the cleanups and simplification of the libcfs module. Remove > * functions the kernel provides. Possible intergrate some of the functionality > * into the kernel proper. > * > ****************************************************************************** > > https://jira.hpdd.intel.com/browse/LU-100086 > > LNET_MINOR conflicts with USERIO_MINOR > > ------------------------------------------------------------------------------ > > https://jira.hpdd.intel.com/browse/LU-8130 > > Fix and simplify libcfs hash handling > > ------------------------------------------------------------------------------ > > https://jira.hpdd.intel.com/browse/LU-8703 > > The current way we handle SMP is wrong. Platforms like ARM and KNL can have > core and NUMA setups with things like NUMA nodes with no cores. We need to > handle such cases. This work also greatly simplified the lustre SMP code. > > ------------------------------------------------------------------------------ > > https://jira.hpdd.intel.com/browse/LU-9019 > > Replace libcfs time API with standard kernel APIs. Also migrate away from > jiffies. We found jiffies can vary on nodes which can lead to corner cases > that can break the file system due to nodes having inconsistent behavior. > So move to time64_t and ktime_t as much as possible. > > ****************************************************************************** > * Proper IB support for ko2iblnd > ****************************************************************************** > https://jira.hpdd.intel.com/browse/LU-9179 > > Poor performance for the ko2iblnd driver. This is related to many of the > patches below that are missing from the linux client. > ------------------------------------------------------------------------------ > > https://jira.hpdd.intel.com/browse/LU-9886 > > Crash in upstream kiblnd_handle_early_rxs() > ------------------------------------------------------------------------------ > > https://jira.hpdd.intel.com/browse/LU-10394 / LU-10526 / LU-10089 > > Default to default to using MEM_REG > ------------------------------------------------------------------------------ > > https://jira.hpdd.intel.com/browse/LU-10459 > > throttle tx based on queue depth > ------------------------------------------------------------------------------ > > https://jira.hpdd.intel.com/browse/LU-9943 > > correct WR fast reg accounting > ------------------------------------------------------------------------------ > > https://jira.hpdd.intel.com/browse/LU-10291 > > remove concurrent_sends tunable > ------------------------------------------------------------------------------ > > https://jira.hpdd.intel.com/browse/LU-10213 > > calculate qp max_send_wrs properly > ------------------------------------------------------------------------------ > > https://jira.hpdd.intel.com/browse/LU-9810 > > use less CQ entries for each connection > ------------------------------------------------------------------------------ > > https://jira.hpdd.intel.com/browse/LU-10129 / LU-9180 > > rework map_on_demand behavior > ------------------------------------------------------------------------------ > > https://jira.hpdd.intel.com/browse/LU-10129 > > query device capabilities > ------------------------------------------------------------------------------ > > https://jira.hpdd.intel.com/browse/LU-10015 > > fix race at kiblnd_connect_peer > ------------------------------------------------------------------------------ > > https://jira.hpdd.intel.com/browse/LU-9983 > > allow for discontiguous fragments > ------------------------------------------------------------------------------ > > https://jira.hpdd.intel.com/browse/LU-9500 > > Don't Page Align remote_addr with FastReg > ------------------------------------------------------------------------------ > > https://jira.hpdd.intel.com/browse/LU-9448 > > handle empty CPTs > ------------------------------------------------------------------------------ > > https://jira.hpdd.intel.com/browse/LU-9507 > > Don't Assert On Reconnect with MultiQP > ------------------------------------------------------------------------------ > > https://jira.hpdd.intel.com/browse/LU-9472 > > Fix FastReg map/unmap for MLX5 > ------------------------------------------------------------------------------ > > https://jira.hpdd.intel.com/browse/LU-9425 > > Turn on 2 sges by default > ------------------------------------------------------------------------------ > > https://jira.hpdd.intel.com/browse/LU-8943 > > Enable Multiple OPA Endpoints between Nodes > ------------------------------------------------------------------------------ > > https://jira.hpdd.intel.com/browse/LU-5718 > > multiple sges for work request > ------------------------------------------------------------------------------ > > https://jira.hpdd.intel.com/browse/LU-9094 > > kill timedout txs from ibp_tx_queue > ------------------------------------------------------------------------------ > > https://jira.hpdd.intel.com/browse/LU-9094 > > reconnect peer for REJ_INVALID_SERVICE_ID > ------------------------------------------------------------------------------ > > https://jira.hpdd.intel.com/browse/LU-8752 > > Stop MLX5 triggering a dump_cqe > ------------------------------------------------------------------------------ > > https://jira.hpdd.intel.com/browse/LU-8874 > > Move ko2iblnd to latest RDMA changes > ------------------------------------------------------------------------------ > > https://jira.hpdd.intel.com/browse/LU-8875 / LU-8874 > > Change to new RDMA done callback mechanism > > ------------------------------------------------------------------------------ > > https://jira.hpdd.intel.com/browse/LU-9164 / LU-8874 > > Incorporate RDMA map/unamp API's into ko2iblnd > > ****************************************************************************** > * sysfs/debugfs fixes > * > * https://jira.hpdd.intel.com/browse/LU-8066 > * > * The original migration to sysfs was done in haste without properly working > * utilities to test the changes. This covers the work to restore the proper > * behavior. Huge project to make this right. > * > ****************************************************************************** > > https://jira.hpdd.intel.com/browse/LU-9431 > > The function class_process_proc_param was used for our mass updates of proc > tunables. It didn't work with sysfs and it was just ugly so it was removed. > In the process the ability to mass update thousands of clients was lost. This > work restores this in a sane way. > > ------------------------------------------------------------------------------ > https://jira.hpdd.intel.com/browse/LU-9091 > > One the major request of users is the ability to pass in parameters into a > sysfs file in various different units. For example we can set max_pages_per_rpc > but this can vary on platforms due to different platform sizes. So you can > set this like max_pages_per_rpc=16MiB. The original code to handle this written > before the string helpers were created so the code doesn't follow that format > but it would be easy to move to. Currently the string helpers does the reverse > of what we need, changing bytes to string. We need to change a string to bytes. > > ****************************************************************************** > * Proper user land to kernel space interface for Lustre > * > * https://jira.hpdd.intel.com/browse/LU-9680 > * > ****************************************************************************** > > https://jira.hpdd.intel.com/browse/LU-8915 > > Don't use linux list structure as user land arguments for lnet selftest. > This code is pretty poor quality and really needs to be reworked. > > ------------------------------------------------------------------------------ > > https://jira.hpdd.intel.com/browse/LU-8834 > > The lustre ioctl LL_IOC_FUTIMES_3 is very generic. Need to either work with > other file systems with similar functionality and make a common syscall > interface or rework our server code to automagically do it for us. > > ------------------------------------------------------------------------------ > > https://jira.hpdd.intel.com/browse/LU-6202 > > Cleanup up ioctl handling. We have many obsolete ioctls. Also the way we do > ioctls can be changed over to netlink. This also has the benefit of working > better with HPC systems that do IO forwarding. Such systems don't like ioctls > very well. > > ------------------------------------------------------------------------------ > > https://jira.hpdd.intel.com/browse/LU-9667 > > More cleanups by making our utilities use sysfs instead of ioctls for LNet. > Also it has been requested to move the remaining ioctls to the netlink API. > > ****************************************************************************** > * Misc > ****************************************************************************** > > ------------------------------------------------------------------------------ > https://jira.hpdd.intel.com/browse/LU-9855 > > Clean up obdclass preprocessor code. One of the major eye sores is the various > pointer redirections and macros used by the obdclass. This makes the code very > difficult to understand. It was requested by the Al Viro to clean this up before > we leave staging. > > ------------------------------------------------------------------------------ > > https://jira.hpdd.intel.com/browse/LU-9633 > > Migrate to sphinx kernel-doc style comments. Add documents in Documentation. > > ------------------------------------------------------------------------------ > > https://jira.hpdd.intel.com/browse/LU-6142 > > Possible remaining coding style fix. Remove deadcode. Enforce kernel code > style. Other minor misc cleanups... > > ------------------------------------------------------------------------------ > > https://jira.hpdd.intel.com/browse/LU-8837 > > Separate client/server functionality. Functions only used by server can be > removed from client. Most of this has been done but we need a inspect of the > code to make sure. > > ------------------------------------------------------------------------------ > > https://jira.hpdd.intel.com/browse/LU-8964 > > Lustre client readahead/writeback control needs to better suit kernel providings. > Currently its being explored. We could end up replacing the CLIO read ahead > abstract with the kernel proper version. > > ------------------------------------------------------------------------------ > > https://jira.hpdd.intel.com/browse/LU-9862 > > Patch that landed for LU-7890 leads to static checker errors > ------------------------------------------------------------------------------ > > https://jira.hpdd.intel.com/browse/LU-9868 > > dcache/namei fixes for lustre > ------------------------------------------------------------------------------ > > https://jira.hpdd.intel.com/browse/LU-10467 > > use standard linux wait_events macros work by Neil Brown > > ------------------------------------------------------------------------------ -------------- next part -------------- An HTML attachment was scrubbed... URL: From neilb at suse.com Mon Feb 12 01:20:43 2018 From: neilb at suse.com (NeilBrown) Date: Mon, 12 Feb 2018 12:20:43 +1100 Subject: [lustre-devel] [e2fsprogs PATCH] tune2fs: don't recover journal if device is busy. Message-ID: <871shrrok4.fsf@notabene.neil.brown.name> tune2fs currently replays the journal if it needs recovery and the filesystem isn't mounted. The test for "is the filesystem mounted" isn't completely robust. Lustre makes use of ext4 filesystems in a way that they are mounted without being visible in /proc/mounts or similar. This usage can easily be detected by attempting to open the device with O_EXCL. tune2fs already does this and the EXT2_MF_BUSY flag is set if open(O_EXCL) fails. Several uses other than lustre mounts could cause O_EXCL to fail, but in any case it seems unwise to recover the journal when something else is keeping the device busy. So add an extra test to avoid journal recovery when the device is busy. This fixes some problems with lustre usage. Signed-off-by: NeilBrown -- Note: it seems wrong to recover the journal *after* making changes to the superblock - there is a good chance that recovering the journal will over-write those changes. This is what was happening that lead me to this problem. Shouldn't journal recovery happen *first*?? Thanks, NeilBrown --- misc/tune2fs.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/misc/tune2fs.c b/misc/tune2fs.c index c33fb9d80b10..703e55b6b972 100644 --- a/misc/tune2fs.c +++ b/misc/tune2fs.c @@ -3337,7 +3337,7 @@ _("Warning: The journal is dirty. You may wish to replay the journal like:\n\n" } #else /* Recover the journal if possible. */ - if ((open_flag & EXT2_FLAG_RW) && !(mount_flags & EXT2_MF_MOUNTED) && + if ((open_flag & EXT2_FLAG_RW) && !(mount_flags & (EXT2_MF_BUSY | EXT2_MF_MOUNTED)) && ext2fs_has_feature_journal_needs_recovery(fs->super)) { errcode_t err; -- 2.14.0.rc0.dirty -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 832 bytes Desc: not available URL: From jsimmons at infradead.org Mon Feb 12 02:06:07 2018 From: jsimmons at infradead.org (James Simmons) Date: Mon, 12 Feb 2018 02:06:07 +0000 (GMT) Subject: [lustre-devel] [PATCH] staging: lustre: update the TODO list In-Reply-To: <1518390961.4139.1.camel@perches.com> References: <1518390008-17876-1-git-send-email-jsimmons@infradead.org> <1518390961.4139.1.camel@perches.com> Message-ID: > On Sun, 2018-02-11 at 18:00 -0500, James Simmons wrote: > > As more people become involved with the progression of the lustre > > client it needs to more clear what needs to be done to leave > > staging. Update the TODO list with the various bugs and changes > > to accomplish this. Some are simple bugs and others are far more > > complex task that will change many lines of code. Some even cover > > updating the user land utilities to meet the kernel requirements. > > Several bugs have already been addressed and just need to be > > pushed to the staging tree. > > Nice list. Thanks. Its nice to have a direction :-) > Are or should those entries that have been addressed > also in this new list? Do you mean by those entries what originally was in the TODO list or what has been fixed over the last 2 years? From neilb at suse.com Mon Feb 12 02:09:37 2018 From: neilb at suse.com (NeilBrown) Date: Mon, 12 Feb 2018 13:09:37 +1100 Subject: [lustre-devel] Lustre upstream client TODO list In-Reply-To: References: <87bmgvrsj8.fsf@notabene.neil.brown.name> Message-ID: <87wozjq7q6.fsf@notabene.neil.brown.name> On Mon, Feb 12 2018, Patrick Farrell wrote: > Neil, > > Apologies if you've answered this elsewhere, but what's the genesis of > your current (extremely welcome) interest in Lustre? Some commitment > by SUSE? "commitment" might be too strong a word - certainly too strong for me to use - but "interest" is probably fair. Some interest within SUSE. NeilBrown > > Regards, > - Patrick > ________________________________ > From: lustre-devel on behalf of NeilBrown > Sent: Sunday, February 11, 2018 5:54:51 PM > To: James Simmons; Lustre Development List > Cc: Oleg Drokin > Subject: Re: [lustre-devel] Lustre upstream client TODO list > > On Sun, Feb 11 2018, James Simmons wrote: > >> So I sent a patch upstream that laid out what most needs to be done for >> the linux lustre client to leave staging. I placed the new text here for >> ease of read so you don't have to go searching for it. Feed back is >> welcomed. Hoepfully posting it will make it clear what needs to be done. > > > Thanks so much for putting this together and pushing it out. I really > appreciated it and hope to show that appreciation with patches :-) > > NeilBrown > >> >> Currently all the work directed toward the lustre upstream client is tracked >> at the following link: >> >> https://jira.hpdd.intel.com/browse/LU-9679 >> >> Under this ticket you will see the following work items that need to be >> addressed: >> >> ****************************************************************************** >> * libcfs cleanup >> * >> * https://jira.hpdd.intel.com/browse/LU-9859 >> * >> * Track all the cleanups and simplification of the libcfs module. Remove >> * functions the kernel provides. Possible intergrate some of the functionality >> * into the kernel proper. >> * >> ****************************************************************************** >> >> https://jira.hpdd.intel.com/browse/LU-100086 >> >> LNET_MINOR conflicts with USERIO_MINOR >> >> ------------------------------------------------------------------------------ >> >> https://jira.hpdd.intel.com/browse/LU-8130 >> >> Fix and simplify libcfs hash handling >> >> ------------------------------------------------------------------------------ >> >> https://jira.hpdd.intel.com/browse/LU-8703 >> >> The current way we handle SMP is wrong. Platforms like ARM and KNL can have >> core and NUMA setups with things like NUMA nodes with no cores. We need to >> handle such cases. This work also greatly simplified the lustre SMP code. >> >> ------------------------------------------------------------------------------ >> >> https://jira.hpdd.intel.com/browse/LU-9019 >> >> Replace libcfs time API with standard kernel APIs. Also migrate away from >> jiffies. We found jiffies can vary on nodes which can lead to corner cases >> that can break the file system due to nodes having inconsistent behavior. >> So move to time64_t and ktime_t as much as possible. >> >> ****************************************************************************** >> * Proper IB support for ko2iblnd >> ****************************************************************************** >> https://jira.hpdd.intel.com/browse/LU-9179 >> >> Poor performance for the ko2iblnd driver. This is related to many of the >> patches below that are missing from the linux client. >> ------------------------------------------------------------------------------ >> >> https://jira.hpdd.intel.com/browse/LU-9886 >> >> Crash in upstream kiblnd_handle_early_rxs() >> ------------------------------------------------------------------------------ >> >> https://jira.hpdd.intel.com/browse/LU-10394 / LU-10526 / LU-10089 >> >> Default to default to using MEM_REG >> ------------------------------------------------------------------------------ >> >> https://jira.hpdd.intel.com/browse/LU-10459 >> >> throttle tx based on queue depth >> ------------------------------------------------------------------------------ >> >> https://jira.hpdd.intel.com/browse/LU-9943 >> >> correct WR fast reg accounting >> ------------------------------------------------------------------------------ >> >> https://jira.hpdd.intel.com/browse/LU-10291 >> >> remove concurrent_sends tunable >> ------------------------------------------------------------------------------ >> >> https://jira.hpdd.intel.com/browse/LU-10213 >> >> calculate qp max_send_wrs properly >> ------------------------------------------------------------------------------ >> >> https://jira.hpdd.intel.com/browse/LU-9810 >> >> use less CQ entries for each connection >> ------------------------------------------------------------------------------ >> >> https://jira.hpdd.intel.com/browse/LU-10129 / LU-9180 >> >> rework map_on_demand behavior >> ------------------------------------------------------------------------------ >> >> https://jira.hpdd.intel.com/browse/LU-10129 >> >> query device capabilities >> ------------------------------------------------------------------------------ >> >> https://jira.hpdd.intel.com/browse/LU-10015 >> >> fix race at kiblnd_connect_peer >> ------------------------------------------------------------------------------ >> >> https://jira.hpdd.intel.com/browse/LU-9983 >> >> allow for discontiguous fragments >> ------------------------------------------------------------------------------ >> >> https://jira.hpdd.intel.com/browse/LU-9500 >> >> Don't Page Align remote_addr with FastReg >> ------------------------------------------------------------------------------ >> >> https://jira.hpdd.intel.com/browse/LU-9448 >> >> handle empty CPTs >> ------------------------------------------------------------------------------ >> >> https://jira.hpdd.intel.com/browse/LU-9507 >> >> Don't Assert On Reconnect with MultiQP >> ------------------------------------------------------------------------------ >> >> https://jira.hpdd.intel.com/browse/LU-9472 >> >> Fix FastReg map/unmap for MLX5 >> ------------------------------------------------------------------------------ >> >> https://jira.hpdd.intel.com/browse/LU-9425 >> >> Turn on 2 sges by default >> ------------------------------------------------------------------------------ >> >> https://jira.hpdd.intel.com/browse/LU-8943 >> >> Enable Multiple OPA Endpoints between Nodes >> ------------------------------------------------------------------------------ >> >> https://jira.hpdd.intel.com/browse/LU-5718 >> >> multiple sges for work request >> ------------------------------------------------------------------------------ >> >> https://jira.hpdd.intel.com/browse/LU-9094 >> >> kill timedout txs from ibp_tx_queue >> ------------------------------------------------------------------------------ >> >> https://jira.hpdd.intel.com/browse/LU-9094 >> >> reconnect peer for REJ_INVALID_SERVICE_ID >> ------------------------------------------------------------------------------ >> >> https://jira.hpdd.intel.com/browse/LU-8752 >> >> Stop MLX5 triggering a dump_cqe >> ------------------------------------------------------------------------------ >> >> https://jira.hpdd.intel.com/browse/LU-8874 >> >> Move ko2iblnd to latest RDMA changes >> ------------------------------------------------------------------------------ >> >> https://jira.hpdd.intel.com/browse/LU-8875 / LU-8874 >> >> Change to new RDMA done callback mechanism >> >> ------------------------------------------------------------------------------ >> >> https://jira.hpdd.intel.com/browse/LU-9164 / LU-8874 >> >> Incorporate RDMA map/unamp API's into ko2iblnd >> >> ****************************************************************************** >> * sysfs/debugfs fixes >> * >> * https://jira.hpdd.intel.com/browse/LU-8066 >> * >> * The original migration to sysfs was done in haste without properly working >> * utilities to test the changes. This covers the work to restore the proper >> * behavior. Huge project to make this right. >> * >> ****************************************************************************** >> >> https://jira.hpdd.intel.com/browse/LU-9431 >> >> The function class_process_proc_param was used for our mass updates of proc >> tunables. It didn't work with sysfs and it was just ugly so it was removed. >> In the process the ability to mass update thousands of clients was lost. This >> work restores this in a sane way. >> >> ------------------------------------------------------------------------------ >> https://jira.hpdd.intel.com/browse/LU-9091 >> >> One the major request of users is the ability to pass in parameters into a >> sysfs file in various different units. For example we can set max_pages_per_rpc >> but this can vary on platforms due to different platform sizes. So you can >> set this like max_pages_per_rpc=16MiB. The original code to handle this written >> before the string helpers were created so the code doesn't follow that format >> but it would be easy to move to. Currently the string helpers does the reverse >> of what we need, changing bytes to string. We need to change a string to bytes. >> >> ****************************************************************************** >> * Proper user land to kernel space interface for Lustre >> * >> * https://jira.hpdd.intel.com/browse/LU-9680 >> * >> ****************************************************************************** >> >> https://jira.hpdd.intel.com/browse/LU-8915 >> >> Don't use linux list structure as user land arguments for lnet selftest. >> This code is pretty poor quality and really needs to be reworked. >> >> ------------------------------------------------------------------------------ >> >> https://jira.hpdd.intel.com/browse/LU-8834 >> >> The lustre ioctl LL_IOC_FUTIMES_3 is very generic. Need to either work with >> other file systems with similar functionality and make a common syscall >> interface or rework our server code to automagically do it for us. >> >> ------------------------------------------------------------------------------ >> >> https://jira.hpdd.intel.com/browse/LU-6202 >> >> Cleanup up ioctl handling. We have many obsolete ioctls. Also the way we do >> ioctls can be changed over to netlink. This also has the benefit of working >> better with HPC systems that do IO forwarding. Such systems don't like ioctls >> very well. >> >> ------------------------------------------------------------------------------ >> >> https://jira.hpdd.intel.com/browse/LU-9667 >> >> More cleanups by making our utilities use sysfs instead of ioctls for LNet. >> Also it has been requested to move the remaining ioctls to the netlink API. >> >> ****************************************************************************** >> * Misc >> ****************************************************************************** >> >> ------------------------------------------------------------------------------ >> https://jira.hpdd.intel.com/browse/LU-9855 >> >> Clean up obdclass preprocessor code. One of the major eye sores is the various >> pointer redirections and macros used by the obdclass. This makes the code very >> difficult to understand. It was requested by the Al Viro to clean this up before >> we leave staging. >> >> ------------------------------------------------------------------------------ >> >> https://jira.hpdd.intel.com/browse/LU-9633 >> >> Migrate to sphinx kernel-doc style comments. Add documents in Documentation. >> >> ------------------------------------------------------------------------------ >> >> https://jira.hpdd.intel.com/browse/LU-6142 >> >> Possible remaining coding style fix. Remove deadcode. Enforce kernel code >> style. Other minor misc cleanups... >> >> ------------------------------------------------------------------------------ >> >> https://jira.hpdd.intel.com/browse/LU-8837 >> >> Separate client/server functionality. Functions only used by server can be >> removed from client. Most of this has been done but we need a inspect of the >> code to make sure. >> >> ------------------------------------------------------------------------------ >> >> https://jira.hpdd.intel.com/browse/LU-8964 >> >> Lustre client readahead/writeback control needs to better suit kernel providings. >> Currently its being explored. We could end up replacing the CLIO read ahead >> abstract with the kernel proper version. >> >> ------------------------------------------------------------------------------ >> >> https://jira.hpdd.intel.com/browse/LU-9862 >> >> Patch that landed for LU-7890 leads to static checker errors >> ------------------------------------------------------------------------------ >> >> https://jira.hpdd.intel.com/browse/LU-9868 >> >> dcache/namei fixes for lustre >> ------------------------------------------------------------------------------ >> >> https://jira.hpdd.intel.com/browse/LU-10467 >> >> use standard linux wait_events macros work by Neil Brown >> >> ------------------------------------------------------------------------------ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 832 bytes Desc: not available URL: From darrick.wong at oracle.com Mon Feb 12 02:16:09 2018 From: darrick.wong at oracle.com (Darrick J. Wong) Date: Sun, 11 Feb 2018 18:16:09 -0800 Subject: [lustre-devel] [e2fsprogs PATCH] tune2fs: don't recover journal if device is busy. In-Reply-To: <871shrrok4.fsf@notabene.neil.brown.name> References: <871shrrok4.fsf@notabene.neil.brown.name> Message-ID: <20180212021609.GA5204@magnolia> On Mon, Feb 12, 2018 at 12:20:43PM +1100, NeilBrown wrote: > > tune2fs currently replays the journal if it needs > recovery and the filesystem isn't mounted. > > The test for "is the filesystem mounted" isn't completely robust. > Lustre makes use of ext4 filesystems in a way that they are mounted > without being visible in /proc/mounts or similar. > This usage can easily be detected by attempting to open the device > with O_EXCL. tune2fs already does this and the EXT2_MF_BUSY flag > is set if open(O_EXCL) fails. > Several uses other than lustre mounts could cause O_EXCL to fail, > but in any case it seems unwise to recover the journal when something > else is keeping the device busy. > > So add an extra test to avoid journal recovery when the device > is busy. This fixes some problems with lustre usage. > > Signed-off-by: NeilBrown > > -- > Note: it seems wrong to recover the journal *after* making > changes to the superblock - there is a good chance that > recovering the journal will over-write those changes. > This is what was happening that lead me to this problem. > Shouldn't journal recovery happen *first*?? Yes. Oops. :/ This whole hunk ought to move up to be right after ext2fs_check_if_mounted, I think. As for this patch itself, Reviewed-by: Darrick J. Wong --D From joe at perches.com Mon Feb 12 03:21:06 2018 From: joe at perches.com (Joe Perches) Date: Sun, 11 Feb 2018 19:21:06 -0800 Subject: [lustre-devel] [PATCH] staging: lustre: update the TODO list In-Reply-To: References: <1518390008-17876-1-git-send-email-jsimmons@infradead.org> <1518390961.4139.1.camel@perches.com> Message-ID: <1518405666.4139.8.camel@perches.com> On Mon, 2018-02-12 at 02:06 +0000, James Simmons wrote: > > On Sun, 2018-02-11 at 18:00 -0500, James Simmons wrote: > > > As more people become involved with the progression of the lustre > > > client it needs to more clear what needs to be done to leave > > > staging. Update the TODO list with the various bugs and changes > > > to accomplish this. Some are simple bugs and others are far more > > > complex task that will change many lines of code. Some even cover > > > updating the user land utilities to meet the kernel requirements. > > > Several bugs have already been addressed and just need to be > > > pushed to the staging tree. > > > > Nice list. > > Thanks. Its nice to have a direction :-) > > > Are or should those entries that have been addressed > > also in this new list? > > Do you mean by those entries what originally was in the > TODO list or what has been fixed over the last 2 years? My reading of > Several bugs have already been addressed and just need to be > pushed to the staging tree. is that the new list contains entries for things that have already been fixed but have not yet been pushed to the staging tree. So, if there are entries in the new TODO list that really are fixed, but not yet posted, then marking those entries as TODO seems inappropriate. From dan.carpenter at oracle.com Mon Feb 12 07:41:07 2018 From: dan.carpenter at oracle.com (Dan Carpenter) Date: Mon, 12 Feb 2018 10:41:07 +0300 Subject: [lustre-devel] [PATCH 41/80] staging: lustre: lmv: separate master object with master stripe In-Reply-To: <87r2pvkkl5.fsf@notabene.neil.brown.name> References: <1471378773-24590-1-git-send-email-jsimmons@infradead.org> <1471378773-24590-42-git-send-email-jsimmons@infradead.org> <87r2pvkkl5.fsf@notabene.neil.brown.name> Message-ID: <20180212074107.ulp52nikjv5m5mf2@mwanda> On Fri, Feb 09, 2018 at 12:39:18PM +1100, NeilBrown wrote: > On Tue, Aug 16 2016, James Simmons wrote: > > > > > +static inline bool > > +lsm_md_eq(const struct lmv_stripe_md *lsm1, const struct lmv_stripe_md *lsm2) > > +{ > > + int idx; > > + > > + if (lsm1->lsm_md_magic != lsm2->lsm_md_magic || > > + lsm1->lsm_md_stripe_count != lsm2->lsm_md_stripe_count || > > + lsm1->lsm_md_master_mdt_index != lsm2->lsm_md_master_mdt_index || > > + lsm1->lsm_md_hash_type != lsm2->lsm_md_hash_type || > > + lsm1->lsm_md_layout_version != lsm2->lsm_md_layout_version || > > + !strcmp(lsm1->lsm_md_pool_name, lsm2->lsm_md_pool_name)) > > + return false; > > Hi James and all, > This patch (8f18c8a48b736c2f in linux) is different from the > corresponding patch in lustre-release (60e07b972114df). > > In that patch, the last clause in the 'if' condition is > > + strcmp(lsm1->lsm_md_pool_name, > + lsm2->lsm_md_pool_name) != 0) > > Whoever converted it to "!strcmp()" inverted the condition. This is a > perfect example of why I absolutely *loathe* the "!strcmp()" construct!! People think that "if (!strcmp()) " is prefered kernel style but it's not. if (foo != NULL) { The != NULL is a double negative. I don't think it adds anything. Some kernel developers like this style because it's explicit about the type. I have never seen any bugs caused by this format or solved by this format. Anyway checkpatch complains. if (ret != 0) { In this situation "ret" is not a number, it's an error code. The != 0 is a double negative and complicated to think about. Btw, I sort of prefer "if (ret)" to "if (ret < 0)", not because of style but it's easier for Smatch. No subsystems are totally consistent so the (by definition inconsistent) "if (ret < 0)" checks cause false positives in Smatch. if (len != 0) This is OK. "len" is a number. if (strcmp(one, two) != 0) { With strcmp() I really prefer == 0 and != 0 because it works like this: strcmp(one, two) == 0 --> means one == two strcmp(one, two) < 0 --> means one < two strcmp(one, two) != 0 --> means one != two Either style is accepted in the kernel but I think == 0 just makes so much sense. I mostly see bugs from this when people are "fixing" the style from == 0 to !strcmp() so my sample is very biased. Normally, if the original author writes the code any bugs are caught in testing so either way is going to be bug free. But the only thing that checkpatch complains about is == NULL and != NULL. regards, dan carpenter From neilb at suse.com Mon Feb 12 21:09:25 2018 From: neilb at suse.com (NeilBrown) Date: Tue, 13 Feb 2018 08:09:25 +1100 Subject: [lustre-devel] [PATCH 0/2] Two lustre bug fixes Message-ID: <151846966702.17168.15765227765350491300.stgit@noble> These patches fix 2 issues I found during testing. Together they cause lots of path-name-lookup syscalls to return -ENOMEM. The first bug fixes triggers and error The second bug fixes causes that error to be reported as -ENOMEM. Thanks, NeilBrown --- NeilBrown (2): staging: lustre: fix inverted test on strcmp staging: lustre: honor error code from ll_iget(). drivers/staging/lustre/lustre/include/lustre_lmv.h | 2 +- drivers/staging/lustre/lustre/llite/llite_lib.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) -- Signature From neilb at suse.com Mon Feb 12 21:09:25 2018 From: neilb at suse.com (NeilBrown) Date: Tue, 13 Feb 2018 08:09:25 +1100 Subject: [lustre-devel] [PATCH 1/2] staging: lustre: fix inverted test on strcmp In-Reply-To: <151846966702.17168.15765227765350491300.stgit@noble> References: <151846966702.17168.15765227765350491300.stgit@noble> Message-ID: <151846976573.17168.10557259719846165376.stgit@noble> This code tests various fields to see if they are different, except for one where there test is if they are the same. This is clearly wrong for a function that is tesding for equality. So change "!strcmp()" which I always find hard to read, to "strcmp() != 0" which obviously means that the strings are not equal. Signed-off-by: NeilBrown --- drivers/staging/lustre/lustre/include/lustre_lmv.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/staging/lustre/lustre/include/lustre_lmv.h b/drivers/staging/lustre/lustre/include/lustre_lmv.h index f4298e5f7543..080ec1f8e19f 100644 --- a/drivers/staging/lustre/lustre/include/lustre_lmv.h +++ b/drivers/staging/lustre/lustre/include/lustre_lmv.h @@ -63,7 +63,7 @@ lsm_md_eq(const struct lmv_stripe_md *lsm1, const struct lmv_stripe_md *lsm2) lsm1->lsm_md_master_mdt_index != lsm2->lsm_md_master_mdt_index || lsm1->lsm_md_hash_type != lsm2->lsm_md_hash_type || lsm1->lsm_md_layout_version != lsm2->lsm_md_layout_version || - !strcmp(lsm1->lsm_md_pool_name, lsm2->lsm_md_pool_name)) + strcmp(lsm1->lsm_md_pool_name, lsm2->lsm_md_pool_name) != 0) return false; for (idx = 0; idx < lsm1->lsm_md_stripe_count; idx++) { From neilb at suse.com Mon Feb 12 21:09:25 2018 From: neilb at suse.com (NeilBrown) Date: Tue, 13 Feb 2018 08:09:25 +1100 Subject: [lustre-devel] [PATCH 2/2] staging: lustre: honor error code from ll_iget(). In-Reply-To: <151846966702.17168.15765227765350491300.stgit@noble> References: <151846966702.17168.15765227765350491300.stgit@noble> Message-ID: <151846976581.17168.2838881093184873250.stgit@noble> Commit 020ecc6f3229 ("staging: lustre: llite: Remove IS_ERR tests") changed ll_prep_inode to assume any error from ll_iget() meant -ENOMEM because at that time it only returned NULL for errors. Commit c3397e7e677b ("staging: lustre: llite: add error handler in inode prepare phase") changed ll_iget() to once again return meaningful codes, but nobody told ll_prep_inode(). So change ll_prep_inode() back to using PTR_ERR(*inode). Signed-off-by: NeilBrown --- drivers/staging/lustre/lustre/llite/llite_lib.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/staging/lustre/lustre/llite/llite_lib.c b/drivers/staging/lustre/lustre/llite/llite_lib.c index 6735a6f006d2..020f0faeb750 100644 --- a/drivers/staging/lustre/lustre/llite/llite_lib.c +++ b/drivers/staging/lustre/lustre/llite/llite_lib.c @@ -2143,7 +2143,7 @@ int ll_prep_inode(struct inode **inode, struct ptlrpc_request *req, md.posix_acl = NULL; } #endif - rc = -ENOMEM; + rc = PTR_ERR(*inode); CERROR("new_inode -fatal: rc %d\n", rc); goto out; } From neilb at suse.com Mon Feb 12 21:22:36 2018 From: neilb at suse.com (NeilBrown) Date: Tue, 13 Feb 2018 08:22:36 +1100 Subject: [lustre-devel] [PATCH 00/19] RESEND staging: lustre: use standard wait_event macros Message-ID: <151847037709.22826.16175867257667686132.stgit@noble> Hi, this is a resend of patch series that I sent a little over a month ago. Since then I've recieved Reviewed-By:from James Simmons (thanks) and confirmation that it passes his testing. I cannot quite say that all tests pass for me with these patch, but the tests produce the same results both with and without the patches. No non-trivial changes - a couple of typos in change descriptions fixes, and reviewed-by's added. Previous intro text: Hi, this is a revised version of the patch series I sent under a similar subject in mid December. Improvements are: - new wait_event_idle* macros are now in include/linux/wait.h which Ack from peterz. - *all* waits are now TASK_IDLE or TASK_INTERRUPTIBLE and so don't affect the load average. There is no need to choose whether load is appropriate or not in each case. - all l_wait_event() users are handled so l_wait_event() is removed. The one case I had left out before uses wait_event_idle_exclusive() with and option of using wait_event_idle_exclusive_lifo() is that ever gets approved. I think this set is ready to go. If you only review two patches, please review staging: lustre: simplify waiting in ldlm_completion_ast() and staging: lustre: remove back_to_sleep() as in both of those, the actual behaviour of the current code (as I understand it) doesn't seem to agree with comments/debug message, or just generally looks odd. Thanks, NeilBrown --- NeilBrown (19): sched/wait: add wait_event_idle() functions. staging: lustre: discard SVC_SIGNAL and related functions staging: lustre: replace simple cases of l_wait_event() with wait_event(). staging: lustre: discard cfs_time_seconds() staging: lustre: use wait_event_idle_timeout() where appropriate. staging: lustre: introduce and use l_wait_event_abortable() staging: lustre: simplify l_wait_event when intr handler but no timeout. staging: lustre: simplify waiting in ldlm_completion_ast() staging: lustre: open code polling loop instead of using l_wait_event() staging: lustre: simplify waiting in ptlrpc_invalidate_import() staging: lustre: remove back_to_sleep() staging: lustre: make polling loop in ptlrpc_unregister_bulk more obvious staging: lustre: use wait_event_idle_timeout in ptlrpcd() staging: lustre: improve waiting in sptlrpc_req_refresh_ctx staging: lustre: use explicit poll loop in ptlrpc_service_unlink_rqbd staging: lustre: use explicit poll loop in ptlrpc_unregister_reply staging: lustre: remove l_wait_event from ptlrpc_set_wait staging: lustre: replace l_wait_event_exclusive_head() with wait_event_idle_exclusive staging: lustre: remove l_wait_event() and related code .../lustre/include/linux/libcfs/libcfs_debug.h | 4 .../lustre/include/linux/libcfs/libcfs_time.h | 2 .../lustre/include/linux/libcfs/linux/linux-time.h | 7 .../staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c | 8 - .../staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c | 4 .../staging/lustre/lnet/klnds/socklnd/socklnd.c | 6 .../staging/lustre/lnet/klnds/socklnd/socklnd_cb.c | 22 + drivers/staging/lustre/lnet/libcfs/debug.c | 2 drivers/staging/lustre/lnet/libcfs/fail.c | 2 drivers/staging/lustre/lnet/libcfs/tracefile.c | 4 drivers/staging/lustre/lnet/lnet/acceptor.c | 2 drivers/staging/lustre/lnet/lnet/api-ni.c | 4 drivers/staging/lustre/lnet/lnet/lib-move.c | 4 drivers/staging/lustre/lnet/lnet/net_fault.c | 14 - drivers/staging/lustre/lnet/lnet/peer.c | 2 drivers/staging/lustre/lnet/lnet/router.c | 8 - drivers/staging/lustre/lnet/selftest/conrpc.c | 4 drivers/staging/lustre/lnet/selftest/rpc.c | 2 drivers/staging/lustre/lnet/selftest/selftest.h | 2 drivers/staging/lustre/lnet/selftest/timer.c | 2 drivers/staging/lustre/lustre/include/lustre_dlm.h | 2 drivers/staging/lustre/lustre/include/lustre_lib.h | 296 ++------------------ drivers/staging/lustre/lustre/include/lustre_mdc.h | 2 drivers/staging/lustre/lustre/include/lustre_net.h | 8 - drivers/staging/lustre/lustre/ldlm/ldlm_flock.c | 30 -- drivers/staging/lustre/lustre/ldlm/ldlm_lock.c | 14 - drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c | 12 - drivers/staging/lustre/lustre/ldlm/ldlm_pool.c | 17 - drivers/staging/lustre/lustre/ldlm/ldlm_request.c | 55 +--- drivers/staging/lustre/lustre/ldlm/ldlm_resource.c | 14 - drivers/staging/lustre/lustre/llite/llite_lib.c | 23 +- drivers/staging/lustre/lustre/llite/statahead.c | 60 ++-- drivers/staging/lustre/lustre/lov/lov_object.c | 6 drivers/staging/lustre/lustre/lov/lov_request.c | 12 - drivers/staging/lustre/lustre/mdc/mdc_request.c | 5 drivers/staging/lustre/lustre/mgc/mgc_request.c | 19 - drivers/staging/lustre/lustre/obdclass/cl_io.c | 23 +- drivers/staging/lustre/lustre/obdclass/genops.c | 24 +- drivers/staging/lustre/lustre/obdclass/llog_obd.c | 5 .../staging/lustre/lustre/obdecho/echo_client.c | 2 drivers/staging/lustre/lustre/osc/osc_cache.c | 28 +- drivers/staging/lustre/lustre/osc/osc_object.c | 6 drivers/staging/lustre/lustre/osc/osc_page.c | 6 drivers/staging/lustre/lustre/osc/osc_request.c | 6 drivers/staging/lustre/lustre/ptlrpc/client.c | 101 +++---- drivers/staging/lustre/lustre/ptlrpc/events.c | 7 drivers/staging/lustre/lustre/ptlrpc/import.c | 51 +-- drivers/staging/lustre/lustre/ptlrpc/niobuf.c | 15 + .../staging/lustre/lustre/ptlrpc/pack_generic.c | 9 - drivers/staging/lustre/lustre/ptlrpc/pinger.c | 28 +- .../staging/lustre/lustre/ptlrpc/ptlrpc_internal.h | 2 drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c | 18 + drivers/staging/lustre/lustre/ptlrpc/recover.c | 12 - drivers/staging/lustre/lustre/ptlrpc/sec.c | 34 ++ drivers/staging/lustre/lustre/ptlrpc/sec_gc.c | 23 -- drivers/staging/lustre/lustre/ptlrpc/service.c | 84 +++--- include/linux/wait.h | 114 ++++++++ 57 files changed, 516 insertions(+), 762 deletions(-) -- Signature From neilb at suse.com Mon Feb 12 21:22:36 2018 From: neilb at suse.com (NeilBrown) Date: Tue, 13 Feb 2018 08:22:36 +1100 Subject: [lustre-devel] [PATCH 02/19] staging: lustre: discard SVC_SIGNAL and related functions In-Reply-To: <151847037709.22826.16175867257667686132.stgit@noble> References: <151847037709.22826.16175867257667686132.stgit@noble> Message-ID: <151847055648.22826.5480938397433602648.stgit@noble> This flag is never set, so remove checks and remove the flag. Reviewed-by: James Simmons Signed-off-by: NeilBrown --- drivers/staging/lustre/lustre/include/lustre_net.h | 6 ------ drivers/staging/lustre/lustre/ptlrpc/sec_gc.c | 4 +--- 2 files changed, 1 insertion(+), 9 deletions(-) diff --git a/drivers/staging/lustre/lustre/include/lustre_net.h b/drivers/staging/lustre/lustre/include/lustre_net.h index 3ff5de4770e8..4c665eca2467 100644 --- a/drivers/staging/lustre/lustre/include/lustre_net.h +++ b/drivers/staging/lustre/lustre/include/lustre_net.h @@ -1260,7 +1260,6 @@ enum { SVC_STARTING = 1 << 2, SVC_RUNNING = 1 << 3, SVC_EVENT = 1 << 4, - SVC_SIGNAL = 1 << 5, }; #define PTLRPC_THR_NAME_LEN 32 @@ -1333,11 +1332,6 @@ static inline int thread_is_event(struct ptlrpc_thread *thread) return !!(thread->t_flags & SVC_EVENT); } -static inline int thread_is_signal(struct ptlrpc_thread *thread) -{ - return !!(thread->t_flags & SVC_SIGNAL); -} - static inline void thread_clear_flags(struct ptlrpc_thread *thread, __u32 flags) { thread->t_flags &= ~flags; diff --git a/drivers/staging/lustre/lustre/ptlrpc/sec_gc.c b/drivers/staging/lustre/lustre/ptlrpc/sec_gc.c index 8d1e0edfcede..d85c8638c009 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/sec_gc.c +++ b/drivers/staging/lustre/lustre/ptlrpc/sec_gc.c @@ -153,7 +153,6 @@ static int sec_gc_main(void *arg) while (1) { struct ptlrpc_sec *sec; - thread_clear_flags(thread, SVC_SIGNAL); sec_process_ctx_list(); again: /* go through sec list do gc. @@ -184,8 +183,7 @@ static int sec_gc_main(void *arg) lwi = LWI_TIMEOUT(msecs_to_jiffies(SEC_GC_INTERVAL * MSEC_PER_SEC), NULL, NULL); l_wait_event(thread->t_ctl_waitq, - thread_is_stopping(thread) || - thread_is_signal(thread), + thread_is_stopping(thread), &lwi); if (thread_test_and_clear_flags(thread, SVC_STOPPING)) From neilb at suse.com Mon Feb 12 21:22:36 2018 From: neilb at suse.com (NeilBrown) Date: Tue, 13 Feb 2018 08:22:36 +1100 Subject: [lustre-devel] [PATCH 03/19] staging: lustre: replace simple cases of l_wait_event() with wait_event(). In-Reply-To: <151847037709.22826.16175867257667686132.stgit@noble> References: <151847037709.22826.16175867257667686132.stgit@noble> Message-ID: <151847055651.22826.1296202136075732095.stgit@noble> When the lwi arg is full of zeros, l_wait_event() behaves almost identically to the standard wait_event_idle() interface, so use that instead. l_wait_event() uses TASK_INTERRUPTIBLE, but blocks all signals. wait_event_idle() uses the new TASK_IDLE and so avoids adding to the load average without needing to block signals. In one case, wait_event_idle_exclusive() is needed. Also remove all l_wait_condition*() macros which were short-cuts for setting lwi to {0}. Reviewed-by: James Simmons Signed-off-by: NeilBrown --- drivers/staging/lustre/lustre/include/lustre_lib.h | 19 -------- drivers/staging/lustre/lustre/ldlm/ldlm_lock.c | 4 -- drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c | 8 +-- drivers/staging/lustre/lustre/ldlm/ldlm_pool.c | 5 +- drivers/staging/lustre/lustre/llite/statahead.c | 50 ++++++++------------ drivers/staging/lustre/lustre/lov/lov_object.c | 6 +- drivers/staging/lustre/lustre/mgc/mgc_request.c | 4 -- drivers/staging/lustre/lustre/obdclass/cl_io.c | 6 +- drivers/staging/lustre/lustre/obdclass/genops.c | 15 ++---- drivers/staging/lustre/lustre/osc/osc_cache.c | 5 +- drivers/staging/lustre/lustre/osc/osc_object.c | 4 -- drivers/staging/lustre/lustre/ptlrpc/pinger.c | 10 ++-- drivers/staging/lustre/lustre/ptlrpc/sec_gc.c | 11 ++-- drivers/staging/lustre/lustre/ptlrpc/service.c | 13 ++--- 14 files changed, 53 insertions(+), 107 deletions(-) diff --git a/drivers/staging/lustre/lustre/include/lustre_lib.h b/drivers/staging/lustre/lustre/include/lustre_lib.h index ca1dce15337e..7d950c53e962 100644 --- a/drivers/staging/lustre/lustre/include/lustre_lib.h +++ b/drivers/staging/lustre/lustre/include/lustre_lib.h @@ -333,24 +333,7 @@ do { \ __ret; \ }) -#define l_wait_condition(wq, condition) \ -({ \ - struct l_wait_info lwi = { 0 }; \ - l_wait_event(wq, condition, &lwi); \ -}) - -#define l_wait_condition_exclusive(wq, condition) \ -({ \ - struct l_wait_info lwi = { 0 }; \ - l_wait_event_exclusive(wq, condition, &lwi); \ -}) - -#define l_wait_condition_exclusive_head(wq, condition) \ -({ \ - struct l_wait_info lwi = { 0 }; \ - l_wait_event_exclusive_head(wq, condition, &lwi); \ -}) - /** @} lib */ + #endif /* _LUSTRE_LIB_H */ diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c b/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c index 7cbc6a06afec..4f700ddb47c6 100644 --- a/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c @@ -1913,14 +1913,12 @@ void ldlm_cancel_callback(struct ldlm_lock *lock) ldlm_set_bl_done(lock); wake_up_all(&lock->l_waitq); } else if (!ldlm_is_bl_done(lock)) { - struct l_wait_info lwi = { 0 }; - /* * The lock is guaranteed to have been canceled once * returning from this function. */ unlock_res_and_lock(lock); - l_wait_event(lock->l_waitq, is_bl_done(lock), &lwi); + wait_event_idle(lock->l_waitq, is_bl_done(lock)); lock_res_and_lock(lock); } } diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c b/drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c index 5f6e7c933b81..6c7c4b19a0a0 100644 --- a/drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c @@ -833,17 +833,15 @@ static int ldlm_bl_thread_main(void *arg) /* cannot use bltd after this, it is only on caller's stack */ while (1) { - struct l_wait_info lwi = { 0 }; struct ldlm_bl_work_item *blwi = NULL; struct obd_export *exp = NULL; int rc; rc = ldlm_bl_get_work(blp, &blwi, &exp); if (!rc) - l_wait_event_exclusive(blp->blp_waitq, - ldlm_bl_get_work(blp, &blwi, - &exp), - &lwi); + wait_event_idle_exclusive(blp->blp_waitq, + ldlm_bl_get_work(blp, &blwi, + &exp)); atomic_inc(&blp->blp_busy_threads); if (ldlm_bl_thread_need_create(blp, blwi)) diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c b/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c index 8563bd32befa..f27c2694793a 100644 --- a/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c @@ -1031,7 +1031,6 @@ static int ldlm_pools_thread_main(void *arg) static int ldlm_pools_thread_start(void) { - struct l_wait_info lwi = { 0 }; struct task_struct *task; if (ldlm_pools_thread) @@ -1052,8 +1051,8 @@ static int ldlm_pools_thread_start(void) ldlm_pools_thread = NULL; return PTR_ERR(task); } - l_wait_event(ldlm_pools_thread->t_ctl_waitq, - thread_is_running(ldlm_pools_thread), &lwi); + wait_event_idle(ldlm_pools_thread->t_ctl_waitq, + thread_is_running(ldlm_pools_thread)); return 0; } diff --git a/drivers/staging/lustre/lustre/llite/statahead.c b/drivers/staging/lustre/lustre/llite/statahead.c index 90c7324575e4..78005cc6e831 100644 --- a/drivers/staging/lustre/lustre/llite/statahead.c +++ b/drivers/staging/lustre/lustre/llite/statahead.c @@ -864,7 +864,6 @@ static int ll_agl_thread(void *arg) struct ll_sb_info *sbi = ll_i2sbi(dir); struct ll_statahead_info *sai; struct ptlrpc_thread *thread; - struct l_wait_info lwi = { 0 }; sai = ll_sai_get(dir); thread = &sai->sai_agl_thread; @@ -885,10 +884,9 @@ static int ll_agl_thread(void *arg) wake_up(&thread->t_ctl_waitq); while (1) { - l_wait_event(thread->t_ctl_waitq, - !list_empty(&sai->sai_agls) || - !thread_is_running(thread), - &lwi); + wait_event_idle(thread->t_ctl_waitq, + !list_empty(&sai->sai_agls) || + !thread_is_running(thread)); if (!thread_is_running(thread)) break; @@ -932,7 +930,6 @@ static int ll_agl_thread(void *arg) static void ll_start_agl(struct dentry *parent, struct ll_statahead_info *sai) { struct ptlrpc_thread *thread = &sai->sai_agl_thread; - struct l_wait_info lwi = { 0 }; struct ll_inode_info *plli; struct task_struct *task; @@ -948,9 +945,8 @@ static void ll_start_agl(struct dentry *parent, struct ll_statahead_info *sai) return; } - l_wait_event(thread->t_ctl_waitq, - thread_is_running(thread) || thread_is_stopped(thread), - &lwi); + wait_event_idle(thread->t_ctl_waitq, + thread_is_running(thread) || thread_is_stopped(thread)); } /* statahead thread main function */ @@ -968,7 +964,6 @@ static int ll_statahead_thread(void *arg) int first = 0; int rc = 0; struct md_op_data *op_data; - struct l_wait_info lwi = { 0 }; sai = ll_sai_get(dir); sa_thread = &sai->sai_thread; @@ -1069,12 +1064,11 @@ static int ll_statahead_thread(void *arg) /* wait for spare statahead window */ do { - l_wait_event(sa_thread->t_ctl_waitq, - !sa_sent_full(sai) || - sa_has_callback(sai) || - !list_empty(&sai->sai_agls) || - !thread_is_running(sa_thread), - &lwi); + wait_event_idle(sa_thread->t_ctl_waitq, + !sa_sent_full(sai) || + sa_has_callback(sai) || + !list_empty(&sai->sai_agls) || + !thread_is_running(sa_thread)); sa_handle_callback(sai); spin_lock(&lli->lli_agl_lock); @@ -1128,11 +1122,10 @@ static int ll_statahead_thread(void *arg) * for file release to stop me. */ while (thread_is_running(sa_thread)) { - l_wait_event(sa_thread->t_ctl_waitq, - sa_has_callback(sai) || - !agl_list_empty(sai) || - !thread_is_running(sa_thread), - &lwi); + wait_event_idle(sa_thread->t_ctl_waitq, + sa_has_callback(sai) || + !agl_list_empty(sai) || + !thread_is_running(sa_thread)); sa_handle_callback(sai); } @@ -1145,9 +1138,8 @@ static int ll_statahead_thread(void *arg) CDEBUG(D_READA, "stop agl thread: sai %p pid %u\n", sai, (unsigned int)agl_thread->t_pid); - l_wait_event(agl_thread->t_ctl_waitq, - thread_is_stopped(agl_thread), - &lwi); + wait_event_idle(agl_thread->t_ctl_waitq, + thread_is_stopped(agl_thread)); } else { /* Set agl_thread flags anyway. */ thread_set_flags(agl_thread, SVC_STOPPED); @@ -1159,8 +1151,8 @@ static int ll_statahead_thread(void *arg) */ while (sai->sai_sent != sai->sai_replied) { /* in case we're not woken up, timeout wait */ - lwi = LWI_TIMEOUT(msecs_to_jiffies(MSEC_PER_SEC >> 3), - NULL, NULL); + struct l_wait_info lwi = LWI_TIMEOUT(msecs_to_jiffies(MSEC_PER_SEC >> 3), + NULL, NULL); l_wait_event(sa_thread->t_ctl_waitq, sai->sai_sent == sai->sai_replied, &lwi); } @@ -1520,7 +1512,6 @@ static int start_statahead_thread(struct inode *dir, struct dentry *dentry) { struct ll_inode_info *lli = ll_i2info(dir); struct ll_statahead_info *sai = NULL; - struct l_wait_info lwi = { 0 }; struct ptlrpc_thread *thread; struct task_struct *task; struct dentry *parent = dentry->d_parent; @@ -1570,9 +1561,8 @@ static int start_statahead_thread(struct inode *dir, struct dentry *dentry) goto out; } - l_wait_event(thread->t_ctl_waitq, - thread_is_running(thread) || thread_is_stopped(thread), - &lwi); + wait_event_idle(thread->t_ctl_waitq, + thread_is_running(thread) || thread_is_stopped(thread)); ll_sai_put(sai); /* diff --git a/drivers/staging/lustre/lustre/lov/lov_object.c b/drivers/staging/lustre/lustre/lov/lov_object.c index 897cf2cd4a24..86cd4f9fbd0c 100644 --- a/drivers/staging/lustre/lustre/lov/lov_object.c +++ b/drivers/staging/lustre/lustre/lov/lov_object.c @@ -723,15 +723,13 @@ static void lov_conf_unlock(struct lov_object *lov) static int lov_layout_wait(const struct lu_env *env, struct lov_object *lov) { - struct l_wait_info lwi = { 0 }; - while (atomic_read(&lov->lo_active_ios) > 0) { CDEBUG(D_INODE, "file:" DFID " wait for active IO, now: %d.\n", PFID(lu_object_fid(lov2lu(lov))), atomic_read(&lov->lo_active_ios)); - l_wait_event(lov->lo_waitq, - atomic_read(&lov->lo_active_ios) == 0, &lwi); + wait_event_idle(lov->lo_waitq, + atomic_read(&lov->lo_active_ios) == 0); } return 0; } diff --git a/drivers/staging/lustre/lustre/mgc/mgc_request.c b/drivers/staging/lustre/lustre/mgc/mgc_request.c index 79ff85feab64..b743aee62349 100644 --- a/drivers/staging/lustre/lustre/mgc/mgc_request.c +++ b/drivers/staging/lustre/lustre/mgc/mgc_request.c @@ -601,9 +601,7 @@ static int mgc_requeue_thread(void *data) config_log_put(cld_prev); /* Wait a bit to see if anyone else needs a requeue */ - lwi = (struct l_wait_info) { 0 }; - l_wait_event(rq_waitq, rq_state & (RQ_NOW | RQ_STOP), - &lwi); + wait_event_idle(rq_waitq, rq_state & (RQ_NOW | RQ_STOP)); spin_lock(&config_list_lock); } diff --git a/drivers/staging/lustre/lustre/obdclass/cl_io.c b/drivers/staging/lustre/lustre/obdclass/cl_io.c index 6ec5218a18c1..902bad22013b 100644 --- a/drivers/staging/lustre/lustre/obdclass/cl_io.c +++ b/drivers/staging/lustre/lustre/obdclass/cl_io.c @@ -1110,10 +1110,8 @@ int cl_sync_io_wait(const struct lu_env *env, struct cl_sync_io *anchor, CERROR("IO failed: %d, still wait for %d remaining entries\n", rc, atomic_read(&anchor->csi_sync_nr)); - lwi = (struct l_wait_info) { 0 }; - (void)l_wait_event(anchor->csi_waitq, - atomic_read(&anchor->csi_sync_nr) == 0, - &lwi); + wait_event_idle(anchor->csi_waitq, + atomic_read(&anchor->csi_sync_nr) == 0); } else { rc = anchor->csi_sync_rc; } diff --git a/drivers/staging/lustre/lustre/obdclass/genops.c b/drivers/staging/lustre/lustre/obdclass/genops.c index b1d6ba4a3190..3ff25b8d3b48 100644 --- a/drivers/staging/lustre/lustre/obdclass/genops.c +++ b/drivers/staging/lustre/lustre/obdclass/genops.c @@ -1237,12 +1237,10 @@ static int obd_zombie_is_idle(void) */ void obd_zombie_barrier(void) { - struct l_wait_info lwi = { 0 }; - if (obd_zombie_pid == current_pid()) /* don't wait for myself */ return; - l_wait_event(obd_zombie_waitq, obd_zombie_is_idle(), &lwi); + wait_event_idle(obd_zombie_waitq, obd_zombie_is_idle()); } EXPORT_SYMBOL(obd_zombie_barrier); @@ -1257,10 +1255,8 @@ static int obd_zombie_impexp_thread(void *unused) obd_zombie_pid = current_pid(); while (!test_bit(OBD_ZOMBIE_STOP, &obd_zombie_flags)) { - struct l_wait_info lwi = { 0 }; - - l_wait_event(obd_zombie_waitq, - !obd_zombie_impexp_check(NULL), &lwi); + wait_event_idle(obd_zombie_waitq, + !obd_zombie_impexp_check(NULL)); obd_zombie_impexp_cull(); /* @@ -1593,7 +1589,6 @@ static inline bool obd_mod_rpc_slot_avail(struct client_obd *cli, u16 obd_get_mod_rpc_slot(struct client_obd *cli, __u32 opc, struct lookup_intent *it) { - struct l_wait_info lwi = LWI_INTR(NULL, NULL); bool close_req = false; u16 i, max; @@ -1631,8 +1626,8 @@ u16 obd_get_mod_rpc_slot(struct client_obd *cli, __u32 opc, CDEBUG(D_RPCTRACE, "%s: sleeping for a modify RPC slot opc %u, max %hu\n", cli->cl_import->imp_obd->obd_name, opc, max); - l_wait_event(cli->cl_mod_rpcs_waitq, - obd_mod_rpc_slot_avail(cli, close_req), &lwi); + wait_event_idle(cli->cl_mod_rpcs_waitq, + obd_mod_rpc_slot_avail(cli, close_req)); } while (true); } EXPORT_SYMBOL(obd_get_mod_rpc_slot); diff --git a/drivers/staging/lustre/lustre/osc/osc_cache.c b/drivers/staging/lustre/lustre/osc/osc_cache.c index 5767ac2a7d16..b8d5adca94e1 100644 --- a/drivers/staging/lustre/lustre/osc/osc_cache.c +++ b/drivers/staging/lustre/lustre/osc/osc_cache.c @@ -964,9 +964,8 @@ static int osc_extent_wait(const struct lu_env *env, struct osc_extent *ext, "%s: wait ext to %u timedout, recovery in progress?\n", cli_name(osc_cli(obj)), state); - lwi = LWI_INTR(NULL, NULL); - rc = l_wait_event(ext->oe_waitq, extent_wait_cb(ext, state), - &lwi); + wait_event_idle(ext->oe_waitq, extent_wait_cb(ext, state)); + rc = 0; } if (rc == 0 && ext->oe_rc < 0) rc = ext->oe_rc; diff --git a/drivers/staging/lustre/lustre/osc/osc_object.c b/drivers/staging/lustre/lustre/osc/osc_object.c index f82c87a77550..6c424f0290bb 100644 --- a/drivers/staging/lustre/lustre/osc/osc_object.c +++ b/drivers/staging/lustre/lustre/osc/osc_object.c @@ -454,12 +454,10 @@ struct lu_object *osc_object_alloc(const struct lu_env *env, int osc_object_invalidate(const struct lu_env *env, struct osc_object *osc) { - struct l_wait_info lwi = { 0 }; - CDEBUG(D_INODE, "Invalidate osc object: %p, # of active IOs: %d\n", osc, atomic_read(&osc->oo_nr_ios)); - l_wait_event(osc->oo_io_waitq, !atomic_read(&osc->oo_nr_ios), &lwi); + wait_event_idle(osc->oo_io_waitq, !atomic_read(&osc->oo_nr_ios)); /* Discard all dirty pages of this object. */ osc_cache_truncate_start(env, osc, 0, NULL); diff --git a/drivers/staging/lustre/lustre/ptlrpc/pinger.c b/drivers/staging/lustre/lustre/ptlrpc/pinger.c index fe6b47bfe8be..af707cb2b62b 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/pinger.c +++ b/drivers/staging/lustre/lustre/ptlrpc/pinger.c @@ -291,7 +291,6 @@ static struct ptlrpc_thread pinger_thread; int ptlrpc_start_pinger(void) { - struct l_wait_info lwi = { 0 }; struct task_struct *task; int rc; @@ -310,8 +309,8 @@ int ptlrpc_start_pinger(void) CERROR("cannot start pinger thread: rc = %d\n", rc); return rc; } - l_wait_event(pinger_thread.t_ctl_waitq, - thread_is_running(&pinger_thread), &lwi); + wait_event_idle(pinger_thread.t_ctl_waitq, + thread_is_running(&pinger_thread)); return 0; } @@ -320,7 +319,6 @@ static int ptlrpc_pinger_remove_timeouts(void); int ptlrpc_stop_pinger(void) { - struct l_wait_info lwi = { 0 }; int rc = 0; if (thread_is_init(&pinger_thread) || @@ -331,8 +329,8 @@ int ptlrpc_stop_pinger(void) thread_set_flags(&pinger_thread, SVC_STOPPING); wake_up(&pinger_thread.t_ctl_waitq); - l_wait_event(pinger_thread.t_ctl_waitq, - thread_is_stopped(&pinger_thread), &lwi); + wait_event_idle(pinger_thread.t_ctl_waitq, + thread_is_stopped(&pinger_thread)); return rc; } diff --git a/drivers/staging/lustre/lustre/ptlrpc/sec_gc.c b/drivers/staging/lustre/lustre/ptlrpc/sec_gc.c index d85c8638c009..b61e1aa25e8c 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/sec_gc.c +++ b/drivers/staging/lustre/lustre/ptlrpc/sec_gc.c @@ -197,7 +197,6 @@ static int sec_gc_main(void *arg) int sptlrpc_gc_init(void) { - struct l_wait_info lwi = { 0 }; struct task_struct *task; mutex_init(&sec_gc_mutex); @@ -214,18 +213,16 @@ int sptlrpc_gc_init(void) return PTR_ERR(task); } - l_wait_event(sec_gc_thread.t_ctl_waitq, - thread_is_running(&sec_gc_thread), &lwi); + wait_event_idle(sec_gc_thread.t_ctl_waitq, + thread_is_running(&sec_gc_thread)); return 0; } void sptlrpc_gc_fini(void) { - struct l_wait_info lwi = { 0 }; - thread_set_flags(&sec_gc_thread, SVC_STOPPING); wake_up(&sec_gc_thread.t_ctl_waitq); - l_wait_event(sec_gc_thread.t_ctl_waitq, - thread_is_stopped(&sec_gc_thread), &lwi); + wait_event_idle(sec_gc_thread.t_ctl_waitq, + thread_is_stopped(&sec_gc_thread)); } diff --git a/drivers/staging/lustre/lustre/ptlrpc/service.c b/drivers/staging/lustre/lustre/ptlrpc/service.c index 63be6e7273f3..1f22926c1355 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/service.c +++ b/drivers/staging/lustre/lustre/ptlrpc/service.c @@ -2233,7 +2233,7 @@ static int ptlrpc_hr_main(void *arg) wake_up(&ptlrpc_hr.hr_waitq); while (!ptlrpc_hr.hr_stopping) { - l_wait_condition(hrt->hrt_waitq, hrt_dont_sleep(hrt, &replies)); + wait_event_idle(hrt->hrt_waitq, hrt_dont_sleep(hrt, &replies)); while (!list_empty(&replies)) { struct ptlrpc_reply_state *rs; @@ -2312,7 +2312,6 @@ static int ptlrpc_start_hr_threads(void) static void ptlrpc_svcpt_stop_threads(struct ptlrpc_service_part *svcpt) { - struct l_wait_info lwi = { 0 }; struct ptlrpc_thread *thread; LIST_HEAD(zombie); @@ -2341,8 +2340,8 @@ static void ptlrpc_svcpt_stop_threads(struct ptlrpc_service_part *svcpt) CDEBUG(D_INFO, "waiting for stopping-thread %s #%u\n", svcpt->scp_service->srv_thread_name, thread->t_id); - l_wait_event(thread->t_ctl_waitq, - thread_is_stopped(thread), &lwi); + wait_event_idle(thread->t_ctl_waitq, + thread_is_stopped(thread)); spin_lock(&svcpt->scp_lock); } @@ -2403,7 +2402,6 @@ int ptlrpc_start_threads(struct ptlrpc_service *svc) int ptlrpc_start_thread(struct ptlrpc_service_part *svcpt, int wait) { - struct l_wait_info lwi = { 0 }; struct ptlrpc_thread *thread; struct ptlrpc_service *svc; struct task_struct *task; @@ -2499,9 +2497,8 @@ int ptlrpc_start_thread(struct ptlrpc_service_part *svcpt, int wait) if (!wait) return 0; - l_wait_event(thread->t_ctl_waitq, - thread_is_running(thread) || thread_is_stopped(thread), - &lwi); + wait_event_idle(thread->t_ctl_waitq, + thread_is_running(thread) || thread_is_stopped(thread)); rc = thread_is_stopped(thread) ? thread->t_id : 0; return rc; From neilb at suse.com Mon Feb 12 21:22:36 2018 From: neilb at suse.com (NeilBrown) Date: Tue, 13 Feb 2018 08:22:36 +1100 Subject: [lustre-devel] [PATCH 01/19] sched/wait: add wait_event_idle() functions. In-Reply-To: <151847037709.22826.16175867257667686132.stgit@noble> References: <151847037709.22826.16175867257667686132.stgit@noble> Message-ID: <151847055644.22826.9130972646394182562.stgit@noble> The new TASK_IDLE state (TASK_UNINTERRUPTIBLE | __TASK_NOLOAD) is not much used. One way to make it easier to use is to add wait_event*() family functions that make use of it. This patch adds: wait_event_idle() wait_event_idle_timeout() wait_event_idle_exclusive() wait_event_idle_exclusive_timeout() This set was chosen because lustre needs them before it can discard its own l_wait_event() macro. Acked-by: Peter Zijlstra (Intel) Reviewed-by: James Simmons Signed-off-by: NeilBrown --- include/linux/wait.h | 114 ++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 114 insertions(+) diff --git a/include/linux/wait.h b/include/linux/wait.h index 55a611486bac..d9f131ecf708 100644 --- a/include/linux/wait.h +++ b/include/linux/wait.h @@ -599,6 +599,120 @@ do { \ __ret; \ }) +/** + * wait_event_idle - wait for a condition without contributing to system load + * @wq_head: the waitqueue to wait on + * @condition: a C expression for the event to wait for + * + * The process is put to sleep (TASK_IDLE) until the + * @condition evaluates to true. + * The @condition is checked each time the waitqueue @wq_head is woken up. + * + * wake_up() has to be called after changing any variable that could + * change the result of the wait condition. + * + */ +#define wait_event_idle(wq_head, condition) \ +do { \ + might_sleep(); \ + if (!(condition)) \ + ___wait_event(wq_head, condition, TASK_IDLE, 0, 0, schedule()); \ +} while (0) + +/** + * wait_event_idle_exclusive - wait for a condition with contributing to system load + * @wq_head: the waitqueue to wait on + * @condition: a C expression for the event to wait for + * + * The process is put to sleep (TASK_IDLE) until the + * @condition evaluates to true. + * The @condition is checked each time the waitqueue @wq_head is woken up. + * + * The process is put on the wait queue with an WQ_FLAG_EXCLUSIVE flag + * set thus if other processes wait on the same list, when this + * process is woken further processes are not considered. + * + * wake_up() has to be called after changing any variable that could + * change the result of the wait condition. + * + */ +#define wait_event_idle_exclusive(wq_head, condition) \ +do { \ + might_sleep(); \ + if (!(condition)) \ + ___wait_event(wq_head, condition, TASK_IDLE, 1, 0, schedule()); \ +} while (0) + +#define __wait_event_idle_timeout(wq_head, condition, timeout) \ + ___wait_event(wq_head, ___wait_cond_timeout(condition), \ + TASK_IDLE, 0, timeout, \ + __ret = schedule_timeout(__ret)) + +/** + * wait_event_idle_timeout - sleep without load until a condition becomes true or a timeout elapses + * @wq_head: the waitqueue to wait on + * @condition: a C expression for the event to wait for + * @timeout: timeout, in jiffies + * + * The process is put to sleep (TASK_IDLE) until the + * @condition evaluates to true. The @condition is checked each time + * the waitqueue @wq_head is woken up. + * + * wake_up() has to be called after changing any variable that could + * change the result of the wait condition. + * + * Returns: + * 0 if the @condition evaluated to %false after the @timeout elapsed, + * 1 if the @condition evaluated to %true after the @timeout elapsed, + * or the remaining jiffies (at least 1) if the @condition evaluated + * to %true before the @timeout elapsed. + */ +#define wait_event_idle_timeout(wq_head, condition, timeout) \ +({ \ + long __ret = timeout; \ + might_sleep(); \ + if (!___wait_cond_timeout(condition)) \ + __ret = __wait_event_idle_timeout(wq_head, condition, timeout); \ + __ret; \ +}) + +#define __wait_event_idle_exclusive_timeout(wq_head, condition, timeout) \ + ___wait_event(wq_head, ___wait_cond_timeout(condition), \ + TASK_IDLE, 1, timeout, \ + __ret = schedule_timeout(__ret)) + +/** + * wait_event_idle_exclusive_timeout - sleep without load until a condition becomes true or a timeout elapses + * @wq_head: the waitqueue to wait on + * @condition: a C expression for the event to wait for + * @timeout: timeout, in jiffies + * + * The process is put to sleep (TASK_IDLE) until the + * @condition evaluates to true. The @condition is checked each time + * the waitqueue @wq_head is woken up. + * + * The process is put on the wait queue with an WQ_FLAG_EXCLUSIVE flag + * set thus if other processes wait on the same list, when this + * process is woken further processes are not considered. + * + * wake_up() has to be called after changing any variable that could + * change the result of the wait condition. + * + * Returns: + * 0 if the @condition evaluated to %false after the @timeout elapsed, + * 1 if the @condition evaluated to %true after the @timeout elapsed, + * or the remaining jiffies (at least 1) if the @condition evaluated + * to %true before the @timeout elapsed. + */ +#define wait_event_idle_exclusive_timeout(wq_head, condition, timeout) \ +({ \ + long __ret = timeout; \ + might_sleep(); \ + if (!___wait_cond_timeout(condition)) \ + __ret = __wait_event_idle_exclusive_timeout(wq_head, condition, timeout);\ + __ret; \ +}) + extern int do_wait_intr(wait_queue_head_t *, wait_queue_entry_t *); extern int do_wait_intr_irq(wait_queue_head_t *, wait_queue_entry_t *); From neilb at suse.com Mon Feb 12 21:22:36 2018 From: neilb at suse.com (NeilBrown) Date: Tue, 13 Feb 2018 08:22:36 +1100 Subject: [lustre-devel] [PATCH 04/19] staging: lustre: discard cfs_time_seconds() In-Reply-To: <151847037709.22826.16175867257667686132.stgit@noble> References: <151847037709.22826.16175867257667686132.stgit@noble> Message-ID: <151847055655.22826.3028342292810517097.stgit@noble> cfs_time_seconds() converts a number of seconds to the matching number of jiffies. The standard way to do this in Linux is "* HZ". So discard cfs_time_seconds() and use "* HZ" instead. Reviewed-by: James Simmons Signed-off-by: NeilBrown --- .../lustre/include/linux/libcfs/libcfs_debug.h | 4 ++-- .../lustre/include/linux/libcfs/libcfs_time.h | 2 +- .../lustre/include/linux/libcfs/linux/linux-time.h | 7 +----- .../staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c | 8 ++++--- .../staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c | 4 ++-- .../staging/lustre/lnet/klnds/socklnd/socklnd.c | 6 +++-- .../staging/lustre/lnet/klnds/socklnd/socklnd_cb.c | 22 ++++++++++---------- drivers/staging/lustre/lnet/libcfs/debug.c | 2 +- drivers/staging/lustre/lnet/libcfs/fail.c | 2 +- drivers/staging/lustre/lnet/libcfs/tracefile.c | 4 ++-- drivers/staging/lustre/lnet/lnet/acceptor.c | 2 +- drivers/staging/lustre/lnet/lnet/api-ni.c | 4 ++-- drivers/staging/lustre/lnet/lnet/lib-move.c | 4 ++-- drivers/staging/lustre/lnet/lnet/net_fault.c | 14 +++++-------- drivers/staging/lustre/lnet/lnet/peer.c | 2 +- drivers/staging/lustre/lnet/lnet/router.c | 8 ++++--- drivers/staging/lustre/lnet/selftest/conrpc.c | 4 ++-- drivers/staging/lustre/lnet/selftest/rpc.c | 2 +- drivers/staging/lustre/lnet/selftest/selftest.h | 2 +- drivers/staging/lustre/lnet/selftest/timer.c | 2 +- drivers/staging/lustre/lustre/include/lustre_dlm.h | 2 +- drivers/staging/lustre/lustre/include/lustre_mdc.h | 2 +- drivers/staging/lustre/lustre/include/lustre_net.h | 2 +- drivers/staging/lustre/lustre/ldlm/ldlm_lock.c | 2 +- drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c | 4 ++-- drivers/staging/lustre/lustre/ldlm/ldlm_pool.c | 2 +- drivers/staging/lustre/lustre/ldlm/ldlm_request.c | 2 +- drivers/staging/lustre/lustre/ldlm/ldlm_resource.c | 2 +- drivers/staging/lustre/lustre/llite/llite_lib.c | 4 ++-- drivers/staging/lustre/lustre/llite/statahead.c | 2 +- drivers/staging/lustre/lustre/lov/lov_request.c | 4 ++-- drivers/staging/lustre/lustre/mdc/mdc_request.c | 2 +- drivers/staging/lustre/lustre/mgc/mgc_request.c | 2 +- drivers/staging/lustre/lustre/obdclass/cl_io.c | 2 +- .../staging/lustre/lustre/obdecho/echo_client.c | 2 +- drivers/staging/lustre/lustre/osc/osc_cache.c | 4 ++-- drivers/staging/lustre/lustre/osc/osc_object.c | 2 +- drivers/staging/lustre/lustre/ptlrpc/client.c | 10 +++++---- drivers/staging/lustre/lustre/ptlrpc/events.c | 2 +- drivers/staging/lustre/lustre/ptlrpc/import.c | 15 ++++++-------- drivers/staging/lustre/lustre/ptlrpc/niobuf.c | 4 ++-- .../staging/lustre/lustre/ptlrpc/pack_generic.c | 2 +- drivers/staging/lustre/lustre/ptlrpc/pinger.c | 8 ++++--- drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c | 4 ++-- drivers/staging/lustre/lustre/ptlrpc/recover.c | 2 +- drivers/staging/lustre/lustre/ptlrpc/service.c | 8 ++++--- 46 files changed, 96 insertions(+), 106 deletions(-) diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_debug.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_debug.h index 1b98f0953afb..9290a19429e7 100644 --- a/drivers/staging/lustre/include/linux/libcfs/libcfs_debug.h +++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_debug.h @@ -66,8 +66,8 @@ extern unsigned int libcfs_panic_on_lbug; # define DEBUG_SUBSYSTEM S_UNDEFINED #endif -#define CDEBUG_DEFAULT_MAX_DELAY (cfs_time_seconds(600)) /* jiffies */ -#define CDEBUG_DEFAULT_MIN_DELAY ((cfs_time_seconds(1) + 1) / 2) /* jiffies */ +#define CDEBUG_DEFAULT_MAX_DELAY (600 * HZ) /* jiffies */ +#define CDEBUG_DEFAULT_MIN_DELAY ((HZ + 1) / 2) /* jiffies */ #define CDEBUG_DEFAULT_BACKOFF 2 struct cfs_debug_limit_state { unsigned long cdls_next; diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_time.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_time.h index 9699646decb9..c4f25be78268 100644 --- a/drivers/staging/lustre/include/linux/libcfs/libcfs_time.h +++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_time.h @@ -62,7 +62,7 @@ static inline int cfs_time_aftereq(unsigned long t1, unsigned long t2) static inline unsigned long cfs_time_shift(int seconds) { - return cfs_time_add(cfs_time_current(), cfs_time_seconds(seconds)); + return cfs_time_add(cfs_time_current(), seconds * HZ); } /* diff --git a/drivers/staging/lustre/include/linux/libcfs/linux/linux-time.h b/drivers/staging/lustre/include/linux/libcfs/linux/linux-time.h index aece13698eb4..805cb326af86 100644 --- a/drivers/staging/lustre/include/linux/libcfs/linux/linux-time.h +++ b/drivers/staging/lustre/include/linux/libcfs/linux/linux-time.h @@ -65,11 +65,6 @@ static inline unsigned long cfs_time_current(void) return jiffies; } -static inline long cfs_time_seconds(int seconds) -{ - return ((long)seconds) * msecs_to_jiffies(MSEC_PER_SEC); -} - static inline long cfs_duration_sec(long d) { return d / msecs_to_jiffies(MSEC_PER_SEC); @@ -85,7 +80,7 @@ static inline u64 cfs_time_add_64(u64 t, u64 d) static inline u64 cfs_time_shift_64(int seconds) { return cfs_time_add_64(cfs_time_current_64(), - cfs_time_seconds(seconds)); + seconds * HZ); } static inline int cfs_time_before_64(u64 t1, u64 t2) diff --git a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c index ec84edfda271..7ae2955c4db6 100644 --- a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c +++ b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c @@ -1211,7 +1211,7 @@ static struct kib_hca_dev *kiblnd_current_hdev(struct kib_dev *dev) CDEBUG(D_NET, "%s: Wait for failover\n", dev->ibd_ifname); set_current_state(TASK_INTERRUPTIBLE); - schedule_timeout(cfs_time_seconds(1) / 100); + schedule_timeout(HZ / 100); read_lock_irqsave(&kiblnd_data.kib_global_lock, flags); } @@ -1921,7 +1921,7 @@ struct list_head *kiblnd_pool_alloc_node(struct kib_poolset *ps) set_current_state(TASK_INTERRUPTIBLE); schedule_timeout(interval); - if (interval < cfs_time_seconds(1)) + if (interval < HZ) interval *= 2; goto again; @@ -2541,7 +2541,7 @@ static void kiblnd_base_shutdown(void) "Waiting for %d threads to terminate\n", atomic_read(&kiblnd_data.kib_nthreads)); set_current_state(TASK_UNINTERRUPTIBLE); - schedule_timeout(cfs_time_seconds(1)); + schedule_timeout(HZ); } /* fall through */ @@ -2592,7 +2592,7 @@ static void kiblnd_shutdown(struct lnet_ni *ni) libcfs_nid2str(ni->ni_nid), atomic_read(&net->ibn_npeers)); set_current_state(TASK_UNINTERRUPTIBLE); - schedule_timeout(cfs_time_seconds(1)); + schedule_timeout(HZ); } kiblnd_net_fini_pools(net); diff --git a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c index b3e7f28eb978..ca094555e04b 100644 --- a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c +++ b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c @@ -3728,8 +3728,8 @@ kiblnd_failover_thread(void *arg) add_wait_queue(&kiblnd_data.kib_failover_waitq, &wait); write_unlock_irqrestore(glock, flags); - rc = schedule_timeout(long_sleep ? cfs_time_seconds(10) : - cfs_time_seconds(1)); + rc = schedule_timeout(long_sleep ? 10 * HZ : + HZ); remove_wait_queue(&kiblnd_data.kib_failover_waitq, &wait); write_lock_irqsave(glock, flags); diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c index ff292216290d..7086678e1c3e 100644 --- a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c +++ b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c @@ -1677,7 +1677,7 @@ ksocknal_destroy_conn(struct ksock_conn *conn) switch (conn->ksnc_rx_state) { case SOCKNAL_RX_LNET_PAYLOAD: last_rcv = conn->ksnc_rx_deadline - - cfs_time_seconds(*ksocknal_tunables.ksnd_timeout); + *ksocknal_tunables.ksnd_timeout * HZ; CERROR("Completing partial receive from %s[%d], ip %pI4h:%d, with error, wanted: %zd, left: %d, last alive is %ld secs ago\n", libcfs_id2str(conn->ksnc_peer->ksnp_id), conn->ksnc_type, &conn->ksnc_ipaddr, conn->ksnc_port, @@ -2356,7 +2356,7 @@ ksocknal_base_shutdown(void) ksocknal_data.ksnd_nthreads); read_unlock(&ksocknal_data.ksnd_global_lock); set_current_state(TASK_UNINTERRUPTIBLE); - schedule_timeout(cfs_time_seconds(1)); + schedule_timeout(HZ); read_lock(&ksocknal_data.ksnd_global_lock); } read_unlock(&ksocknal_data.ksnd_global_lock); @@ -2599,7 +2599,7 @@ ksocknal_shutdown(struct lnet_ni *ni) "waiting for %d peers to disconnect\n", net->ksnn_npeers); set_current_state(TASK_UNINTERRUPTIBLE); - schedule_timeout(cfs_time_seconds(1)); + schedule_timeout(HZ); ksocknal_debug_peerhash(ni); diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c index 11fd3a36424f..63e452f666bf 100644 --- a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c +++ b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c @@ -189,7 +189,7 @@ ksocknal_transmit(struct ksock_conn *conn, struct ksock_tx *tx) if (ksocknal_data.ksnd_stall_tx) { set_current_state(TASK_UNINTERRUPTIBLE); - schedule_timeout(cfs_time_seconds(ksocknal_data.ksnd_stall_tx)); + schedule_timeout(ksocknal_data.ksnd_stall_tx * HZ); } LASSERT(tx->tx_resid); @@ -294,7 +294,7 @@ ksocknal_receive(struct ksock_conn *conn) if (ksocknal_data.ksnd_stall_rx) { set_current_state(TASK_UNINTERRUPTIBLE); - schedule_timeout(cfs_time_seconds(ksocknal_data.ksnd_stall_rx)); + schedule_timeout(ksocknal_data.ksnd_stall_rx * HZ); } rc = ksocknal_connsock_addref(conn); @@ -1780,7 +1780,7 @@ ksocknal_connect(struct ksock_route *route) int rc = 0; deadline = cfs_time_add(cfs_time_current(), - cfs_time_seconds(*ksocknal_tunables.ksnd_timeout)); + *ksocknal_tunables.ksnd_timeout * HZ); write_lock_bh(&ksocknal_data.ksnd_global_lock); @@ -1878,7 +1878,7 @@ ksocknal_connect(struct ksock_route *route) * so min_reconnectms should be good heuristic */ route->ksnr_retry_interval = - cfs_time_seconds(*ksocknal_tunables.ksnd_min_reconnectms) / 1000; + *ksocknal_tunables.ksnd_min_reconnectms * HZ / 1000; route->ksnr_timeout = cfs_time_add(cfs_time_current(), route->ksnr_retry_interval); } @@ -1899,10 +1899,10 @@ ksocknal_connect(struct ksock_route *route) route->ksnr_retry_interval *= 2; route->ksnr_retry_interval = max(route->ksnr_retry_interval, - cfs_time_seconds(*ksocknal_tunables.ksnd_min_reconnectms) / 1000); + (long)*ksocknal_tunables.ksnd_min_reconnectms * HZ / 1000); route->ksnr_retry_interval = min(route->ksnr_retry_interval, - cfs_time_seconds(*ksocknal_tunables.ksnd_max_reconnectms) / 1000); + (long)*ksocknal_tunables.ksnd_max_reconnectms * HZ / 1000); LASSERT(route->ksnr_retry_interval); route->ksnr_timeout = cfs_time_add(cfs_time_current(), @@ -1972,7 +1972,7 @@ ksocknal_connd_check_start(time64_t sec, long *timeout) if (sec - ksocknal_data.ksnd_connd_failed_stamp <= 1) { /* may run out of resource, retry later */ - *timeout = cfs_time_seconds(1); + *timeout = HZ; return 0; } @@ -2031,8 +2031,8 @@ ksocknal_connd_check_stop(time64_t sec, long *timeout) val = (int)(ksocknal_data.ksnd_connd_starting_stamp + SOCKNAL_CONND_TIMEOUT - sec); - *timeout = (val > 0) ? cfs_time_seconds(val) : - cfs_time_seconds(SOCKNAL_CONND_TIMEOUT); + *timeout = (val > 0) ? val * HZ : + SOCKNAL_CONND_TIMEOUT * HZ; if (val > 0) return 0; @@ -2307,7 +2307,7 @@ ksocknal_send_keepalive_locked(struct ksock_peer *peer) if (*ksocknal_tunables.ksnd_keepalive <= 0 || time_before(cfs_time_current(), cfs_time_add(peer->ksnp_last_alive, - cfs_time_seconds(*ksocknal_tunables.ksnd_keepalive)))) + *ksocknal_tunables.ksnd_keepalive * HZ))) return 0; if (time_before(cfs_time_current(), peer->ksnp_send_keepalive)) @@ -2563,7 +2563,7 @@ ksocknal_reaper(void *arg) ksocknal_data.ksnd_peer_hash_size; } - deadline = cfs_time_add(deadline, cfs_time_seconds(p)); + deadline = cfs_time_add(deadline, p * HZ); } if (nenomem_conns) { diff --git a/drivers/staging/lustre/lnet/libcfs/debug.c b/drivers/staging/lustre/lnet/libcfs/debug.c index 551c45bf4108..c70d2ae29b11 100644 --- a/drivers/staging/lustre/lnet/libcfs/debug.c +++ b/drivers/staging/lustre/lnet/libcfs/debug.c @@ -113,7 +113,7 @@ static int param_set_delay_minmax(const char *val, if (rc) return -EINVAL; - d = cfs_time_seconds(sec) / 100; + d = sec * HZ / 100; if (d < min || d > max) return -EINVAL; diff --git a/drivers/staging/lustre/lnet/libcfs/fail.c b/drivers/staging/lustre/lnet/libcfs/fail.c index 39439b303d65..d3f1e866c6a7 100644 --- a/drivers/staging/lustre/lnet/libcfs/fail.c +++ b/drivers/staging/lustre/lnet/libcfs/fail.c @@ -134,7 +134,7 @@ int __cfs_fail_timeout_set(u32 id, u32 value, int ms, int set) CERROR("cfs_fail_timeout id %x sleeping for %dms\n", id, ms); set_current_state(TASK_UNINTERRUPTIBLE); - schedule_timeout(cfs_time_seconds(ms) / 1000); + schedule_timeout(ms * HZ / 1000); CERROR("cfs_fail_timeout id %x awake\n", id); } return ret; diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.c b/drivers/staging/lustre/lnet/libcfs/tracefile.c index 57913aae1d88..4affca750bc5 100644 --- a/drivers/staging/lustre/lnet/libcfs/tracefile.c +++ b/drivers/staging/lustre/lnet/libcfs/tracefile.c @@ -441,7 +441,7 @@ int libcfs_debug_vmsg2(struct libcfs_debug_msg_data *msgdata, if (cfs_time_after(cfs_time_current(), cdls->cdls_next + libcfs_console_max_delay + - cfs_time_seconds(10))) { + 10 * HZ)) { /* last timeout was a long time ago */ cdls->cdls_delay /= libcfs_console_backoff * 4; } else { @@ -1071,7 +1071,7 @@ static int tracefiled(void *arg) init_waitqueue_entry(&__wait, current); add_wait_queue(&tctl->tctl_waitq, &__wait); set_current_state(TASK_INTERRUPTIBLE); - schedule_timeout(cfs_time_seconds(1)); + schedule_timeout(HZ); remove_wait_queue(&tctl->tctl_waitq, &__wait); } complete(&tctl->tctl_stop); diff --git a/drivers/staging/lustre/lnet/lnet/acceptor.c b/drivers/staging/lustre/lnet/lnet/acceptor.c index ee85cab6f437..6c1f4941d4ba 100644 --- a/drivers/staging/lustre/lnet/lnet/acceptor.c +++ b/drivers/staging/lustre/lnet/lnet/acceptor.c @@ -365,7 +365,7 @@ lnet_acceptor(void *arg) if (rc != -EAGAIN) { CWARN("Accept error %d: pausing...\n", rc); set_current_state(TASK_UNINTERRUPTIBLE); - schedule_timeout(cfs_time_seconds(1)); + schedule_timeout(HZ); } continue; } diff --git a/drivers/staging/lustre/lnet/lnet/api-ni.c b/drivers/staging/lustre/lnet/lnet/api-ni.c index 2c7abad57104..93e6274e9dac 100644 --- a/drivers/staging/lustre/lnet/lnet/api-ni.c +++ b/drivers/staging/lustre/lnet/lnet/api-ni.c @@ -970,7 +970,7 @@ lnet_ping_md_unlink(struct lnet_ping_info *pinfo, while (pinfo->pi_features != LNET_PING_FEAT_INVAL) { CDEBUG(D_NET, "Still waiting for ping MD to unlink\n"); set_current_state(TASK_UNINTERRUPTIBLE); - schedule_timeout(cfs_time_seconds(1)); + schedule_timeout(HZ); } cfs_restore_sigs(blocked); @@ -1109,7 +1109,7 @@ lnet_clear_zombies_nis_locked(void) libcfs_nid2str(ni->ni_nid)); } set_current_state(TASK_UNINTERRUPTIBLE); - schedule_timeout(cfs_time_seconds(1)); + schedule_timeout(HZ); lnet_net_lock(LNET_LOCK_EX); continue; } diff --git a/drivers/staging/lustre/lnet/lnet/lib-move.c b/drivers/staging/lustre/lnet/lnet/lib-move.c index c673037dbce4..ed43b3f4b114 100644 --- a/drivers/staging/lustre/lnet/lnet/lib-move.c +++ b/drivers/staging/lustre/lnet/lnet/lib-move.c @@ -524,7 +524,7 @@ lnet_peer_is_alive(struct lnet_peer *lp, unsigned long now) return 0; deadline = cfs_time_add(lp->lp_last_alive, - cfs_time_seconds(lp->lp_ni->ni_peertimeout)); + lp->lp_ni->ni_peertimeout * HZ); alive = cfs_time_after(deadline, now); /* Update obsolete lp_alive except for routers assumed to be dead @@ -562,7 +562,7 @@ lnet_peer_alive_locked(struct lnet_peer *lp) unsigned long next_query = cfs_time_add(lp->lp_last_query, - cfs_time_seconds(lnet_queryinterval)); + lnet_queryinterval * HZ); if (time_before(now, next_query)) { if (lp->lp_alive) diff --git a/drivers/staging/lustre/lnet/lnet/net_fault.c b/drivers/staging/lustre/lnet/lnet/net_fault.c index e3468cef273b..a63b7941d435 100644 --- a/drivers/staging/lustre/lnet/lnet/net_fault.c +++ b/drivers/staging/lustre/lnet/lnet/net_fault.c @@ -315,9 +315,8 @@ drop_rule_match(struct lnet_drop_rule *rule, lnet_nid_t src, rule->dr_time_base = now; rule->dr_drop_time = rule->dr_time_base + - cfs_time_seconds( - prandom_u32_max(attr->u.drop.da_interval)); - rule->dr_time_base += cfs_time_seconds(attr->u.drop.da_interval); + prandom_u32_max(attr->u.drop.da_interval) * HZ; + rule->dr_time_base += attr->u.drop.da_interval * HZ; CDEBUG(D_NET, "Drop Rule %s->%s: next drop : %lu\n", libcfs_nid2str(attr->fa_src), @@ -440,8 +439,7 @@ static struct delay_daemon_data delay_dd; static unsigned long round_timeout(unsigned long timeout) { - return cfs_time_seconds((unsigned int) - cfs_duration_sec(cfs_time_sub(timeout, 0)) + 1); + return (unsigned int)rounddown(timeout, HZ) + HZ; } static void @@ -483,10 +481,8 @@ delay_rule_match(struct lnet_delay_rule *rule, lnet_nid_t src, rule->dl_time_base = now; rule->dl_delay_time = rule->dl_time_base + - cfs_time_seconds( - prandom_u32_max( - attr->u.delay.la_interval)); - rule->dl_time_base += cfs_time_seconds(attr->u.delay.la_interval); + prandom_u32_max(attr->u.delay.la_interval) * HZ; + rule->dl_time_base += attr->u.delay.la_interval * HZ; CDEBUG(D_NET, "Delay Rule %s->%s: next delay : %lu\n", libcfs_nid2str(attr->fa_src), diff --git a/drivers/staging/lustre/lnet/lnet/peer.c b/drivers/staging/lustre/lnet/lnet/peer.c index 3e157c10fec4..3d4caa609c83 100644 --- a/drivers/staging/lustre/lnet/lnet/peer.c +++ b/drivers/staging/lustre/lnet/lnet/peer.c @@ -137,7 +137,7 @@ lnet_peer_table_deathrow_wait_locked(struct lnet_peer_table *ptable, ptable->pt_zombies); } set_current_state(TASK_UNINTERRUPTIBLE); - schedule_timeout(cfs_time_seconds(1) >> 1); + schedule_timeout(HZ >> 1); lnet_net_lock(cpt_locked); } } diff --git a/drivers/staging/lustre/lnet/lnet/router.c b/drivers/staging/lustre/lnet/lnet/router.c index 6504761ca598..ec1d3e58519a 100644 --- a/drivers/staging/lustre/lnet/lnet/router.c +++ b/drivers/staging/lustre/lnet/lnet/router.c @@ -808,7 +808,7 @@ lnet_wait_known_routerstate(void) return; set_current_state(TASK_UNINTERRUPTIBLE); - schedule_timeout(cfs_time_seconds(1)); + schedule_timeout(HZ); } } @@ -1011,7 +1011,7 @@ lnet_ping_router_locked(struct lnet_peer *rtr) if (secs && !rtr->lp_ping_notsent && cfs_time_after(now, cfs_time_add(rtr->lp_ping_timestamp, - cfs_time_seconds(secs)))) { + secs * HZ))) { int rc; struct lnet_process_id id; struct lnet_handle_md mdh; @@ -1185,7 +1185,7 @@ lnet_prune_rc_data(int wait_unlink) CDEBUG(((i & (-i)) == i) ? D_WARNING : D_NET, "Waiting for rc buffers to unlink\n"); set_current_state(TASK_UNINTERRUPTIBLE); - schedule_timeout(cfs_time_seconds(1) / 4); + schedule_timeout(HZ / 4); lnet_net_lock(LNET_LOCK_EX); } @@ -1282,7 +1282,7 @@ lnet_router_checker(void *arg) else wait_event_interruptible_timeout(the_lnet.ln_rc_waitq, false, - cfs_time_seconds(1)); + HZ); } lnet_prune_rc_data(1); /* wait for UNLINK */ diff --git a/drivers/staging/lustre/lnet/selftest/conrpc.c b/drivers/staging/lustre/lnet/selftest/conrpc.c index 7aa515c34594..6dcc966b293b 100644 --- a/drivers/staging/lustre/lnet/selftest/conrpc.c +++ b/drivers/staging/lustre/lnet/selftest/conrpc.c @@ -359,7 +359,7 @@ lstcon_rpc_trans_postwait(struct lstcon_rpc_trans *trans, int timeout) rc = wait_event_interruptible_timeout(trans->tas_waitq, lstcon_rpc_trans_check(trans), - cfs_time_seconds(timeout)); + timeout * HZ); rc = (rc > 0) ? 0 : ((rc < 0) ? -EINTR : -ETIMEDOUT); mutex_lock(&console_session.ses_mutex); @@ -1350,7 +1350,7 @@ lstcon_rpc_cleanup_wait(void) CWARN("Session is shutting down, waiting for termination of transactions\n"); set_current_state(TASK_UNINTERRUPTIBLE); - schedule_timeout(cfs_time_seconds(1)); + schedule_timeout(HZ); mutex_lock(&console_session.ses_mutex); } diff --git a/drivers/staging/lustre/lnet/selftest/rpc.c b/drivers/staging/lustre/lnet/selftest/rpc.c index f8198ad1046e..9613b0a77007 100644 --- a/drivers/staging/lustre/lnet/selftest/rpc.c +++ b/drivers/staging/lustre/lnet/selftest/rpc.c @@ -1604,7 +1604,7 @@ srpc_startup(void) /* 1 second pause to avoid timestamp reuse */ set_current_state(TASK_UNINTERRUPTIBLE); - schedule_timeout(cfs_time_seconds(1)); + schedule_timeout(HZ); srpc_data.rpc_matchbits = ((__u64)ktime_get_real_seconds()) << 48; srpc_data.rpc_state = SRPC_STATE_NONE; diff --git a/drivers/staging/lustre/lnet/selftest/selftest.h b/drivers/staging/lustre/lnet/selftest/selftest.h index ad04534f000c..05466b85e1c0 100644 --- a/drivers/staging/lustre/lnet/selftest/selftest.h +++ b/drivers/staging/lustre/lnet/selftest/selftest.h @@ -575,7 +575,7 @@ swi_state2str(int state) #define selftest_wait_events() \ do { \ set_current_state(TASK_UNINTERRUPTIBLE); \ - schedule_timeout(cfs_time_seconds(1) / 10); \ + schedule_timeout(HZ / 10); \ } while (0) #define lst_wait_until(cond, lock, fmt, ...) \ diff --git a/drivers/staging/lustre/lnet/selftest/timer.c b/drivers/staging/lustre/lnet/selftest/timer.c index ab125a8524c5..9716afeb3c94 100644 --- a/drivers/staging/lustre/lnet/selftest/timer.c +++ b/drivers/staging/lustre/lnet/selftest/timer.c @@ -177,7 +177,7 @@ stt_timer_main(void *arg) rc = wait_event_timeout(stt_data.stt_waitq, stt_data.stt_shuttingdown, - cfs_time_seconds(STTIMER_SLOTTIME)); + STTIMER_SLOTTIME * HZ); } spin_lock(&stt_data.stt_lock); diff --git a/drivers/staging/lustre/lustre/include/lustre_dlm.h b/drivers/staging/lustre/lustre/include/lustre_dlm.h index e0b17052b2ea..239aa2b1268f 100644 --- a/drivers/staging/lustre/lustre/include/lustre_dlm.h +++ b/drivers/staging/lustre/lustre/include/lustre_dlm.h @@ -60,7 +60,7 @@ struct obd_device; #define OBD_LDLM_DEVICENAME "ldlm" #define LDLM_DEFAULT_LRU_SIZE (100 * num_online_cpus()) -#define LDLM_DEFAULT_MAX_ALIVE (cfs_time_seconds(3900)) /* 65 min */ +#define LDLM_DEFAULT_MAX_ALIVE (65 * 60 * HZ) /* 65 min */ #define LDLM_DEFAULT_PARALLEL_AST_LIMIT 1024 /** diff --git a/drivers/staging/lustre/lustre/include/lustre_mdc.h b/drivers/staging/lustre/lustre/include/lustre_mdc.h index 007e1ec3f0f4..a9c9992a2502 100644 --- a/drivers/staging/lustre/lustre/include/lustre_mdc.h +++ b/drivers/staging/lustre/lustre/include/lustre_mdc.h @@ -124,7 +124,7 @@ static inline void mdc_get_rpc_lock(struct mdc_rpc_lock *lck, */ while (unlikely(lck->rpcl_it == MDC_FAKE_RPCL_IT)) { mutex_unlock(&lck->rpcl_mutex); - schedule_timeout(cfs_time_seconds(1) / 4); + schedule_timeout(HZ / 4); goto again; } diff --git a/drivers/staging/lustre/lustre/include/lustre_net.h b/drivers/staging/lustre/lustre/include/lustre_net.h index 4c665eca2467..5a4434e7c85a 100644 --- a/drivers/staging/lustre/lustre/include/lustre_net.h +++ b/drivers/staging/lustre/lustre/include/lustre_net.h @@ -2262,7 +2262,7 @@ static inline int ptlrpc_send_limit_expired(struct ptlrpc_request *req) { if (req->rq_delay_limit != 0 && time_before(cfs_time_add(req->rq_queued_time, - cfs_time_seconds(req->rq_delay_limit)), + req->rq_delay_limit * HZ), cfs_time_current())) { return 1; } diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c b/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c index 4f700ddb47c6..773abe78708a 100644 --- a/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c @@ -1366,7 +1366,7 @@ enum ldlm_mode ldlm_lock_match(struct ldlm_namespace *ns, __u64 flags, } } - lwi = LWI_TIMEOUT_INTR(cfs_time_seconds(obd_timeout), + lwi = LWI_TIMEOUT_INTR(obd_timeout * HZ, NULL, LWI_ON_SIGNAL_NOOP, NULL); /* XXX FIXME see comment on CAN_MATCH in lustre_dlm.h */ diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c b/drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c index 6c7c4b19a0a0..58913e628124 100644 --- a/drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c @@ -163,7 +163,7 @@ static void ldlm_handle_cp_callback(struct ptlrpc_request *req, LDLM_DEBUG(lock, "client completion callback handler START"); if (OBD_FAIL_CHECK(OBD_FAIL_LDLM_CANCEL_BL_CB_RACE)) { - int to = cfs_time_seconds(1); + int to = HZ; while (to > 0) { set_current_state(TASK_INTERRUPTIBLE); @@ -327,7 +327,7 @@ static void ldlm_handle_gl_callback(struct ptlrpc_request *req, !lock->l_readers && !lock->l_writers && cfs_time_after(cfs_time_current(), cfs_time_add(lock->l_last_used, - cfs_time_seconds(10)))) { + 10 * HZ))) { unlock_res_and_lock(lock); if (ldlm_bl_to_thread_lock(ns, NULL, lock)) ldlm_handle_bl_callback(ns, NULL, lock); diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c b/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c index f27c2694793a..622245a5f049 100644 --- a/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c @@ -1008,7 +1008,7 @@ static int ldlm_pools_thread_main(void *arg) * Wait until the next check time, or until we're * stopped. */ - lwi = LWI_TIMEOUT(cfs_time_seconds(c_time), + lwi = LWI_TIMEOUT(c_time * HZ, NULL, NULL); l_wait_event(thread->t_ctl_waitq, thread_is_stopping(thread) || diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_request.c b/drivers/staging/lustre/lustre/ldlm/ldlm_request.c index 6aa37463db46..a244fa717134 100644 --- a/drivers/staging/lustre/lustre/ldlm/ldlm_request.c +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_request.c @@ -288,7 +288,7 @@ int ldlm_completion_ast(struct ldlm_lock *lock, __u64 flags, void *data) LDLM_DEBUG(lock, "waiting indefinitely because of NO_TIMEOUT"); lwi = LWI_INTR(interrupted_completion_wait, &lwd); } else { - lwi = LWI_TIMEOUT_INTR(cfs_time_seconds(timeout), + lwi = LWI_TIMEOUT_INTR(timeout * HZ, ldlm_expired_completion_wait, interrupted_completion_wait, &lwd); } diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_resource.c b/drivers/staging/lustre/lustre/ldlm/ldlm_resource.c index 9958533cc227..2e66825c8f4b 100644 --- a/drivers/staging/lustre/lustre/ldlm/ldlm_resource.c +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_resource.c @@ -799,7 +799,7 @@ static void cleanup_resource(struct ldlm_resource *res, struct list_head *q, LDLM_DEBUG(lock, "setting FL_LOCAL_ONLY"); if (lock->l_flags & LDLM_FL_FAIL_LOC) { set_current_state(TASK_UNINTERRUPTIBLE); - schedule_timeout(cfs_time_seconds(4)); + schedule_timeout(4 * HZ); set_current_state(TASK_RUNNING); } if (lock->l_completion_ast) diff --git a/drivers/staging/lustre/lustre/llite/llite_lib.c b/drivers/staging/lustre/lustre/llite/llite_lib.c index 020f0faeb750..c820b201af71 100644 --- a/drivers/staging/lustre/lustre/llite/llite_lib.c +++ b/drivers/staging/lustre/lustre/llite/llite_lib.c @@ -2026,8 +2026,8 @@ void ll_umount_begin(struct super_block *sb) * to decrement mnt_cnt and hope to finish it within 10sec. */ init_waitqueue_head(&waitq); - lwi = LWI_TIMEOUT_INTERVAL(cfs_time_seconds(10), - cfs_time_seconds(1), NULL, NULL); + lwi = LWI_TIMEOUT_INTERVAL(10 * HZ, + HZ, NULL, NULL); l_wait_event(waitq, may_umount(sbi->ll_mnt.mnt), &lwi); schedule(); diff --git a/drivers/staging/lustre/lustre/llite/statahead.c b/drivers/staging/lustre/lustre/llite/statahead.c index 78005cc6e831..96360f104b92 100644 --- a/drivers/staging/lustre/lustre/llite/statahead.c +++ b/drivers/staging/lustre/lustre/llite/statahead.c @@ -1424,7 +1424,7 @@ static int revalidate_statahead_dentry(struct inode *dir, spin_lock(&lli->lli_sa_lock); sai->sai_index_wait = entry->se_index; spin_unlock(&lli->lli_sa_lock); - lwi = LWI_TIMEOUT_INTR(cfs_time_seconds(30), NULL, + lwi = LWI_TIMEOUT_INTR(30 * HZ, NULL, LWI_ON_SIGNAL_NOOP, NULL); rc = l_wait_event(sai->sai_waitq, sa_ready(entry), &lwi); if (rc < 0) { diff --git a/drivers/staging/lustre/lustre/lov/lov_request.c b/drivers/staging/lustre/lustre/lov/lov_request.c index cfa1d7f92b0f..fb3b7a7fa32a 100644 --- a/drivers/staging/lustre/lustre/lov/lov_request.c +++ b/drivers/staging/lustre/lustre/lov/lov_request.c @@ -126,8 +126,8 @@ static int lov_check_and_wait_active(struct lov_obd *lov, int ost_idx) mutex_unlock(&lov->lov_lock); init_waitqueue_head(&waitq); - lwi = LWI_TIMEOUT_INTERVAL(cfs_time_seconds(obd_timeout), - cfs_time_seconds(1), NULL, NULL); + lwi = LWI_TIMEOUT_INTERVAL(obd_timeout * HZ, + HZ, NULL, NULL); rc = l_wait_event(waitq, lov_check_set(lov, ost_idx), &lwi); if (tgt->ltd_active) diff --git a/drivers/staging/lustre/lustre/mdc/mdc_request.c b/drivers/staging/lustre/lustre/mdc/mdc_request.c index 03e55bca4ada..b12518ba5ae9 100644 --- a/drivers/staging/lustre/lustre/mdc/mdc_request.c +++ b/drivers/staging/lustre/lustre/mdc/mdc_request.c @@ -888,7 +888,7 @@ static int mdc_getpage(struct obd_export *exp, const struct lu_fid *fid, exp->exp_obd->obd_name, -EIO); return -EIO; } - lwi = LWI_TIMEOUT_INTR(cfs_time_seconds(resends), NULL, NULL, + lwi = LWI_TIMEOUT_INTR(resends * HZ, NULL, NULL, NULL); l_wait_event(waitq, 0, &lwi); diff --git a/drivers/staging/lustre/lustre/mgc/mgc_request.c b/drivers/staging/lustre/lustre/mgc/mgc_request.c index b743aee62349..a01d13bde102 100644 --- a/drivers/staging/lustre/lustre/mgc/mgc_request.c +++ b/drivers/staging/lustre/lustre/mgc/mgc_request.c @@ -1628,7 +1628,7 @@ int mgc_process_log(struct obd_device *mgc, struct config_llog_data *cld) if (rcl == -ESHUTDOWN && atomic_read(&mgc->u.cli.cl_mgc_refcount) > 0 && !retry) { - int secs = cfs_time_seconds(obd_timeout); + int secs = obd_timeout * HZ; struct obd_import *imp; struct l_wait_info lwi; diff --git a/drivers/staging/lustre/lustre/obdclass/cl_io.c b/drivers/staging/lustre/lustre/obdclass/cl_io.c index 902bad22013b..ce5e7bdda692 100644 --- a/drivers/staging/lustre/lustre/obdclass/cl_io.c +++ b/drivers/staging/lustre/lustre/obdclass/cl_io.c @@ -1097,7 +1097,7 @@ EXPORT_SYMBOL(cl_sync_io_init); int cl_sync_io_wait(const struct lu_env *env, struct cl_sync_io *anchor, long timeout) { - struct l_wait_info lwi = LWI_TIMEOUT_INTR(cfs_time_seconds(timeout), + struct l_wait_info lwi = LWI_TIMEOUT_INTR(timeout * HZ, NULL, NULL, NULL); int rc; diff --git a/drivers/staging/lustre/lustre/obdecho/echo_client.c b/drivers/staging/lustre/lustre/obdecho/echo_client.c index b9c1dc7e61b0..9c5ce5074b66 100644 --- a/drivers/staging/lustre/lustre/obdecho/echo_client.c +++ b/drivers/staging/lustre/lustre/obdecho/echo_client.c @@ -752,7 +752,7 @@ static struct lu_device *echo_device_free(const struct lu_env *env, spin_unlock(&ec->ec_lock); CERROR("echo_client still has objects at cleanup time, wait for 1 second\n"); set_current_state(TASK_UNINTERRUPTIBLE); - schedule_timeout(cfs_time_seconds(1)); + schedule_timeout(HZ); lu_site_purge(env, ed->ed_site, -1); spin_lock(&ec->ec_lock); } diff --git a/drivers/staging/lustre/lustre/osc/osc_cache.c b/drivers/staging/lustre/lustre/osc/osc_cache.c index b8d5adca94e1..0797e671f667 100644 --- a/drivers/staging/lustre/lustre/osc/osc_cache.c +++ b/drivers/staging/lustre/lustre/osc/osc_cache.c @@ -934,7 +934,7 @@ static int osc_extent_wait(const struct lu_env *env, struct osc_extent *ext, enum osc_extent_state state) { struct osc_object *obj = ext->oe_obj; - struct l_wait_info lwi = LWI_TIMEOUT_INTR(cfs_time_seconds(600), NULL, + struct l_wait_info lwi = LWI_TIMEOUT_INTR(600 * HZ, NULL, LWI_ON_SIGNAL_NOOP, NULL); int rc = 0; @@ -1571,7 +1571,7 @@ static int osc_enter_cache(const struct lu_env *env, struct client_obd *cli, struct l_wait_info lwi; int rc = -EDQUOT; - lwi = LWI_TIMEOUT_INTR(cfs_time_seconds(AT_OFF ? obd_timeout : at_max), + lwi = LWI_TIMEOUT_INTR((AT_OFF ? obd_timeout : at_max) * HZ, NULL, LWI_ON_SIGNAL_NOOP, NULL); OSC_DUMP_GRANT(D_CACHE, cli, "need:%d\n", bytes); diff --git a/drivers/staging/lustre/lustre/osc/osc_object.c b/drivers/staging/lustre/lustre/osc/osc_object.c index 6c424f0290bb..6baa8e2e00c9 100644 --- a/drivers/staging/lustre/lustre/osc/osc_object.c +++ b/drivers/staging/lustre/lustre/osc/osc_object.c @@ -328,7 +328,7 @@ int osc_object_is_contended(struct osc_object *obj) * ll_file_is_contended. */ retry_time = cfs_time_add(obj->oo_contention_time, - cfs_time_seconds(osc_contention_time)); + osc_contention_time * HZ); if (cfs_time_after(cur_time, retry_time)) { osc_object_clear_contended(obj); return 0; diff --git a/drivers/staging/lustre/lustre/ptlrpc/client.c b/drivers/staging/lustre/lustre/ptlrpc/client.c index bac4b2304bad..0ab13f8e5993 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/client.c +++ b/drivers/staging/lustre/lustre/ptlrpc/client.c @@ -766,7 +766,7 @@ int ptlrpc_request_bufs_pack(struct ptlrpc_request *request, * fail_loc */ set_current_state(TASK_UNINTERRUPTIBLE); - schedule_timeout(cfs_time_seconds(2)); + schedule_timeout(2 * HZ); set_current_state(TASK_RUNNING); } } @@ -2284,7 +2284,7 @@ int ptlrpc_set_wait(struct ptlrpc_request_set *set) * We still want to block for a limited time, * so we allow interrupts during the timeout. */ - lwi = LWI_TIMEOUT_INTR_ALL(cfs_time_seconds(1), + lwi = LWI_TIMEOUT_INTR_ALL(HZ, ptlrpc_expired_set, ptlrpc_interrupted_set, set); else @@ -2293,7 +2293,7 @@ int ptlrpc_set_wait(struct ptlrpc_request_set *set) * interrupts are allowed. Wait until all * complete, or an in-flight req times out. */ - lwi = LWI_TIMEOUT(cfs_time_seconds(timeout ? timeout : 1), + lwi = LWI_TIMEOUT((timeout ? timeout : 1) * HZ, ptlrpc_expired_set, set); rc = l_wait_event(set->set_waitq, ptlrpc_check_set(NULL, set), &lwi); @@ -2538,8 +2538,8 @@ static int ptlrpc_unregister_reply(struct ptlrpc_request *request, int async) * Network access will complete in finite time but the HUGE * timeout lets us CWARN for visibility of sluggish NALs */ - lwi = LWI_TIMEOUT_INTERVAL(cfs_time_seconds(LONG_UNLINK), - cfs_time_seconds(1), NULL, NULL); + lwi = LWI_TIMEOUT_INTERVAL(LONG_UNLINK * HZ, + HZ, NULL, NULL); rc = l_wait_event(*wq, !ptlrpc_client_recv_or_unlink(request), &lwi); if (rc == 0) { diff --git a/drivers/staging/lustre/lustre/ptlrpc/events.c b/drivers/staging/lustre/lustre/ptlrpc/events.c index 811b7ab3a582..71f7588570ef 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/events.c +++ b/drivers/staging/lustre/lustre/ptlrpc/events.c @@ -517,7 +517,7 @@ static void ptlrpc_ni_fini(void) /* Wait for a bit */ init_waitqueue_head(&waitq); - lwi = LWI_TIMEOUT(cfs_time_seconds(2), NULL, NULL); + lwi = LWI_TIMEOUT(2 * HZ, NULL, NULL); l_wait_event(waitq, 0, &lwi); break; } diff --git a/drivers/staging/lustre/lustre/ptlrpc/import.c b/drivers/staging/lustre/lustre/ptlrpc/import.c index 5b0f65536c29..0eba5f18bd3b 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/import.c +++ b/drivers/staging/lustre/lustre/ptlrpc/import.c @@ -307,9 +307,9 @@ void ptlrpc_invalidate_import(struct obd_import *imp) * have been locally cancelled by ptlrpc_abort_inflight. */ lwi = LWI_TIMEOUT_INTERVAL( - cfs_timeout_cap(cfs_time_seconds(timeout)), - (timeout > 1) ? cfs_time_seconds(1) : - cfs_time_seconds(1) / 2, + cfs_timeout_cap(timeout * HZ), + (timeout > 1) ? HZ : + HZ / 2, NULL, NULL); rc = l_wait_event(imp->imp_recovery_waitq, (atomic_read(&imp->imp_inflight) == 0), @@ -431,7 +431,7 @@ void ptlrpc_fail_import(struct obd_import *imp, __u32 conn_cnt) int ptlrpc_reconnect_import(struct obd_import *imp) { struct l_wait_info lwi; - int secs = cfs_time_seconds(obd_timeout); + int secs = obd_timeout * HZ; int rc; ptlrpc_pinger_force(imp); @@ -1508,14 +1508,13 @@ int ptlrpc_disconnect_import(struct obd_import *imp, int noclose) if (AT_OFF) { if (imp->imp_server_timeout) - timeout = cfs_time_seconds(obd_timeout / 2); + timeout = obd_timeout * HZ / 2; else - timeout = cfs_time_seconds(obd_timeout); + timeout = obd_timeout * HZ; } else { int idx = import_at_get_index(imp, imp->imp_client->cli_request_portal); - timeout = cfs_time_seconds( - at_get(&imp->imp_at.iat_service_estimate[idx])); + timeout = at_get(&imp->imp_at.iat_service_estimate[idx]) * HZ; } lwi = LWI_TIMEOUT_INTR(cfs_timeout_cap(timeout), diff --git a/drivers/staging/lustre/lustre/ptlrpc/niobuf.c b/drivers/staging/lustre/lustre/ptlrpc/niobuf.c index 047d712e850c..0c2ded721c49 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/niobuf.c +++ b/drivers/staging/lustre/lustre/ptlrpc/niobuf.c @@ -270,8 +270,8 @@ int ptlrpc_unregister_bulk(struct ptlrpc_request *req, int async) /* Network access will complete in finite time but the HUGE * timeout lets us CWARN for visibility of sluggish LNDs */ - lwi = LWI_TIMEOUT_INTERVAL(cfs_time_seconds(LONG_UNLINK), - cfs_time_seconds(1), NULL, NULL); + lwi = LWI_TIMEOUT_INTERVAL(LONG_UNLINK * HZ, + HZ, NULL, NULL); rc = l_wait_event(*wq, !ptlrpc_client_bulk_active(req), &lwi); if (rc == 0) { ptlrpc_rqphase_move(req, req->rq_next_phase); diff --git a/drivers/staging/lustre/lustre/ptlrpc/pack_generic.c b/drivers/staging/lustre/lustre/ptlrpc/pack_generic.c index a64e125df95f..c060d6f5015a 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/pack_generic.c +++ b/drivers/staging/lustre/lustre/ptlrpc/pack_generic.c @@ -267,7 +267,7 @@ lustre_get_emerg_rs(struct ptlrpc_service_part *svcpt) /* If we cannot get anything for some long time, we better * bail out instead of waiting infinitely */ - lwi = LWI_TIMEOUT(cfs_time_seconds(10), NULL, NULL); + lwi = LWI_TIMEOUT(10 * HZ, NULL, NULL); rc = l_wait_event(svcpt->scp_rep_waitq, !list_empty(&svcpt->scp_rep_idle), &lwi); if (rc != 0) diff --git a/drivers/staging/lustre/lustre/ptlrpc/pinger.c b/drivers/staging/lustre/lustre/ptlrpc/pinger.c index af707cb2b62b..010a1cdf05fa 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/pinger.c +++ b/drivers/staging/lustre/lustre/ptlrpc/pinger.c @@ -141,7 +141,7 @@ static long pinger_check_timeout(unsigned long time) } mutex_unlock(&pinger_mutex); - return cfs_time_sub(cfs_time_add(time, cfs_time_seconds(timeout)), + return cfs_time_sub(cfs_time_add(time, timeout * HZ), cfs_time_current()); } @@ -247,7 +247,7 @@ static int ptlrpc_pinger_main(void *arg) if (imp->imp_pingable && imp->imp_next_ping && cfs_time_after(imp->imp_next_ping, cfs_time_add(this_ping, - cfs_time_seconds(PING_INTERVAL)))) + PING_INTERVAL * HZ))) ptlrpc_update_next_ping(imp, 0); } mutex_unlock(&pinger_mutex); @@ -264,10 +264,10 @@ static int ptlrpc_pinger_main(void *arg) CDEBUG(D_INFO, "next wakeup in " CFS_DURATION_T " (%ld)\n", time_to_next_wake, cfs_time_add(this_ping, - cfs_time_seconds(PING_INTERVAL))); + PING_INTERVAL * HZ)); if (time_to_next_wake > 0) { lwi = LWI_TIMEOUT(max_t(long, time_to_next_wake, - cfs_time_seconds(1)), + HZ), NULL, NULL); l_wait_event(thread->t_ctl_waitq, thread_is_stopping(thread) || diff --git a/drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c b/drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c index 8b865294d933..dad2f9290f70 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c +++ b/drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c @@ -230,7 +230,7 @@ void ptlrpcd_add_req(struct ptlrpc_request *req) spin_lock(&req->rq_lock); if (req->rq_invalid_rqset) { - struct l_wait_info lwi = LWI_TIMEOUT(cfs_time_seconds(5), + struct l_wait_info lwi = LWI_TIMEOUT(5 * HZ, back_to_sleep, NULL); req->rq_invalid_rqset = 0; @@ -438,7 +438,7 @@ static int ptlrpcd(void *arg) int timeout; timeout = ptlrpc_set_next_timeout(set); - lwi = LWI_TIMEOUT(cfs_time_seconds(timeout ? timeout : 1), + lwi = LWI_TIMEOUT((timeout ? timeout : 1) * HZ, ptlrpc_expired_set, set); lu_context_enter(&env.le_ctx); diff --git a/drivers/staging/lustre/lustre/ptlrpc/recover.c b/drivers/staging/lustre/lustre/ptlrpc/recover.c index e4d3f23e9f3a..5bbd23eebfa6 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/recover.c +++ b/drivers/staging/lustre/lustre/ptlrpc/recover.c @@ -347,7 +347,7 @@ int ptlrpc_recover_import(struct obd_import *imp, char *new_uuid, int async) if (!async) { struct l_wait_info lwi; - int secs = cfs_time_seconds(obd_timeout); + int secs = obd_timeout * HZ; CDEBUG(D_HA, "%s: recovery started, waiting %u seconds\n", obd2cli_tgt(imp->imp_obd), secs); diff --git a/drivers/staging/lustre/lustre/ptlrpc/service.c b/drivers/staging/lustre/lustre/ptlrpc/service.c index 1f22926c1355..6d4229ebc9d9 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/service.c +++ b/drivers/staging/lustre/lustre/ptlrpc/service.c @@ -2149,7 +2149,7 @@ static int ptlrpc_main(void *arg) * Wait for a timeout (unless something else * happens) before I try again */ - svcpt->scp_rqbd_timeout = cfs_time_seconds(1) / 10; + svcpt->scp_rqbd_timeout = HZ / 10; CDEBUG(D_RPCTRACE, "Posted buffers: %d\n", svcpt->scp_nrqbds_posted); } @@ -2588,7 +2588,7 @@ static void ptlrpc_wait_replies(struct ptlrpc_service_part *svcpt) { while (1) { int rc; - struct l_wait_info lwi = LWI_TIMEOUT(cfs_time_seconds(10), + struct l_wait_info lwi = LWI_TIMEOUT(10 * HZ, NULL, NULL); rc = l_wait_event(svcpt->scp_waitq, @@ -2660,8 +2660,8 @@ ptlrpc_service_unlink_rqbd(struct ptlrpc_service *svc) * of sluggish LNDs */ lwi = LWI_TIMEOUT_INTERVAL( - cfs_time_seconds(LONG_UNLINK), - cfs_time_seconds(1), NULL, NULL); + LONG_UNLINK * HZ, + HZ, NULL, NULL); rc = l_wait_event(svcpt->scp_waitq, svcpt->scp_nrqbds_posted == 0, &lwi); if (rc == -ETIMEDOUT) { From neilb at suse.com Mon Feb 12 21:22:36 2018 From: neilb at suse.com (NeilBrown) Date: Tue, 13 Feb 2018 08:22:36 +1100 Subject: [lustre-devel] [PATCH 05/19] staging: lustre: use wait_event_idle_timeout() where appropriate. In-Reply-To: <151847037709.22826.16175867257667686132.stgit@noble> References: <151847037709.22826.16175867257667686132.stgit@noble> Message-ID: <151847055663.22826.6145062511637064173.stgit@noble> When the lwi arg has a timeout, but no timeout callback function, l_wait_event() acts much the same as wait_event_idle_timeout() - the wait is not interruptible and simply waits for the event or the timeouts. The most noticable difference is that the return value is -ETIMEDOUT or 0, rather than 0 or non-zero. Another difference is that if the timeout is zero, l_wait_event() will not time out at all. In the one case where that is possible we need to conditionally use wait_event_idle(). So replace all such calls with wait_event_idle_timeout(), being careful of the return value. In one case, there is no event expected, only the timeout is needed. So use schedule_timeout_uninterruptible(). Note that the presence or absence of LWI_ON_SIGNAL_NOOP has no effect in these cases. It only has effect if the timeout callback is non-NULL, or the timeout is zero, or LWI_TIMEOUT_INTR_ALL() is used. Reviewed-by: James Simmons Signed-off-by: NeilBrown --- drivers/staging/lustre/lustre/ldlm/ldlm_lock.c | 10 ++------ drivers/staging/lustre/lustre/ldlm/ldlm_pool.c | 12 +++------- drivers/staging/lustre/lustre/llite/statahead.c | 14 ++++------- drivers/staging/lustre/lustre/mdc/mdc_request.c | 5 +--- drivers/staging/lustre/lustre/mgc/mgc_request.c | 15 +++++------- drivers/staging/lustre/lustre/obdclass/cl_io.c | 17 ++++++++------ drivers/staging/lustre/lustre/osc/osc_cache.c | 25 ++++++++++---------- drivers/staging/lustre/lustre/ptlrpc/events.c | 7 +----- drivers/staging/lustre/lustre/ptlrpc/import.c | 12 ++++------ .../staging/lustre/lustre/ptlrpc/pack_generic.c | 9 +++---- drivers/staging/lustre/lustre/ptlrpc/pinger.c | 12 +++------- drivers/staging/lustre/lustre/ptlrpc/recover.c | 12 ++++------ drivers/staging/lustre/lustre/ptlrpc/sec_gc.c | 10 ++------ drivers/staging/lustre/lustre/ptlrpc/service.c | 11 ++++----- 14 files changed, 68 insertions(+), 103 deletions(-) diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c b/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c index 773abe78708a..95bea351d21d 100644 --- a/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c @@ -1349,7 +1349,6 @@ enum ldlm_mode ldlm_lock_match(struct ldlm_namespace *ns, __u64 flags, if ((flags & LDLM_FL_LVB_READY) && !ldlm_is_lvb_ready(lock)) { __u64 wait_flags = LDLM_FL_LVB_READY | LDLM_FL_DESTROYED | LDLM_FL_FAIL_NOTIFIED; - struct l_wait_info lwi; if (lock->l_completion_ast) { int err = lock->l_completion_ast(lock, @@ -1366,13 +1365,10 @@ enum ldlm_mode ldlm_lock_match(struct ldlm_namespace *ns, __u64 flags, } } - lwi = LWI_TIMEOUT_INTR(obd_timeout * HZ, - NULL, LWI_ON_SIGNAL_NOOP, NULL); - /* XXX FIXME see comment on CAN_MATCH in lustre_dlm.h */ - l_wait_event(lock->l_waitq, - lock->l_flags & wait_flags, - &lwi); + wait_event_idle_timeout(lock->l_waitq, + lock->l_flags & wait_flags, + obd_timeout * HZ); if (!ldlm_is_lvb_ready(lock)) { if (flags & LDLM_FL_TEST_LOCK) LDLM_LOCK_RELEASE(lock); diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c b/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c index 622245a5f049..a0e486b57e08 100644 --- a/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c @@ -997,8 +997,6 @@ static int ldlm_pools_thread_main(void *arg) "ldlm_poold", current_pid()); while (1) { - struct l_wait_info lwi; - /* * Recal all pools on this tick. */ @@ -1008,12 +1006,10 @@ static int ldlm_pools_thread_main(void *arg) * Wait until the next check time, or until we're * stopped. */ - lwi = LWI_TIMEOUT(c_time * HZ, - NULL, NULL); - l_wait_event(thread->t_ctl_waitq, - thread_is_stopping(thread) || - thread_is_event(thread), - &lwi); + wait_event_idle_timeout(thread->t_ctl_waitq, + thread_is_stopping(thread) || + thread_is_event(thread), + c_time * HZ); if (thread_test_and_clear_flags(thread, SVC_STOPPING)) break; diff --git a/drivers/staging/lustre/lustre/llite/statahead.c b/drivers/staging/lustre/lustre/llite/statahead.c index 96360f104b92..6052bfd7ff05 100644 --- a/drivers/staging/lustre/lustre/llite/statahead.c +++ b/drivers/staging/lustre/lustre/llite/statahead.c @@ -1151,10 +1151,9 @@ static int ll_statahead_thread(void *arg) */ while (sai->sai_sent != sai->sai_replied) { /* in case we're not woken up, timeout wait */ - struct l_wait_info lwi = LWI_TIMEOUT(msecs_to_jiffies(MSEC_PER_SEC >> 3), - NULL, NULL); - l_wait_event(sa_thread->t_ctl_waitq, - sai->sai_sent == sai->sai_replied, &lwi); + wait_event_idle_timeout(sa_thread->t_ctl_waitq, + sai->sai_sent == sai->sai_replied, + HZ>>3); } /* release resources held by statahead RPCs */ @@ -1374,7 +1373,6 @@ static int revalidate_statahead_dentry(struct inode *dir, { struct ll_inode_info *lli = ll_i2info(dir); struct sa_entry *entry = NULL; - struct l_wait_info lwi = { 0 }; struct ll_dentry_data *ldd; int rc = 0; @@ -1424,10 +1422,8 @@ static int revalidate_statahead_dentry(struct inode *dir, spin_lock(&lli->lli_sa_lock); sai->sai_index_wait = entry->se_index; spin_unlock(&lli->lli_sa_lock); - lwi = LWI_TIMEOUT_INTR(30 * HZ, NULL, - LWI_ON_SIGNAL_NOOP, NULL); - rc = l_wait_event(sai->sai_waitq, sa_ready(entry), &lwi); - if (rc < 0) { + if (0 == wait_event_idle_timeout(sai->sai_waitq, + sa_ready(entry), 30 * HZ)) { /* * entry may not be ready, so it may be used by inflight * statahead RPC, don't free it. diff --git a/drivers/staging/lustre/lustre/mdc/mdc_request.c b/drivers/staging/lustre/lustre/mdc/mdc_request.c index b12518ba5ae9..ab48746ce433 100644 --- a/drivers/staging/lustre/lustre/mdc/mdc_request.c +++ b/drivers/staging/lustre/lustre/mdc/mdc_request.c @@ -838,7 +838,6 @@ static int mdc_getpage(struct obd_export *exp, const struct lu_fid *fid, struct ptlrpc_bulk_desc *desc; struct ptlrpc_request *req; wait_queue_head_t waitq; - struct l_wait_info lwi; int resends = 0; int rc; int i; @@ -888,9 +887,7 @@ static int mdc_getpage(struct obd_export *exp, const struct lu_fid *fid, exp->exp_obd->obd_name, -EIO); return -EIO; } - lwi = LWI_TIMEOUT_INTR(resends * HZ, NULL, NULL, - NULL); - l_wait_event(waitq, 0, &lwi); + wait_event_idle_timeout(waitq, 0, resends * HZ); goto restart_bulk; } diff --git a/drivers/staging/lustre/lustre/mgc/mgc_request.c b/drivers/staging/lustre/lustre/mgc/mgc_request.c index a01d13bde102..c61cd23a96df 100644 --- a/drivers/staging/lustre/lustre/mgc/mgc_request.c +++ b/drivers/staging/lustre/lustre/mgc/mgc_request.c @@ -535,7 +535,6 @@ static int mgc_requeue_thread(void *data) spin_lock(&config_list_lock); rq_state |= RQ_RUNNING; while (!(rq_state & RQ_STOP)) { - struct l_wait_info lwi; struct config_llog_data *cld, *cld_prev; int rand = prandom_u32_max(MGC_TIMEOUT_RAND_CENTISEC); int to; @@ -556,9 +555,9 @@ static int mgc_requeue_thread(void *data) to = msecs_to_jiffies(MGC_TIMEOUT_MIN_SECONDS * MSEC_PER_SEC); /* rand is centi-seconds */ to += msecs_to_jiffies(rand * MSEC_PER_SEC / 100); - lwi = LWI_TIMEOUT(to, NULL, NULL); - l_wait_event(rq_waitq, rq_state & (RQ_STOP | RQ_PRECLEANUP), - &lwi); + wait_event_idle_timeout(rq_waitq, + rq_state & (RQ_STOP | RQ_PRECLEANUP), + to); /* * iterate & processing through the list. for each cld, process @@ -1628,9 +1627,7 @@ int mgc_process_log(struct obd_device *mgc, struct config_llog_data *cld) if (rcl == -ESHUTDOWN && atomic_read(&mgc->u.cli.cl_mgc_refcount) > 0 && !retry) { - int secs = obd_timeout * HZ; struct obd_import *imp; - struct l_wait_info lwi; mutex_unlock(&cld->cld_lock); imp = class_exp2cliimp(mgc->u.cli.cl_mgc_mgsexp); @@ -1645,9 +1642,9 @@ int mgc_process_log(struct obd_device *mgc, struct config_llog_data *cld) */ ptlrpc_pinger_force(imp); - lwi = LWI_TIMEOUT(secs, NULL, NULL); - l_wait_event(imp->imp_recovery_waitq, - !mgc_import_in_recovery(imp), &lwi); + wait_event_idle_timeout(imp->imp_recovery_waitq, + !mgc_import_in_recovery(imp), + obd_timeout * HZ); if (imp->imp_state == LUSTRE_IMP_FULL) { retry = true; diff --git a/drivers/staging/lustre/lustre/obdclass/cl_io.c b/drivers/staging/lustre/lustre/obdclass/cl_io.c index ce5e7bdda692..ab84e011b560 100644 --- a/drivers/staging/lustre/lustre/obdclass/cl_io.c +++ b/drivers/staging/lustre/lustre/obdclass/cl_io.c @@ -1097,16 +1097,19 @@ EXPORT_SYMBOL(cl_sync_io_init); int cl_sync_io_wait(const struct lu_env *env, struct cl_sync_io *anchor, long timeout) { - struct l_wait_info lwi = LWI_TIMEOUT_INTR(timeout * HZ, - NULL, NULL, NULL); - int rc; + int rc = 1; LASSERT(timeout >= 0); - rc = l_wait_event(anchor->csi_waitq, - atomic_read(&anchor->csi_sync_nr) == 0, - &lwi); - if (rc < 0) { + if (timeout == 0) + wait_event_idle(anchor->csi_waitq, + atomic_read(&anchor->csi_sync_nr) == 0); + else + rc = wait_event_idle_timeout(anchor->csi_waitq, + atomic_read(&anchor->csi_sync_nr) == 0, + timeout * HZ); + if (rc == 0) { + rc = -ETIMEDOUT; CERROR("IO failed: %d, still wait for %d remaining entries\n", rc, atomic_read(&anchor->csi_sync_nr)); diff --git a/drivers/staging/lustre/lustre/osc/osc_cache.c b/drivers/staging/lustre/lustre/osc/osc_cache.c index 0797e671f667..dacfab12c501 100644 --- a/drivers/staging/lustre/lustre/osc/osc_cache.c +++ b/drivers/staging/lustre/lustre/osc/osc_cache.c @@ -934,8 +934,6 @@ static int osc_extent_wait(const struct lu_env *env, struct osc_extent *ext, enum osc_extent_state state) { struct osc_object *obj = ext->oe_obj; - struct l_wait_info lwi = LWI_TIMEOUT_INTR(600 * HZ, NULL, - LWI_ON_SIGNAL_NOOP, NULL); int rc = 0; osc_object_lock(obj); @@ -958,17 +956,19 @@ static int osc_extent_wait(const struct lu_env *env, struct osc_extent *ext, osc_extent_release(env, ext); /* wait for the extent until its state becomes @state */ - rc = l_wait_event(ext->oe_waitq, extent_wait_cb(ext, state), &lwi); - if (rc == -ETIMEDOUT) { + rc = wait_event_idle_timeout(ext->oe_waitq, + extent_wait_cb(ext, state), 600 * HZ); + if (rc == 0) { OSC_EXTENT_DUMP(D_ERROR, ext, "%s: wait ext to %u timedout, recovery in progress?\n", cli_name(osc_cli(obj)), state); wait_event_idle(ext->oe_waitq, extent_wait_cb(ext, state)); - rc = 0; } - if (rc == 0 && ext->oe_rc < 0) + if (ext->oe_rc < 0) rc = ext->oe_rc; + else + rc = 0; return rc; } @@ -1568,12 +1568,9 @@ static int osc_enter_cache(const struct lu_env *env, struct client_obd *cli, struct osc_object *osc = oap->oap_obj; struct lov_oinfo *loi = osc->oo_oinfo; struct osc_cache_waiter ocw; - struct l_wait_info lwi; + unsigned long timeout = (AT_OFF ? obd_timeout : at_max) * HZ; int rc = -EDQUOT; - lwi = LWI_TIMEOUT_INTR((AT_OFF ? obd_timeout : at_max) * HZ, - NULL, LWI_ON_SIGNAL_NOOP, NULL); - OSC_DUMP_GRANT(D_CACHE, cli, "need:%d\n", bytes); spin_lock(&cli->cl_loi_list_lock); @@ -1616,13 +1613,15 @@ static int osc_enter_cache(const struct lu_env *env, struct client_obd *cli, CDEBUG(D_CACHE, "%s: sleeping for cache space @ %p for %p\n", cli_name(cli), &ocw, oap); - rc = l_wait_event(ocw.ocw_waitq, ocw_granted(cli, &ocw), &lwi); + rc = wait_event_idle_timeout(ocw.ocw_waitq, + ocw_granted(cli, &ocw), timeout); spin_lock(&cli->cl_loi_list_lock); - if (rc < 0) { - /* l_wait_event is interrupted by signal, or timed out */ + if (rc == 0) { + /* wait_event is interrupted by signal, or timed out */ list_del_init(&ocw.ocw_entry); + rc = -ETIMEDOUT; break; } LASSERT(list_empty(&ocw.ocw_entry)); diff --git a/drivers/staging/lustre/lustre/ptlrpc/events.c b/drivers/staging/lustre/lustre/ptlrpc/events.c index 71f7588570ef..130bacc2c891 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/events.c +++ b/drivers/staging/lustre/lustre/ptlrpc/events.c @@ -490,8 +490,6 @@ int ptlrpc_uuid_to_peer(struct obd_uuid *uuid, static void ptlrpc_ni_fini(void) { - wait_queue_head_t waitq; - struct l_wait_info lwi; int rc; int retries; @@ -515,10 +513,7 @@ static void ptlrpc_ni_fini(void) if (retries != 0) CWARN("Event queue still busy\n"); - /* Wait for a bit */ - init_waitqueue_head(&waitq); - lwi = LWI_TIMEOUT(2 * HZ, NULL, NULL); - l_wait_event(waitq, 0, &lwi); + schedule_timeout_uninterruptible(2 * HZ); break; } } diff --git a/drivers/staging/lustre/lustre/ptlrpc/import.c b/drivers/staging/lustre/lustre/ptlrpc/import.c index 0eba5f18bd3b..ed210550f61f 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/import.c +++ b/drivers/staging/lustre/lustre/ptlrpc/import.c @@ -430,21 +430,19 @@ void ptlrpc_fail_import(struct obd_import *imp, __u32 conn_cnt) int ptlrpc_reconnect_import(struct obd_import *imp) { - struct l_wait_info lwi; - int secs = obd_timeout * HZ; int rc; ptlrpc_pinger_force(imp); CDEBUG(D_HA, "%s: recovery started, waiting %u seconds\n", - obd2cli_tgt(imp->imp_obd), secs); + obd2cli_tgt(imp->imp_obd), obd_timeout); - lwi = LWI_TIMEOUT(secs, NULL, NULL); - rc = l_wait_event(imp->imp_recovery_waitq, - !ptlrpc_import_in_recovery(imp), &lwi); + rc = wait_event_idle_timeout(imp->imp_recovery_waitq, + !ptlrpc_import_in_recovery(imp), + obd_timeout * HZ); CDEBUG(D_HA, "%s: recovery finished s:%s\n", obd2cli_tgt(imp->imp_obd), ptlrpc_import_state_name(imp->imp_state)); - return rc; + return rc == 0 ? -ETIMEDOUT : 0; } EXPORT_SYMBOL(ptlrpc_reconnect_import); diff --git a/drivers/staging/lustre/lustre/ptlrpc/pack_generic.c b/drivers/staging/lustre/lustre/ptlrpc/pack_generic.c index c060d6f5015a..f73463ac401f 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/pack_generic.c +++ b/drivers/staging/lustre/lustre/ptlrpc/pack_generic.c @@ -260,17 +260,16 @@ lustre_get_emerg_rs(struct ptlrpc_service_part *svcpt) /* See if we have anything in a pool, and wait if nothing */ while (list_empty(&svcpt->scp_rep_idle)) { - struct l_wait_info lwi; int rc; spin_unlock(&svcpt->scp_rep_lock); /* If we cannot get anything for some long time, we better * bail out instead of waiting infinitely */ - lwi = LWI_TIMEOUT(10 * HZ, NULL, NULL); - rc = l_wait_event(svcpt->scp_rep_waitq, - !list_empty(&svcpt->scp_rep_idle), &lwi); - if (rc != 0) + rc = wait_event_idle_timeout(svcpt->scp_rep_waitq, + !list_empty(&svcpt->scp_rep_idle), + 10 * HZ); + if (rc == 0) goto out; spin_lock(&svcpt->scp_rep_lock); } diff --git a/drivers/staging/lustre/lustre/ptlrpc/pinger.c b/drivers/staging/lustre/lustre/ptlrpc/pinger.c index 010a1cdf05fa..639070f6e68e 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/pinger.c +++ b/drivers/staging/lustre/lustre/ptlrpc/pinger.c @@ -228,7 +228,6 @@ static int ptlrpc_pinger_main(void *arg) /* And now, loop forever, pinging as needed. */ while (1) { unsigned long this_ping = cfs_time_current(); - struct l_wait_info lwi; long time_to_next_wake; struct timeout_item *item; struct list_head *iter; @@ -266,13 +265,10 @@ static int ptlrpc_pinger_main(void *arg) cfs_time_add(this_ping, PING_INTERVAL * HZ)); if (time_to_next_wake > 0) { - lwi = LWI_TIMEOUT(max_t(long, time_to_next_wake, - HZ), - NULL, NULL); - l_wait_event(thread->t_ctl_waitq, - thread_is_stopping(thread) || - thread_is_event(thread), - &lwi); + wait_event_idle_timeout(thread->t_ctl_waitq, + thread_is_stopping(thread) || + thread_is_event(thread), + max_t(long, time_to_next_wake, HZ)); if (thread_test_and_clear_flags(thread, SVC_STOPPING)) break; /* woken after adding import to reset timer */ diff --git a/drivers/staging/lustre/lustre/ptlrpc/recover.c b/drivers/staging/lustre/lustre/ptlrpc/recover.c index 5bbd23eebfa6..7b5f2429d144 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/recover.c +++ b/drivers/staging/lustre/lustre/ptlrpc/recover.c @@ -346,17 +346,15 @@ int ptlrpc_recover_import(struct obd_import *imp, char *new_uuid, int async) goto out; if (!async) { - struct l_wait_info lwi; - int secs = obd_timeout * HZ; - CDEBUG(D_HA, "%s: recovery started, waiting %u seconds\n", - obd2cli_tgt(imp->imp_obd), secs); + obd2cli_tgt(imp->imp_obd), obd_timeout); - lwi = LWI_TIMEOUT(secs, NULL, NULL); - rc = l_wait_event(imp->imp_recovery_waitq, - !ptlrpc_import_in_recovery(imp), &lwi); + rc = wait_event_idle_timeout(imp->imp_recovery_waitq, + !ptlrpc_import_in_recovery(imp), + obd_timeout * HZ); CDEBUG(D_HA, "%s: recovery finished\n", obd2cli_tgt(imp->imp_obd)); + rc = rc? 0 : -ETIMEDOUT; } out: diff --git a/drivers/staging/lustre/lustre/ptlrpc/sec_gc.c b/drivers/staging/lustre/lustre/ptlrpc/sec_gc.c index b61e1aa25e8c..48f1a72afd77 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/sec_gc.c +++ b/drivers/staging/lustre/lustre/ptlrpc/sec_gc.c @@ -142,7 +142,6 @@ static void sec_do_gc(struct ptlrpc_sec *sec) static int sec_gc_main(void *arg) { struct ptlrpc_thread *thread = arg; - struct l_wait_info lwi; unshare_fs_struct(); @@ -179,12 +178,9 @@ static int sec_gc_main(void *arg) /* check ctx list again before sleep */ sec_process_ctx_list(); - - lwi = LWI_TIMEOUT(msecs_to_jiffies(SEC_GC_INTERVAL * MSEC_PER_SEC), - NULL, NULL); - l_wait_event(thread->t_ctl_waitq, - thread_is_stopping(thread), - &lwi); + wait_event_idle_timeout(thread->t_ctl_waitq, + thread_is_stopping(thread), + SEC_GC_INTERVAL * HZ); if (thread_test_and_clear_flags(thread, SVC_STOPPING)) break; diff --git a/drivers/staging/lustre/lustre/ptlrpc/service.c b/drivers/staging/lustre/lustre/ptlrpc/service.c index 6d4229ebc9d9..5c41297d23d2 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/service.c +++ b/drivers/staging/lustre/lustre/ptlrpc/service.c @@ -2588,13 +2588,12 @@ static void ptlrpc_wait_replies(struct ptlrpc_service_part *svcpt) { while (1) { int rc; - struct l_wait_info lwi = LWI_TIMEOUT(10 * HZ, - NULL, NULL); - rc = l_wait_event(svcpt->scp_waitq, - atomic_read(&svcpt->scp_nreps_difficult) == 0, - &lwi); - if (rc == 0) + rc = wait_event_idle_timeout( + svcpt->scp_waitq, + atomic_read(&svcpt->scp_nreps_difficult) == 0, + 10 * HZ); + if (rc > 0) break; CWARN("Unexpectedly long timeout %s %p\n", svcpt->scp_service->srv_name, svcpt->scp_service); From neilb at suse.com Mon Feb 12 21:22:36 2018 From: neilb at suse.com (NeilBrown) Date: Tue, 13 Feb 2018 08:22:36 +1100 Subject: [lustre-devel] [PATCH 06/19] staging: lustre: introduce and use l_wait_event_abortable() In-Reply-To: <151847037709.22826.16175867257667686132.stgit@noble> References: <151847037709.22826.16175867257667686132.stgit@noble> Message-ID: <151847055667.22826.8544979478231411275.stgit@noble> lustre sometimes wants to wait for an event, but abort if one of a specific list of signals arrives. This is a little bit like wait_event_killable(), except that the signals are identified a different way. So introduce l_wait_event_abortable() which provides this functionality. Having separate functions for separate needs is more in line with the pattern set by include/linux/wait.h, than having a single function which tries to include all possible needs. Also introduce l_wait_event_abortable_exclusive(). Note that l_wait_event() return -EINTR on a signal, while Linux wait_event functions return -ERESTARTSYS. l_wait_event_{abortable_,}exclusive follow the Linux pattern. Reviewed-by: James Simmons Signed-off-by: NeilBrown --- drivers/staging/lustre/lustre/include/lustre_lib.h | 24 ++++++++++++++++++++ drivers/staging/lustre/lustre/ldlm/ldlm_resource.c | 12 +++++----- drivers/staging/lustre/lustre/llite/llite_lib.c | 12 +++------- drivers/staging/lustre/lustre/obdclass/genops.c | 9 +++----- drivers/staging/lustre/lustre/obdclass/llog_obd.c | 5 ++-- drivers/staging/lustre/lustre/osc/osc_page.c | 6 ++--- drivers/staging/lustre/lustre/osc/osc_request.c | 6 ++--- 7 files changed, 43 insertions(+), 31 deletions(-) diff --git a/drivers/staging/lustre/lustre/include/lustre_lib.h b/drivers/staging/lustre/lustre/include/lustre_lib.h index 7d950c53e962..b2a64d0e682c 100644 --- a/drivers/staging/lustre/lustre/include/lustre_lib.h +++ b/drivers/staging/lustre/lustre/include/lustre_lib.h @@ -336,4 +336,28 @@ do { \ /** @} lib */ + +/* l_wait_event_abortable() is a bit like wait_event_killable() + * except there is a fixed set of signals which will abort: + * LUSTRE_FATAL_SIGS + */ +#define l_wait_event_abortable(wq, condition) \ +({ \ + sigset_t __blocked; \ + int __ret = 0; \ + __blocked = cfs_block_sigsinv(LUSTRE_FATAL_SIGS); \ + __ret = wait_event_interruptible(wq, condition); \ + cfs_restore_sigs(__blocked); \ + __ret; \ +}) + +#define l_wait_event_abortable_exclusive(wq, condition) \ +({ \ + sigset_t __blocked; \ + int __ret = 0; \ + __blocked = cfs_block_sigsinv(LUSTRE_FATAL_SIGS); \ + __ret = wait_event_interruptible_exclusive(wq, condition); \ + cfs_restore_sigs(__blocked); \ + __ret; \ +}) #endif /* _LUSTRE_LIB_H */ diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_resource.c b/drivers/staging/lustre/lustre/ldlm/ldlm_resource.c index 2e66825c8f4b..4c44603ab6f9 100644 --- a/drivers/staging/lustre/lustre/ldlm/ldlm_resource.c +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_resource.c @@ -879,7 +879,6 @@ static int __ldlm_namespace_free(struct ldlm_namespace *ns, int force) ldlm_namespace_cleanup(ns, force ? LDLM_FL_LOCAL_ONLY : 0); if (atomic_read(&ns->ns_bref) > 0) { - struct l_wait_info lwi = LWI_INTR(LWI_ON_SIGNAL_NOOP, NULL); int rc; CDEBUG(D_DLMTRACE, @@ -887,11 +886,12 @@ static int __ldlm_namespace_free(struct ldlm_namespace *ns, int force) ldlm_ns_name(ns), atomic_read(&ns->ns_bref)); force_wait: if (force) - lwi = LWI_TIMEOUT(msecs_to_jiffies(obd_timeout * - MSEC_PER_SEC) / 4, NULL, NULL); - - rc = l_wait_event(ns->ns_waitq, - atomic_read(&ns->ns_bref) == 0, &lwi); + rc = wait_event_idle_timeout(ns->ns_waitq, + atomic_read(&ns->ns_bref) == 0, + obd_timeout * HZ / 4) ? 0 : -ETIMEDOUT; + else + rc = l_wait_event_abortable(ns->ns_waitq, + atomic_read(&ns->ns_bref) == 0); /* Forced cleanups should be able to reclaim all references, * so it's safe to wait forever... we can't leak locks... diff --git a/drivers/staging/lustre/lustre/llite/llite_lib.c b/drivers/staging/lustre/lustre/llite/llite_lib.c index c820b201af71..ccb614bd7f53 100644 --- a/drivers/staging/lustre/lustre/llite/llite_lib.c +++ b/drivers/staging/lustre/lustre/llite/llite_lib.c @@ -986,16 +986,12 @@ void ll_put_super(struct super_block *sb) } /* Wait for unstable pages to be committed to stable storage */ - if (!force) { - struct l_wait_info lwi = LWI_INTR(LWI_ON_SIGNAL_NOOP, NULL); - - rc = l_wait_event(sbi->ll_cache->ccc_unstable_waitq, - !atomic_long_read(&sbi->ll_cache->ccc_unstable_nr), - &lwi); - } + if (!force) + rc = l_wait_event_abortable(sbi->ll_cache->ccc_unstable_waitq, + !atomic_long_read(&sbi->ll_cache->ccc_unstable_nr)); ccc_count = atomic_long_read(&sbi->ll_cache->ccc_unstable_nr); - if (!force && rc != -EINTR) + if (!force && rc != -ERESTARTSYS) LASSERTF(!ccc_count, "count: %li\n", ccc_count); /* We need to set force before the lov_disconnect in diff --git a/drivers/staging/lustre/lustre/obdclass/genops.c b/drivers/staging/lustre/lustre/obdclass/genops.c index 3ff25b8d3b48..8f776a4058a9 100644 --- a/drivers/staging/lustre/lustre/obdclass/genops.c +++ b/drivers/staging/lustre/lustre/obdclass/genops.c @@ -1332,7 +1332,6 @@ static bool obd_request_slot_avail(struct client_obd *cli, int obd_get_request_slot(struct client_obd *cli) { struct obd_request_slot_waiter orsw; - struct l_wait_info lwi; int rc; spin_lock(&cli->cl_loi_list_lock); @@ -1347,11 +1346,9 @@ int obd_get_request_slot(struct client_obd *cli) orsw.orsw_signaled = false; spin_unlock(&cli->cl_loi_list_lock); - lwi = LWI_INTR(LWI_ON_SIGNAL_NOOP, NULL); - rc = l_wait_event(orsw.orsw_waitq, - obd_request_slot_avail(cli, &orsw) || - orsw.orsw_signaled, - &lwi); + rc = l_wait_event_abortable(orsw.orsw_waitq, + obd_request_slot_avail(cli, &orsw) || + orsw.orsw_signaled); /* * Here, we must take the lock to avoid the on-stack 'orsw' to be diff --git a/drivers/staging/lustre/lustre/obdclass/llog_obd.c b/drivers/staging/lustre/lustre/obdclass/llog_obd.c index 28bbaa2136ac..26aea114a29b 100644 --- a/drivers/staging/lustre/lustre/obdclass/llog_obd.c +++ b/drivers/staging/lustre/lustre/obdclass/llog_obd.c @@ -104,7 +104,6 @@ EXPORT_SYMBOL(__llog_ctxt_put); int llog_cleanup(const struct lu_env *env, struct llog_ctxt *ctxt) { - struct l_wait_info lwi = LWI_INTR(LWI_ON_SIGNAL_NOOP, NULL); struct obd_llog_group *olg; int rc, idx; @@ -129,8 +128,8 @@ int llog_cleanup(const struct lu_env *env, struct llog_ctxt *ctxt) CERROR("Error %d while cleaning up ctxt %p\n", rc, ctxt); - l_wait_event(olg->olg_waitq, - llog_group_ctxt_null(olg, idx), &lwi); + l_wait_event_abortable(olg->olg_waitq, + llog_group_ctxt_null(olg, idx)); return rc; } diff --git a/drivers/staging/lustre/lustre/osc/osc_page.c b/drivers/staging/lustre/lustre/osc/osc_page.c index 20094b6309f9..6fdd521feb21 100644 --- a/drivers/staging/lustre/lustre/osc/osc_page.c +++ b/drivers/staging/lustre/lustre/osc/osc_page.c @@ -759,7 +759,6 @@ static long osc_lru_reclaim(struct client_obd *cli, unsigned long npages) static int osc_lru_alloc(const struct lu_env *env, struct client_obd *cli, struct osc_page *opg) { - struct l_wait_info lwi = LWI_INTR(LWI_ON_SIGNAL_NOOP, NULL); struct osc_io *oio = osc_env_io(env); int rc = 0; @@ -782,9 +781,8 @@ static int osc_lru_alloc(const struct lu_env *env, struct client_obd *cli, cond_resched(); - rc = l_wait_event(osc_lru_waitq, - atomic_long_read(cli->cl_lru_left) > 0, - &lwi); + rc = l_wait_event_abortable(osc_lru_waitq, + atomic_long_read(cli->cl_lru_left) > 0); if (rc < 0) break; diff --git a/drivers/staging/lustre/lustre/osc/osc_request.c b/drivers/staging/lustre/lustre/osc/osc_request.c index 45b1ebf33363..074b5ce6284c 100644 --- a/drivers/staging/lustre/lustre/osc/osc_request.c +++ b/drivers/staging/lustre/lustre/osc/osc_request.c @@ -552,14 +552,12 @@ static int osc_destroy(const struct lu_env *env, struct obd_export *exp, req->rq_interpret_reply = osc_destroy_interpret; if (!osc_can_send_destroy(cli)) { - struct l_wait_info lwi = LWI_INTR(LWI_ON_SIGNAL_NOOP, NULL); - /* * Wait until the number of on-going destroy RPCs drops * under max_rpc_in_flight */ - l_wait_event_exclusive(cli->cl_destroy_waitq, - osc_can_send_destroy(cli), &lwi); + l_wait_event_abortable_exclusive(cli->cl_destroy_waitq, + osc_can_send_destroy(cli)); } /* Do not wait for response */ From neilb at suse.com Mon Feb 12 21:22:36 2018 From: neilb at suse.com (NeilBrown) Date: Tue, 13 Feb 2018 08:22:36 +1100 Subject: [lustre-devel] [PATCH 08/19] staging: lustre: simplify waiting in ldlm_completion_ast() In-Reply-To: <151847037709.22826.16175867257667686132.stgit@noble> References: <151847037709.22826.16175867257667686132.stgit@noble> Message-ID: <151847055674.22826.12912356108048917428.stgit@noble> If a signal-callback (lwi_on_signal) is set without lwi_allow_intr, as is the case in ldlm_completion_ast(), the behavior depends on the timeout set. If a timeout is set, then signals are ignored. If the timeout is reached, the timeout handler is called. If the timeout handler return 0, which ldlm_expired_completion_wait() always does, the l_wait_event() switches to exactly the behavior if no timeout was set. If no timeout is set, then "fatal" signals are not ignored. If one arrives the callback is run, but as the callback is empty in this case, that is not relevant. This can be simplified to: if a timeout is wanted wait_event_idle_timeout() if that timed out, call the timeout handler l_wait_event_abortable() i.e. the code always waits indefinitely. Sometimes it performs a non-abortable wait first. Sometimes it doesn't. But it only aborts before the condition is true if it is signaled. This doesn't quite agree with the comments and debug messages. Reviewed-by: James Simmons Signed-off-by: NeilBrown --- drivers/staging/lustre/lustre/ldlm/ldlm_request.c | 55 +++++++-------------- 1 file changed, 18 insertions(+), 37 deletions(-) diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_request.c b/drivers/staging/lustre/lustre/ldlm/ldlm_request.c index a244fa717134..f1233d844bbd 100644 --- a/drivers/staging/lustre/lustre/ldlm/ldlm_request.c +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_request.c @@ -72,15 +72,6 @@ MODULE_PARM_DESC(ldlm_enqueue_min, "lock enqueue timeout minimum"); /* in client side, whether the cached locks will be canceled before replay */ unsigned int ldlm_cancel_unused_locks_before_replay = 1; -static void interrupted_completion_wait(void *data) -{ -} - -struct lock_wait_data { - struct ldlm_lock *lwd_lock; - __u32 lwd_conn_cnt; -}; - struct ldlm_async_args { struct lustre_handle lock_handle; }; @@ -112,10 +103,8 @@ static int ldlm_request_bufsize(int count, int type) return sizeof(struct ldlm_request) + avail; } -static int ldlm_expired_completion_wait(void *data) +static void ldlm_expired_completion_wait(struct ldlm_lock *lock, struct obd_import *imp2) { - struct lock_wait_data *lwd = data; - struct ldlm_lock *lock = lwd->lwd_lock; struct obd_import *imp; struct obd_device *obd; @@ -135,19 +124,17 @@ static int ldlm_expired_completion_wait(void *data) if (last_dump == 0) libcfs_debug_dumplog(); } - return 0; + return; } obd = lock->l_conn_export->exp_obd; imp = obd->u.cli.cl_import; - ptlrpc_fail_import(imp, lwd->lwd_conn_cnt); + ptlrpc_fail_import(imp, imp2 ? imp2->imp_conn_cnt : 0); LDLM_ERROR(lock, "lock timed out (enqueued at %lld, %llds ago), entering recovery for %s@%s", (s64)lock->l_last_activity, (s64)(ktime_get_real_seconds() - lock->l_last_activity), obd2cli_tgt(obd), imp->imp_connection->c_remote_uuid.uuid); - - return 0; } /** @@ -251,10 +238,8 @@ EXPORT_SYMBOL(ldlm_completion_ast_async); int ldlm_completion_ast(struct ldlm_lock *lock, __u64 flags, void *data) { /* XXX ALLOCATE - 160 bytes */ - struct lock_wait_data lwd; struct obd_device *obd; struct obd_import *imp = NULL; - struct l_wait_info lwi; __u32 timeout; int rc = 0; @@ -281,32 +266,28 @@ int ldlm_completion_ast(struct ldlm_lock *lock, __u64 flags, void *data) timeout = ldlm_cp_timeout(lock); - lwd.lwd_lock = lock; lock->l_last_activity = ktime_get_real_seconds(); - if (ldlm_is_no_timeout(lock)) { - LDLM_DEBUG(lock, "waiting indefinitely because of NO_TIMEOUT"); - lwi = LWI_INTR(interrupted_completion_wait, &lwd); - } else { - lwi = LWI_TIMEOUT_INTR(timeout * HZ, - ldlm_expired_completion_wait, - interrupted_completion_wait, &lwd); - } - - if (imp) { - spin_lock(&imp->imp_lock); - lwd.lwd_conn_cnt = imp->imp_conn_cnt; - spin_unlock(&imp->imp_lock); - } - if (OBD_FAIL_CHECK_RESET(OBD_FAIL_LDLM_INTR_CP_AST, OBD_FAIL_LDLM_CP_BL_RACE | OBD_FAIL_ONCE)) { ldlm_set_fail_loc(lock); rc = -EINTR; } else { - /* Go to sleep until the lock is granted or cancelled. */ - rc = l_wait_event(lock->l_waitq, - is_granted_or_cancelled(lock), &lwi); + /* Go to sleep until the lock is granted or canceled. */ + if (!ldlm_is_no_timeout(lock)) { + /* Wait uninterruptible for a while first */ + rc = wait_event_idle_timeout(lock->l_waitq, + is_granted_or_cancelled(lock), + timeout * HZ); + if (rc == 0) + ldlm_expired_completion_wait(lock, imp); + } + /* Now wait abortable */ + if (rc == 0) + rc = l_wait_event_abortable(lock->l_waitq, + is_granted_or_cancelled(lock)); + else + rc = 0; } if (rc) { From neilb at suse.com Mon Feb 12 21:22:36 2018 From: neilb at suse.com (NeilBrown) Date: Tue, 13 Feb 2018 08:22:36 +1100 Subject: [lustre-devel] [PATCH 07/19] staging: lustre: simplify l_wait_event when intr handler but no timeout. In-Reply-To: <151847037709.22826.16175867257667686132.stgit@noble> References: <151847037709.22826.16175867257667686132.stgit@noble> Message-ID: <151847055671.22826.8688568641693262055.stgit@noble> If l_wait_event() is given a function to be called on a signal, but no timeout or timeout handler, then the intr function is simply called at the end if the wait was aborted by a signal. So a simpler way to write the code (in the one place this case is used) it to open-code the body of the function after the wait_event, if -ERESTARTSYS was returned. Reviewed-by: James Simmons Signed-off-by: NeilBrown --- drivers/staging/lustre/lustre/ldlm/ldlm_flock.c | 30 +++++------------------ 1 file changed, 7 insertions(+), 23 deletions(-) diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_flock.c b/drivers/staging/lustre/lustre/ldlm/ldlm_flock.c index 657ab95091a0..411b540b96d9 100644 --- a/drivers/staging/lustre/lustre/ldlm/ldlm_flock.c +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_flock.c @@ -310,24 +310,6 @@ static int ldlm_process_flock_lock(struct ldlm_lock *req) return LDLM_ITER_CONTINUE; } -struct ldlm_flock_wait_data { - struct ldlm_lock *fwd_lock; -}; - -static void -ldlm_flock_interrupted_wait(void *data) -{ - struct ldlm_lock *lock; - - lock = ((struct ldlm_flock_wait_data *)data)->fwd_lock; - - lock_res_and_lock(lock); - - /* client side - set flag to prevent lock from being put on LRU list */ - ldlm_set_cbpending(lock); - unlock_res_and_lock(lock); -} - /** * Flock completion callback function. * @@ -342,8 +324,6 @@ int ldlm_flock_completion_ast(struct ldlm_lock *lock, __u64 flags, void *data) { struct file_lock *getlk = lock->l_ast_data; - struct ldlm_flock_wait_data fwd; - struct l_wait_info lwi; int rc = 0; OBD_FAIL_TIMEOUT(OBD_FAIL_LDLM_CP_CB_WAIT2, 4); @@ -372,13 +352,17 @@ ldlm_flock_completion_ast(struct ldlm_lock *lock, __u64 flags, void *data) LDLM_DEBUG(lock, "client-side enqueue returned a blocked lock, sleeping"); - fwd.fwd_lock = lock; - lwi = LWI_TIMEOUT_INTR(0, NULL, ldlm_flock_interrupted_wait, &fwd); /* Go to sleep until the lock is granted. */ - rc = l_wait_event(lock->l_waitq, is_granted_or_cancelled(lock), &lwi); + rc = l_wait_event_abortable(lock->l_waitq, is_granted_or_cancelled(lock)); if (rc) { + lock_res_and_lock(lock); + + /* client side - set flag to prevent lock from being put on LRU list */ + ldlm_set_cbpending(lock); + unlock_res_and_lock(lock); + LDLM_DEBUG(lock, "client-side enqueue waking up: failed (%d)", rc); return rc; From neilb at suse.com Mon Feb 12 21:30:47 2018 From: neilb at suse.com (NeilBrown) Date: Tue, 13 Feb 2018 08:30:47 +1100 Subject: [lustre-devel] [PATCH 0/5] RESEND dcache/namei fixes for lustre Message-ID: <151847081541.26699.3911837371411314092.stgit@noble> Hi, this is a resend of some patches I send in August 2017. They fix a couple of little bugs in the namei-related code and clean up other bits. Only change is that one patch was dropped as other dcache changes made it irrelevant. There will be compiler warnings about a 'const' until a patch that I sent to Al Viro - and which he acknowledged - lands in the tree. Thanks, NeilBrown --- NeilBrown (5): staging: lustre: llite: handle DCACHE_PAR_LOOKUP in ll_dcompare staging: lustre: llite: use d_splice_alias for directories. staging: lustre: llite: remove directory-specific code from ll_find_alias() staging: lluste: llite: simplify ll_find_alias() staging: lustre: llite: refine ll_find_alias based on d_exact_alias drivers/staging/lustre/lustre/llite/dcache.c | 10 ++++ drivers/staging/lustre/lustre/llite/namei.c | 60 +++++++++++++------------- 2 files changed, 39 insertions(+), 31 deletions(-) -- Signature From neilb at suse.com Mon Feb 12 21:30:47 2018 From: neilb at suse.com (NeilBrown) Date: Tue, 13 Feb 2018 08:30:47 +1100 Subject: [lustre-devel] [PATCH 1/5] staging: lustre: llite: handle DCACHE_PAR_LOOKUP in ll_dcompare In-Reply-To: <151847081541.26699.3911837371411314092.stgit@noble> References: <151847081541.26699.3911837371411314092.stgit@noble> Message-ID: <151847104793.26699.5632904371432185858.stgit@noble> ll_dcompare is used in two slightly different contexts. It is called (from __d_lookup, __d_lookup_rcu, and d_exact_alias) to compare a name against a dentry that is already in the dcache. It is also called (from d_alloc_parallel) to compare a name against a dentry that is not in the dcache yet, but is part of an active "lookup" or "atomic_open" call. In the first case we need to avoid matching against "invalid" dentries as a match implies something about ldlm locks which is not accurate. In the second case we need to allow matching against "invalid" dentries as the dentry will always be invalid (set by ll_d_init()) but we still want to guard against multiple concurrent lookups of the same name. d_alloc_parallel() will repeat the call to ll_dcompare() after the lookup has finished, and if the dentry is still invalid, the whole d_alloc_parallel() process is repeated. This assures us that it is safe to report success whenever d_in_lookup(). With this patch, there will never be two threads concurrently in ll_lookup_nd(), looking up the same name in the same directory. Signed-off-by: NeilBrown --- drivers/staging/lustre/lustre/llite/dcache.c | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/drivers/staging/lustre/lustre/llite/dcache.c b/drivers/staging/lustre/lustre/llite/dcache.c index 549369739d80..dc30b4582234 100644 --- a/drivers/staging/lustre/lustre/llite/dcache.c +++ b/drivers/staging/lustre/lustre/llite/dcache.c @@ -74,6 +74,12 @@ static void ll_release(struct dentry *de) * an AST before calling d_revalidate_it(). The dentry still exists (marked * INVALID) so d_lookup() matches it, but we have no lock on it (so * lock_match() fails) and we spin around real_lookup(). + * + * This race doesn't apply to lookups in d_alloc_parallel(), and for + * those we want to ensure that only one dentry with a given name is + * in ll_lookup_nd() at a time. So allow invalid dentries to match + * while d_in_lookup(). We will be called again when the lookup + * completes, and can give a different answer then. */ static int ll_dcompare(const struct dentry *dentry, unsigned int len, const char *str, @@ -93,6 +99,10 @@ static int ll_dcompare(const struct dentry *dentry, if (d_mountpoint((struct dentry *)dentry)) return 0; + /* ensure exclusion against parallel lookup of the same name */ + if (d_in_lookup((struct dentry*)dentry)) + return 0; + if (d_lustre_invalid(dentry)) return 1; From neilb at suse.com Mon Feb 12 21:30:48 2018 From: neilb at suse.com (NeilBrown) Date: Tue, 13 Feb 2018 08:30:48 +1100 Subject: [lustre-devel] [PATCH 2/5] staging: lustre: llite: use d_splice_alias for directories. In-Reply-To: <151847081541.26699.3911837371411314092.stgit@noble> References: <151847081541.26699.3911837371411314092.stgit@noble> Message-ID: <151847104797.26699.15971881529877617500.stgit@noble> In the Linux dcache a directory only ever has one dentry, so d_splice_alias() can be used by ll_splice_alias() for directories. It will find the one dentry whether it is DCACHE_DISCONNECTED or IS_ROOT() or d_lustre_invalid(). Separating out the directories from non-directories will allow us to simplify the non-directory code. Signed-off-by: NeilBrown --- drivers/staging/lustre/lustre/llite/namei.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/drivers/staging/lustre/lustre/llite/namei.c b/drivers/staging/lustre/lustre/llite/namei.c index a2687f46a16d..60fb18f83bf8 100644 --- a/drivers/staging/lustre/lustre/llite/namei.c +++ b/drivers/staging/lustre/lustre/llite/namei.c @@ -434,7 +434,7 @@ static struct dentry *ll_find_alias(struct inode *inode, struct dentry *dentry) */ struct dentry *ll_splice_alias(struct inode *inode, struct dentry *de) { - if (inode) { + if (inode && !S_ISDIR(inode->i_mode)) { struct dentry *new = ll_find_alias(inode, de); if (new) { @@ -445,8 +445,13 @@ struct dentry *ll_splice_alias(struct inode *inode, struct dentry *de) new, d_inode(new), d_count(new), new->d_flags); return new; } + d_add(de, inode); + } else { + struct dentry *new = d_splice_alias(inode, de); + + if (new) + de = new; } - d_add(de, inode); CDEBUG(D_DENTRY, "Add dentry %p inode %p refc %d flags %#x\n", de, d_inode(de), d_count(de), de->d_flags); return de; From neilb at suse.com Mon Feb 12 21:30:48 2018 From: neilb at suse.com (NeilBrown) Date: Tue, 13 Feb 2018 08:30:48 +1100 Subject: [lustre-devel] [PATCH 3/5] staging: lustre: llite: remove directory-specific code from ll_find_alias() In-Reply-To: <151847081541.26699.3911837371411314092.stgit@noble> References: <151847081541.26699.3911837371411314092.stgit@noble> Message-ID: <151847104800.26699.1742105991834584712.stgit@noble> Now that ll_find_alias() is never called for directories, we can remove code that only applies to directories. Signed-off-by: NeilBrown --- drivers/staging/lustre/lustre/llite/namei.c | 26 ++++++++------------------ 1 file changed, 8 insertions(+), 18 deletions(-) diff --git a/drivers/staging/lustre/lustre/llite/namei.c b/drivers/staging/lustre/lustre/llite/namei.c index 60fb18f83bf8..2322328155d6 100644 --- a/drivers/staging/lustre/lustre/llite/namei.c +++ b/drivers/staging/lustre/lustre/llite/namei.c @@ -380,21 +380,15 @@ void ll_i2gids(__u32 *suppgids, struct inode *i1, struct inode *i2) } /* - * try to reuse three types of dentry: - * 1. unhashed alias, this one is unhashed by d_invalidate (but it may be valid - * by concurrent .revalidate). - * 2. INVALID alias (common case for no valid ldlm lock held, but this flag may - * be cleared by others calling d_lustre_revalidate). - * 3. DISCONNECTED alias. + * Try to reuse unhashed or invalidated dentries. */ static struct dentry *ll_find_alias(struct inode *inode, struct dentry *dentry) { - struct dentry *alias, *discon_alias, *invalid_alias; + struct dentry *alias, *invalid_alias; if (hlist_empty(&inode->i_dentry)) return NULL; - discon_alias = NULL; invalid_alias = NULL; spin_lock(&inode->i_lock); @@ -402,22 +396,18 @@ static struct dentry *ll_find_alias(struct inode *inode, struct dentry *dentry) LASSERT(alias != dentry); spin_lock(&alias->d_lock); - if ((alias->d_flags & DCACHE_DISCONNECTED) && - S_ISDIR(inode->i_mode)) - /* LASSERT(last_discon == NULL); LU-405, bz 20055 */ - discon_alias = alias; - else if (alias->d_parent == dentry->d_parent && - alias->d_name.hash == dentry->d_name.hash && - alias->d_name.len == dentry->d_name.len && - memcmp(alias->d_name.name, dentry->d_name.name, - dentry->d_name.len) == 0) + if (alias->d_parent == dentry->d_parent && + alias->d_name.hash == dentry->d_name.hash && + alias->d_name.len == dentry->d_name.len && + memcmp(alias->d_name.name, dentry->d_name.name, + dentry->d_name.len) == 0) invalid_alias = alias; spin_unlock(&alias->d_lock); if (invalid_alias) break; } - alias = invalid_alias ?: discon_alias ?: NULL; + alias = invalid_alias ?: NULL; if (alias) { spin_lock(&alias->d_lock); dget_dlock(alias); From neilb at suse.com Mon Feb 12 21:30:48 2018 From: neilb at suse.com (NeilBrown) Date: Tue, 13 Feb 2018 08:30:48 +1100 Subject: [lustre-devel] [PATCH 4/5] staging: lluste: llite: simplify ll_find_alias() In-Reply-To: <151847081541.26699.3911837371411314092.stgit@noble> References: <151847081541.26699.3911837371411314092.stgit@noble> Message-ID: <151847104803.26699.4826721577063033284.stgit@noble> Now that ll_find_alias is only searching for one type of dentry, we can return as soon as we find it. This allows substantial simplification, and brings the bonus that we don't need to take the d_lock again just to increment the ref-count. We can increment it immediately that the dentry is found. Signed-off-by: NeilBrown --- drivers/staging/lustre/lustre/llite/namei.c | 23 ++++++++--------------- 1 file changed, 8 insertions(+), 15 deletions(-) diff --git a/drivers/staging/lustre/lustre/llite/namei.c b/drivers/staging/lustre/lustre/llite/namei.c index 2322328155d6..baf94f4bcee9 100644 --- a/drivers/staging/lustre/lustre/llite/namei.c +++ b/drivers/staging/lustre/lustre/llite/namei.c @@ -384,13 +384,11 @@ void ll_i2gids(__u32 *suppgids, struct inode *i1, struct inode *i2) */ static struct dentry *ll_find_alias(struct inode *inode, struct dentry *dentry) { - struct dentry *alias, *invalid_alias; + struct dentry *alias; if (hlist_empty(&inode->i_dentry)) return NULL; - invalid_alias = NULL; - spin_lock(&inode->i_lock); hlist_for_each_entry(alias, &inode->i_dentry, d_u.d_alias) { LASSERT(alias != dentry); @@ -400,22 +398,17 @@ static struct dentry *ll_find_alias(struct inode *inode, struct dentry *dentry) alias->d_name.hash == dentry->d_name.hash && alias->d_name.len == dentry->d_name.len && memcmp(alias->d_name.name, dentry->d_name.name, - dentry->d_name.len) == 0) - invalid_alias = alias; - spin_unlock(&alias->d_lock); - - if (invalid_alias) - break; - } - alias = invalid_alias ?: NULL; - if (alias) { - spin_lock(&alias->d_lock); - dget_dlock(alias); + dentry->d_name.len) == 0) { + dget_dlock(alias); + spin_unlock(&alias->d_lock); + spin_unlock(&inode->i_lock); + return alias; + } spin_unlock(&alias->d_lock); } spin_unlock(&inode->i_lock); - return alias; + return NULL; } /* From neilb at suse.com Mon Feb 12 21:30:48 2018 From: neilb at suse.com (NeilBrown) Date: Tue, 13 Feb 2018 08:30:48 +1100 Subject: [lustre-devel] [PATCH 5/5] staging: lustre: llite: refine ll_find_alias based on d_exact_alias In-Reply-To: <151847081541.26699.3911837371411314092.stgit@noble> References: <151847081541.26699.3911837371411314092.stgit@noble> Message-ID: <151847104807.26699.140278585898570948.stgit@noble> The task of ll_find_alias() is now very similar to d_exact_alias(). We cannot use that function directly, but we can copy much of the structure so that the similarities and differences are more obvious. Examining d_exact_alias() shows that the d_lock spinlock does not need to be held in ll_find_alias as much as it currently is. Signed-off-by: NeilBrown --- drivers/staging/lustre/lustre/llite/namei.c | 30 ++++++++++++++++++--------- 1 file changed, 20 insertions(+), 10 deletions(-) diff --git a/drivers/staging/lustre/lustre/llite/namei.c b/drivers/staging/lustre/lustre/llite/namei.c index baf94f4bcee9..6c9ec462eb41 100644 --- a/drivers/staging/lustre/lustre/llite/namei.c +++ b/drivers/staging/lustre/lustre/llite/namei.c @@ -381,6 +381,10 @@ void ll_i2gids(__u32 *suppgids, struct inode *i1, struct inode *i2) /* * Try to reuse unhashed or invalidated dentries. + * This is very similar to d_exact_alias(), and any changes in one should be + * considered for inclusion in the other. The differences are that we don't + * need an unhashed alias, and we don't want d_compare to be used for + * comparison. */ static struct dentry *ll_find_alias(struct inode *inode, struct dentry *dentry) { @@ -392,19 +396,25 @@ static struct dentry *ll_find_alias(struct inode *inode, struct dentry *dentry) spin_lock(&inode->i_lock); hlist_for_each_entry(alias, &inode->i_dentry, d_u.d_alias) { LASSERT(alias != dentry); + /* + * Don't need alias->d_lock here, because aliases with + * d_parent == entry->d_parent are not subject to name or + * parent changes, because the parent inode i_mutex is held. + */ - spin_lock(&alias->d_lock); - if (alias->d_parent == dentry->d_parent && - alias->d_name.hash == dentry->d_name.hash && - alias->d_name.len == dentry->d_name.len && + if (alias->d_parent != dentry->d_parent) + continue; + if (alias->d_name.hash != dentry->d_name.hash) + continue; + if (alias->d_name.len != dentry->d_name.len || memcmp(alias->d_name.name, dentry->d_name.name, - dentry->d_name.len) == 0) { - dget_dlock(alias); - spin_unlock(&alias->d_lock); - spin_unlock(&inode->i_lock); - return alias; - } + dentry->d_name.len) != 0) + continue; + spin_lock(&alias->d_lock); + dget_dlock(alias); spin_unlock(&alias->d_lock); + spin_unlock(&inode->i_lock); + return alias; } spin_unlock(&inode->i_lock); From paf at cray.com Mon Feb 12 21:36:23 2018 From: paf at cray.com (Patrick Farrell) Date: Mon, 12 Feb 2018 21:36:23 +0000 Subject: [lustre-devel] [PATCH 06/19] staging: lustre: introduce and use l_wait_event_abortable() In-Reply-To: <151847055667.22826.8544979478231411275.stgit@noble> References: <151847037709.22826.16175867257667686132.stgit@noble> <151847055667.22826.8544979478231411275.stgit@noble> Message-ID: <81AFF1FD-E398-4B69-9B20-1D202747AB25@cray.com> It's worth noting that the change from -EINTR to -ERESTARTSYS will modify the behavior of userspace slightly. Specifically, when a signal handler is setup with retry set (SA_RESTART flag set), the syscall will be restarted rather than aborted. This should be fine. It may eventually shake out some stuble bugs in Lustre when we abort an operation and then restart it from a point where we didn't in the past - past instances where we changed from -EINTR to -ERESTARTSYS certainly have - but it's for the best. As I understand it from past conversations with Andreas and others, Lustre is not really intending to claim it's not restartable, it's just an artifact of people copying older code that was written without awareness of the difference in EINTR vs ERESTARTSYS. Ideally someone should go through and audit the remaining uses of -EINTR and replace most of them with -ERESTARTSYS. James, maybe you want to add that to the TODO list? On 2/12/18, 3:24 PM, "lustre-devel on behalf of NeilBrown" wrote: lustre sometimes wants to wait for an event, but abort if one of a specific list of signals arrives. This is a little bit like wait_event_killable(), except that the signals are identified a different way. So introduce l_wait_event_abortable() which provides this functionality. Having separate functions for separate needs is more in line with the pattern set by include/linux/wait.h, than having a single function which tries to include all possible needs. Also introduce l_wait_event_abortable_exclusive(). Note that l_wait_event() return -EINTR on a signal, while Linux wait_event functions return -ERESTARTSYS. l_wait_event_{abortable_,}exclusive follow the Linux pattern. Reviewed-by: James Simmons Signed-off-by: NeilBrown --- drivers/staging/lustre/lustre/include/lustre_lib.h | 24 ++++++++++++++++++++ drivers/staging/lustre/lustre/ldlm/ldlm_resource.c | 12 +++++----- drivers/staging/lustre/lustre/llite/llite_lib.c | 12 +++------- drivers/staging/lustre/lustre/obdclass/genops.c | 9 +++----- drivers/staging/lustre/lustre/obdclass/llog_obd.c | 5 ++-- drivers/staging/lustre/lustre/osc/osc_page.c | 6 ++--- drivers/staging/lustre/lustre/osc/osc_request.c | 6 ++--- 7 files changed, 43 insertions(+), 31 deletions(-) diff --git a/drivers/staging/lustre/lustre/include/lustre_lib.h b/drivers/staging/lustre/lustre/include/lustre_lib.h index 7d950c53e962..b2a64d0e682c 100644 --- a/drivers/staging/lustre/lustre/include/lustre_lib.h +++ b/drivers/staging/lustre/lustre/include/lustre_lib.h @@ -336,4 +336,28 @@ do { \ /** @} lib */ + +/* l_wait_event_abortable() is a bit like wait_event_killable() + * except there is a fixed set of signals which will abort: + * LUSTRE_FATAL_SIGS + */ +#define l_wait_event_abortable(wq, condition) \ +({ \ + sigset_t __blocked; \ + int __ret = 0; \ + __blocked = cfs_block_sigsinv(LUSTRE_FATAL_SIGS); \ + __ret = wait_event_interruptible(wq, condition); \ + cfs_restore_sigs(__blocked); \ + __ret; \ +}) + +#define l_wait_event_abortable_exclusive(wq, condition) \ +({ \ + sigset_t __blocked; \ + int __ret = 0; \ + __blocked = cfs_block_sigsinv(LUSTRE_FATAL_SIGS); \ + __ret = wait_event_interruptible_exclusive(wq, condition); \ + cfs_restore_sigs(__blocked); \ + __ret; \ +}) #endif /* _LUSTRE_LIB_H */ diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_resource.c b/drivers/staging/lustre/lustre/ldlm/ldlm_resource.c index 2e66825c8f4b..4c44603ab6f9 100644 --- a/drivers/staging/lustre/lustre/ldlm/ldlm_resource.c +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_resource.c @@ -879,7 +879,6 @@ static int __ldlm_namespace_free(struct ldlm_namespace *ns, int force) ldlm_namespace_cleanup(ns, force ? LDLM_FL_LOCAL_ONLY : 0); if (atomic_read(&ns->ns_bref) > 0) { - struct l_wait_info lwi = LWI_INTR(LWI_ON_SIGNAL_NOOP, NULL); int rc; CDEBUG(D_DLMTRACE, @@ -887,11 +886,12 @@ static int __ldlm_namespace_free(struct ldlm_namespace *ns, int force) ldlm_ns_name(ns), atomic_read(&ns->ns_bref)); force_wait: if (force) - lwi = LWI_TIMEOUT(msecs_to_jiffies(obd_timeout * - MSEC_PER_SEC) / 4, NULL, NULL); - - rc = l_wait_event(ns->ns_waitq, - atomic_read(&ns->ns_bref) == 0, &lwi); + rc = wait_event_idle_timeout(ns->ns_waitq, + atomic_read(&ns->ns_bref) == 0, + obd_timeout * HZ / 4) ? 0 : -ETIMEDOUT; + else + rc = l_wait_event_abortable(ns->ns_waitq, + atomic_read(&ns->ns_bref) == 0); /* Forced cleanups should be able to reclaim all references, * so it's safe to wait forever... we can't leak locks... diff --git a/drivers/staging/lustre/lustre/llite/llite_lib.c b/drivers/staging/lustre/lustre/llite/llite_lib.c index c820b201af71..ccb614bd7f53 100644 --- a/drivers/staging/lustre/lustre/llite/llite_lib.c +++ b/drivers/staging/lustre/lustre/llite/llite_lib.c @@ -986,16 +986,12 @@ void ll_put_super(struct super_block *sb) } /* Wait for unstable pages to be committed to stable storage */ - if (!force) { - struct l_wait_info lwi = LWI_INTR(LWI_ON_SIGNAL_NOOP, NULL); - - rc = l_wait_event(sbi->ll_cache->ccc_unstable_waitq, - !atomic_long_read(&sbi->ll_cache->ccc_unstable_nr), - &lwi); - } + if (!force) + rc = l_wait_event_abortable(sbi->ll_cache->ccc_unstable_waitq, + !atomic_long_read(&sbi->ll_cache->ccc_unstable_nr)); ccc_count = atomic_long_read(&sbi->ll_cache->ccc_unstable_nr); - if (!force && rc != -EINTR) + if (!force && rc != -ERESTARTSYS) LASSERTF(!ccc_count, "count: %li\n", ccc_count); /* We need to set force before the lov_disconnect in diff --git a/drivers/staging/lustre/lustre/obdclass/genops.c b/drivers/staging/lustre/lustre/obdclass/genops.c index 3ff25b8d3b48..8f776a4058a9 100644 --- a/drivers/staging/lustre/lustre/obdclass/genops.c +++ b/drivers/staging/lustre/lustre/obdclass/genops.c @@ -1332,7 +1332,6 @@ static bool obd_request_slot_avail(struct client_obd *cli, int obd_get_request_slot(struct client_obd *cli) { struct obd_request_slot_waiter orsw; - struct l_wait_info lwi; int rc; spin_lock(&cli->cl_loi_list_lock); @@ -1347,11 +1346,9 @@ int obd_get_request_slot(struct client_obd *cli) orsw.orsw_signaled = false; spin_unlock(&cli->cl_loi_list_lock); - lwi = LWI_INTR(LWI_ON_SIGNAL_NOOP, NULL); - rc = l_wait_event(orsw.orsw_waitq, - obd_request_slot_avail(cli, &orsw) || - orsw.orsw_signaled, - &lwi); + rc = l_wait_event_abortable(orsw.orsw_waitq, + obd_request_slot_avail(cli, &orsw) || + orsw.orsw_signaled); /* * Here, we must take the lock to avoid the on-stack 'orsw' to be diff --git a/drivers/staging/lustre/lustre/obdclass/llog_obd.c b/drivers/staging/lustre/lustre/obdclass/llog_obd.c index 28bbaa2136ac..26aea114a29b 100644 --- a/drivers/staging/lustre/lustre/obdclass/llog_obd.c +++ b/drivers/staging/lustre/lustre/obdclass/llog_obd.c @@ -104,7 +104,6 @@ EXPORT_SYMBOL(__llog_ctxt_put); int llog_cleanup(const struct lu_env *env, struct llog_ctxt *ctxt) { - struct l_wait_info lwi = LWI_INTR(LWI_ON_SIGNAL_NOOP, NULL); struct obd_llog_group *olg; int rc, idx; @@ -129,8 +128,8 @@ int llog_cleanup(const struct lu_env *env, struct llog_ctxt *ctxt) CERROR("Error %d while cleaning up ctxt %p\n", rc, ctxt); - l_wait_event(olg->olg_waitq, - llog_group_ctxt_null(olg, idx), &lwi); + l_wait_event_abortable(olg->olg_waitq, + llog_group_ctxt_null(olg, idx)); return rc; } diff --git a/drivers/staging/lustre/lustre/osc/osc_page.c b/drivers/staging/lustre/lustre/osc/osc_page.c index 20094b6309f9..6fdd521feb21 100644 --- a/drivers/staging/lustre/lustre/osc/osc_page.c +++ b/drivers/staging/lustre/lustre/osc/osc_page.c @@ -759,7 +759,6 @@ static long osc_lru_reclaim(struct client_obd *cli, unsigned long npages) static int osc_lru_alloc(const struct lu_env *env, struct client_obd *cli, struct osc_page *opg) { - struct l_wait_info lwi = LWI_INTR(LWI_ON_SIGNAL_NOOP, NULL); struct osc_io *oio = osc_env_io(env); int rc = 0; @@ -782,9 +781,8 @@ static int osc_lru_alloc(const struct lu_env *env, struct client_obd *cli, cond_resched(); - rc = l_wait_event(osc_lru_waitq, - atomic_long_read(cli->cl_lru_left) > 0, - &lwi); + rc = l_wait_event_abortable(osc_lru_waitq, + atomic_long_read(cli->cl_lru_left) > 0); if (rc < 0) break; diff --git a/drivers/staging/lustre/lustre/osc/osc_request.c b/drivers/staging/lustre/lustre/osc/osc_request.c index 45b1ebf33363..074b5ce6284c 100644 --- a/drivers/staging/lustre/lustre/osc/osc_request.c +++ b/drivers/staging/lustre/lustre/osc/osc_request.c @@ -552,14 +552,12 @@ static int osc_destroy(const struct lu_env *env, struct obd_export *exp, req->rq_interpret_reply = osc_destroy_interpret; if (!osc_can_send_destroy(cli)) { - struct l_wait_info lwi = LWI_INTR(LWI_ON_SIGNAL_NOOP, NULL); - /* * Wait until the number of on-going destroy RPCs drops * under max_rpc_in_flight */ - l_wait_event_exclusive(cli->cl_destroy_waitq, - osc_can_send_destroy(cli), &lwi); + l_wait_event_abortable_exclusive(cli->cl_destroy_waitq, + osc_can_send_destroy(cli)); } /* Do not wait for response */ _______________________________________________ lustre-devel mailing list lustre-devel at lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org From paf at cray.com Mon Feb 12 22:08:56 2018 From: paf at cray.com (Patrick Farrell) Date: Mon, 12 Feb 2018 22:08:56 +0000 Subject: [lustre-devel] [PATCH 08/19] staging: lustre: simplify waiting in ldlm_completion_ast() In-Reply-To: <151847055674.22826.12912356108048917428.stgit@noble> References: <151847037709.22826.16175867257667686132.stgit@noble> <151847055674.22826.12912356108048917428.stgit@noble> Message-ID: <4AC4BB93-85C5-4CA1-B062-1274833C51C6@cray.com> Neil, I didn't get anything after 8/19 in this series. Is this just me? (I'd keep waiting, except I also found a few things in this patch.) Minor: The line XXX ALLOCATE is out of date and could go. (It refers to a mix of things you eliminated and things that were already gone.) Less minor: You remove use of the imp_lock when reading the connection count. While that'll work on x86, it's probably wrong on some architecture to read that without taking the lock...? Bug: The existing code uses the imp_conn_cnt from *before* the wait, rather than after. I think that's quite important. So you'll want to read it out before the wait. I think the main reason we'd hit the timeout is a disconnect, which should cause a reconnect, so it's very important to use the value from *before* the wait. (See comment on ptlrpc_set_import_discon for more of an explanation. Basically it's tracking a connection 'epoch', if it's changed, someone else already went through the reconnect code for this 'connection epoch' and we shouldn't start that process.) - Patrick On 2/12/18, 3:24 PM, "lustre-devel on behalf of NeilBrown" wrote: If a signal-callback (lwi_on_signal) is set without lwi_allow_intr, as is the case in ldlm_completion_ast(), the behavior depends on the timeout set. If a timeout is set, then signals are ignored. If the timeout is reached, the timeout handler is called. If the timeout handler return 0, which ldlm_expired_completion_wait() always does, the l_wait_event() switches to exactly the behavior if no timeout was set. If no timeout is set, then "fatal" signals are not ignored. If one arrives the callback is run, but as the callback is empty in this case, that is not relevant. This can be simplified to: if a timeout is wanted wait_event_idle_timeout() if that timed out, call the timeout handler l_wait_event_abortable() i.e. the code always waits indefinitely. Sometimes it performs a non-abortable wait first. Sometimes it doesn't. But it only aborts before the condition is true if it is signaled. This doesn't quite agree with the comments and debug messages. Reviewed-by: James Simmons Signed-off-by: NeilBrown --- drivers/staging/lustre/lustre/ldlm/ldlm_request.c | 55 +++++++-------------- 1 file changed, 18 insertions(+), 37 deletions(-) diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_request.c b/drivers/staging/lustre/lustre/ldlm/ldlm_request.c index a244fa717134..f1233d844bbd 100644 --- a/drivers/staging/lustre/lustre/ldlm/ldlm_request.c +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_request.c @@ -72,15 +72,6 @@ MODULE_PARM_DESC(ldlm_enqueue_min, "lock enqueue timeout minimum"); /* in client side, whether the cached locks will be canceled before replay */ unsigned int ldlm_cancel_unused_locks_before_replay = 1; -static void interrupted_completion_wait(void *data) -{ -} - -struct lock_wait_data { - struct ldlm_lock *lwd_lock; - __u32 lwd_conn_cnt; -}; - struct ldlm_async_args { struct lustre_handle lock_handle; }; @@ -112,10 +103,8 @@ static int ldlm_request_bufsize(int count, int type) return sizeof(struct ldlm_request) + avail; } -static int ldlm_expired_completion_wait(void *data) +static void ldlm_expired_completion_wait(struct ldlm_lock *lock, struct obd_import *imp2) { - struct lock_wait_data *lwd = data; - struct ldlm_lock *lock = lwd->lwd_lock; struct obd_import *imp; struct obd_device *obd; @@ -135,19 +124,17 @@ static int ldlm_expired_completion_wait(void *data) if (last_dump == 0) libcfs_debug_dumplog(); } - return 0; + return; } obd = lock->l_conn_export->exp_obd; imp = obd->u.cli.cl_import; - ptlrpc_fail_import(imp, lwd->lwd_conn_cnt); + ptlrpc_fail_import(imp, imp2 ? imp2->imp_conn_cnt : 0); LDLM_ERROR(lock, "lock timed out (enqueued at %lld, %llds ago), entering recovery for %s@%s", (s64)lock->l_last_activity, (s64)(ktime_get_real_seconds() - lock->l_last_activity), obd2cli_tgt(obd), imp->imp_connection->c_remote_uuid.uuid); - - return 0; } /** @@ -251,10 +238,8 @@ EXPORT_SYMBOL(ldlm_completion_ast_async); int ldlm_completion_ast(struct ldlm_lock *lock, __u64 flags, void *data) { /* XXX ALLOCATE - 160 bytes */ - struct lock_wait_data lwd; struct obd_device *obd; struct obd_import *imp = NULL; - struct l_wait_info lwi; __u32 timeout; int rc = 0; @@ -281,32 +266,28 @@ int ldlm_completion_ast(struct ldlm_lock *lock, __u64 flags, void *data) timeout = ldlm_cp_timeout(lock); - lwd.lwd_lock = lock; lock->l_last_activity = ktime_get_real_seconds(); - if (ldlm_is_no_timeout(lock)) { - LDLM_DEBUG(lock, "waiting indefinitely because of NO_TIMEOUT"); - lwi = LWI_INTR(interrupted_completion_wait, &lwd); - } else { - lwi = LWI_TIMEOUT_INTR(timeout * HZ, - ldlm_expired_completion_wait, - interrupted_completion_wait, &lwd); - } - - if (imp) { - spin_lock(&imp->imp_lock); - lwd.lwd_conn_cnt = imp->imp_conn_cnt; - spin_unlock(&imp->imp_lock); - } - if (OBD_FAIL_CHECK_RESET(OBD_FAIL_LDLM_INTR_CP_AST, OBD_FAIL_LDLM_CP_BL_RACE | OBD_FAIL_ONCE)) { ldlm_set_fail_loc(lock); rc = -EINTR; } else { - /* Go to sleep until the lock is granted or cancelled. */ - rc = l_wait_event(lock->l_waitq, - is_granted_or_cancelled(lock), &lwi); + /* Go to sleep until the lock is granted or canceled. */ + if (!ldlm_is_no_timeout(lock)) { + /* Wait uninterruptible for a while first */ + rc = wait_event_idle_timeout(lock->l_waitq, + is_granted_or_cancelled(lock), + timeout * HZ); + if (rc == 0) + ldlm_expired_completion_wait(lock, imp); + } + /* Now wait abortable */ + if (rc == 0) + rc = l_wait_event_abortable(lock->l_waitq, + is_granted_or_cancelled(lock)); + else + rc = 0; } if (rc) { _______________________________________________ lustre-devel mailing list lustre-devel at lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org From neilb at suse.com Mon Feb 12 23:47:59 2018 From: neilb at suse.com (NeilBrown) Date: Tue, 13 Feb 2018 10:47:59 +1100 Subject: [lustre-devel] [PATCH 09/19] staging: lustre: open code polling loop instead of using l_wait_event() In-Reply-To: <151847037709.22826.16175867257667686132.stgit@noble> References: <151847037709.22826.16175867257667686132.stgit@noble> Message-ID: <151847927897.19840.8283814090245785502.stgit@noble> Two places that LWI_TIMEOUT_INTERVAL() is used, the outcome is a simple polling loop that polls every second for some event (with a limit). So write a simple loop to make this more apparent. Reviewed-by: James Simmons Signed-off-by: NeilBrown --- drivers/staging/lustre/lustre/llite/llite_lib.c | 11 +++++------ drivers/staging/lustre/lustre/lov/lov_request.c | 12 +++++------- 2 files changed, 10 insertions(+), 13 deletions(-) diff --git a/drivers/staging/lustre/lustre/llite/llite_lib.c b/drivers/staging/lustre/lustre/llite/llite_lib.c index ccb614bd7f53..9e96a8ee1783 100644 --- a/drivers/staging/lustre/lustre/llite/llite_lib.c +++ b/drivers/staging/lustre/lustre/llite/llite_lib.c @@ -1984,8 +1984,7 @@ void ll_umount_begin(struct super_block *sb) struct ll_sb_info *sbi = ll_s2sbi(sb); struct obd_device *obd; struct obd_ioctl_data *ioc_data; - wait_queue_head_t waitq; - struct l_wait_info lwi; + int cnt = 0; CDEBUG(D_VFSTRACE, "VFS Op: superblock %p count %d active %d\n", sb, sb->s_count, atomic_read(&sb->s_active)); @@ -2021,10 +2020,10 @@ void ll_umount_begin(struct super_block *sb) * and then continue. For now, we just periodically checking for vfs * to decrement mnt_cnt and hope to finish it within 10sec. */ - init_waitqueue_head(&waitq); - lwi = LWI_TIMEOUT_INTERVAL(10 * HZ, - HZ, NULL, NULL); - l_wait_event(waitq, may_umount(sbi->ll_mnt.mnt), &lwi); + while (cnt < 10 && !may_umount(sbi->ll_mnt.mnt)) { + schedule_timeout_uninterruptible(HZ); + cnt ++; + } schedule(); } diff --git a/drivers/staging/lustre/lustre/lov/lov_request.c b/drivers/staging/lustre/lustre/lov/lov_request.c index fb3b7a7fa32a..c1e58fcc30b3 100644 --- a/drivers/staging/lustre/lustre/lov/lov_request.c +++ b/drivers/staging/lustre/lustre/lov/lov_request.c @@ -99,8 +99,7 @@ static int lov_check_set(struct lov_obd *lov, int idx) */ static int lov_check_and_wait_active(struct lov_obd *lov, int ost_idx) { - wait_queue_head_t waitq; - struct l_wait_info lwi; + int cnt = 0; struct lov_tgt_desc *tgt; int rc = 0; @@ -125,11 +124,10 @@ static int lov_check_and_wait_active(struct lov_obd *lov, int ost_idx) mutex_unlock(&lov->lov_lock); - init_waitqueue_head(&waitq); - lwi = LWI_TIMEOUT_INTERVAL(obd_timeout * HZ, - HZ, NULL, NULL); - - rc = l_wait_event(waitq, lov_check_set(lov, ost_idx), &lwi); + while (cnt < obd_timeout && !lov_check_set(lov, ost_idx)) { + schedule_timeout_uninterruptible(HZ); + cnt ++; + } if (tgt->ltd_active) return 1; From neilb at suse.com Mon Feb 12 23:47:59 2018 From: neilb at suse.com (NeilBrown) Date: Tue, 13 Feb 2018 10:47:59 +1100 Subject: [lustre-devel] [PATCH 10/19] staging: lustre: simplify waiting in ptlrpc_invalidate_import() In-Reply-To: <151847037709.22826.16175867257667686132.stgit@noble> References: <151847037709.22826.16175867257667686132.stgit@noble> Message-ID: <151847927900.19840.18432480230541523283.stgit@noble> This waiter currently wakes up every second to re-test if imp_flight is zero. If we ensure wakeup is called whenever imp_flight is decremented to zero, we can just have a simple wait_event_idle_timeout(). So add a wake_up_all to the one place it is missing, and simplify the wait_event. Reviewed-by: James Simmons Signed-off-by: NeilBrown --- drivers/staging/lustre/lustre/ptlrpc/client.c | 3 ++- drivers/staging/lustre/lustre/ptlrpc/import.c | 21 ++++++++------------- 2 files changed, 10 insertions(+), 14 deletions(-) diff --git a/drivers/staging/lustre/lustre/ptlrpc/client.c b/drivers/staging/lustre/lustre/ptlrpc/client.c index 0ab13f8e5993..81b7a7046d82 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/client.c +++ b/drivers/staging/lustre/lustre/ptlrpc/client.c @@ -1588,7 +1588,8 @@ static int ptlrpc_send_new_req(struct ptlrpc_request *req) spin_lock(&imp->imp_lock); if (!list_empty(&req->rq_list)) { list_del_init(&req->rq_list); - atomic_dec(&req->rq_import->imp_inflight); + if (atomic_dec_and_test(&req->rq_import->imp_inflight)) + wake_up_all(&req->rq_import->imp_recovery_waitq); } spin_unlock(&imp->imp_lock); ptlrpc_rqphase_move(req, RQ_PHASE_NEW); diff --git a/drivers/staging/lustre/lustre/ptlrpc/import.c b/drivers/staging/lustre/lustre/ptlrpc/import.c index ed210550f61f..5d62c9de27eb 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/import.c +++ b/drivers/staging/lustre/lustre/ptlrpc/import.c @@ -265,7 +265,6 @@ void ptlrpc_invalidate_import(struct obd_import *imp) { struct list_head *tmp, *n; struct ptlrpc_request *req; - struct l_wait_info lwi; unsigned int timeout; int rc; @@ -306,19 +305,15 @@ void ptlrpc_invalidate_import(struct obd_import *imp) * callbacks. Cap it at obd_timeout -- these should all * have been locally cancelled by ptlrpc_abort_inflight. */ - lwi = LWI_TIMEOUT_INTERVAL( - cfs_timeout_cap(timeout * HZ), - (timeout > 1) ? HZ : - HZ / 2, - NULL, NULL); - rc = l_wait_event(imp->imp_recovery_waitq, - (atomic_read(&imp->imp_inflight) == 0), - &lwi); - if (rc) { + rc = wait_event_idle_timeout(imp->imp_recovery_waitq, + atomic_read(&imp->imp_inflight) == 0, + obd_timeout * HZ); + + if (rc == 0) { const char *cli_tgt = obd2cli_tgt(imp->imp_obd); - CERROR("%s: rc = %d waiting for callback (%d != 0)\n", - cli_tgt, rc, + CERROR("%s: timeout waiting for callback (%d != 0)\n", + cli_tgt, atomic_read(&imp->imp_inflight)); spin_lock(&imp->imp_lock); @@ -365,7 +360,7 @@ void ptlrpc_invalidate_import(struct obd_import *imp) } spin_unlock(&imp->imp_lock); } - } while (rc != 0); + } while (rc == 0); /* * Let's additionally check that no new rpcs added to import in From neilb at suse.com Mon Feb 12 23:47:59 2018 From: neilb at suse.com (NeilBrown) Date: Tue, 13 Feb 2018 10:47:59 +1100 Subject: [lustre-devel] [PATCH 12/19] staging: lustre: make polling loop in ptlrpc_unregister_bulk more obvious In-Reply-To: <151847037709.22826.16175867257667686132.stgit@noble> References: <151847037709.22826.16175867257667686132.stgit@noble> Message-ID: <151847927906.19840.2239613687503242193.stgit@noble> This use of l_wait_event() is a polling loop that re-checks every second. Make this more obvious with a while loop and wait_event_idle_timeout(). Reviewed-by: James Simmons Signed-off-by: NeilBrown --- drivers/staging/lustre/lustre/ptlrpc/niobuf.c | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/drivers/staging/lustre/lustre/ptlrpc/niobuf.c b/drivers/staging/lustre/lustre/ptlrpc/niobuf.c index 0c2ded721c49..86883abaad2c 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/niobuf.c +++ b/drivers/staging/lustre/lustre/ptlrpc/niobuf.c @@ -229,7 +229,6 @@ int ptlrpc_unregister_bulk(struct ptlrpc_request *req, int async) { struct ptlrpc_bulk_desc *desc = req->rq_bulk; wait_queue_head_t *wq; - struct l_wait_info lwi; int rc; LASSERT(!in_interrupt()); /* might sleep */ @@ -246,7 +245,7 @@ int ptlrpc_unregister_bulk(struct ptlrpc_request *req, int async) /* the unlink ensures the callback happens ASAP and is the last * one. If it fails, it must be because completion just happened, - * but we must still l_wait_event() in this case to give liblustre + * but we must still wait_event() in this case to give liblustre * a chance to run client_bulk_callback() */ mdunlink_iterate_helper(desc->bd_mds, desc->bd_md_max_brw); @@ -270,15 +269,17 @@ int ptlrpc_unregister_bulk(struct ptlrpc_request *req, int async) /* Network access will complete in finite time but the HUGE * timeout lets us CWARN for visibility of sluggish LNDs */ - lwi = LWI_TIMEOUT_INTERVAL(LONG_UNLINK * HZ, - HZ, NULL, NULL); - rc = l_wait_event(*wq, !ptlrpc_client_bulk_active(req), &lwi); - if (rc == 0) { + int cnt = 0; + while (cnt < LONG_UNLINK && + (rc = wait_event_idle_timeout(*wq, + !ptlrpc_client_bulk_active(req), + HZ)) == 0) + cnt += 1; + if (rc > 0) { ptlrpc_rqphase_move(req, req->rq_next_phase); return 1; } - LASSERT(rc == -ETIMEDOUT); DEBUG_REQ(D_WARNING, req, "Unexpectedly long timeout: desc %p", desc); } From neilb at suse.com Mon Feb 12 23:47:59 2018 From: neilb at suse.com (NeilBrown) Date: Tue, 13 Feb 2018 10:47:59 +1100 Subject: [lustre-devel] [PATCH 11/19] staging: lustre: remove back_to_sleep() In-Reply-To: <151847037709.22826.16175867257667686132.stgit@noble> References: <151847037709.22826.16175867257667686132.stgit@noble> Message-ID: <151847927903.19840.11717617846481237765.stgit@noble> When 'back_to_sleep()' is passed as the 'timeout' function, the effect is to wait indefinitely for the event, polling once after the timeout. If LWI_ON_SIGNAL_NOOP is given, then after the timeout we allow fatal signals to interrupt the wait. Make this more obvious in both places "back_to_sleep()" is used but using two explicit sleeps. The code in ptlrpcd_add_req() looks odd - why not just have one wait_event_idle()? However I believe this is a faithful transformation of the existing code. Reviewed-by: James Simmons Signed-off-by: NeilBrown --- drivers/staging/lustre/lustre/include/lustre_lib.h | 4 ---- drivers/staging/lustre/lustre/ptlrpc/import.c | 11 ++++++----- drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c | 9 +++++---- 3 files changed, 11 insertions(+), 13 deletions(-) diff --git a/drivers/staging/lustre/lustre/include/lustre_lib.h b/drivers/staging/lustre/lustre/include/lustre_lib.h index b2a64d0e682c..1939e959b92a 100644 --- a/drivers/staging/lustre/lustre/include/lustre_lib.h +++ b/drivers/staging/lustre/lustre/include/lustre_lib.h @@ -140,10 +140,6 @@ void target_send_reply(struct ptlrpc_request *req, int rc, int fail_id); * XXX nikita: some ptlrpc daemon threads have races of that sort. * */ -static inline int back_to_sleep(void *arg) -{ - return 0; -} #define LWI_ON_SIGNAL_NOOP ((void (*)(void *))(-1)) diff --git a/drivers/staging/lustre/lustre/ptlrpc/import.c b/drivers/staging/lustre/lustre/ptlrpc/import.c index 5d62c9de27eb..faf0f606f013 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/import.c +++ b/drivers/staging/lustre/lustre/ptlrpc/import.c @@ -1496,7 +1496,6 @@ int ptlrpc_disconnect_import(struct obd_import *imp, int noclose) } if (ptlrpc_import_in_recovery(imp)) { - struct l_wait_info lwi; long timeout; if (AT_OFF) { @@ -1510,10 +1509,12 @@ int ptlrpc_disconnect_import(struct obd_import *imp, int noclose) timeout = at_get(&imp->imp_at.iat_service_estimate[idx]) * HZ; } - lwi = LWI_TIMEOUT_INTR(cfs_timeout_cap(timeout), - back_to_sleep, LWI_ON_SIGNAL_NOOP, NULL); - rc = l_wait_event(imp->imp_recovery_waitq, - !ptlrpc_import_in_recovery(imp), &lwi); + if (wait_event_idle_timeout(imp->imp_recovery_waitq, + !ptlrpc_import_in_recovery(imp), + cfs_timeout_cap(timeout)) == 0) + l_wait_event_abortable( + imp->imp_recovery_waitq, + !ptlrpc_import_in_recovery(imp)); } spin_lock(&imp->imp_lock); diff --git a/drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c b/drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c index dad2f9290f70..437b4b2a9072 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c +++ b/drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c @@ -230,12 +230,13 @@ void ptlrpcd_add_req(struct ptlrpc_request *req) spin_lock(&req->rq_lock); if (req->rq_invalid_rqset) { - struct l_wait_info lwi = LWI_TIMEOUT(5 * HZ, - back_to_sleep, NULL); - req->rq_invalid_rqset = 0; spin_unlock(&req->rq_lock); - l_wait_event(req->rq_set_waitq, !req->rq_set, &lwi); + if (wait_event_idle_timeout(req->rq_set_waitq, + !req->rq_set, + 5 * HZ) == 0) + wait_event_idle(req->rq_set_waitq, + !req->rq_set); } else if (req->rq_set) { /* If we have a valid "rq_set", just reuse it to avoid double * linked. From neilb at suse.com Mon Feb 12 23:47:59 2018 From: neilb at suse.com (NeilBrown) Date: Tue, 13 Feb 2018 10:47:59 +1100 Subject: [lustre-devel] [PATCH 13/19] staging: lustre: use wait_event_idle_timeout in ptlrpcd() In-Reply-To: <151847037709.22826.16175867257667686132.stgit@noble> References: <151847037709.22826.16175867257667686132.stgit@noble> Message-ID: <151847927909.19840.2781972409839568492.stgit@noble> We can replace l_wait_event() with wait_event_idle_timeout() here providing we call the timeout function when wait_event_idle_timeout() returns zero. As ptlrpc_expired_set() returns 1, the l_wait_event() aborts of the first timeout. Reviewed-by: James Simmons Signed-off-by: NeilBrown --- drivers/staging/lustre/lustre/ptlrpc/client.c | 12 ++++++++---- .../staging/lustre/lustre/ptlrpc/ptlrpc_internal.h | 2 +- drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c | 9 +++++---- 3 files changed, 14 insertions(+), 9 deletions(-) diff --git a/drivers/staging/lustre/lustre/ptlrpc/client.c b/drivers/staging/lustre/lustre/ptlrpc/client.c index 81b7a7046d82..f70176c6db08 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/client.c +++ b/drivers/staging/lustre/lustre/ptlrpc/client.c @@ -2125,9 +2125,8 @@ int ptlrpc_expire_one_request(struct ptlrpc_request *req, int async_unlink) * Callback used when waiting on sets with l_wait_event. * Always returns 1. */ -int ptlrpc_expired_set(void *data) +void ptlrpc_expired_set(struct ptlrpc_request_set *set) { - struct ptlrpc_request_set *set = data; struct list_head *tmp; time64_t now = ktime_get_real_seconds(); @@ -2156,7 +2155,12 @@ int ptlrpc_expired_set(void *data) */ ptlrpc_expire_one_request(req, 1); } +} +static int ptlrpc_expired_set_void(void *data) +{ + struct ptlrpc_request_set *set = data; + ptlrpc_expired_set(set); /* * When waiting for a whole set, we always break out of the * sleep so we can recalculate the timeout, or enable interrupts @@ -2286,7 +2290,7 @@ int ptlrpc_set_wait(struct ptlrpc_request_set *set) * so we allow interrupts during the timeout. */ lwi = LWI_TIMEOUT_INTR_ALL(HZ, - ptlrpc_expired_set, + ptlrpc_expired_set_void, ptlrpc_interrupted_set, set); else /* @@ -2295,7 +2299,7 @@ int ptlrpc_set_wait(struct ptlrpc_request_set *set) * complete, or an in-flight req times out. */ lwi = LWI_TIMEOUT((timeout ? timeout : 1) * HZ, - ptlrpc_expired_set, set); + ptlrpc_expired_set_void, set); rc = l_wait_event(set->set_waitq, ptlrpc_check_set(NULL, set), &lwi); diff --git a/drivers/staging/lustre/lustre/ptlrpc/ptlrpc_internal.h b/drivers/staging/lustre/lustre/ptlrpc/ptlrpc_internal.h index f9decbd1459d..b7a8d7537a66 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/ptlrpc_internal.h +++ b/drivers/staging/lustre/lustre/ptlrpc/ptlrpc_internal.h @@ -68,7 +68,7 @@ void ptlrpc_request_cache_free(struct ptlrpc_request *req); void ptlrpc_init_xid(void); void ptlrpc_set_add_new_req(struct ptlrpcd_ctl *pc, struct ptlrpc_request *req); -int ptlrpc_expired_set(void *data); +void ptlrpc_expired_set(struct ptlrpc_request_set *set); int ptlrpc_set_next_timeout(struct ptlrpc_request_set *set); void ptlrpc_resend_req(struct ptlrpc_request *request); void ptlrpc_set_bulk_mbits(struct ptlrpc_request *req); diff --git a/drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c b/drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c index 437b4b2a9072..6ed77521d025 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c +++ b/drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c @@ -435,16 +435,17 @@ static int ptlrpcd(void *arg) * new_req_list and ptlrpcd_check() moves them into the set. */ do { - struct l_wait_info lwi; int timeout; timeout = ptlrpc_set_next_timeout(set); - lwi = LWI_TIMEOUT((timeout ? timeout : 1) * HZ, - ptlrpc_expired_set, set); lu_context_enter(&env.le_ctx); lu_context_enter(env.le_ses); - l_wait_event(set->set_waitq, ptlrpcd_check(&env, pc), &lwi); + if (wait_event_idle_timeout(set->set_waitq, + ptlrpcd_check(&env, pc), + (timeout ? timeout : 1) * HZ) == 0) + ptlrpc_expired_set(set); + lu_context_exit(&env.le_ctx); lu_context_exit(env.le_ses); From neilb at suse.com Mon Feb 12 23:47:59 2018 From: neilb at suse.com (NeilBrown) Date: Tue, 13 Feb 2018 10:47:59 +1100 Subject: [lustre-devel] [PATCH 14/19] staging: lustre: improve waiting in sptlrpc_req_refresh_ctx In-Reply-To: <151847037709.22826.16175867257667686132.stgit@noble> References: <151847037709.22826.16175867257667686132.stgit@noble> Message-ID: <151847927912.19840.9343278155250610050.stgit@noble> Replace l_wait_event with wait_event_idle_timeout() and call the handler function explicitly. This makes it more clear what is happening. Reviewed-by: James Simmons Signed-off-by: NeilBrown --- drivers/staging/lustre/lustre/ptlrpc/sec.c | 34 ++++++++++++++++++++-------- 1 file changed, 24 insertions(+), 10 deletions(-) diff --git a/drivers/staging/lustre/lustre/ptlrpc/sec.c b/drivers/staging/lustre/lustre/ptlrpc/sec.c index 617e004d00f8..90e3b3022106 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/sec.c +++ b/drivers/staging/lustre/lustre/ptlrpc/sec.c @@ -554,9 +554,8 @@ int ctx_check_refresh(struct ptlrpc_cli_ctx *ctx) } static -int ctx_refresh_timeout(void *data) +int ctx_refresh_timeout(struct ptlrpc_request *req) { - struct ptlrpc_request *req = data; int rc; /* conn_cnt is needed in expire_one_request */ @@ -575,10 +574,8 @@ int ctx_refresh_timeout(void *data) } static -void ctx_refresh_interrupt(void *data) +void ctx_refresh_interrupt(struct ptlrpc_request *req) { - struct ptlrpc_request *req = data; - spin_lock(&req->rq_lock); req->rq_intr = 1; spin_unlock(&req->rq_lock); @@ -611,7 +608,6 @@ int sptlrpc_req_refresh_ctx(struct ptlrpc_request *req, long timeout) { struct ptlrpc_cli_ctx *ctx = req->rq_cli_ctx; struct ptlrpc_sec *sec; - struct l_wait_info lwi; int rc; LASSERT(ctx); @@ -743,10 +739,28 @@ int sptlrpc_req_refresh_ctx(struct ptlrpc_request *req, long timeout) req->rq_restart = 0; spin_unlock(&req->rq_lock); - lwi = LWI_TIMEOUT_INTR(msecs_to_jiffies(timeout * MSEC_PER_SEC), - ctx_refresh_timeout, ctx_refresh_interrupt, - req); - rc = l_wait_event(req->rq_reply_waitq, ctx_check_refresh(ctx), &lwi); + rc = wait_event_idle_timeout(req->rq_reply_waitq, + ctx_check_refresh(ctx), + timeout * HZ); + if (rc == 0 && ctx_refresh_timeout(req) == 0) { + /* Keep waiting, but enable some signals */ + rc = l_wait_event_abortable(req->rq_reply_waitq, + ctx_check_refresh(ctx)); + if (rc == 0) + rc = 1; + } + + if (rc > 0) + /* condition is true */ + rc = 0; + else if (rc == 0) + /* Timed out */ + rc = -ETIMEDOUT; + else { + /* Aborted by signal */ + rc = -EINTR; + ctx_refresh_interrupt(req); + } /* * following cases could lead us here: From neilb at suse.com Mon Feb 12 23:47:59 2018 From: neilb at suse.com (NeilBrown) Date: Tue, 13 Feb 2018 10:47:59 +1100 Subject: [lustre-devel] [PATCH 17/19] staging: lustre: remove l_wait_event from ptlrpc_set_wait In-Reply-To: <151847037709.22826.16175867257667686132.stgit@noble> References: <151847037709.22826.16175867257667686132.stgit@noble> Message-ID: <151847927923.19840.5570834902527836784.stgit@noble> This is the last remaining use of l_wait_event(). It is the only use of LWI_TIMEOUT_INTR_ALL() which has a meaning that timeouts can be interrupted. Only interrupts by "fatal" signals are allowed, so introduce l_wait_event_abortable_timeout() to support this. Reviewed-by: James Simmons Signed-off-by: NeilBrown --- drivers/staging/lustre/lustre/include/lustre_lib.h | 14 +++ drivers/staging/lustre/lustre/ptlrpc/client.c | 84 ++++++++------------ 2 files changed, 49 insertions(+), 49 deletions(-) diff --git a/drivers/staging/lustre/lustre/include/lustre_lib.h b/drivers/staging/lustre/lustre/include/lustre_lib.h index 1939e959b92a..ccc1a329e42b 100644 --- a/drivers/staging/lustre/lustre/include/lustre_lib.h +++ b/drivers/staging/lustre/lustre/include/lustre_lib.h @@ -196,6 +196,10 @@ struct l_wait_info { #define LUSTRE_FATAL_SIGS (sigmask(SIGKILL) | sigmask(SIGINT) | \ sigmask(SIGTERM) | sigmask(SIGQUIT) | \ sigmask(SIGALRM)) +static inline int l_fatal_signal_pending(struct task_struct *p) +{ + return signal_pending(p) && sigtestsetmask(&p->pending.signal, LUSTRE_FATAL_SIGS); +} /** * wait_queue_entry_t of Linux (version < 2.6.34) is a FIFO list for exclusively @@ -347,6 +351,16 @@ do { \ __ret; \ }) +#define l_wait_event_abortable_timeout(wq, condition, timeout) \ +({ \ + sigset_t __blocked; \ + int __ret = 0; \ + __blocked = cfs_block_sigsinv(LUSTRE_FATAL_SIGS); \ + __ret = wait_event_interruptible_timeout(wq, condition, timeout);\ + cfs_restore_sigs(__blocked); \ + __ret; \ +}) + #define l_wait_event_abortable_exclusive(wq, condition) \ ({ \ sigset_t __blocked; \ diff --git a/drivers/staging/lustre/lustre/ptlrpc/client.c b/drivers/staging/lustre/lustre/ptlrpc/client.c index ffdd3ffd62c6..3d689d6100bc 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/client.c +++ b/drivers/staging/lustre/lustre/ptlrpc/client.c @@ -1774,7 +1774,7 @@ int ptlrpc_check_set(const struct lu_env *env, struct ptlrpc_request_set *set) } /* - * ptlrpc_set_wait->l_wait_event sets lwi_allow_intr + * ptlrpc_set_wait allow signal to abort the timeout * so it sets rq_intr regardless of individual rpc * timeouts. The synchronous IO waiting path sets * rq_intr irrespective of whether ptlrpcd @@ -2122,8 +2122,7 @@ int ptlrpc_expire_one_request(struct ptlrpc_request *req, int async_unlink) /** * Time out all uncompleted requests in request set pointed by \a data - * Callback used when waiting on sets with l_wait_event. - * Always returns 1. + * Called when wait_event_idle_timeout times out. */ void ptlrpc_expired_set(struct ptlrpc_request_set *set) { @@ -2156,18 +2155,6 @@ void ptlrpc_expired_set(struct ptlrpc_request_set *set) ptlrpc_expire_one_request(req, 1); } } -static int ptlrpc_expired_set_void(void *data) -{ - struct ptlrpc_request_set *set = data; - - ptlrpc_expired_set(set); - /* - * When waiting for a whole set, we always break out of the - * sleep so we can recalculate the timeout, or enable interrupts - * if everyone's timed out. - */ - return 1; -} /** * Sets rq_intr flag in \a req under spinlock. @@ -2182,11 +2169,10 @@ EXPORT_SYMBOL(ptlrpc_mark_interrupted); /** * Interrupts (sets interrupted flag) all uncompleted requests in - * a set \a data. Callback for l_wait_event for interruptible waits. + * a set \a data. Called when l_wait_event_abortable_timeout receives signal. */ -static void ptlrpc_interrupted_set(void *data) +static void ptlrpc_interrupted_set(struct ptlrpc_request_set *set) { - struct ptlrpc_request_set *set = data; struct list_head *tmp; CDEBUG(D_RPCTRACE, "INTERRUPTED SET %p\n", set); @@ -2256,7 +2242,6 @@ int ptlrpc_set_wait(struct ptlrpc_request_set *set) { struct list_head *tmp; struct ptlrpc_request *req; - struct l_wait_info lwi; int rc, timeout; if (set->set_producer) @@ -2282,46 +2267,47 @@ int ptlrpc_set_wait(struct ptlrpc_request_set *set) CDEBUG(D_RPCTRACE, "set %p going to sleep for %d seconds\n", set, timeout); - if (timeout == 0 && !signal_pending(current)) + if (timeout == 0 && !signal_pending(current)) { /* * No requests are in-flight (ether timed out * or delayed), so we can allow interrupts. * We still want to block for a limited time, * so we allow interrupts during the timeout. */ - lwi = LWI_TIMEOUT_INTR_ALL(HZ, - ptlrpc_expired_set_void, - ptlrpc_interrupted_set, set); - else + rc = l_wait_event_abortable_timeout(set->set_waitq, + ptlrpc_check_set(NULL, set), + HZ); + if (rc == 0) { + rc = -ETIMEDOUT; + ptlrpc_expired_set(set); + } else if (rc < 0) { + rc = -EINTR; + ptlrpc_interrupted_set(set); + } else + rc = 0; + } else { /* * At least one request is in flight, so no * interrupts are allowed. Wait until all * complete, or an in-flight req times out. */ - lwi = LWI_TIMEOUT((timeout ? timeout : 1) * HZ, - ptlrpc_expired_set_void, set); - - rc = l_wait_event(set->set_waitq, ptlrpc_check_set(NULL, set), &lwi); - - /* - * LU-769 - if we ignored the signal because it was already - * pending when we started, we need to handle it now or we risk - * it being ignored forever - */ - if (rc == -ETIMEDOUT && !lwi.lwi_allow_intr && - signal_pending(current)) { - sigset_t blocked_sigs = - cfs_block_sigsinv(LUSTRE_FATAL_SIGS); - - /* - * In fact we only interrupt for the "fatal" signals - * like SIGINT or SIGKILL. We still ignore less - * important signals since ptlrpc set is not easily - * reentrant from userspace again - */ - if (signal_pending(current)) - ptlrpc_interrupted_set(set); - cfs_restore_sigs(blocked_sigs); + rc = wait_event_idle_timeout(set->set_waitq, + ptlrpc_check_set(NULL, set), + (timeout ? timeout : 1) * HZ); + if (rc == 0) { + ptlrpc_expired_set(set); + rc = -ETIMEDOUT; + /* + * LU-769 - if we ignored the signal + * because it was already pending when + * we started, we need to handle it + * now or we risk it being ignored + * forever + */ + if (l_fatal_signal_pending(current)) + ptlrpc_interrupted_set(set); + } else + rc = 0; } LASSERT(rc == 0 || rc == -EINTR || rc == -ETIMEDOUT); @@ -2528,7 +2514,7 @@ static int ptlrpc_unregister_reply(struct ptlrpc_request *request, int async) return 0; /* - * We have to l_wait_event() whatever the result, to give liblustre + * We have to wait_event_idle_timeout() whatever the result, to give liblustre * a chance to run reply_in_callback(), and to make sure we've * unlinked before returning a req to the pool. */ From neilb at suse.com Mon Feb 12 23:47:59 2018 From: neilb at suse.com (NeilBrown) Date: Tue, 13 Feb 2018 10:47:59 +1100 Subject: [lustre-devel] [PATCH 16/19] staging: lustre: use explicit poll loop in ptlrpc_unregister_reply In-Reply-To: <151847037709.22826.16175867257667686132.stgit@noble> References: <151847037709.22826.16175867257667686132.stgit@noble> Message-ID: <151847927920.19840.8201423653881194896.stgit@noble> replace l_wait_event() with wait_event_idle_timeout() and explicit loop. This approach is easier to understand. Reviewed-by: James Simmons Signed-off-by: NeilBrown --- drivers/staging/lustre/lustre/ptlrpc/client.c | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/drivers/staging/lustre/lustre/ptlrpc/client.c b/drivers/staging/lustre/lustre/ptlrpc/client.c index f70176c6db08..ffdd3ffd62c6 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/client.c +++ b/drivers/staging/lustre/lustre/ptlrpc/client.c @@ -2500,7 +2500,6 @@ static int ptlrpc_unregister_reply(struct ptlrpc_request *request, int async) { int rc; wait_queue_head_t *wq; - struct l_wait_info lwi; /* Might sleep. */ LASSERT(!in_interrupt()); @@ -2543,16 +2542,17 @@ static int ptlrpc_unregister_reply(struct ptlrpc_request *request, int async) * Network access will complete in finite time but the HUGE * timeout lets us CWARN for visibility of sluggish NALs */ - lwi = LWI_TIMEOUT_INTERVAL(LONG_UNLINK * HZ, - HZ, NULL, NULL); - rc = l_wait_event(*wq, !ptlrpc_client_recv_or_unlink(request), - &lwi); - if (rc == 0) { + int cnt = 0; + while (cnt < LONG_UNLINK && + (rc = wait_event_idle_timeout(*wq, + !ptlrpc_client_recv_or_unlink(request), + HZ)) == 0) + cnt += 1; + if (rc > 0) { ptlrpc_rqphase_move(request, request->rq_next_phase); return 1; } - LASSERT(rc == -ETIMEDOUT); DEBUG_REQ(D_WARNING, request, "Unexpectedly long timeout receiving_reply=%d req_ulinked=%d reply_unlinked=%d", request->rq_receiving_reply, From neilb at suse.com Mon Feb 12 23:47:59 2018 From: neilb at suse.com (NeilBrown) Date: Tue, 13 Feb 2018 10:47:59 +1100 Subject: [lustre-devel] [PATCH 18/19] staging: lustre: replace l_wait_event_exclusive_head() with wait_event_idle_exclusive In-Reply-To: <151847037709.22826.16175867257667686132.stgit@noble> References: <151847037709.22826.16175867257667686132.stgit@noble> Message-ID: <151847927926.19840.9872301796680320791.stgit@noble> This l_wait_event_exclusive_head() will wait indefinitely if the timeout is zero. If it does wait with a timeout and times out, the timeout for next time is set to zero. The can be mapped to a call to either wait_event_idle_exclusive() or wait_event_idle_exclusive_timeout() depending in the timeout setting. The current code arranges for LIFO queuing of waiters, but include/event.h doesn't support that yet. Until it does, fall back on FIFO with wait_event_idle_exclusive{,_timeout}(). Reviewed-by: James Simmons Signed-off-by: NeilBrown --- drivers/staging/lustre/lustre/ptlrpc/service.c | 43 ++++++++++++++---------- 1 file changed, 25 insertions(+), 18 deletions(-) diff --git a/drivers/staging/lustre/lustre/ptlrpc/service.c b/drivers/staging/lustre/lustre/ptlrpc/service.c index 6e3403417434..29fdb54f16ca 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/service.c +++ b/drivers/staging/lustre/lustre/ptlrpc/service.c @@ -1897,15 +1897,6 @@ ptlrpc_check_rqbd_pool(struct ptlrpc_service_part *svcpt) } } -static int -ptlrpc_retry_rqbds(void *arg) -{ - struct ptlrpc_service_part *svcpt = arg; - - svcpt->scp_rqbd_timeout = 0; - return -ETIMEDOUT; -} - static inline int ptlrpc_threads_enough(struct ptlrpc_service_part *svcpt) { @@ -1968,13 +1959,17 @@ ptlrpc_server_request_incoming(struct ptlrpc_service_part *svcpt) return !list_empty(&svcpt->scp_req_incoming); } +/* We perfer lifo queuing, but kernel doesn't provide that yet. */ +#ifndef wait_event_idle_exclusive_lifo +#define wait_event_idle_exclusive_lifo wait_event_idle_exclusive +#define wait_event_idle_exclusive_lifo_timeout wait_event_idle_exclusive_timeout +#endif + static __attribute__((__noinline__)) int ptlrpc_wait_event(struct ptlrpc_service_part *svcpt, struct ptlrpc_thread *thread) { /* Don't exit while there are replies to be handled */ - struct l_wait_info lwi = LWI_TIMEOUT(svcpt->scp_rqbd_timeout, - ptlrpc_retry_rqbds, svcpt); /* XXX: Add this back when libcfs watchdog is merged upstream lc_watchdog_disable(thread->t_watchdog); @@ -1982,13 +1977,25 @@ ptlrpc_wait_event(struct ptlrpc_service_part *svcpt, cond_resched(); - l_wait_event_exclusive_head(svcpt->scp_waitq, - ptlrpc_thread_stopping(thread) || - ptlrpc_server_request_incoming(svcpt) || - ptlrpc_server_request_pending(svcpt, - false) || - ptlrpc_rqbd_pending(svcpt) || - ptlrpc_at_check(svcpt), &lwi); + if (svcpt->scp_rqbd_timeout == 0) + wait_event_idle_exclusive_lifo( + svcpt->scp_waitq, + ptlrpc_thread_stopping(thread) || + ptlrpc_server_request_incoming(svcpt) || + ptlrpc_server_request_pending(svcpt, + false) || + ptlrpc_rqbd_pending(svcpt) || + ptlrpc_at_check(svcpt)); + else if (0 == wait_event_idle_exclusive_lifo_timeout( + svcpt->scp_waitq, + ptlrpc_thread_stopping(thread) || + ptlrpc_server_request_incoming(svcpt) || + ptlrpc_server_request_pending(svcpt, + false) || + ptlrpc_rqbd_pending(svcpt) || + ptlrpc_at_check(svcpt), + svcpt->scp_rqbd_timeout)) + svcpt->scp_rqbd_timeout = 0; if (ptlrpc_thread_stopping(thread)) return -EINTR; From neilb at suse.com Mon Feb 12 23:47:59 2018 From: neilb at suse.com (NeilBrown) Date: Tue, 13 Feb 2018 10:47:59 +1100 Subject: [lustre-devel] [PATCH 15/19] staging: lustre: use explicit poll loop in ptlrpc_service_unlink_rqbd In-Reply-To: <151847037709.22826.16175867257667686132.stgit@noble> References: <151847037709.22826.16175867257667686132.stgit@noble> Message-ID: <151847927917.19840.6241020862557231833.stgit@noble> Rather an using l_wait_event(), use wait_event_idle_timeout() with an explicit loop so it is easier to see what is happening. Reviewed-by: James Simmons Signed-off-by: NeilBrown --- drivers/staging/lustre/lustre/ptlrpc/service.c | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/drivers/staging/lustre/lustre/ptlrpc/service.c b/drivers/staging/lustre/lustre/ptlrpc/service.c index 5c41297d23d2..6e3403417434 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/service.c +++ b/drivers/staging/lustre/lustre/ptlrpc/service.c @@ -2618,7 +2618,7 @@ ptlrpc_service_unlink_rqbd(struct ptlrpc_service *svc) { struct ptlrpc_service_part *svcpt; struct ptlrpc_request_buffer_desc *rqbd; - struct l_wait_info lwi; + int cnt; int rc; int i; @@ -2658,12 +2658,13 @@ ptlrpc_service_unlink_rqbd(struct ptlrpc_service *svc) * the HUGE timeout lets us CWARN for visibility * of sluggish LNDs */ - lwi = LWI_TIMEOUT_INTERVAL( - LONG_UNLINK * HZ, - HZ, NULL, NULL); - rc = l_wait_event(svcpt->scp_waitq, - svcpt->scp_nrqbds_posted == 0, &lwi); - if (rc == -ETIMEDOUT) { + cnt = 0; + while (cnt < LONG_UNLINK && + (rc = wait_event_idle_timeout(svcpt->scp_waitq, + svcpt->scp_nrqbds_posted == 0, + HZ)) == 0) + cnt ++; + if (rc == 0) { CWARN("Service %s waiting for request buffers\n", svcpt->scp_service->srv_name); } From neilb at suse.com Mon Feb 12 23:47:59 2018 From: neilb at suse.com (NeilBrown) Date: Tue, 13 Feb 2018 10:47:59 +1100 Subject: [lustre-devel] [PATCH 19/19] staging: lustre: remove l_wait_event() and related code In-Reply-To: <151847037709.22826.16175867257667686132.stgit@noble> References: <151847037709.22826.16175867257667686132.stgit@noble> Message-ID: <151847927929.19840.14894766883488647956.stgit@noble> These macros are no longer used, so they can be removed. Reviewed-by: James Simmons Signed-off-by: NeilBrown --- drivers/staging/lustre/lustre/include/lustre_lib.h | 249 -------------------- 1 file changed, 249 deletions(-) diff --git a/drivers/staging/lustre/lustre/include/lustre_lib.h b/drivers/staging/lustre/lustre/include/lustre_lib.h index ccc1a329e42b..1efd86f18c1f 100644 --- a/drivers/staging/lustre/lustre/include/lustre_lib.h +++ b/drivers/staging/lustre/lustre/include/lustre_lib.h @@ -76,123 +76,6 @@ int do_set_info_async(struct obd_import *imp, void target_send_reply(struct ptlrpc_request *req, int rc, int fail_id); -/* - * l_wait_event is a flexible sleeping function, permitting simple caller - * configuration of interrupt and timeout sensitivity along with actions to - * be performed in the event of either exception. - * - * The first form of usage looks like this: - * - * struct l_wait_info lwi = LWI_TIMEOUT_INTR(timeout, timeout_handler, - * intr_handler, callback_data); - * rc = l_wait_event(waitq, condition, &lwi); - * - * l_wait_event() makes the current process wait on 'waitq' until 'condition' - * is TRUE or a "killable" signal (SIGTERM, SIKGILL, SIGINT) is pending. It - * returns 0 to signify 'condition' is TRUE, but if a signal wakes it before - * 'condition' becomes true, it optionally calls the specified 'intr_handler' - * if not NULL, and returns -EINTR. - * - * If a non-zero timeout is specified, signals are ignored until the timeout - * has expired. At this time, if 'timeout_handler' is not NULL it is called. - * If it returns FALSE l_wait_event() continues to wait as described above with - * signals enabled. Otherwise it returns -ETIMEDOUT. - * - * LWI_INTR(intr_handler, callback_data) is shorthand for - * LWI_TIMEOUT_INTR(0, NULL, intr_handler, callback_data) - * - * The second form of usage looks like this: - * - * struct l_wait_info lwi = LWI_TIMEOUT(timeout, timeout_handler); - * rc = l_wait_event(waitq, condition, &lwi); - * - * This form is the same as the first except that it COMPLETELY IGNORES - * SIGNALS. The caller must therefore beware that if 'timeout' is zero, or if - * 'timeout_handler' is not NULL and returns FALSE, then the ONLY thing that - * can unblock the current process is 'condition' becoming TRUE. - * - * Another form of usage is: - * struct l_wait_info lwi = LWI_TIMEOUT_INTERVAL(timeout, interval, - * timeout_handler); - * rc = l_wait_event(waitq, condition, &lwi); - * This is the same as previous case, but condition is checked once every - * 'interval' jiffies (if non-zero). - * - * Subtle synchronization point: this macro does *not* necessary takes - * wait-queue spin-lock before returning, and, hence, following idiom is safe - * ONLY when caller provides some external locking: - * - * Thread1 Thread2 - * - * l_wait_event(&obj->wq, ....); (1) - * - * wake_up(&obj->wq): (2) - * spin_lock(&q->lock); (2.1) - * __wake_up_common(q, ...); (2.2) - * spin_unlock(&q->lock, flags); (2.3) - * - * kfree(obj); (3) - * - * As l_wait_event() may "short-cut" execution and return without taking - * wait-queue spin-lock, some additional synchronization is necessary to - * guarantee that step (3) can begin only after (2.3) finishes. - * - * XXX nikita: some ptlrpc daemon threads have races of that sort. - * - */ - -#define LWI_ON_SIGNAL_NOOP ((void (*)(void *))(-1)) - -struct l_wait_info { - long lwi_timeout; - long lwi_interval; - int lwi_allow_intr; - int (*lwi_on_timeout)(void *); - void (*lwi_on_signal)(void *); - void *lwi_cb_data; -}; - -/* NB: LWI_TIMEOUT ignores signals completely */ -#define LWI_TIMEOUT(time, cb, data) \ -((struct l_wait_info) { \ - .lwi_timeout = time, \ - .lwi_on_timeout = cb, \ - .lwi_cb_data = data, \ - .lwi_interval = 0, \ - .lwi_allow_intr = 0 \ -}) - -#define LWI_TIMEOUT_INTERVAL(time, interval, cb, data) \ -((struct l_wait_info) { \ - .lwi_timeout = time, \ - .lwi_on_timeout = cb, \ - .lwi_cb_data = data, \ - .lwi_interval = interval, \ - .lwi_allow_intr = 0 \ -}) - -#define LWI_TIMEOUT_INTR(time, time_cb, sig_cb, data) \ -((struct l_wait_info) { \ - .lwi_timeout = time, \ - .lwi_on_timeout = time_cb, \ - .lwi_on_signal = sig_cb, \ - .lwi_cb_data = data, \ - .lwi_interval = 0, \ - .lwi_allow_intr = 0 \ -}) - -#define LWI_TIMEOUT_INTR_ALL(time, time_cb, sig_cb, data) \ -((struct l_wait_info) { \ - .lwi_timeout = time, \ - .lwi_on_timeout = time_cb, \ - .lwi_on_signal = sig_cb, \ - .lwi_cb_data = data, \ - .lwi_interval = 0, \ - .lwi_allow_intr = 1 \ -}) - -#define LWI_INTR(cb, data) LWI_TIMEOUT_INTR(0, NULL, cb, data) - #define LUSTRE_FATAL_SIGS (sigmask(SIGKILL) | sigmask(SIGINT) | \ sigmask(SIGTERM) | sigmask(SIGQUIT) | \ sigmask(SIGALRM)) @@ -201,138 +84,6 @@ static inline int l_fatal_signal_pending(struct task_struct *p) return signal_pending(p) && sigtestsetmask(&p->pending.signal, LUSTRE_FATAL_SIGS); } -/** - * wait_queue_entry_t of Linux (version < 2.6.34) is a FIFO list for exclusively - * waiting threads, which is not always desirable because all threads will - * be waken up again and again, even user only needs a few of them to be - * active most time. This is not good for performance because cache can - * be polluted by different threads. - * - * LIFO list can resolve this problem because we always wakeup the most - * recent active thread by default. - * - * NB: please don't call non-exclusive & exclusive wait on the same - * waitq if add_wait_queue_exclusive_head is used. - */ -#define add_wait_queue_exclusive_head(waitq, link) \ -{ \ - unsigned long flags; \ - \ - spin_lock_irqsave(&((waitq)->lock), flags); \ - __add_wait_queue_exclusive(waitq, link); \ - spin_unlock_irqrestore(&((waitq)->lock), flags); \ -} - -/* - * wait for @condition to become true, but no longer than timeout, specified - * by @info. - */ -#define __l_wait_event(wq, condition, info, ret, l_add_wait) \ -do { \ - wait_queue_entry_t __wait; \ - long __timeout = info->lwi_timeout; \ - sigset_t __blocked; \ - int __allow_intr = info->lwi_allow_intr; \ - \ - ret = 0; \ - if (condition) \ - break; \ - \ - init_waitqueue_entry(&__wait, current); \ - l_add_wait(&wq, &__wait); \ - \ - /* Block all signals (just the non-fatal ones if no timeout). */ \ - if (info->lwi_on_signal && (__timeout == 0 || __allow_intr)) \ - __blocked = cfs_block_sigsinv(LUSTRE_FATAL_SIGS); \ - else \ - __blocked = cfs_block_sigsinv(0); \ - \ - for (;;) { \ - if (condition) \ - break; \ - \ - set_current_state(TASK_INTERRUPTIBLE); \ - \ - if (__timeout == 0) { \ - schedule(); \ - } else { \ - long interval = info->lwi_interval ? \ - min_t(long, \ - info->lwi_interval, __timeout) : \ - __timeout; \ - long remaining = schedule_timeout(interval);\ - __timeout = cfs_time_sub(__timeout, \ - cfs_time_sub(interval, remaining));\ - if (__timeout == 0) { \ - if (!info->lwi_on_timeout || \ - info->lwi_on_timeout(info->lwi_cb_data)) { \ - ret = -ETIMEDOUT; \ - break; \ - } \ - /* Take signals after the timeout expires. */ \ - if (info->lwi_on_signal) \ - (void)cfs_block_sigsinv(LUSTRE_FATAL_SIGS);\ - } \ - } \ - \ - set_current_state(TASK_RUNNING); \ - \ - if (condition) \ - break; \ - if (signal_pending(current)) { \ - if (info->lwi_on_signal && \ - (__timeout == 0 || __allow_intr)) { \ - if (info->lwi_on_signal != LWI_ON_SIGNAL_NOOP) \ - info->lwi_on_signal(info->lwi_cb_data);\ - ret = -EINTR; \ - break; \ - } \ - /* We have to do this here because some signals */ \ - /* are not blockable - ie from strace(1). */ \ - /* In these cases we want to schedule_timeout() */ \ - /* again, because we don't want that to return */ \ - /* -EINTR when the RPC actually succeeded. */ \ - /* the recalc_sigpending() below will deliver the */ \ - /* signal properly. */ \ - cfs_clear_sigpending(); \ - } \ - } \ - \ - cfs_restore_sigs(__blocked); \ - \ - remove_wait_queue(&wq, &__wait); \ -} while (0) - -#define l_wait_event(wq, condition, info) \ -({ \ - int __ret; \ - struct l_wait_info *__info = (info); \ - \ - __l_wait_event(wq, condition, __info, \ - __ret, add_wait_queue); \ - __ret; \ -}) - -#define l_wait_event_exclusive(wq, condition, info) \ -({ \ - int __ret; \ - struct l_wait_info *__info = (info); \ - \ - __l_wait_event(wq, condition, __info, \ - __ret, add_wait_queue_exclusive); \ - __ret; \ -}) - -#define l_wait_event_exclusive_head(wq, condition, info) \ -({ \ - int __ret; \ - struct l_wait_info *__info = (info); \ - \ - __l_wait_event(wq, condition, __info, \ - __ret, add_wait_queue_exclusive_head); \ - __ret; \ -}) - /** @} lib */ From neilb at suse.com Mon Feb 12 23:58:09 2018 From: neilb at suse.com (NeilBrown) Date: Tue, 13 Feb 2018 10:58:09 +1100 Subject: [lustre-devel] [PATCH 06/19] staging: lustre: introduce and use l_wait_event_abortable() In-Reply-To: <81AFF1FD-E398-4B69-9B20-1D202747AB25@cray.com> References: <151847037709.22826.16175867257667686132.stgit@noble> <151847055667.22826.8544979478231411275.stgit@noble> <81AFF1FD-E398-4B69-9B20-1D202747AB25@cray.com> Message-ID: <87r2pprca6.fsf@notabene.neil.brown.name> On Mon, Feb 12 2018, Patrick Farrell wrote: > It's worth noting that the change from -EINTR to -ERESTARTSYS will > modify the behavior of userspace slightly. Specifically, when a > signal handler is setup with retry set (SA_RESTART flag set), the > syscall will be restarted rather than aborted. Thanks for the review. This state is true if the error status ever gets all the way up to user space. I don't think it does, though I haven't carefully audited every l_wait_event_abortable() call site. I did look at a few and the return value is only used locally in the same function. I suspect I would have checked for error codes escaping when I wrote the patch, but I don't have a clear memory. Some sort of audit wouldn't hurt of course. Thanks, NeilBrown > > This should be fine. It may eventually shake out some stuble bugs in Lustre when we abort an operation and then restart it from a point where we didn't in the past - past instances where we changed from -EINTR to -ERESTARTSYS certainly have - but it's for the best. As I understand it from past conversations with Andreas and others, Lustre is not really intending to claim it's not restartable, it's just an artifact of people copying older code that was written without awareness of the difference in EINTR vs ERESTARTSYS. > > Ideally someone should go through and audit the remaining uses of -EINTR and replace most of them with -ERESTARTSYS. > > James, maybe you want to add that to the TODO list? > > On 2/12/18, 3:24 PM, "lustre-devel on behalf of NeilBrown" wrote: > > lustre sometimes wants to wait for an event, but abort if > one of a specific list of signals arrives. This is a little > bit like wait_event_killable(), except that the signals are > identified a different way. > > So introduce l_wait_event_abortable() which provides this > functionality. > Having separate functions for separate needs is more in line > with the pattern set by include/linux/wait.h, than having a > single function which tries to include all possible needs. > > Also introduce l_wait_event_abortable_exclusive(). > > Note that l_wait_event() return -EINTR on a signal, while > Linux wait_event functions return -ERESTARTSYS. > l_wait_event_{abortable_,}exclusive follow the Linux pattern. > > Reviewed-by: James Simmons > Signed-off-by: NeilBrown > --- > drivers/staging/lustre/lustre/include/lustre_lib.h | 24 ++++++++++++++++++++ > drivers/staging/lustre/lustre/ldlm/ldlm_resource.c | 12 +++++----- > drivers/staging/lustre/lustre/llite/llite_lib.c | 12 +++------- > drivers/staging/lustre/lustre/obdclass/genops.c | 9 +++----- > drivers/staging/lustre/lustre/obdclass/llog_obd.c | 5 ++-- > drivers/staging/lustre/lustre/osc/osc_page.c | 6 ++--- > drivers/staging/lustre/lustre/osc/osc_request.c | 6 ++--- > 7 files changed, 43 insertions(+), 31 deletions(-) > > diff --git a/drivers/staging/lustre/lustre/include/lustre_lib.h b/drivers/staging/lustre/lustre/include/lustre_lib.h > index 7d950c53e962..b2a64d0e682c 100644 > --- a/drivers/staging/lustre/lustre/include/lustre_lib.h > +++ b/drivers/staging/lustre/lustre/include/lustre_lib.h > @@ -336,4 +336,28 @@ do { \ > /** @} lib */ > > > + > +/* l_wait_event_abortable() is a bit like wait_event_killable() > + * except there is a fixed set of signals which will abort: > + * LUSTRE_FATAL_SIGS > + */ > +#define l_wait_event_abortable(wq, condition) \ > +({ \ > + sigset_t __blocked; \ > + int __ret = 0; \ > + __blocked = cfs_block_sigsinv(LUSTRE_FATAL_SIGS); \ > + __ret = wait_event_interruptible(wq, condition); \ > + cfs_restore_sigs(__blocked); \ > + __ret; \ > +}) > + > +#define l_wait_event_abortable_exclusive(wq, condition) \ > +({ \ > + sigset_t __blocked; \ > + int __ret = 0; \ > + __blocked = cfs_block_sigsinv(LUSTRE_FATAL_SIGS); \ > + __ret = wait_event_interruptible_exclusive(wq, condition); \ > + cfs_restore_sigs(__blocked); \ > + __ret; \ > +}) > #endif /* _LUSTRE_LIB_H */ > diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_resource.c b/drivers/staging/lustre/lustre/ldlm/ldlm_resource.c > index 2e66825c8f4b..4c44603ab6f9 100644 > --- a/drivers/staging/lustre/lustre/ldlm/ldlm_resource.c > +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_resource.c > @@ -879,7 +879,6 @@ static int __ldlm_namespace_free(struct ldlm_namespace *ns, int force) > ldlm_namespace_cleanup(ns, force ? LDLM_FL_LOCAL_ONLY : 0); > > if (atomic_read(&ns->ns_bref) > 0) { > - struct l_wait_info lwi = LWI_INTR(LWI_ON_SIGNAL_NOOP, NULL); > int rc; > > CDEBUG(D_DLMTRACE, > @@ -887,11 +886,12 @@ static int __ldlm_namespace_free(struct ldlm_namespace *ns, int force) > ldlm_ns_name(ns), atomic_read(&ns->ns_bref)); > force_wait: > if (force) > - lwi = LWI_TIMEOUT(msecs_to_jiffies(obd_timeout * > - MSEC_PER_SEC) / 4, NULL, NULL); > - > - rc = l_wait_event(ns->ns_waitq, > - atomic_read(&ns->ns_bref) == 0, &lwi); > + rc = wait_event_idle_timeout(ns->ns_waitq, > + atomic_read(&ns->ns_bref) == 0, > + obd_timeout * HZ / 4) ? 0 : -ETIMEDOUT; > + else > + rc = l_wait_event_abortable(ns->ns_waitq, > + atomic_read(&ns->ns_bref) == 0); > > /* Forced cleanups should be able to reclaim all references, > * so it's safe to wait forever... we can't leak locks... > diff --git a/drivers/staging/lustre/lustre/llite/llite_lib.c b/drivers/staging/lustre/lustre/llite/llite_lib.c > index c820b201af71..ccb614bd7f53 100644 > --- a/drivers/staging/lustre/lustre/llite/llite_lib.c > +++ b/drivers/staging/lustre/lustre/llite/llite_lib.c > @@ -986,16 +986,12 @@ void ll_put_super(struct super_block *sb) > } > > /* Wait for unstable pages to be committed to stable storage */ > - if (!force) { > - struct l_wait_info lwi = LWI_INTR(LWI_ON_SIGNAL_NOOP, NULL); > - > - rc = l_wait_event(sbi->ll_cache->ccc_unstable_waitq, > - !atomic_long_read(&sbi->ll_cache->ccc_unstable_nr), > - &lwi); > - } > + if (!force) > + rc = l_wait_event_abortable(sbi->ll_cache->ccc_unstable_waitq, > + !atomic_long_read(&sbi->ll_cache->ccc_unstable_nr)); > > ccc_count = atomic_long_read(&sbi->ll_cache->ccc_unstable_nr); > - if (!force && rc != -EINTR) > + if (!force && rc != -ERESTARTSYS) > LASSERTF(!ccc_count, "count: %li\n", ccc_count); > > /* We need to set force before the lov_disconnect in > diff --git a/drivers/staging/lustre/lustre/obdclass/genops.c b/drivers/staging/lustre/lustre/obdclass/genops.c > index 3ff25b8d3b48..8f776a4058a9 100644 > --- a/drivers/staging/lustre/lustre/obdclass/genops.c > +++ b/drivers/staging/lustre/lustre/obdclass/genops.c > @@ -1332,7 +1332,6 @@ static bool obd_request_slot_avail(struct client_obd *cli, > int obd_get_request_slot(struct client_obd *cli) > { > struct obd_request_slot_waiter orsw; > - struct l_wait_info lwi; > int rc; > > spin_lock(&cli->cl_loi_list_lock); > @@ -1347,11 +1346,9 @@ int obd_get_request_slot(struct client_obd *cli) > orsw.orsw_signaled = false; > spin_unlock(&cli->cl_loi_list_lock); > > - lwi = LWI_INTR(LWI_ON_SIGNAL_NOOP, NULL); > - rc = l_wait_event(orsw.orsw_waitq, > - obd_request_slot_avail(cli, &orsw) || > - orsw.orsw_signaled, > - &lwi); > + rc = l_wait_event_abortable(orsw.orsw_waitq, > + obd_request_slot_avail(cli, &orsw) || > + orsw.orsw_signaled); > > /* > * Here, we must take the lock to avoid the on-stack 'orsw' to be > diff --git a/drivers/staging/lustre/lustre/obdclass/llog_obd.c b/drivers/staging/lustre/lustre/obdclass/llog_obd.c > index 28bbaa2136ac..26aea114a29b 100644 > --- a/drivers/staging/lustre/lustre/obdclass/llog_obd.c > +++ b/drivers/staging/lustre/lustre/obdclass/llog_obd.c > @@ -104,7 +104,6 @@ EXPORT_SYMBOL(__llog_ctxt_put); > > int llog_cleanup(const struct lu_env *env, struct llog_ctxt *ctxt) > { > - struct l_wait_info lwi = LWI_INTR(LWI_ON_SIGNAL_NOOP, NULL); > struct obd_llog_group *olg; > int rc, idx; > > @@ -129,8 +128,8 @@ int llog_cleanup(const struct lu_env *env, struct llog_ctxt *ctxt) > CERROR("Error %d while cleaning up ctxt %p\n", > rc, ctxt); > > - l_wait_event(olg->olg_waitq, > - llog_group_ctxt_null(olg, idx), &lwi); > + l_wait_event_abortable(olg->olg_waitq, > + llog_group_ctxt_null(olg, idx)); > > return rc; > } > diff --git a/drivers/staging/lustre/lustre/osc/osc_page.c b/drivers/staging/lustre/lustre/osc/osc_page.c > index 20094b6309f9..6fdd521feb21 100644 > --- a/drivers/staging/lustre/lustre/osc/osc_page.c > +++ b/drivers/staging/lustre/lustre/osc/osc_page.c > @@ -759,7 +759,6 @@ static long osc_lru_reclaim(struct client_obd *cli, unsigned long npages) > static int osc_lru_alloc(const struct lu_env *env, struct client_obd *cli, > struct osc_page *opg) > { > - struct l_wait_info lwi = LWI_INTR(LWI_ON_SIGNAL_NOOP, NULL); > struct osc_io *oio = osc_env_io(env); > int rc = 0; > > @@ -782,9 +781,8 @@ static int osc_lru_alloc(const struct lu_env *env, struct client_obd *cli, > > cond_resched(); > > - rc = l_wait_event(osc_lru_waitq, > - atomic_long_read(cli->cl_lru_left) > 0, > - &lwi); > + rc = l_wait_event_abortable(osc_lru_waitq, > + atomic_long_read(cli->cl_lru_left) > 0); > > if (rc < 0) > break; > diff --git a/drivers/staging/lustre/lustre/osc/osc_request.c b/drivers/staging/lustre/lustre/osc/osc_request.c > index 45b1ebf33363..074b5ce6284c 100644 > --- a/drivers/staging/lustre/lustre/osc/osc_request.c > +++ b/drivers/staging/lustre/lustre/osc/osc_request.c > @@ -552,14 +552,12 @@ static int osc_destroy(const struct lu_env *env, struct obd_export *exp, > > req->rq_interpret_reply = osc_destroy_interpret; > if (!osc_can_send_destroy(cli)) { > - struct l_wait_info lwi = LWI_INTR(LWI_ON_SIGNAL_NOOP, NULL); > - > /* > * Wait until the number of on-going destroy RPCs drops > * under max_rpc_in_flight > */ > - l_wait_event_exclusive(cli->cl_destroy_waitq, > - osc_can_send_destroy(cli), &lwi); > + l_wait_event_abortable_exclusive(cli->cl_destroy_waitq, > + osc_can_send_destroy(cli)); > } > > /* Do not wait for response */ > > > _______________________________________________ > lustre-devel mailing list > lustre-devel at lists.lustre.org > http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 832 bytes Desc: not available URL: From neilb at suse.com Tue Feb 13 00:17:22 2018 From: neilb at suse.com (NeilBrown) Date: Tue, 13 Feb 2018 11:17:22 +1100 Subject: [lustre-devel] [PATCH 08/19] staging: lustre: simplify waiting in ldlm_completion_ast() In-Reply-To: <4AC4BB93-85C5-4CA1-B062-1274833C51C6@cray.com> References: <151847037709.22826.16175867257667686132.stgit@noble> <151847055674.22826.12912356108048917428.stgit@noble> <4AC4BB93-85C5-4CA1-B062-1274833C51C6@cray.com> Message-ID: <87o9ktrbe5.fsf@notabene.neil.brown.name> On Mon, Feb 12 2018, Patrick Farrell wrote: > Neil, > > I didn't get anything after 8/19 in this series. Is this just me? (I'd keep waiting, except I also found a few things in this patch.) Not just you. My fault. They are appearing now. > > Minor: > The line XXX ALLOCATE is out of date and could go. (It refers to a > mix of things you eliminated and things that were already gone.) What does the line even mean? Some comment about stack usage? I think we have a look that looks for large stack frames. I wonder how to run it... > > Less minor: > You remove use of the imp_lock when reading the connection count. While that'll work on x86, it's probably wrong on some architecture to read that without taking the lock...? It was my understanding that on all architectures which Linux support, a 32bit aligned read is atomic wrt any 32bit write. I have trouble imagining how it could be otherwise. I probably should have highlighted the removal of the spinlock in the patch description though - it was intentional. > > Bug: > The existing code uses the imp_conn_cnt from *before* the wait, rather > than after. I think that's quite important. So you'll want to read > it out before the wait. I think the main reason we'd hit the timeout > is a disconnect, which should cause a reconnect, so it's very > important to use the value from *before* the wait. (See comment on > ptlrpc_set_import_discon for more of an explanation. Basically it's > tracking a connection 'epoch', if it's changed, someone else already > went through the reconnect code for this 'connection epoch' and we > shouldn't start that process.) > That wasn't intentional though - thanks for catching! Looking at ptlrpc_set_import_discon(), which is where the number eventually gets used, it is only used to compare with the new value of imp->imp_conn_cnt. This would fix both (assuming the locking issue needs fixing). Thanks, NeilBrown diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_request.c b/drivers/staging/lustre/lustre/ldlm/ldlm_request.c index f1233d844bbd..c3c9186b74ce 100644 --- a/drivers/staging/lustre/lustre/ldlm/ldlm_request.c +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_request.c @@ -103,7 +103,7 @@ static int ldlm_request_bufsize(int count, int type) return sizeof(struct ldlm_request) + avail; } -static void ldlm_expired_completion_wait(struct ldlm_lock *lock, struct obd_import *imp2) +static void ldlm_expired_completion_wait(struct ldlm_lock *lock, __u32 conn_cnt) { struct obd_import *imp; struct obd_device *obd; @@ -129,7 +129,7 @@ static void ldlm_expired_completion_wait(struct ldlm_lock *lock, struct obd_impo obd = lock->l_conn_export->exp_obd; imp = obd->u.cli.cl_import; - ptlrpc_fail_import(imp, imp2 ? imp2->imp_conn_cnt : 0); + ptlrpc_fail_import(imp, conn_cnt); LDLM_ERROR(lock, "lock timed out (enqueued at %lld, %llds ago), entering recovery for %s@%s", (s64)lock->l_last_activity, @@ -241,6 +241,7 @@ int ldlm_completion_ast(struct ldlm_lock *lock, __u64 flags, void *data) struct obd_device *obd; struct obd_import *imp = NULL; __u32 timeout; + __u32 conn_cnt = 0; int rc = 0; if (flags == LDLM_FL_WAIT_NOREPROC) { @@ -268,6 +269,11 @@ int ldlm_completion_ast(struct ldlm_lock *lock, __u64 flags, void *data) lock->l_last_activity = ktime_get_real_seconds(); + if (imp) { + spin_lock(&imp->imp_lock); + conn_cnt = imp->imp_conn_cnt; + spin_unlock(&imp->imp_lock); + } if (OBD_FAIL_CHECK_RESET(OBD_FAIL_LDLM_INTR_CP_AST, OBD_FAIL_LDLM_CP_BL_RACE | OBD_FAIL_ONCE)) { ldlm_set_fail_loc(lock); @@ -280,7 +286,7 @@ int ldlm_completion_ast(struct ldlm_lock *lock, __u64 flags, void *data) is_granted_or_cancelled(lock), timeout * HZ); if (rc == 0) - ldlm_expired_completion_wait(lock, imp); + ldlm_expired_completion_wait(lock, conn_cnt); } /* Now wait abortable */ if (rc == 0) -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 832 bytes Desc: not available URL: From paf at cray.com Tue Feb 13 00:46:51 2018 From: paf at cray.com (Patrick Farrell) Date: Tue, 13 Feb 2018 00:46:51 +0000 Subject: [lustre-devel] [PATCH 08/19] staging: lustre: simplify waiting in ldlm_completion_ast() In-Reply-To: <87o9ktrbe5.fsf@notabene.neil.brown.name> References: <151847037709.22826.16175867257667686132.stgit@noble> <151847055674.22826.12912356108048917428.stgit@noble> <4AC4BB93-85C5-4CA1-B062-1274833C51C6@cray.com>, <87o9ktrbe5.fsf@notabene.neil.brown.name> Message-ID: Yes, I assume it’s a comment about stack usage (or even more out of date and unrelated to the current code entirely. :p) Like you, I can’t imagine what else it would be... You’re probably right about the locking issue, I’m happy to defer. (In any case, this timed out path should be very far from hot...) Otherwise that fix looks good. I’ll review the rest of these tomorrow. ________________________________ From: NeilBrown Sent: Monday, February 12, 2018 6:17:22 PM To: Patrick Farrell; Oleg Drokin; Andreas Dilger; James Simmons; Greg Kroah-Hartman Cc: lkml; lustre Subject: Re: [lustre-devel] [PATCH 08/19] staging: lustre: simplify waiting in ldlm_completion_ast() On Mon, Feb 12 2018, Patrick Farrell wrote: > Neil, > > I didn't get anything after 8/19 in this series. Is this just me? (I'd keep waiting, except I also found a few things in this patch.) Not just you. My fault. They are appearing now. > > Minor: > The line XXX ALLOCATE is out of date and could go. (It refers to a > mix of things you eliminated and things that were already gone.) What does the line even mean? Some comment about stack usage? I think we have a look that looks for large stack frames. I wonder how to run it... > > Less minor: > You remove use of the imp_lock when reading the connection count. While that'll work on x86, it's probably wrong on some architecture to read that without taking the lock...? It was my understanding that on all architectures which Linux support, a 32bit aligned read is atomic wrt any 32bit write. I have trouble imagining how it could be otherwise. I probably should have highlighted the removal of the spinlock in the patch description though - it was intentional. > > Bug: > The existing code uses the imp_conn_cnt from *before* the wait, rather > than after. I think that's quite important. So you'll want to read > it out before the wait. I think the main reason we'd hit the timeout > is a disconnect, which should cause a reconnect, so it's very > important to use the value from *before* the wait. (See comment on > ptlrpc_set_import_discon for more of an explanation. Basically it's > tracking a connection 'epoch', if it's changed, someone else already > went through the reconnect code for this 'connection epoch' and we > shouldn't start that process.) > That wasn't intentional though - thanks for catching! Looking at ptlrpc_set_import_discon(), which is where the number eventually gets used, it is only used to compare with the new value of imp->imp_conn_cnt. This would fix both (assuming the locking issue needs fixing). Thanks, NeilBrown diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_request.c b/drivers/staging/lustre/lustre/ldlm/ldlm_request.c index f1233d844bbd..c3c9186b74ce 100644 --- a/drivers/staging/lustre/lustre/ldlm/ldlm_request.c +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_request.c @@ -103,7 +103,7 @@ static int ldlm_request_bufsize(int count, int type) return sizeof(struct ldlm_request) + avail; } -static void ldlm_expired_completion_wait(struct ldlm_lock *lock, struct obd_import *imp2) +static void ldlm_expired_completion_wait(struct ldlm_lock *lock, __u32 conn_cnt) { struct obd_import *imp; struct obd_device *obd; @@ -129,7 +129,7 @@ static void ldlm_expired_completion_wait(struct ldlm_lock *lock, struct obd_impo obd = lock->l_conn_export->exp_obd; imp = obd->u.cli.cl_import; - ptlrpc_fail_import(imp, imp2 ? imp2->imp_conn_cnt : 0); + ptlrpc_fail_import(imp, conn_cnt); LDLM_ERROR(lock, "lock timed out (enqueued at %lld, %llds ago), entering recovery for %s@%s", (s64)lock->l_last_activity, @@ -241,6 +241,7 @@ int ldlm_completion_ast(struct ldlm_lock *lock, __u64 flags, void *data) struct obd_device *obd; struct obd_import *imp = NULL; __u32 timeout; + __u32 conn_cnt = 0; int rc = 0; if (flags == LDLM_FL_WAIT_NOREPROC) { @@ -268,6 +269,11 @@ int ldlm_completion_ast(struct ldlm_lock *lock, __u64 flags, void *data) lock->l_last_activity = ktime_get_real_seconds(); + if (imp) { + spin_lock(&imp->imp_lock); + conn_cnt = imp->imp_conn_cnt; + spin_unlock(&imp->imp_lock); + } if (OBD_FAIL_CHECK_RESET(OBD_FAIL_LDLM_INTR_CP_AST, OBD_FAIL_LDLM_CP_BL_RACE | OBD_FAIL_ONCE)) { ldlm_set_fail_loc(lock); @@ -280,7 +286,7 @@ int ldlm_completion_ast(struct ldlm_lock *lock, __u64 flags, void *data) is_granted_or_cancelled(lock), timeout * HZ); if (rc == 0) - ldlm_expired_completion_wait(lock, imp); + ldlm_expired_completion_wait(lock, conn_cnt); } /* Now wait abortable */ if (rc == 0) -------------- next part -------------- An HTML attachment was scrubbed... URL: From paf at cray.com Tue Feb 13 18:15:58 2018 From: paf at cray.com (Patrick Farrell) Date: Tue, 13 Feb 2018 18:15:58 +0000 Subject: [lustre-devel] [PATCH 19/19] staging: lustre: remove l_wait_event() and related code In-Reply-To: <151847927929.19840.14894766883488647956.stgit@noble> References: <151847037709.22826.16175867257667686132.stgit@noble> <151847927929.19840.14894766883488647956.stgit@noble> Message-ID: <7207F5AF-1798-4C53-8EBF-6A8316004D1C@cray.com> With the fix from yesterday, these look good. Thanks, Neil. Reviewed-by: Patrick Farrell - Patrick On 2/12/18, 5:52 PM, "lustre-devel on behalf of NeilBrown" wrote: These macros are no longer used, so they can be removed. Reviewed-by: James Simmons Signed-off-by: NeilBrown --- drivers/staging/lustre/lustre/include/lustre_lib.h | 249 -------------------- 1 file changed, 249 deletions(-) diff --git a/drivers/staging/lustre/lustre/include/lustre_lib.h b/drivers/staging/lustre/lustre/include/lustre_lib.h index ccc1a329e42b..1efd86f18c1f 100644 --- a/drivers/staging/lustre/lustre/include/lustre_lib.h +++ b/drivers/staging/lustre/lustre/include/lustre_lib.h @@ -76,123 +76,6 @@ int do_set_info_async(struct obd_import *imp, void target_send_reply(struct ptlrpc_request *req, int rc, int fail_id); -/* - * l_wait_event is a flexible sleeping function, permitting simple caller - * configuration of interrupt and timeout sensitivity along with actions to - * be performed in the event of either exception. - * - * The first form of usage looks like this: - * - * struct l_wait_info lwi = LWI_TIMEOUT_INTR(timeout, timeout_handler, - * intr_handler, callback_data); - * rc = l_wait_event(waitq, condition, &lwi); - * - * l_wait_event() makes the current process wait on 'waitq' until 'condition' - * is TRUE or a "killable" signal (SIGTERM, SIKGILL, SIGINT) is pending. It - * returns 0 to signify 'condition' is TRUE, but if a signal wakes it before - * 'condition' becomes true, it optionally calls the specified 'intr_handler' - * if not NULL, and returns -EINTR. - * - * If a non-zero timeout is specified, signals are ignored until the timeout - * has expired. At this time, if 'timeout_handler' is not NULL it is called. - * If it returns FALSE l_wait_event() continues to wait as described above with - * signals enabled. Otherwise it returns -ETIMEDOUT. - * - * LWI_INTR(intr_handler, callback_data) is shorthand for - * LWI_TIMEOUT_INTR(0, NULL, intr_handler, callback_data) - * - * The second form of usage looks like this: - * - * struct l_wait_info lwi = LWI_TIMEOUT(timeout, timeout_handler); - * rc = l_wait_event(waitq, condition, &lwi); - * - * This form is the same as the first except that it COMPLETELY IGNORES - * SIGNALS. The caller must therefore beware that if 'timeout' is zero, or if - * 'timeout_handler' is not NULL and returns FALSE, then the ONLY thing that - * can unblock the current process is 'condition' becoming TRUE. - * - * Another form of usage is: - * struct l_wait_info lwi = LWI_TIMEOUT_INTERVAL(timeout, interval, - * timeout_handler); - * rc = l_wait_event(waitq, condition, &lwi); - * This is the same as previous case, but condition is checked once every - * 'interval' jiffies (if non-zero). - * - * Subtle synchronization point: this macro does *not* necessary takes - * wait-queue spin-lock before returning, and, hence, following idiom is safe - * ONLY when caller provides some external locking: - * - * Thread1 Thread2 - * - * l_wait_event(&obj->wq, ....); (1) - * - * wake_up(&obj->wq): (2) - * spin_lock(&q->lock); (2.1) - * __wake_up_common(q, ...); (2.2) - * spin_unlock(&q->lock, flags); (2.3) - * - * kfree(obj); (3) - * - * As l_wait_event() may "short-cut" execution and return without taking - * wait-queue spin-lock, some additional synchronization is necessary to - * guarantee that step (3) can begin only after (2.3) finishes. - * - * XXX nikita: some ptlrpc daemon threads have races of that sort. - * - */ - -#define LWI_ON_SIGNAL_NOOP ((void (*)(void *))(-1)) - -struct l_wait_info { - long lwi_timeout; - long lwi_interval; - int lwi_allow_intr; - int (*lwi_on_timeout)(void *); - void (*lwi_on_signal)(void *); - void *lwi_cb_data; -}; - -/* NB: LWI_TIMEOUT ignores signals completely */ -#define LWI_TIMEOUT(time, cb, data) \ -((struct l_wait_info) { \ - .lwi_timeout = time, \ - .lwi_on_timeout = cb, \ - .lwi_cb_data = data, \ - .lwi_interval = 0, \ - .lwi_allow_intr = 0 \ -}) - -#define LWI_TIMEOUT_INTERVAL(time, interval, cb, data) \ -((struct l_wait_info) { \ - .lwi_timeout = time, \ - .lwi_on_timeout = cb, \ - .lwi_cb_data = data, \ - .lwi_interval = interval, \ - .lwi_allow_intr = 0 \ -}) - -#define LWI_TIMEOUT_INTR(time, time_cb, sig_cb, data) \ -((struct l_wait_info) { \ - .lwi_timeout = time, \ - .lwi_on_timeout = time_cb, \ - .lwi_on_signal = sig_cb, \ - .lwi_cb_data = data, \ - .lwi_interval = 0, \ - .lwi_allow_intr = 0 \ -}) - -#define LWI_TIMEOUT_INTR_ALL(time, time_cb, sig_cb, data) \ -((struct l_wait_info) { \ - .lwi_timeout = time, \ - .lwi_on_timeout = time_cb, \ - .lwi_on_signal = sig_cb, \ - .lwi_cb_data = data, \ - .lwi_interval = 0, \ - .lwi_allow_intr = 1 \ -}) - -#define LWI_INTR(cb, data) LWI_TIMEOUT_INTR(0, NULL, cb, data) - #define LUSTRE_FATAL_SIGS (sigmask(SIGKILL) | sigmask(SIGINT) | \ sigmask(SIGTERM) | sigmask(SIGQUIT) | \ sigmask(SIGALRM)) @@ -201,138 +84,6 @@ static inline int l_fatal_signal_pending(struct task_struct *p) return signal_pending(p) && sigtestsetmask(&p->pending.signal, LUSTRE_FATAL_SIGS); } -/** - * wait_queue_entry_t of Linux (version < 2.6.34) is a FIFO list for exclusively - * waiting threads, which is not always desirable because all threads will - * be waken up again and again, even user only needs a few of them to be - * active most time. This is not good for performance because cache can - * be polluted by different threads. - * - * LIFO list can resolve this problem because we always wakeup the most - * recent active thread by default. - * - * NB: please don't call non-exclusive & exclusive wait on the same - * waitq if add_wait_queue_exclusive_head is used. - */ -#define add_wait_queue_exclusive_head(waitq, link) \ -{ \ - unsigned long flags; \ - \ - spin_lock_irqsave(&((waitq)->lock), flags); \ - __add_wait_queue_exclusive(waitq, link); \ - spin_unlock_irqrestore(&((waitq)->lock), flags); \ -} - -/* - * wait for @condition to become true, but no longer than timeout, specified - * by @info. - */ -#define __l_wait_event(wq, condition, info, ret, l_add_wait) \ -do { \ - wait_queue_entry_t __wait; \ - long __timeout = info->lwi_timeout; \ - sigset_t __blocked; \ - int __allow_intr = info->lwi_allow_intr; \ - \ - ret = 0; \ - if (condition) \ - break; \ - \ - init_waitqueue_entry(&__wait, current); \ - l_add_wait(&wq, &__wait); \ - \ - /* Block all signals (just the non-fatal ones if no timeout). */ \ - if (info->lwi_on_signal && (__timeout == 0 || __allow_intr)) \ - __blocked = cfs_block_sigsinv(LUSTRE_FATAL_SIGS); \ - else \ - __blocked = cfs_block_sigsinv(0); \ - \ - for (;;) { \ - if (condition) \ - break; \ - \ - set_current_state(TASK_INTERRUPTIBLE); \ - \ - if (__timeout == 0) { \ - schedule(); \ - } else { \ - long interval = info->lwi_interval ? \ - min_t(long, \ - info->lwi_interval, __timeout) : \ - __timeout; \ - long remaining = schedule_timeout(interval);\ - __timeout = cfs_time_sub(__timeout, \ - cfs_time_sub(interval, remaining));\ - if (__timeout == 0) { \ - if (!info->lwi_on_timeout || \ - info->lwi_on_timeout(info->lwi_cb_data)) { \ - ret = -ETIMEDOUT; \ - break; \ - } \ - /* Take signals after the timeout expires. */ \ - if (info->lwi_on_signal) \ - (void)cfs_block_sigsinv(LUSTRE_FATAL_SIGS);\ - } \ - } \ - \ - set_current_state(TASK_RUNNING); \ - \ - if (condition) \ - break; \ - if (signal_pending(current)) { \ - if (info->lwi_on_signal && \ - (__timeout == 0 || __allow_intr)) { \ - if (info->lwi_on_signal != LWI_ON_SIGNAL_NOOP) \ - info->lwi_on_signal(info->lwi_cb_data);\ - ret = -EINTR; \ - break; \ - } \ - /* We have to do this here because some signals */ \ - /* are not blockable - ie from strace(1). */ \ - /* In these cases we want to schedule_timeout() */ \ - /* again, because we don't want that to return */ \ - /* -EINTR when the RPC actually succeeded. */ \ - /* the recalc_sigpending() below will deliver the */ \ - /* signal properly. */ \ - cfs_clear_sigpending(); \ - } \ - } \ - \ - cfs_restore_sigs(__blocked); \ - \ - remove_wait_queue(&wq, &__wait); \ -} while (0) - -#define l_wait_event(wq, condition, info) \ -({ \ - int __ret; \ - struct l_wait_info *__info = (info); \ - \ - __l_wait_event(wq, condition, __info, \ - __ret, add_wait_queue); \ - __ret; \ -}) - -#define l_wait_event_exclusive(wq, condition, info) \ -({ \ - int __ret; \ - struct l_wait_info *__info = (info); \ - \ - __l_wait_event(wq, condition, __info, \ - __ret, add_wait_queue_exclusive); \ - __ret; \ -}) - -#define l_wait_event_exclusive_head(wq, condition, info) \ -({ \ - int __ret; \ - struct l_wait_info *__info = (info); \ - \ - __l_wait_event(wq, condition, __info, \ - __ret, add_wait_queue_exclusive_head); \ - __ret; \ -}) - /** @} lib */ _______________________________________________ lustre-devel mailing list lustre-devel at lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org From neilb at suse.com Tue Feb 13 20:17:30 2018 From: neilb at suse.com (NeilBrown) Date: Wed, 14 Feb 2018 07:17:30 +1100 Subject: [lustre-devel] [PATCH 08/19 - v2] staging: lustre: simplify waiting in ldlm_completion_ast() In-Reply-To: <151847055674.22826.12912356108048917428.stgit@noble> References: <151847037709.22826.16175867257667686132.stgit@noble> <151847055674.22826.12912356108048917428.stgit@noble> Message-ID: <87fu64r6ed.fsf@notabene.neil.brown.name> If a signal-callback (lwi_on_signal) is set without lwi_allow_intr, as is the case in ldlm_completion_ast(), the behavior depends on the timeout set. If a timeout is set, then signals are ignored. If the timeout is reached, the timeout handler is called. If the timeout handler return 0, which ldlm_expired_completion_wait() always does, the l_wait_event() switches to exactly the behavior if no timeout was set. If no timeout is set, then "fatal" signals are not ignored. If one arrives the callback is run, but as the callback is empty in this case, that is not relevant. This can be simplified to: if a timeout is wanted wait_event_idle_timeout() if that timed out, call the timeout handler l_wait_event_abortable() i.e. the code always waits indefinitely. Sometimes it performs a non-abortable wait first. Sometimes it doesn't. But it only aborts before the condition is true if it is signaled. This doesn't quite agree with the comments and debug messages. Now that we call the timeout handler (ldlm_expired_completion_wait()) wait directly, we can pass the two args directly rather then using a special-purpose struct. Reviewed-by: Patrick Farrell Reviewed-by: James Simmons Signed-off-by: NeilBrown --- Patrick discovered a bug in v1, which this v2 fixes. Greg - do you need me to resend the whole series, or are you ok with taking this replacement in the rest of the original series? Thanks, NeilBrown drivers/staging/lustre/lustre/ldlm/ldlm_request.c | 53 +++++++++-------------- 1 file changed, 20 insertions(+), 33 deletions(-) diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_request.c b/drivers/staging/lustre/lustre/ldlm/ldlm_request.c index a244fa717134..c3c9186b74ce 100644 --- a/drivers/staging/lustre/lustre/ldlm/ldlm_request.c +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_request.c @@ -72,15 +72,6 @@ MODULE_PARM_DESC(ldlm_enqueue_min, "lock enqueue timeout minimum"); /* in client side, whether the cached locks will be canceled before replay */ unsigned int ldlm_cancel_unused_locks_before_replay = 1; -static void interrupted_completion_wait(void *data) -{ -} - -struct lock_wait_data { - struct ldlm_lock *lwd_lock; - __u32 lwd_conn_cnt; -}; - struct ldlm_async_args { struct lustre_handle lock_handle; }; @@ -112,10 +103,8 @@ static int ldlm_request_bufsize(int count, int type) return sizeof(struct ldlm_request) + avail; } -static int ldlm_expired_completion_wait(void *data) +static void ldlm_expired_completion_wait(struct ldlm_lock *lock, __u32 conn_cnt) { - struct lock_wait_data *lwd = data; - struct ldlm_lock *lock = lwd->lwd_lock; struct obd_import *imp; struct obd_device *obd; @@ -135,19 +124,17 @@ static int ldlm_expired_completion_wait(void *data) if (last_dump == 0) libcfs_debug_dumplog(); } - return 0; + return; } obd = lock->l_conn_export->exp_obd; imp = obd->u.cli.cl_import; - ptlrpc_fail_import(imp, lwd->lwd_conn_cnt); + ptlrpc_fail_import(imp, conn_cnt); LDLM_ERROR(lock, "lock timed out (enqueued at %lld, %llds ago), entering recovery for %s@%s", (s64)lock->l_last_activity, (s64)(ktime_get_real_seconds() - lock->l_last_activity), obd2cli_tgt(obd), imp->imp_connection->c_remote_uuid.uuid); - - return 0; } /** @@ -251,11 +238,10 @@ EXPORT_SYMBOL(ldlm_completion_ast_async); int ldlm_completion_ast(struct ldlm_lock *lock, __u64 flags, void *data) { /* XXX ALLOCATE - 160 bytes */ - struct lock_wait_data lwd; struct obd_device *obd; struct obd_import *imp = NULL; - struct l_wait_info lwi; __u32 timeout; + __u32 conn_cnt = 0; int rc = 0; if (flags == LDLM_FL_WAIT_NOREPROC) { @@ -281,32 +267,33 @@ int ldlm_completion_ast(struct ldlm_lock *lock, __u64 flags, void *data) timeout = ldlm_cp_timeout(lock); - lwd.lwd_lock = lock; lock->l_last_activity = ktime_get_real_seconds(); - if (ldlm_is_no_timeout(lock)) { - LDLM_DEBUG(lock, "waiting indefinitely because of NO_TIMEOUT"); - lwi = LWI_INTR(interrupted_completion_wait, &lwd); - } else { - lwi = LWI_TIMEOUT_INTR(timeout * HZ, - ldlm_expired_completion_wait, - interrupted_completion_wait, &lwd); - } - if (imp) { spin_lock(&imp->imp_lock); - lwd.lwd_conn_cnt = imp->imp_conn_cnt; + conn_cnt = imp->imp_conn_cnt; spin_unlock(&imp->imp_lock); } - if (OBD_FAIL_CHECK_RESET(OBD_FAIL_LDLM_INTR_CP_AST, OBD_FAIL_LDLM_CP_BL_RACE | OBD_FAIL_ONCE)) { ldlm_set_fail_loc(lock); rc = -EINTR; } else { - /* Go to sleep until the lock is granted or cancelled. */ - rc = l_wait_event(lock->l_waitq, - is_granted_or_cancelled(lock), &lwi); + /* Go to sleep until the lock is granted or canceled. */ + if (!ldlm_is_no_timeout(lock)) { + /* Wait uninterruptible for a while first */ + rc = wait_event_idle_timeout(lock->l_waitq, + is_granted_or_cancelled(lock), + timeout * HZ); + if (rc == 0) + ldlm_expired_completion_wait(lock, conn_cnt); + } + /* Now wait abortable */ + if (rc == 0) + rc = l_wait_event_abortable(lock->l_waitq, + is_granted_or_cancelled(lock)); + else + rc = 0; } if (rc) { -- 2.14.0.rc0.dirty -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 832 bytes Desc: not available URL: From gregkh at linuxfoundation.org Fri Feb 16 14:18:43 2018 From: gregkh at linuxfoundation.org (Greg Kroah-Hartman) Date: Fri, 16 Feb 2018 15:18:43 +0100 Subject: [lustre-devel] [PATCH 08/19 - v2] staging: lustre: simplify waiting in ldlm_completion_ast() In-Reply-To: <87fu64r6ed.fsf@notabene.neil.brown.name> References: <151847037709.22826.16175867257667686132.stgit@noble> <151847055674.22826.12912356108048917428.stgit@noble> <87fu64r6ed.fsf@notabene.neil.brown.name> Message-ID: <20180216141843.GA22540@kroah.com> On Wed, Feb 14, 2018 at 07:17:30AM +1100, NeilBrown wrote: > > If a signal-callback (lwi_on_signal) is set without lwi_allow_intr, as > is the case in ldlm_completion_ast(), the behavior depends on the > timeout set. > > If a timeout is set, then signals are ignored. If the timeout is > reached, the timeout handler is called. If the timeout handler > return 0, which ldlm_expired_completion_wait() always does, the > l_wait_event() switches to exactly the behavior if no timeout was set. > > If no timeout is set, then "fatal" signals are not ignored. If one > arrives the callback is run, but as the callback is empty in this > case, that is not relevant. > > This can be simplified to: > if a timeout is wanted > wait_event_idle_timeout() > if that timed out, call the timeout handler > l_wait_event_abortable() > > i.e. the code always waits indefinitely. Sometimes it performs a > non-abortable wait first. Sometimes it doesn't. But it only > aborts before the condition is true if it is signaled. > This doesn't quite agree with the comments and debug messages. > > Now that we call the timeout handler (ldlm_expired_completion_wait()) > wait directly, we can pass the two args directly rather then > using a special-purpose struct. > > Reviewed-by: Patrick Farrell > Reviewed-by: James Simmons > Signed-off-by: NeilBrown > --- > > Patrick discovered a bug in v1, which this v2 fixes. > > Greg - do you need me to resend the whole series, or are you ok with > taking this replacement in the rest of the original series? I can take this replacement, thanks. greg k-h From neilb at suse.com Tue Feb 20 02:23:37 2018 From: neilb at suse.com (NeilBrown) Date: Tue, 20 Feb 2018 13:23:37 +1100 Subject: [lustre-devel] [PATCH 00/21] staging: assorted lustre clean-up Message-ID: <151909308778.25814.9912662268627044409.stgit@noble> Following are assorted patches which clean up parts of lustre. One or two tiny buglets are fixed, but mostly the patches just make the code more readable, often by removing distractions. The first several patches remove content from libcfs, completely removing 3 files. There are some list_for_each improvements in ptlrpc, and some code simplification in fid. Finally a small simplification in socklnd. (lnd ?? Lustre Networking Domain??) Thanks, NeilBrown --- NeilBrown (21): staging: lustre: replace all CFS_CAP_* macros with CAP_* staging: lustre: opencode cfs_cap_{raise,lower,raised} staging: lustre: remove linux-curproc.c staging: lustre: remove unnecessary cfs_block_allsigs() calls staging: lustre: lnet: remove cfs_block_allsigs calls. staging: lustre: simplify linux-prim.c staging: lustre: improve API and implementation of blocking signals. staging: lustre: make signal-blocking functions inline staging: lustre: discard libcfs_kvzalloc_cpt() staging: lustre: discard lu_buf allocation library. staging: lustre: improve some libcfs_kvzalloc calls. staging: lustre: discard libcfs_kvzalloc and linux-mem.c staging: lustre: remove phantom struct cfs_crypto_hash_desc staging: lustre: fix assorted checkpatch errors staging: lustre: ptlrpc: list_for_each improvements. staging: lustre: fid: convert lcs_mutex to a spinlock staging: lustre: fid: use wait_event_cmd() staging: lustre: fid: remove seq_fid_alloc_fini() and simplify staging: lustre: fid: fix up debugfs access to ->lcs_space staging: lustre: fid: perform sanity checks before commiting staging: lustre: socklnd: simplify ksnc_rx_iov_space .../staging/lustre/include/linux/libcfs/curproc.h | 37 ++----- .../staging/lustre/include/linux/libcfs/libcfs.h | 27 +++-- .../lustre/include/linux/libcfs/libcfs_crypto.h | 11 +- drivers/staging/lustre/include/linux/lnet/api.h | 1 .../staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c | 6 - .../staging/lustre/lnet/klnds/socklnd/socklnd.h | 11 -- .../staging/lustre/lnet/klnds/socklnd/socklnd_cb.c | 10 -- .../lustre/lnet/klnds/socklnd/socklnd_lib.c | 2 drivers/staging/lustre/lnet/libcfs/Makefile | 4 - .../staging/lustre/lnet/libcfs/linux/linux-cpu.c | 2 .../lustre/lnet/libcfs/linux/linux-crypto.c | 29 ++--- .../lustre/lnet/libcfs/linux/linux-curproc.c | 108 ------------------- .../staging/lustre/lnet/libcfs/linux/linux-mem.c | 51 --------- .../staging/lustre/lnet/libcfs/linux/linux-prim.c | 113 -------------------- drivers/staging/lustre/lnet/lnet/acceptor.c | 2 drivers/staging/lustre/lnet/lnet/api-ni.c | 15 +-- drivers/staging/lustre/lnet/lnet/lib-eq.c | 10 +- drivers/staging/lustre/lnet/lnet/router.c | 2 drivers/staging/lustre/lnet/selftest/timer.c | 2 drivers/staging/lustre/lustre/fid/fid_request.c | 106 ++++++------------- drivers/staging/lustre/lustre/fid/lproc_fid.c | 44 +++++--- drivers/staging/lustre/lustre/include/lu_object.h | 7 - drivers/staging/lustre/lustre/include/lustre_fid.h | 2 drivers/staging/lustre/lustre/include/lustre_lib.h | 18 ++- drivers/staging/lustre/lustre/include/lustre_sec.h | 3 - drivers/staging/lustre/lustre/include/obd_class.h | 8 + .../staging/lustre/lustre/include/obd_support.h | 2 drivers/staging/lustre/lustre/llite/dcache.c | 2 drivers/staging/lustre/lustre/llite/dir.c | 10 +- drivers/staging/lustre/lustre/llite/file.c | 12 +- drivers/staging/lustre/lustre/llite/llite_lib.c | 8 + drivers/staging/lustre/lustre/llite/llite_mmap.c | 8 + drivers/staging/lustre/lustre/llite/xattr.c | 2 drivers/staging/lustre/lustre/lmv/lmv_obd.c | 4 - drivers/staging/lustre/lustre/lov/lov_ea.c | 2 drivers/staging/lustre/lustre/lov/lov_io.c | 2 drivers/staging/lustre/lustre/lov/lov_lock.c | 2 drivers/staging/lustre/lustre/lov/lov_object.c | 4 - drivers/staging/lustre/lustre/lov/lov_pack.c | 2 drivers/staging/lustre/lustre/lov/lov_request.c | 2 drivers/staging/lustre/lustre/mdc/mdc_locks.c | 2 drivers/staging/lustre/lustre/mdc/mdc_request.c | 6 + drivers/staging/lustre/lustre/obdclass/linkea.c | 16 ++- .../lustre/lustre/obdclass/linux/linux-module.c | 4 - drivers/staging/lustre/lustre/obdclass/llog.c | 22 ++-- drivers/staging/lustre/lustre/obdclass/lu_object.c | 70 ------------ .../lustre/lustre/obdclass/lustre_handles.c | 4 - .../staging/lustre/lustre/obdecho/echo_client.c | 8 + drivers/staging/lustre/lustre/osc/osc_cache.c | 4 - drivers/staging/lustre/lustre/osc/osc_page.c | 2 drivers/staging/lustre/lustre/osc/osc_request.c | 2 drivers/staging/lustre/lustre/ptlrpc/client.c | 93 +++++----------- drivers/staging/lustre/lustre/ptlrpc/import.c | 34 ++---- drivers/staging/lustre/lustre/ptlrpc/layout.c | 2 drivers/staging/lustre/lustre/ptlrpc/pinger.c | 8 - drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c | 15 +-- drivers/staging/lustre/lustre/ptlrpc/recover.c | 28 ++--- drivers/staging/lustre/lustre/ptlrpc/sec.c | 4 - drivers/staging/lustre/lustre/ptlrpc/sec_bulk.c | 4 - drivers/staging/lustre/lustre/ptlrpc/sec_null.c | 8 + drivers/staging/lustre/lustre/ptlrpc/sec_plain.c | 8 + drivers/staging/lustre/lustre/ptlrpc/service.c | 27 ++--- 62 files changed, 295 insertions(+), 769 deletions(-) delete mode 100644 drivers/staging/lustre/lnet/libcfs/linux/linux-curproc.c delete mode 100644 drivers/staging/lustre/lnet/libcfs/linux/linux-mem.c delete mode 100644 drivers/staging/lustre/lnet/libcfs/linux/linux-prim.c -- Signature From neilb at suse.com Tue Feb 20 02:23:37 2018 From: neilb at suse.com (NeilBrown) Date: Tue, 20 Feb 2018 13:23:37 +1100 Subject: [lustre-devel] [PATCH 05/21] staging: lustre: lnet: remove cfs_block_allsigs calls. In-Reply-To: <151909308778.25814.9912662268627044409.stgit@noble> References: <151909308778.25814.9912662268627044409.stgit@noble> Message-ID: <151909341769.25814.4922187814462338726.stgit@noble> Both places that cfs_block_allsigs() is used here, the goal is to turn an interruptible wait into an uninterruptible way. So instead of blocking the signals, change TASK_INTERRUPTIBLE to TASK_NOLOAD. In each case, no other functions called while signals are blocked will sleep - just the one that has been fixed. In one case, an extra 'interruptible' flag needs to be passed down so the waiting decision can be made at the right place. Signed-off-by: NeilBrown --- drivers/staging/lustre/include/linux/lnet/api.h | 1 + drivers/staging/lustre/lnet/lnet/api-ni.c | 15 +++------------ drivers/staging/lustre/lnet/lnet/lib-eq.c | 10 +++++++--- 3 files changed, 11 insertions(+), 15 deletions(-) diff --git a/drivers/staging/lustre/include/linux/lnet/api.h b/drivers/staging/lustre/include/linux/lnet/api.h index 31fcd33171b4..dae2e4f0056c 100644 --- a/drivers/staging/lustre/include/linux/lnet/api.h +++ b/drivers/staging/lustre/include/linux/lnet/api.h @@ -169,6 +169,7 @@ int LNetEQFree(struct lnet_handle_eq eventq_in); int LNetEQPoll(struct lnet_handle_eq *eventqs_in, int neq_in, int timeout_ms, + int interruptible, struct lnet_event *event_out, int *which_eq_out); /** @} lnet_eq */ diff --git a/drivers/staging/lustre/lnet/lnet/api-ni.c b/drivers/staging/lustre/lnet/lnet/api-ni.c index 93e6274e9dac..48d25ccadbb3 100644 --- a/drivers/staging/lustre/lnet/lnet/api-ni.c +++ b/drivers/staging/lustre/lnet/lnet/api-ni.c @@ -961,19 +961,15 @@ static void lnet_ping_md_unlink(struct lnet_ping_info *pinfo, struct lnet_handle_md *md_handle) { - sigset_t blocked = cfs_block_allsigs(); - LNetMDUnlink(*md_handle); LNetInvalidateMDHandle(md_handle); /* NB md could be busy; this just starts the unlink */ while (pinfo->pi_features != LNET_PING_FEAT_INVAL) { CDEBUG(D_NET, "Still waiting for ping MD to unlink\n"); - set_current_state(TASK_UNINTERRUPTIBLE); + set_current_state(TASK_NOLOAD); schedule_timeout(HZ); } - - cfs_restore_sigs(blocked); } static void @@ -2141,7 +2137,6 @@ static int lnet_ping(struct lnet_process_id id, int timeout_ms, int nob; int rc; int rc2; - sigset_t blocked; infosz = offsetof(struct lnet_ping_info, pi_ni[n_ids]); @@ -2197,13 +2192,9 @@ static int lnet_ping(struct lnet_process_id id, int timeout_ms, do { /* MUST block for unlink to complete */ - if (unlinked) - blocked = cfs_block_allsigs(); - - rc2 = LNetEQPoll(&eqh, 1, timeout_ms, &event, &which); - if (unlinked) - cfs_restore_sigs(blocked); + rc2 = LNetEQPoll(&eqh, 1, timeout_ms, !unlinked, + &event, &which); CDEBUG(D_NET, "poll %d(%d %d)%s\n", rc2, (rc2 <= 0) ? -1 : event.type, diff --git a/drivers/staging/lustre/lnet/lnet/lib-eq.c b/drivers/staging/lustre/lnet/lnet/lib-eq.c index a173b69e2f92..ea53b5cb3f72 100644 --- a/drivers/staging/lustre/lnet/lnet/lib-eq.c +++ b/drivers/staging/lustre/lnet/lnet/lib-eq.c @@ -308,7 +308,7 @@ lnet_eq_dequeue_event(struct lnet_eq *eq, struct lnet_event *ev) */ static int -lnet_eq_wait_locked(int *timeout_ms) +lnet_eq_wait_locked(int *timeout_ms, long state) __must_hold(&the_lnet.ln_eq_wait_lock) { int tms = *timeout_ms; @@ -320,7 +320,7 @@ __must_hold(&the_lnet.ln_eq_wait_lock) return -ENXIO; /* don't want to wait and no new event */ init_waitqueue_entry(&wl, current); - set_current_state(TASK_INTERRUPTIBLE); + set_current_state(state); add_wait_queue(&the_lnet.ln_eq_waitq, &wl); lnet_eq_wait_unlock(); @@ -359,6 +359,7 @@ __must_hold(&the_lnet.ln_eq_wait_lock) * \param timeout_ms Time in milliseconds to wait for an event to occur on * one of the EQs. The constant LNET_TIME_FOREVER can be used to indicate an * infinite timeout. + * \param interruptible, if true, use TASK_INTERRUPTIBLE, else TASK_NOLOAD * \param event,which On successful return (1 or -EOVERFLOW), \a event will * hold the next event in the EQs, and \a which will contain the index of the * EQ from which the event was taken. @@ -372,6 +373,7 @@ __must_hold(&the_lnet.ln_eq_wait_lock) */ int LNetEQPoll(struct lnet_handle_eq *eventqs, int neq, int timeout_ms, + int interruptible, struct lnet_event *event, int *which) { int wait = 1; @@ -412,7 +414,9 @@ LNetEQPoll(struct lnet_handle_eq *eventqs, int neq, int timeout_ms, * 0 : don't want to wait anymore, but might have new event * so need to call dequeue again */ - wait = lnet_eq_wait_locked(&timeout_ms); + wait = lnet_eq_wait_locked(&timeout_ms, + interruptible ? TASK_INTERRUPTIBLE + : TASK_NOLOAD); if (wait < 0) /* no new event */ break; } From neilb at suse.com Tue Feb 20 02:23:37 2018 From: neilb at suse.com (NeilBrown) Date: Tue, 20 Feb 2018 13:23:37 +1100 Subject: [lustre-devel] [PATCH 03/21] staging: lustre: remove linux-curproc.c In-Reply-To: <151909308778.25814.9912662268627044409.stgit@noble> References: <151909308778.25814.9912662268627044409.stgit@noble> Message-ID: <151909341763.25814.2230821270457016452.stgit@noble> The only functionality remaining here is cfs_curproc_cap_pack(), and it can be trivially implemented as an inline in curproc.h. So do that and remove the file. Signed-off-by: NeilBrown --- .../staging/lustre/include/linux/libcfs/curproc.h | 6 +- drivers/staging/lustre/lnet/libcfs/Makefile | 1 .../lustre/lnet/libcfs/linux/linux-curproc.c | 78 -------------------- 3 files changed, 5 insertions(+), 80 deletions(-) delete mode 100644 drivers/staging/lustre/lnet/libcfs/linux/linux-curproc.c diff --git a/drivers/staging/lustre/include/linux/libcfs/curproc.h b/drivers/staging/lustre/include/linux/libcfs/curproc.h index 51f2179b7534..4702956805a6 100644 --- a/drivers/staging/lustre/include/linux/libcfs/curproc.h +++ b/drivers/staging/lustre/include/linux/libcfs/curproc.h @@ -66,7 +66,11 @@ typedef u32 cfs_cap_t; BIT(CAP_SYS_BOOT) | \ BIT(CAP_SYS_RESOURCE)) -cfs_cap_t cfs_curproc_cap_pack(void); +static inline cfs_cap_t cfs_curproc_cap_pack(void) +{ + /* cfs_cap_t is only the first word of kernel_cap_t */ + return (cfs_cap_t)(current_cap().cap[0]); +} /* __LIBCFS_CURPROC_H__ */ #endif diff --git a/drivers/staging/lustre/lnet/libcfs/Makefile b/drivers/staging/lustre/lnet/libcfs/Makefile index 730f2c675047..fccea8684f0c 100644 --- a/drivers/staging/lustre/lnet/libcfs/Makefile +++ b/drivers/staging/lustre/lnet/libcfs/Makefile @@ -6,7 +6,6 @@ obj-$(CONFIG_LNET) += libcfs.o libcfs-linux-objs := linux-tracefile.o linux-debug.o libcfs-linux-objs += linux-prim.o linux-cpu.o -libcfs-linux-objs += linux-curproc.o libcfs-linux-objs += linux-module.o libcfs-linux-objs += linux-crypto.o libcfs-linux-objs += linux-crypto-adler.o diff --git a/drivers/staging/lustre/lnet/libcfs/linux/linux-curproc.c b/drivers/staging/lustre/lnet/libcfs/linux/linux-curproc.c deleted file mode 100644 index 6b75c5cdc3ec..000000000000 --- a/drivers/staging/lustre/lnet/libcfs/linux/linux-curproc.c +++ /dev/null @@ -1,78 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0 -/* - * GPL HEADER START - * - * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. - * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License version 2 only, - * as published by the Free Software Foundation. - * - * This program is distributed in the hope that it will be useful, but - * WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - * General Public License version 2 for more details (a copy is included - * in the LICENSE file that accompanied this code). - * - * You should have received a copy of the GNU General Public License - * version 2 along with this program; If not, see - * http://www.gnu.org/licenses/gpl-2.0.html - * - * GPL HEADER END - */ -/* - * Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved. - * Use is subject to license terms. - * - * Copyright (c) 2011, 2015, Intel Corporation. - */ -/* - * This file is part of Lustre, http://www.lustre.org/ - * Lustre is a trademark of Sun Microsystems, Inc. - * - * libcfs/libcfs/linux/linux-curproc.c - * - * Lustre curproc API implementation for Linux kernel - * - * Author: Nikita Danilov - */ - -#include -#include - -#include -#include - -#define DEBUG_SUBSYSTEM S_LNET - -#include - -/* - * Implementation of cfs_curproc API (see portals/include/libcfs/curproc.h) - * for Linux kernel. - */ - -static void cfs_kernel_cap_pack(kernel_cap_t kcap, cfs_cap_t *cap) -{ - /* XXX lost high byte */ - *cap = kcap.cap[0]; -} - -cfs_cap_t cfs_curproc_cap_pack(void) -{ - cfs_cap_t cap; - - cfs_kernel_cap_pack(current_cap(), &cap); - return cap; -} -EXPORT_SYMBOL(cfs_curproc_cap_pack); - -/* - * Local variables: - * c-indentation-style: "K&R" - * c-basic-offset: 8 - * tab-width: 8 - * fill-column: 80 - * scroll-step: 1 - * End: - */ From neilb at suse.com Tue Feb 20 02:23:37 2018 From: neilb at suse.com (NeilBrown) Date: Tue, 20 Feb 2018 13:23:37 +1100 Subject: [lustre-devel] [PATCH 08/21] staging: lustre: make signal-blocking functions inline In-Reply-To: <151909308778.25814.9912662268627044409.stgit@noble> References: <151909308778.25814.9912662268627044409.stgit@noble> Message-ID: <151909341780.25814.17116167688658061622.stgit@noble> cfs_block_sigsinv() and cfs_restore_sigs() are now simple enough to inline them. This means we can discard linux-prim.c Signed-off-by: NeilBrown --- .../staging/lustre/include/linux/libcfs/libcfs.h | 20 +++++- drivers/staging/lustre/lnet/libcfs/Makefile | 2 - .../staging/lustre/lnet/libcfs/linux/linux-prim.c | 62 -------------------- 3 files changed, 16 insertions(+), 68 deletions(-) delete mode 100644 drivers/staging/lustre/lnet/libcfs/linux/linux-prim.c diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs.h b/drivers/staging/lustre/include/linux/libcfs/libcfs.h index 9dff050810c4..4f8c65e6d883 100644 --- a/drivers/staging/lustre/include/linux/libcfs/libcfs.h +++ b/drivers/staging/lustre/include/linux/libcfs/libcfs.h @@ -64,11 +64,21 @@ #define LNET_ACCEPTOR_MIN_RESERVED_PORT 512 #define LNET_ACCEPTOR_MAX_RESERVED_PORT 1023 -/* - * Defined by platform - */ -void cfs_block_sigsinv(unsigned long sigs, sigset_t *sigset); -void cfs_restore_sigs(sigset_t *sigset); +/* Block all signals except for the @sigs */ +static inline void cfs_block_sigsinv(unsigned long sigs, sigset_t *old) +{ + sigset_t new; + + siginitsetinv(&new, sigs); + sigorsets(&new, ¤t->blocked, &new); + sigprocmask(SIG_BLOCK, &new, old); +} + +static inline void +cfs_restore_sigs(sigset_t *old) +{ + sigprocmask(SIG_SETMASK, old, NULL); +} struct libcfs_ioctl_handler { struct list_head item; diff --git a/drivers/staging/lustre/lnet/libcfs/Makefile b/drivers/staging/lustre/lnet/libcfs/Makefile index fccea8684f0c..a51a8b0b9921 100644 --- a/drivers/staging/lustre/lnet/libcfs/Makefile +++ b/drivers/staging/lustre/lnet/libcfs/Makefile @@ -5,7 +5,7 @@ subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/lustre/include obj-$(CONFIG_LNET) += libcfs.o libcfs-linux-objs := linux-tracefile.o linux-debug.o -libcfs-linux-objs += linux-prim.o linux-cpu.o +libcfs-linux-objs += linux-cpu.o libcfs-linux-objs += linux-module.o libcfs-linux-objs += linux-crypto.o libcfs-linux-objs += linux-crypto-adler.o diff --git a/drivers/staging/lustre/lnet/libcfs/linux/linux-prim.c b/drivers/staging/lustre/lnet/libcfs/linux/linux-prim.c deleted file mode 100644 index 0766659ddf70..000000000000 --- a/drivers/staging/lustre/lnet/libcfs/linux/linux-prim.c +++ /dev/null @@ -1,62 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0 -/* - * GPL HEADER START - * - * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. - * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License version 2 only, - * as published by the Free Software Foundation. - * - * This program is distributed in the hope that it will be useful, but - * WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - * General Public License version 2 for more details (a copy is included - * in the LICENSE file that accompanied this code). - * - * You should have received a copy of the GNU General Public License - * version 2 along with this program; If not, see - * http://www.gnu.org/licenses/gpl-2.0.html - * - * GPL HEADER END - */ -/* - * Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved. - * Use is subject to license terms. - * - * Copyright (c) 2011, 2012, Intel Corporation. - */ -/* - * This file is part of Lustre, http://www.lustre.org/ - * Lustre is a trademark of Sun Microsystems, Inc. - */ - -#define DEBUG_SUBSYSTEM S_LNET -#include -#include -#include -#include - -#include - -#if defined(CONFIG_KGDB) -#include -#endif - -/* Block all signals except for the @sigs */ -void cfs_block_sigsinv(unsigned long sigs, sigset_t *old) -{ - sigset_t new; - - siginitsetinv(&new, sigs); - sigorsets(&new, ¤t->blocked, &new); - sigprocmask(SIG_BLOCK, &new, old); -} -EXPORT_SYMBOL(cfs_block_sigsinv); - -void -cfs_restore_sigs(sigset_t *old) -{ - sigprocmask(SIG_SETMASK, old, NULL); -} -EXPORT_SYMBOL(cfs_restore_sigs); From neilb at suse.com Tue Feb 20 02:23:37 2018 From: neilb at suse.com (NeilBrown) Date: Tue, 20 Feb 2018 13:23:37 +1100 Subject: [lustre-devel] [PATCH 09/21] staging: lustre: discard libcfs_kvzalloc_cpt() In-Reply-To: <151909308778.25814.9912662268627044409.stgit@noble> References: <151909308778.25814.9912662268627044409.stgit@noble> Message-ID: <151909341783.25814.12309771139592565643.stgit@noble> This function is used precisely once, and is sufficiently trivial that it may as well be open-coded. Doing so helpfully highlights the similarity between the new kvzalloc_node() call and the already existing kzalloc_node() call in the same function. Signed-off-by: NeilBrown --- .../staging/lustre/include/linux/libcfs/libcfs.h | 2 -- .../staging/lustre/lnet/libcfs/linux/linux-mem.c | 7 ------- drivers/staging/lustre/lustre/ptlrpc/service.c | 8 ++++---- 3 files changed, 4 insertions(+), 13 deletions(-) diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs.h b/drivers/staging/lustre/include/linux/libcfs/libcfs.h index 4f8c65e6d883..039205763021 100644 --- a/drivers/staging/lustre/include/linux/libcfs/libcfs.h +++ b/drivers/staging/lustre/include/linux/libcfs/libcfs.h @@ -113,8 +113,6 @@ static inline void *__container_of(void *ptr, unsigned long shift) #define _LIBCFS_H void *libcfs_kvzalloc(size_t size, gfp_t flags); -void *libcfs_kvzalloc_cpt(struct cfs_cpt_table *cptab, int cpt, size_t size, - gfp_t flags); extern struct miscdevice libcfs_dev; /** diff --git a/drivers/staging/lustre/lnet/libcfs/linux/linux-mem.c b/drivers/staging/lustre/lnet/libcfs/linux/linux-mem.c index 963df0ef4afb..f2c001bac303 100644 --- a/drivers/staging/lustre/lnet/libcfs/linux/linux-mem.c +++ b/drivers/staging/lustre/lnet/libcfs/linux/linux-mem.c @@ -42,10 +42,3 @@ void *libcfs_kvzalloc(size_t size, gfp_t flags) return ret; } EXPORT_SYMBOL(libcfs_kvzalloc); - -void *libcfs_kvzalloc_cpt(struct cfs_cpt_table *cptab, int cpt, size_t size, - gfp_t flags) -{ - return kvzalloc_node(size, flags, cfs_cpt_spread_node(cptab, cpt)); -} -EXPORT_SYMBOL(libcfs_kvzalloc_cpt); diff --git a/drivers/staging/lustre/lustre/ptlrpc/service.c b/drivers/staging/lustre/lustre/ptlrpc/service.c index 29fdb54f16ca..57e41e2cd30a 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/service.c +++ b/drivers/staging/lustre/lustre/ptlrpc/service.c @@ -83,10 +83,10 @@ ptlrpc_alloc_rqbd(struct ptlrpc_service_part *svcpt) rqbd->rqbd_cbid.cbid_fn = request_in_callback; rqbd->rqbd_cbid.cbid_arg = rqbd; INIT_LIST_HEAD(&rqbd->rqbd_reqs); - rqbd->rqbd_buffer = libcfs_kvzalloc_cpt(svc->srv_cptable, - svcpt->scp_cpt, - svc->srv_buf_size, - GFP_KERNEL); + rqbd->rqbd_buffer = kvzalloc_node(svc->srv_buf_size, GFP_KERNEL, + cfs_cpt_spread_node(svc->srv_cptable, + svcpt->scp_cpt)); + if (!rqbd->rqbd_buffer) { kfree(rqbd); return NULL; From neilb at suse.com Tue Feb 20 02:23:37 2018 From: neilb at suse.com (NeilBrown) Date: Tue, 20 Feb 2018 13:23:37 +1100 Subject: [lustre-devel] [PATCH 10/21] staging: lustre: discard lu_buf allocation library. In-Reply-To: <151909308778.25814.9912662268627044409.stgit@noble> References: <151909308778.25814.9912662268627044409.stgit@noble> Message-ID: <151909341787.25814.12742683712598697203.stgit@noble> This library code is unnecessarily generic, but also not generic enough. Library code that performs allocations should always take a gfp_flags argument. So discard the library and in the one file where it is used, just use kzalloc or krealloc as needed. In this context, it is clear that vmalloc is never needed. Signed-off-by: NeilBrown --- drivers/staging/lustre/lustre/include/lu_object.h | 7 -- drivers/staging/lustre/lustre/llite/llite_lib.c | 2 - drivers/staging/lustre/lustre/obdclass/linkea.c | 16 +++-- drivers/staging/lustre/lustre/obdclass/lu_object.c | 70 -------------------- 4 files changed, 12 insertions(+), 83 deletions(-) diff --git a/drivers/staging/lustre/lustre/include/lu_object.h b/drivers/staging/lustre/lustre/include/lu_object.h index 34e35fbff978..35c7b582f36d 100644 --- a/drivers/staging/lustre/lustre/include/lu_object.h +++ b/drivers/staging/lustre/lustre/include/lu_object.h @@ -1328,13 +1328,6 @@ struct lu_kmem_descr { int lu_kmem_init(struct lu_kmem_descr *caches); void lu_kmem_fini(struct lu_kmem_descr *caches); -void lu_buf_free(struct lu_buf *buf); -void lu_buf_alloc(struct lu_buf *buf, size_t size); -void lu_buf_realloc(struct lu_buf *buf, size_t size); - -int lu_buf_check_and_grow(struct lu_buf *buf, size_t len); -struct lu_buf *lu_buf_check_and_alloc(struct lu_buf *buf, size_t len); - extern __u32 lu_context_tags_default; extern __u32 lu_session_tags_default; diff --git a/drivers/staging/lustre/lustre/llite/llite_lib.c b/drivers/staging/lustre/lustre/llite/llite_lib.c index 07072ab92bb6..efbd551e7842 100644 --- a/drivers/staging/lustre/lustre/llite/llite_lib.c +++ b/drivers/staging/lustre/lustre/llite/llite_lib.c @@ -2648,7 +2648,7 @@ int ll_getparent(struct file *file, struct getparent __user *arg) } lb_free: - lu_buf_free(&buf); + kvfree(buf.lb_buf); ldata_free: kfree(ldata); return rc; diff --git a/drivers/staging/lustre/lustre/obdclass/linkea.c b/drivers/staging/lustre/lustre/obdclass/linkea.c index fe1638b0916e..74c99ee216bb 100644 --- a/drivers/staging/lustre/lustre/obdclass/linkea.c +++ b/drivers/staging/lustre/lustre/obdclass/linkea.c @@ -33,9 +33,11 @@ int linkea_data_new(struct linkea_data *ldata, struct lu_buf *buf) { - ldata->ld_buf = lu_buf_check_and_alloc(buf, PAGE_SIZE); - if (!ldata->ld_buf->lb_buf) + buf->lb_buf = kzalloc(PAGE_SIZE, GFP_NOFS); + if (!buf->lb_buf) return -ENOMEM; + buf->lb_len = PAGE_SIZE; + ldata->ld_buf = buf; ldata->ld_leh = ldata->ld_buf->lb_buf; ldata->ld_leh->leh_magic = LINK_EA_MAGIC; ldata->ld_leh->leh_len = sizeof(struct link_ea_header); @@ -158,11 +160,15 @@ int linkea_add_buf(struct linkea_data *ldata, const struct lu_name *lname, } if (leh->leh_len + reclen > ldata->ld_buf->lb_len) { - if (lu_buf_check_and_grow(ldata->ld_buf, - leh->leh_len + reclen) < 0) + /* Note: this never happens as MAX_LINKEA_SIZE is 4096, while + * the initial allocation is PAGE_SIZE. + */ + void *b = krealloc(ldata->ld_buf->lb_buf, leh->leh_len + reclen, GFP_NOFS); + if (!b) return -ENOMEM; - leh = ldata->ld_leh = ldata->ld_buf->lb_buf; + ldata->ld_buf->lb_len = leh->leh_len + reclen; + leh = ldata->ld_leh = ldata->ld_buf->lb_buf = b; } ldata->ld_lee = ldata->ld_buf->lb_buf + leh->leh_len; diff --git a/drivers/staging/lustre/lustre/obdclass/lu_object.c b/drivers/staging/lustre/lustre/obdclass/lu_object.c index 2719abbff85f..cca688175d2d 100644 --- a/drivers/staging/lustre/lustre/obdclass/lu_object.c +++ b/drivers/staging/lustre/lustre/obdclass/lu_object.c @@ -2061,73 +2061,3 @@ void lu_kmem_fini(struct lu_kmem_descr *caches) } } EXPORT_SYMBOL(lu_kmem_fini); - -void lu_buf_free(struct lu_buf *buf) -{ - LASSERT(buf); - if (buf->lb_buf) { - LASSERT(buf->lb_len > 0); - kvfree(buf->lb_buf); - buf->lb_buf = NULL; - buf->lb_len = 0; - } -} -EXPORT_SYMBOL(lu_buf_free); - -void lu_buf_alloc(struct lu_buf *buf, size_t size) -{ - LASSERT(buf); - LASSERT(!buf->lb_buf); - LASSERT(!buf->lb_len); - buf->lb_buf = libcfs_kvzalloc(size, GFP_NOFS); - if (likely(buf->lb_buf)) - buf->lb_len = size; -} -EXPORT_SYMBOL(lu_buf_alloc); - -void lu_buf_realloc(struct lu_buf *buf, size_t size) -{ - lu_buf_free(buf); - lu_buf_alloc(buf, size); -} -EXPORT_SYMBOL(lu_buf_realloc); - -struct lu_buf *lu_buf_check_and_alloc(struct lu_buf *buf, size_t len) -{ - if (!buf->lb_buf && !buf->lb_len) - lu_buf_alloc(buf, len); - - if ((len > buf->lb_len) && buf->lb_buf) - lu_buf_realloc(buf, len); - - return buf; -} -EXPORT_SYMBOL(lu_buf_check_and_alloc); - -/** - * Increase the size of the \a buf. - * preserves old data in buffer - * old buffer remains unchanged on error - * \retval 0 or -ENOMEM - */ -int lu_buf_check_and_grow(struct lu_buf *buf, size_t len) -{ - char *ptr; - - if (len <= buf->lb_len) - return 0; - - ptr = libcfs_kvzalloc(len, GFP_NOFS); - if (!ptr) - return -ENOMEM; - - /* Free the old buf */ - if (buf->lb_buf) { - memcpy(ptr, buf->lb_buf, buf->lb_len); - kvfree(buf->lb_buf); - } - - buf->lb_buf = ptr; - buf->lb_len = len; - return 0; -} From neilb at suse.com Tue Feb 20 02:23:37 2018 From: neilb at suse.com (NeilBrown) Date: Tue, 20 Feb 2018 13:23:37 +1100 Subject: [lustre-devel] [PATCH 11/21] staging: lustre: improve some libcfs_kvzalloc calls. In-Reply-To: <151909308778.25814.9912662268627044409.stgit@noble> References: <151909308778.25814.9912662268627044409.stgit@noble> Message-ID: <151909341790.25814.17500193577187717136.stgit@noble> Using vmalloc with GFP_NOFS is not supported as vmalloc performs some internal allocations with GFP_KERNEL. So in cases where the size passed to libcfs_kvzalloc() is clearly at most 1 page, convert to kzalloc(). In cases where the call clearly doesn't hold any filesystem locks, convert to GFP_KERNEL. Unfortunately there are many more that are not easy to fix. Signed-off-by: NeilBrown --- drivers/staging/lustre/lustre/llite/dir.c | 2 +- drivers/staging/lustre/lustre/llite/file.c | 4 ++-- .../lustre/lustre/obdclass/linux/linux-module.c | 2 +- drivers/staging/lustre/lustre/obdclass/llog.c | 2 +- .../lustre/lustre/obdclass/lustre_handles.c | 2 +- drivers/staging/lustre/lustre/ptlrpc/client.c | 4 ++-- 6 files changed, 8 insertions(+), 8 deletions(-) diff --git a/drivers/staging/lustre/lustre/llite/dir.c b/drivers/staging/lustre/lustre/llite/dir.c index 09e3a4999079..d10d27268323 100644 --- a/drivers/staging/lustre/lustre/llite/dir.c +++ b/drivers/staging/lustre/lustre/llite/dir.c @@ -1497,7 +1497,7 @@ static long ll_dir_ioctl(struct file *file, unsigned int cmd, unsigned long arg) if (totalsize >= MDS_MAXREQSIZE / 3) return -E2BIG; - hur = libcfs_kvzalloc(totalsize, GFP_NOFS); + hur = kzalloc(totalsize, GFP_NOFS); if (!hur) return -ENOMEM; diff --git a/drivers/staging/lustre/lustre/llite/file.c b/drivers/staging/lustre/lustre/llite/file.c index 4aad2e331948..002a56793e89 100644 --- a/drivers/staging/lustre/lustre/llite/file.c +++ b/drivers/staging/lustre/lustre/llite/file.c @@ -1318,7 +1318,7 @@ static int ll_lov_setea(struct inode *inode, struct file *file, if (!capable(CAP_SYS_ADMIN)) return -EPERM; - lump = libcfs_kvzalloc(lum_size, GFP_NOFS); + lump = kzalloc(lum_size, GFP_NOFS); if (!lump) return -ENOMEM; @@ -2998,7 +2998,7 @@ static int ll_fiemap(struct inode *inode, struct fiemap_extent_info *fieinfo, num_bytes = sizeof(*fiemap) + (extent_count * sizeof(struct fiemap_extent)); - fiemap = libcfs_kvzalloc(num_bytes, GFP_NOFS); + fiemap = kvzalloc(num_bytes, GFP_KERNEL); if (!fiemap) return -ENOMEM; diff --git a/drivers/staging/lustre/lustre/obdclass/linux/linux-module.c b/drivers/staging/lustre/lustre/obdclass/linux/linux-module.c index 5b1122c408fb..f8967fd44363 100644 --- a/drivers/staging/lustre/lustre/obdclass/linux/linux-module.c +++ b/drivers/staging/lustre/lustre/obdclass/linux/linux-module.c @@ -180,7 +180,7 @@ int obd_ioctl_getdata(char **buf, int *len, void __user *arg) * obdfilter-survey is an example, which relies on ioctl. So we'd * better avoid vmalloc on ioctl path. LU-66 */ - *buf = libcfs_kvzalloc(hdr.ioc_len, GFP_NOFS); + *buf = libcfs_kvzalloc(hdr.ioc_len, GFP_KERNEL); if (!*buf) { CERROR("Cannot allocate control buffer of len %d\n", hdr.ioc_len); diff --git a/drivers/staging/lustre/lustre/obdclass/llog.c b/drivers/staging/lustre/lustre/obdclass/llog.c index 934f067adb14..ed310696a95d 100644 --- a/drivers/staging/lustre/lustre/obdclass/llog.c +++ b/drivers/staging/lustre/lustre/obdclass/llog.c @@ -155,7 +155,7 @@ int llog_init_handle(const struct lu_env *env, struct llog_handle *handle, LASSERT(!handle->lgh_hdr); LASSERT(chunk_size >= LLOG_MIN_CHUNK_SIZE); - llh = libcfs_kvzalloc(sizeof(*llh), GFP_NOFS); + llh = libcfs_kvzalloc(sizeof(*llh), GFP_KERNEL); if (!llh) return -ENOMEM; handle->lgh_hdr = llh; diff --git a/drivers/staging/lustre/lustre/obdclass/lustre_handles.c b/drivers/staging/lustre/lustre/obdclass/lustre_handles.c index 2d6da2431a09..9a6377502ae4 100644 --- a/drivers/staging/lustre/lustre/obdclass/lustre_handles.c +++ b/drivers/staging/lustre/lustre/obdclass/lustre_handles.c @@ -185,7 +185,7 @@ int class_handle_init(void) LASSERT(!handle_hash); handle_hash = libcfs_kvzalloc(sizeof(*bucket) * HANDLE_HASH_SIZE, - GFP_NOFS); + GFP_KERNEL); if (!handle_hash) return -ENOMEM; diff --git a/drivers/staging/lustre/lustre/ptlrpc/client.c b/drivers/staging/lustre/lustre/ptlrpc/client.c index 3d689d6100bc..781462621d92 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/client.c +++ b/drivers/staging/lustre/lustre/ptlrpc/client.c @@ -544,10 +544,10 @@ int ptlrpc_add_rqs_to_pool(struct ptlrpc_request_pool *pool, int num_rq) struct lustre_msg *msg; spin_unlock(&pool->prp_lock); - req = ptlrpc_request_cache_alloc(GFP_NOFS); + req = ptlrpc_request_cache_alloc(GFP_KERNEL); if (!req) return i; - msg = libcfs_kvzalloc(size, GFP_NOFS); + msg = libcfs_kvzalloc(size, GFP_KERNEL); if (!msg) { ptlrpc_request_cache_free(req); return i; From neilb at suse.com Tue Feb 20 02:23:37 2018 From: neilb at suse.com (NeilBrown) Date: Tue, 20 Feb 2018 13:23:37 +1100 Subject: [lustre-devel] [PATCH 12/21] staging: lustre: discard libcfs_kvzalloc and linux-mem.c In-Reply-To: <151909308778.25814.9912662268627044409.stgit@noble> References: <151909308778.25814.9912662268627044409.stgit@noble> Message-ID: <151909341794.25814.12112685405079106591.stgit@noble> The only interesting difference between libcfs_kvzalloc() and kvzalloc() is that the former appears to work with GFP_NOFS, which the latter gives a WARN_ON_ONCE() when that is attempted. Each libcfs_kvzalloc() should really be analysed and either converted to a kzalloc() call if the size is never more than a page, or to use GFP_KERNEL if no locks are held. If there is ever a case where locks are held and a large allocation is needed, then some other technique should be used. It might be nice to not always blindly zero pages too... For now, just convert libcfs_kvzalloc() calls to kvzalloc(), and let the warning remind us that there is work to do. Signed-off-by: NeilBrown --- .../staging/lustre/include/linux/libcfs/libcfs.h | 2 - drivers/staging/lustre/lnet/libcfs/Makefile | 1 .../staging/lustre/lnet/libcfs/linux/linux-mem.c | 44 -------------------- drivers/staging/lustre/lustre/llite/file.c | 2 - drivers/staging/lustre/lustre/lmv/lmv_obd.c | 4 +- drivers/staging/lustre/lustre/lov/lov_ea.c | 2 - drivers/staging/lustre/lustre/lov/lov_io.c | 2 - drivers/staging/lustre/lustre/lov/lov_lock.c | 2 - drivers/staging/lustre/lustre/lov/lov_object.c | 4 +- drivers/staging/lustre/lustre/lov/lov_pack.c | 2 - drivers/staging/lustre/lustre/mdc/mdc_locks.c | 2 - .../lustre/lustre/obdclass/linux/linux-module.c | 2 - drivers/staging/lustre/lustre/obdclass/llog.c | 4 +- .../lustre/lustre/obdclass/lustre_handles.c | 2 - drivers/staging/lustre/lustre/ptlrpc/client.c | 2 - drivers/staging/lustre/lustre/ptlrpc/sec.c | 4 +- drivers/staging/lustre/lustre/ptlrpc/sec_bulk.c | 2 - drivers/staging/lustre/lustre/ptlrpc/sec_null.c | 8 ++-- drivers/staging/lustre/lustre/ptlrpc/sec_plain.c | 8 ++-- drivers/staging/lustre/lustre/ptlrpc/service.c | 4 +- 20 files changed, 28 insertions(+), 75 deletions(-) delete mode 100644 drivers/staging/lustre/lnet/libcfs/linux/linux-mem.c diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs.h b/drivers/staging/lustre/include/linux/libcfs/libcfs.h index 039205763021..392793582956 100644 --- a/drivers/staging/lustre/include/linux/libcfs/libcfs.h +++ b/drivers/staging/lustre/include/linux/libcfs/libcfs.h @@ -112,8 +112,6 @@ static inline void *__container_of(void *ptr, unsigned long shift) #define _LIBCFS_H -void *libcfs_kvzalloc(size_t size, gfp_t flags); - extern struct miscdevice libcfs_dev; /** * The path of debug log dump upcall script. diff --git a/drivers/staging/lustre/lnet/libcfs/Makefile b/drivers/staging/lustre/lnet/libcfs/Makefile index a51a8b0b9921..b7dc7ac11cc5 100644 --- a/drivers/staging/lustre/lnet/libcfs/Makefile +++ b/drivers/staging/lustre/lnet/libcfs/Makefile @@ -9,7 +9,6 @@ libcfs-linux-objs += linux-cpu.o libcfs-linux-objs += linux-module.o libcfs-linux-objs += linux-crypto.o libcfs-linux-objs += linux-crypto-adler.o -libcfs-linux-objs += linux-mem.o libcfs-linux-objs := $(addprefix linux/,$(libcfs-linux-objs)) diff --git a/drivers/staging/lustre/lnet/libcfs/linux/linux-mem.c b/drivers/staging/lustre/lnet/libcfs/linux/linux-mem.c deleted file mode 100644 index f2c001bac303..000000000000 --- a/drivers/staging/lustre/lnet/libcfs/linux/linux-mem.c +++ /dev/null @@ -1,44 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0 -/* - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License version 2 only, - * as published by the Free Software Foundation. - * - * This program is distributed in the hope that it will be useful, but - * WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - * General Public License version 2 for more details (a copy is included - * in the LICENSE file that accompanied this code). - * - * You should have received a copy of the GNU General Public License - * version 2 along with this program; If not, see - * http://www.gnu.org/licenses/gpl-2.0.html - * - */ -/* - * This file creates a memory allocation primitive for Lustre, that - * allows to fallback to vmalloc allocations should regular kernel allocations - * fail due to size or system memory fragmentation. - * - * Author: Oleg Drokin - * - */ -/* - * This file is part of Lustre, http://www.lustre.org/ - * Lustre is a trademark of Seagate Technology. - */ -#include -#include - -#include - -void *libcfs_kvzalloc(size_t size, gfp_t flags) -{ - void *ret; - - ret = kzalloc(size, flags | __GFP_NOWARN); - if (!ret) - ret = __vmalloc(size, flags | __GFP_ZERO, PAGE_KERNEL); - return ret; -} -EXPORT_SYMBOL(libcfs_kvzalloc); diff --git a/drivers/staging/lustre/lustre/llite/file.c b/drivers/staging/lustre/lustre/llite/file.c index 002a56793e89..ca5faea13b7e 100644 --- a/drivers/staging/lustre/lustre/llite/file.c +++ b/drivers/staging/lustre/lustre/llite/file.c @@ -3361,7 +3361,7 @@ static int ll_layout_fetch(struct inode *inode, struct ldlm_lock *lock) goto out; } - lvbdata = libcfs_kvzalloc(lmmsize, GFP_NOFS); + lvbdata = kvzalloc(lmmsize, GFP_NOFS); if (!lvbdata) { rc = -ENOMEM; goto out; diff --git a/drivers/staging/lustre/lustre/lmv/lmv_obd.c b/drivers/staging/lustre/lustre/lmv/lmv_obd.c index c2c57f65431e..179651531862 100644 --- a/drivers/staging/lustre/lustre/lmv/lmv_obd.c +++ b/drivers/staging/lustre/lustre/lmv/lmv_obd.c @@ -1035,7 +1035,7 @@ static int lmv_iocontrol(unsigned int cmd, struct obd_export *exp, reqlen = offsetof(typeof(*hur), hur_user_item[nr]) + hur->hur_request.hr_data_len; - req = libcfs_kvzalloc(reqlen, GFP_NOFS); + req = kvzalloc(reqlen, GFP_NOFS); if (!req) return -ENOMEM; @@ -2733,7 +2733,7 @@ static int lmv_unpackmd(struct obd_export *exp, struct lmv_stripe_md **lsmp, lsm_size = lmv_stripe_md_size(0); if (!lsm) { - lsm = libcfs_kvzalloc(lsm_size, GFP_NOFS); + lsm = kvzalloc(lsm_size, GFP_NOFS); if (!lsm) return -ENOMEM; allocated = true; diff --git a/drivers/staging/lustre/lustre/lov/lov_ea.c b/drivers/staging/lustre/lustre/lov/lov_ea.c index d563dd73343a..c56a971745e8 100644 --- a/drivers/staging/lustre/lustre/lov/lov_ea.c +++ b/drivers/staging/lustre/lustre/lov/lov_ea.c @@ -89,7 +89,7 @@ struct lov_stripe_md *lsm_alloc_plain(u16 stripe_count) oinfo_ptrs_size = sizeof(struct lov_oinfo *) * stripe_count; lsm_size = sizeof(*lsm) + oinfo_ptrs_size; - lsm = libcfs_kvzalloc(lsm_size, GFP_NOFS); + lsm = kvzalloc(lsm_size, GFP_NOFS); if (!lsm) return NULL; diff --git a/drivers/staging/lustre/lustre/lov/lov_io.c b/drivers/staging/lustre/lustre/lov/lov_io.c index c5f5d1b106dc..c0dbf6cd53b4 100644 --- a/drivers/staging/lustre/lustre/lov/lov_io.c +++ b/drivers/staging/lustre/lustre/lov/lov_io.c @@ -243,7 +243,7 @@ static int lov_io_subio_init(const struct lu_env *env, struct lov_io *lio, * when writing a page. -jay */ lio->lis_subs = - libcfs_kvzalloc(lsm->lsm_stripe_count * + kvzalloc(lsm->lsm_stripe_count * sizeof(lio->lis_subs[0]), GFP_NOFS); if (lio->lis_subs) { diff --git a/drivers/staging/lustre/lustre/lov/lov_lock.c b/drivers/staging/lustre/lustre/lov/lov_lock.c index 2fcdeb707ff9..b0292100bf26 100644 --- a/drivers/staging/lustre/lustre/lov/lov_lock.c +++ b/drivers/staging/lustre/lustre/lov/lov_lock.c @@ -145,7 +145,7 @@ static struct lov_lock *lov_lock_sub_init(const struct lu_env *env, nr++; } LASSERT(nr > 0); - lovlck = libcfs_kvzalloc(offsetof(struct lov_lock, lls_sub[nr]), + lovlck = kvzalloc(offsetof(struct lov_lock, lls_sub[nr]), GFP_NOFS); if (!lovlck) return ERR_PTR(-ENOMEM); diff --git a/drivers/staging/lustre/lustre/lov/lov_object.c b/drivers/staging/lustre/lustre/lov/lov_object.c index 86cd4f9fbd0c..16cf9d584bbc 100644 --- a/drivers/staging/lustre/lustre/lov/lov_object.c +++ b/drivers/staging/lustre/lustre/lov/lov_object.c @@ -242,7 +242,7 @@ static int lov_init_raid0(const struct lu_env *env, struct lov_device *dev, r0->lo_nr = lsm->lsm_stripe_count; LASSERT(r0->lo_nr <= lov_targets_nr(dev)); - r0->lo_sub = libcfs_kvzalloc(r0->lo_nr * sizeof(r0->lo_sub[0]), + r0->lo_sub = kvzalloc(r0->lo_nr * sizeof(r0->lo_sub[0]), GFP_NOFS); if (r0->lo_sub) { int psz = 0; @@ -1375,7 +1375,7 @@ static int lov_object_fiemap(const struct lu_env *env, struct cl_object *obj, if (fiemap_count_to_size(fiemap->fm_extent_count) < buffer_size) buffer_size = fiemap_count_to_size(fiemap->fm_extent_count); - fm_local = libcfs_kvzalloc(buffer_size, GFP_NOFS); + fm_local = kvzalloc(buffer_size, GFP_NOFS); if (!fm_local) { rc = -ENOMEM; goto out; diff --git a/drivers/staging/lustre/lustre/lov/lov_pack.c b/drivers/staging/lustre/lustre/lov/lov_pack.c index e5b11c4085a9..b1060d02a164 100644 --- a/drivers/staging/lustre/lustre/lov/lov_pack.c +++ b/drivers/staging/lustre/lustre/lov/lov_pack.c @@ -333,7 +333,7 @@ int lov_getstripe(struct lov_object *obj, struct lov_stripe_md *lsm, lmmk_size = lov_mds_md_size(stripe_count, lsm->lsm_magic); - lmmk = libcfs_kvzalloc(lmmk_size, GFP_NOFS); + lmmk = kvzalloc(lmmk_size, GFP_NOFS); if (!lmmk) { rc = -ENOMEM; goto out; diff --git a/drivers/staging/lustre/lustre/mdc/mdc_locks.c b/drivers/staging/lustre/lustre/mdc/mdc_locks.c index 3114907ac5ff..695ef44532cf 100644 --- a/drivers/staging/lustre/lustre/mdc/mdc_locks.c +++ b/drivers/staging/lustre/lustre/mdc/mdc_locks.c @@ -660,7 +660,7 @@ static int mdc_finish_enqueue(struct obd_export *exp, LDLM_DEBUG(lock, "layout lock returned by: %s, lvb_len: %d", ldlm_it2str(it->it_op), lvb_len); - lmm = libcfs_kvzalloc(lvb_len, GFP_NOFS); + lmm = kvzalloc(lvb_len, GFP_NOFS); if (!lmm) { LDLM_LOCK_PUT(lock); return -ENOMEM; diff --git a/drivers/staging/lustre/lustre/obdclass/linux/linux-module.c b/drivers/staging/lustre/lustre/obdclass/linux/linux-module.c index f8967fd44363..7bceee7f121e 100644 --- a/drivers/staging/lustre/lustre/obdclass/linux/linux-module.c +++ b/drivers/staging/lustre/lustre/obdclass/linux/linux-module.c @@ -180,7 +180,7 @@ int obd_ioctl_getdata(char **buf, int *len, void __user *arg) * obdfilter-survey is an example, which relies on ioctl. So we'd * better avoid vmalloc on ioctl path. LU-66 */ - *buf = libcfs_kvzalloc(hdr.ioc_len, GFP_KERNEL); + *buf = kvzalloc(hdr.ioc_len, GFP_KERNEL); if (!*buf) { CERROR("Cannot allocate control buffer of len %d\n", hdr.ioc_len); diff --git a/drivers/staging/lustre/lustre/obdclass/llog.c b/drivers/staging/lustre/lustre/obdclass/llog.c index ed310696a95d..693e1129f1f9 100644 --- a/drivers/staging/lustre/lustre/obdclass/llog.c +++ b/drivers/staging/lustre/lustre/obdclass/llog.c @@ -155,7 +155,7 @@ int llog_init_handle(const struct lu_env *env, struct llog_handle *handle, LASSERT(!handle->lgh_hdr); LASSERT(chunk_size >= LLOG_MIN_CHUNK_SIZE); - llh = libcfs_kvzalloc(sizeof(*llh), GFP_KERNEL); + llh = kvzalloc(sizeof(*llh), GFP_KERNEL); if (!llh) return -ENOMEM; handle->lgh_hdr = llh; @@ -240,7 +240,7 @@ static int llog_process_thread(void *arg) /* expect chunk_size to be power of two */ LASSERT(is_power_of_2(chunk_size)); - buf = libcfs_kvzalloc(chunk_size, GFP_NOFS); + buf = kvzalloc(chunk_size, GFP_NOFS); if (!buf) { lpi->lpi_rc = -ENOMEM; return 0; diff --git a/drivers/staging/lustre/lustre/obdclass/lustre_handles.c b/drivers/staging/lustre/lustre/obdclass/lustre_handles.c index 9a6377502ae4..f53b1a3c342e 100644 --- a/drivers/staging/lustre/lustre/obdclass/lustre_handles.c +++ b/drivers/staging/lustre/lustre/obdclass/lustre_handles.c @@ -184,7 +184,7 @@ int class_handle_init(void) LASSERT(!handle_hash); - handle_hash = libcfs_kvzalloc(sizeof(*bucket) * HANDLE_HASH_SIZE, + handle_hash = kvzalloc(sizeof(*bucket) * HANDLE_HASH_SIZE, GFP_KERNEL); if (!handle_hash) return -ENOMEM; diff --git a/drivers/staging/lustre/lustre/ptlrpc/client.c b/drivers/staging/lustre/lustre/ptlrpc/client.c index 781462621d92..d4c641d2480c 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/client.c +++ b/drivers/staging/lustre/lustre/ptlrpc/client.c @@ -547,7 +547,7 @@ int ptlrpc_add_rqs_to_pool(struct ptlrpc_request_pool *pool, int num_rq) req = ptlrpc_request_cache_alloc(GFP_KERNEL); if (!req) return i; - msg = libcfs_kvzalloc(size, GFP_KERNEL); + msg = kvzalloc(size, GFP_KERNEL); if (!msg) { ptlrpc_request_cache_free(req); return i; diff --git a/drivers/staging/lustre/lustre/ptlrpc/sec.c b/drivers/staging/lustre/lustre/ptlrpc/sec.c index 90e3b3022106..f152ba1af0fc 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/sec.c +++ b/drivers/staging/lustre/lustre/ptlrpc/sec.c @@ -442,7 +442,7 @@ int sptlrpc_req_ctx_switch(struct ptlrpc_request *req, /* save request message */ reqmsg_size = req->rq_reqlen; if (reqmsg_size != 0) { - reqmsg = libcfs_kvzalloc(reqmsg_size, GFP_NOFS); + reqmsg = kvzalloc(reqmsg_size, GFP_NOFS); if (!reqmsg) return -ENOMEM; memcpy(reqmsg, req->rq_reqmsg, reqmsg_size); @@ -1089,7 +1089,7 @@ int sptlrpc_cli_unwrap_early_reply(struct ptlrpc_request *req, early_size = req->rq_nob_received; early_bufsz = size_roundup_power2(early_size); - early_buf = libcfs_kvzalloc(early_bufsz, GFP_NOFS); + early_buf = kvzalloc(early_bufsz, GFP_NOFS); if (!early_buf) { rc = -ENOMEM; goto err_req; diff --git a/drivers/staging/lustre/lustre/ptlrpc/sec_bulk.c b/drivers/staging/lustre/lustre/ptlrpc/sec_bulk.c index 134ee727e8b7..2184022ed724 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/sec_bulk.c +++ b/drivers/staging/lustre/lustre/ptlrpc/sec_bulk.c @@ -375,7 +375,7 @@ static inline void enc_pools_alloc(void) { LASSERT(page_pools.epp_max_pools); page_pools.epp_pools = - libcfs_kvzalloc(page_pools.epp_max_pools * + kvzalloc(page_pools.epp_max_pools * sizeof(*page_pools.epp_pools), GFP_NOFS); } diff --git a/drivers/staging/lustre/lustre/ptlrpc/sec_null.c b/drivers/staging/lustre/lustre/ptlrpc/sec_null.c index 80cea0b24693..ecc387d1b9b4 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/sec_null.c +++ b/drivers/staging/lustre/lustre/ptlrpc/sec_null.c @@ -158,7 +158,7 @@ int null_alloc_reqbuf(struct ptlrpc_sec *sec, int alloc_size = size_roundup_power2(msgsize); LASSERT(!req->rq_pool); - req->rq_reqbuf = libcfs_kvzalloc(alloc_size, GFP_NOFS); + req->rq_reqbuf = kvzalloc(alloc_size, GFP_NOFS); if (!req->rq_reqbuf) return -ENOMEM; @@ -201,7 +201,7 @@ int null_alloc_repbuf(struct ptlrpc_sec *sec, msgsize = size_roundup_power2(msgsize); - req->rq_repbuf = libcfs_kvzalloc(msgsize, GFP_NOFS); + req->rq_repbuf = kvzalloc(msgsize, GFP_NOFS); if (!req->rq_repbuf) return -ENOMEM; @@ -246,7 +246,7 @@ int null_enlarge_reqbuf(struct ptlrpc_sec *sec, if (req->rq_reqbuf_len < newmsg_size) { alloc_size = size_roundup_power2(newmsg_size); - newbuf = libcfs_kvzalloc(alloc_size, GFP_NOFS); + newbuf = kvzalloc(alloc_size, GFP_NOFS); if (!newbuf) return -ENOMEM; @@ -317,7 +317,7 @@ int null_alloc_rs(struct ptlrpc_request *req, int msgsize) /* pre-allocated */ LASSERT(rs->rs_size >= rs_size); } else { - rs = libcfs_kvzalloc(rs_size, GFP_NOFS); + rs = kvzalloc(rs_size, GFP_NOFS); if (!rs) return -ENOMEM; diff --git a/drivers/staging/lustre/lustre/ptlrpc/sec_plain.c b/drivers/staging/lustre/lustre/ptlrpc/sec_plain.c index 44e34056515b..ec3d9af76b17 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/sec_plain.c +++ b/drivers/staging/lustre/lustre/ptlrpc/sec_plain.c @@ -562,7 +562,7 @@ int plain_alloc_reqbuf(struct ptlrpc_sec *sec, LASSERT(!req->rq_pool); alloc_len = size_roundup_power2(alloc_len); - req->rq_reqbuf = libcfs_kvzalloc(alloc_len, GFP_NOFS); + req->rq_reqbuf = kvzalloc(alloc_len, GFP_NOFS); if (!req->rq_reqbuf) return -ENOMEM; @@ -620,7 +620,7 @@ int plain_alloc_repbuf(struct ptlrpc_sec *sec, alloc_len = size_roundup_power2(alloc_len); - req->rq_repbuf = libcfs_kvzalloc(alloc_len, GFP_NOFS); + req->rq_repbuf = kvzalloc(alloc_len, GFP_NOFS); if (!req->rq_repbuf) return -ENOMEM; @@ -671,7 +671,7 @@ int plain_enlarge_reqbuf(struct ptlrpc_sec *sec, if (req->rq_reqbuf_len < newbuf_size) { newbuf_size = size_roundup_power2(newbuf_size); - newbuf = libcfs_kvzalloc(newbuf_size, GFP_NOFS); + newbuf = kvzalloc(newbuf_size, GFP_NOFS); if (!newbuf) return -ENOMEM; @@ -808,7 +808,7 @@ int plain_alloc_rs(struct ptlrpc_request *req, int msgsize) /* pre-allocated */ LASSERT(rs->rs_size >= rs_size); } else { - rs = libcfs_kvzalloc(rs_size, GFP_NOFS); + rs = kvzalloc(rs_size, GFP_NOFS); if (!rs) return -ENOMEM; diff --git a/drivers/staging/lustre/lustre/ptlrpc/service.c b/drivers/staging/lustre/lustre/ptlrpc/service.c index 57e41e2cd30a..79d9f3860022 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/service.c +++ b/drivers/staging/lustre/lustre/ptlrpc/service.c @@ -1068,7 +1068,7 @@ static int ptlrpc_at_send_early_reply(struct ptlrpc_request *req) reqcopy = ptlrpc_request_cache_alloc(GFP_NOFS); if (!reqcopy) return -ENOMEM; - reqmsg = libcfs_kvzalloc(req->rq_reqlen, GFP_NOFS); + reqmsg = kvzalloc(req->rq_reqlen, GFP_NOFS); if (!reqmsg) { rc = -ENOMEM; goto out_free; @@ -2077,7 +2077,7 @@ static int ptlrpc_main(void *arg) } /* Alloc reply state structure for this one */ - rs = libcfs_kvzalloc(svc->srv_max_reply_size, GFP_NOFS); + rs = kvzalloc(svc->srv_max_reply_size, GFP_NOFS); if (!rs) { rc = -ENOMEM; goto out_srv_fini; From neilb at suse.com Tue Feb 20 02:23:38 2018 From: neilb at suse.com (NeilBrown) Date: Tue, 20 Feb 2018 13:23:38 +1100 Subject: [lustre-devel] [PATCH 15/21] staging: lustre: ptlrpc: list_for_each improvements. In-Reply-To: <151909308778.25814.9912662268627044409.stgit@noble> References: <151909308778.25814.9912662268627044409.stgit@noble> Message-ID: <151909341806.25814.159210690547707508.stgit@noble> 1/ use list_for_each_entry_safe() instead of list_for_each_safe() and similar. 2/ use list_first_entry() and list_last_entry() where appropriate. 3/ When removing everything from a list, use while ((x = list_first_entry_or_null()) { as it makes the intent clear 4/ No need to take a spinlock in a structure that is about to be freed - we must have exclusive access at this stage. Signed-off-by: NeilBrown --- drivers/staging/lustre/lustre/ptlrpc/client.c | 89 +++++++----------------- drivers/staging/lustre/lustre/ptlrpc/import.c | 34 +++------ drivers/staging/lustre/lustre/ptlrpc/pinger.c | 8 +- drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c | 15 +--- drivers/staging/lustre/lustre/ptlrpc/recover.c | 26 +++---- drivers/staging/lustre/lustre/ptlrpc/service.c | 13 +--- 6 files changed, 59 insertions(+), 126 deletions(-) diff --git a/drivers/staging/lustre/lustre/ptlrpc/client.c b/drivers/staging/lustre/lustre/ptlrpc/client.c index d4c641d2480c..ca096fadb9c0 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/client.c +++ b/drivers/staging/lustre/lustre/ptlrpc/client.c @@ -504,19 +504,16 @@ void ptlrpc_request_cache_free(struct ptlrpc_request *req) */ void ptlrpc_free_rq_pool(struct ptlrpc_request_pool *pool) { - struct list_head *l, *tmp; struct ptlrpc_request *req; - spin_lock(&pool->prp_lock); - list_for_each_safe(l, tmp, &pool->prp_req_list) { - req = list_entry(l, struct ptlrpc_request, rq_list); + while ((req = list_first_entry_or_null(&pool->prp_req_list, + struct ptlrpc_request, rq_list))) { list_del(&req->rq_list); LASSERT(req->rq_reqbuf); LASSERT(req->rq_reqbuf_len == pool->prp_rq_size); kvfree(req->rq_reqbuf); ptlrpc_request_cache_free(req); } - spin_unlock(&pool->prp_lock); kfree(pool); } EXPORT_SYMBOL(ptlrpc_free_rq_pool); @@ -656,16 +653,13 @@ static void __ptlrpc_free_req_to_pool(struct ptlrpc_request *request) void ptlrpc_add_unreplied(struct ptlrpc_request *req) { struct obd_import *imp = req->rq_import; - struct list_head *tmp; struct ptlrpc_request *iter; assert_spin_locked(&imp->imp_lock); LASSERT(list_empty(&req->rq_unreplied_list)); /* unreplied list is sorted by xid in ascending order */ - list_for_each_prev(tmp, &imp->imp_unreplied_list) { - iter = list_entry(tmp, struct ptlrpc_request, - rq_unreplied_list); + list_for_each_entry_reverse(iter, &imp->imp_unreplied_list, rq_unreplied_list) { LASSERT(req->rq_xid != iter->rq_xid); if (req->rq_xid < iter->rq_xid) @@ -1001,18 +995,14 @@ struct ptlrpc_request_set *ptlrpc_prep_fcset(int max, set_producer_func func, */ void ptlrpc_set_destroy(struct ptlrpc_request_set *set) { - struct list_head *tmp; - struct list_head *next; + struct ptlrpc_request *req; int expected_phase; int n = 0; /* Requests on the set should either all be completed, or all be new */ expected_phase = (atomic_read(&set->set_remaining) == 0) ? RQ_PHASE_COMPLETE : RQ_PHASE_NEW; - list_for_each(tmp, &set->set_requests) { - struct ptlrpc_request *req = - list_entry(tmp, struct ptlrpc_request, rq_set_chain); - + list_for_each_entry(req, &set->set_requests, rq_set_chain) { LASSERT(req->rq_phase == expected_phase); n++; } @@ -1021,9 +1011,9 @@ void ptlrpc_set_destroy(struct ptlrpc_request_set *set) atomic_read(&set->set_remaining) == n, "%d / %d\n", atomic_read(&set->set_remaining), n); - list_for_each_safe(tmp, next, &set->set_requests) { - struct ptlrpc_request *req = - list_entry(tmp, struct ptlrpc_request, rq_set_chain); + while ((req = list_first_entry_or_null(&set->set_requests, + struct ptlrpc_request, + rq_set_chain))) { list_del_init(&req->rq_set_chain); LASSERT(req->rq_phase == expected_phase); @@ -1640,7 +1630,7 @@ static inline int ptlrpc_set_producer(struct ptlrpc_request_set *set) */ int ptlrpc_check_set(const struct lu_env *env, struct ptlrpc_request_set *set) { - struct list_head *tmp, *next; + struct ptlrpc_request *req, *next; struct list_head comp_reqs; int force_timer_recalc = 0; @@ -1648,9 +1638,7 @@ int ptlrpc_check_set(const struct lu_env *env, struct ptlrpc_request_set *set) return 1; INIT_LIST_HEAD(&comp_reqs); - list_for_each_safe(tmp, next, &set->set_requests) { - struct ptlrpc_request *req = - list_entry(tmp, struct ptlrpc_request, rq_set_chain); + list_for_each_entry_safe(req, next, &set->set_requests, rq_set_chain) { struct obd_import *imp = req->rq_import; int unregistered = 0; int rc = 0; @@ -2126,13 +2114,11 @@ int ptlrpc_expire_one_request(struct ptlrpc_request *req, int async_unlink) */ void ptlrpc_expired_set(struct ptlrpc_request_set *set) { - struct list_head *tmp; + struct ptlrpc_request *req; time64_t now = ktime_get_real_seconds(); /* A timeout expired. See which reqs it applies to... */ - list_for_each(tmp, &set->set_requests) { - struct ptlrpc_request *req = - list_entry(tmp, struct ptlrpc_request, rq_set_chain); + list_for_each_entry(req, &set->set_requests, rq_set_chain) { /* don't expire request waiting for context */ if (req->rq_wait_ctx) @@ -2173,13 +2159,10 @@ EXPORT_SYMBOL(ptlrpc_mark_interrupted); */ static void ptlrpc_interrupted_set(struct ptlrpc_request_set *set) { - struct list_head *tmp; - + struct ptlrpc_request *req; CDEBUG(D_RPCTRACE, "INTERRUPTED SET %p\n", set); - list_for_each(tmp, &set->set_requests) { - struct ptlrpc_request *req = - list_entry(tmp, struct ptlrpc_request, rq_set_chain); + list_for_each_entry(req, &set->set_requests, rq_set_chain) { if (req->rq_phase != RQ_PHASE_RPC && req->rq_phase != RQ_PHASE_UNREG_RPC) @@ -2194,14 +2177,12 @@ static void ptlrpc_interrupted_set(struct ptlrpc_request_set *set) */ int ptlrpc_set_next_timeout(struct ptlrpc_request_set *set) { - struct list_head *tmp; time64_t now = ktime_get_real_seconds(); int timeout = 0; struct ptlrpc_request *req; time64_t deadline; - list_for_each(tmp, &set->set_requests) { - req = list_entry(tmp, struct ptlrpc_request, rq_set_chain); + list_for_each_entry(req, &set->set_requests, rq_set_chain) { /* Request in-flight? */ if (!(((req->rq_phase == RQ_PHASE_RPC) && !req->rq_waiting) || @@ -2240,16 +2221,13 @@ int ptlrpc_set_next_timeout(struct ptlrpc_request_set *set) */ int ptlrpc_set_wait(struct ptlrpc_request_set *set) { - struct list_head *tmp; struct ptlrpc_request *req; int rc, timeout; if (set->set_producer) (void)ptlrpc_set_producer(set); else - list_for_each(tmp, &set->set_requests) { - req = list_entry(tmp, struct ptlrpc_request, - rq_set_chain); + list_for_each_entry(req, &set->set_requests, rq_set_chain) { if (req->rq_phase == RQ_PHASE_NEW) (void)ptlrpc_send_new_req(req); } @@ -2322,9 +2300,7 @@ int ptlrpc_set_wait(struct ptlrpc_request_set *set) * the error cases -eeb. */ if (rc == 0 && atomic_read(&set->set_remaining) == 0) { - list_for_each(tmp, &set->set_requests) { - req = list_entry(tmp, struct ptlrpc_request, - rq_set_chain); + list_for_each_entry(req, &set->set_requests, rq_set_chain) { spin_lock(&req->rq_lock); req->rq_invalid_rqset = 1; spin_unlock(&req->rq_lock); @@ -2335,9 +2311,7 @@ int ptlrpc_set_wait(struct ptlrpc_request_set *set) LASSERT(atomic_read(&set->set_remaining) == 0); rc = set->set_rc; /* rq_status of already freed requests if any */ - list_for_each(tmp, &set->set_requests) { - req = list_entry(tmp, struct ptlrpc_request, rq_set_chain); - + list_for_each_entry(req, &set->set_requests, rq_set_chain) { LASSERT(req->rq_phase == RQ_PHASE_COMPLETE); if (req->rq_status != 0) rc = req->rq_status; @@ -2716,8 +2690,7 @@ EXPORT_SYMBOL(ptlrpc_request_addref); void ptlrpc_retain_replayable_request(struct ptlrpc_request *req, struct obd_import *imp) { - struct list_head *tmp; - + struct ptlrpc_request *iter; assert_spin_locked(&imp->imp_lock); if (req->rq_transno == 0) { @@ -2744,10 +2717,7 @@ void ptlrpc_retain_replayable_request(struct ptlrpc_request *req, LASSERT(imp->imp_replayable); /* Balanced in ptlrpc_free_committed, usually. */ ptlrpc_request_addref(req); - list_for_each_prev(tmp, &imp->imp_replay_list) { - struct ptlrpc_request *iter = - list_entry(tmp, struct ptlrpc_request, rq_replay_list); - + list_for_each_entry_reverse(iter, &imp->imp_replay_list, rq_replay_list) { /* * We may have duplicate transnos if we create and then * open a file, or for closes retained if to match creating @@ -2955,7 +2925,7 @@ int ptlrpc_replay_req(struct ptlrpc_request *req) */ void ptlrpc_abort_inflight(struct obd_import *imp) { - struct list_head *tmp, *n; + struct ptlrpc_request *req, *n; /* * Make sure that no new requests get processed for this import. @@ -2969,10 +2939,7 @@ void ptlrpc_abort_inflight(struct obd_import *imp) * locked? Also, how do we know if the requests on the list are * being freed at this time? */ - list_for_each_safe(tmp, n, &imp->imp_sending_list) { - struct ptlrpc_request *req = - list_entry(tmp, struct ptlrpc_request, rq_list); - + list_for_each_entry_safe(req, n, &imp->imp_sending_list, rq_list) { DEBUG_REQ(D_RPCTRACE, req, "inflight"); spin_lock(&req->rq_lock); @@ -2984,10 +2951,7 @@ void ptlrpc_abort_inflight(struct obd_import *imp) spin_unlock(&req->rq_lock); } - list_for_each_safe(tmp, n, &imp->imp_delayed_list) { - struct ptlrpc_request *req = - list_entry(tmp, struct ptlrpc_request, rq_list); - + list_for_each_entry_safe(req, n, &imp->imp_delayed_list, rq_list) { DEBUG_REQ(D_RPCTRACE, req, "aborting waiting req"); spin_lock(&req->rq_lock); @@ -3014,12 +2978,9 @@ void ptlrpc_abort_inflight(struct obd_import *imp) */ void ptlrpc_abort_set(struct ptlrpc_request_set *set) { - struct list_head *tmp, *pos; - - list_for_each_safe(pos, tmp, &set->set_requests) { - struct ptlrpc_request *req = - list_entry(pos, struct ptlrpc_request, rq_set_chain); + struct ptlrpc_request *req, *tmp; + list_for_each_entry_safe(req, tmp, &set->set_requests, rq_set_chain) { spin_lock(&req->rq_lock); if (req->rq_phase != RQ_PHASE_RPC) { spin_unlock(&req->rq_lock); diff --git a/drivers/staging/lustre/lustre/ptlrpc/import.c b/drivers/staging/lustre/lustre/ptlrpc/import.c index faf0f606f013..a2c4fc3488b1 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/import.c +++ b/drivers/staging/lustre/lustre/ptlrpc/import.c @@ -242,15 +242,13 @@ ptlrpc_inflight_deadline(struct ptlrpc_request *req, time64_t now) static unsigned int ptlrpc_inflight_timeout(struct obd_import *imp) { time64_t now = ktime_get_real_seconds(); - struct list_head *tmp, *n; - struct ptlrpc_request *req; + struct ptlrpc_request *req, *n; unsigned int timeout = 0; spin_lock(&imp->imp_lock); - list_for_each_safe(tmp, n, &imp->imp_sending_list) { - req = list_entry(tmp, struct ptlrpc_request, rq_list); + list_for_each_entry_safe(req, n, &imp->imp_sending_list, rq_list) timeout = max(ptlrpc_inflight_deadline(req, now), timeout); - } + spin_unlock(&imp->imp_lock); return timeout; } @@ -263,8 +261,7 @@ static unsigned int ptlrpc_inflight_timeout(struct obd_import *imp) */ void ptlrpc_invalidate_import(struct obd_import *imp) { - struct list_head *tmp, *n; - struct ptlrpc_request *req; + struct ptlrpc_request *req, *n; unsigned int timeout; int rc; @@ -336,19 +333,13 @@ void ptlrpc_invalidate_import(struct obd_import *imp) */ rc = 0; } else { - list_for_each_safe(tmp, n, - &imp->imp_sending_list) { - req = list_entry(tmp, - struct ptlrpc_request, - rq_list); + list_for_each_entry_safe(req, n, + &imp->imp_sending_list, rq_list) { DEBUG_REQ(D_ERROR, req, "still on sending list"); } - list_for_each_safe(tmp, n, - &imp->imp_delayed_list) { - req = list_entry(tmp, - struct ptlrpc_request, - rq_list); + list_for_each_entry_safe(req, n, + &imp->imp_delayed_list, rq_list) { DEBUG_REQ(D_ERROR, req, "still on delayed list"); } @@ -557,14 +548,13 @@ static int import_select_connection(struct obd_import *imp) static int ptlrpc_first_transno(struct obd_import *imp, __u64 *transno) { struct ptlrpc_request *req; - struct list_head *tmp; /* The requests in committed_list always have smaller transnos than * the requests in replay_list */ if (!list_empty(&imp->imp_committed_list)) { - tmp = imp->imp_committed_list.next; - req = list_entry(tmp, struct ptlrpc_request, rq_replay_list); + req = list_first_entry(&imp->imp_committed_list, + struct ptlrpc_request, rq_replay_list); *transno = req->rq_transno; if (req->rq_transno == 0) { DEBUG_REQ(D_ERROR, req, @@ -574,8 +564,8 @@ static int ptlrpc_first_transno(struct obd_import *imp, __u64 *transno) return 1; } if (!list_empty(&imp->imp_replay_list)) { - tmp = imp->imp_replay_list.next; - req = list_entry(tmp, struct ptlrpc_request, rq_replay_list); + req = list_first_entry(&imp->imp_replay_list, + struct ptlrpc_request, rq_replay_list); *transno = req->rq_transno; if (req->rq_transno == 0) { DEBUG_REQ(D_ERROR, req, "zero transno in replay_list"); diff --git a/drivers/staging/lustre/lustre/ptlrpc/pinger.c b/drivers/staging/lustre/lustre/ptlrpc/pinger.c index 639070f6e68e..b5f3cfee8e75 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/pinger.c +++ b/drivers/staging/lustre/lustre/ptlrpc/pinger.c @@ -230,17 +230,13 @@ static int ptlrpc_pinger_main(void *arg) unsigned long this_ping = cfs_time_current(); long time_to_next_wake; struct timeout_item *item; - struct list_head *iter; + struct obd_import *imp; mutex_lock(&pinger_mutex); list_for_each_entry(item, &timeout_list, ti_chain) { item->ti_cb(item, item->ti_cb_data); } - list_for_each(iter, &pinger_imports) { - struct obd_import *imp = - list_entry(iter, struct obd_import, - imp_pinger_chain); - + list_for_each_entry(imp, &pinger_imports, imp_pinger_chain) { ptlrpc_pinger_process_import(imp, this_ping); /* obd_timeout might have changed */ if (imp->imp_pingable && imp->imp_next_ping && diff --git a/drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c b/drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c index 6ed77521d025..c0fa13942bd8 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c +++ b/drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c @@ -197,17 +197,14 @@ ptlrpcd_select_pc(struct ptlrpc_request *req) static int ptlrpcd_steal_rqset(struct ptlrpc_request_set *des, struct ptlrpc_request_set *src) { - struct list_head *tmp, *pos; - struct ptlrpc_request *req; + struct ptlrpc_request *req, *tmp; int rc = 0; spin_lock(&src->set_new_req_lock); if (likely(!list_empty(&src->set_new_requests))) { - list_for_each_safe(pos, tmp, &src->set_new_requests) { - req = list_entry(pos, struct ptlrpc_request, - rq_set_chain); + list_for_each_entry_safe(req, tmp, &src->set_new_requests, rq_set_chain) req->rq_set = des; - } + list_splice_init(&src->set_new_requests, &des->set_requests); rc = atomic_read(&src->set_new_count); atomic_add(rc, &des->set_remaining); @@ -273,8 +270,7 @@ static inline void ptlrpc_reqset_get(struct ptlrpc_request_set *set) */ static int ptlrpcd_check(struct lu_env *env, struct ptlrpcd_ctl *pc) { - struct list_head *tmp, *pos; - struct ptlrpc_request *req; + struct ptlrpc_request *req, *tmp; struct ptlrpc_request_set *set = pc->pc_set; int rc = 0; int rc2; @@ -320,8 +316,7 @@ static int ptlrpcd_check(struct lu_env *env, struct ptlrpcd_ctl *pc) /* NB: ptlrpc_check_set has already moved completed request at the * head of seq::set_requests */ - list_for_each_safe(pos, tmp, &set->set_requests) { - req = list_entry(pos, struct ptlrpc_request, rq_set_chain); + list_for_each_entry_safe(req, tmp, &set->set_requests, rq_set_chain) { if (req->rq_phase != RQ_PHASE_COMPLETE) break; diff --git a/drivers/staging/lustre/lustre/ptlrpc/recover.c b/drivers/staging/lustre/lustre/ptlrpc/recover.c index 5bb9f9fe91d8..2ea0a7ff87dd 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/recover.c +++ b/drivers/staging/lustre/lustre/ptlrpc/recover.c @@ -66,8 +66,7 @@ void ptlrpc_initiate_recovery(struct obd_import *imp) int ptlrpc_replay_next(struct obd_import *imp, int *inflight) { int rc = 0; - struct list_head *tmp, *pos; - struct ptlrpc_request *req = NULL; + struct ptlrpc_request *req = NULL, *pos; __u64 last_transno; *inflight = 0; @@ -86,8 +85,8 @@ int ptlrpc_replay_next(struct obd_import *imp, int *inflight) /* Replay all the committed open requests on committed_list first */ if (!list_empty(&imp->imp_committed_list)) { - tmp = imp->imp_committed_list.prev; - req = list_entry(tmp, struct ptlrpc_request, rq_replay_list); + req = list_last_entry(&imp->imp_committed_list, + struct ptlrpc_request, rq_replay_list); /* The last request on committed_list hasn't been replayed */ if (req->rq_transno > last_transno) { @@ -119,13 +118,13 @@ int ptlrpc_replay_next(struct obd_import *imp, int *inflight) * the imp_replay_list */ if (!req) { - list_for_each_safe(tmp, pos, &imp->imp_replay_list) { - req = list_entry(tmp, struct ptlrpc_request, - rq_replay_list); - - if (req->rq_transno > last_transno) + struct ptlrpc_request *tmp; + list_for_each_entry_safe(tmp, pos, &imp->imp_replay_list, + rq_replay_list) { + if (tmp->rq_transno > last_transno) { + req = tmp; break; - req = NULL; + } } } @@ -211,13 +210,10 @@ int ptlrpc_resend(struct obd_import *imp) */ void ptlrpc_wake_delayed(struct obd_import *imp) { - struct list_head *tmp, *pos; - struct ptlrpc_request *req; + struct ptlrpc_request *req, *pos; spin_lock(&imp->imp_lock); - list_for_each_safe(tmp, pos, &imp->imp_delayed_list) { - req = list_entry(tmp, struct ptlrpc_request, rq_list); - + list_for_each_entry_safe(req, pos, &imp->imp_delayed_list, rq_list) { DEBUG_REQ(D_HA, req, "waking (set %p):", req->rq_set); ptlrpc_client_wake_req(req); } diff --git a/drivers/staging/lustre/lustre/ptlrpc/service.c b/drivers/staging/lustre/lustre/ptlrpc/service.c index 99aeb291f3f2..49417228b621 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/service.c +++ b/drivers/staging/lustre/lustre/ptlrpc/service.c @@ -726,8 +726,6 @@ static void ptlrpc_server_drop_request(struct ptlrpc_request *req) struct ptlrpc_service_part *svcpt = rqbd->rqbd_svcpt; struct ptlrpc_service *svc = svcpt->scp_service; int refcount; - struct list_head *tmp; - struct list_head *nxt; if (!atomic_dec_and_test(&req->rq_refcount)) return; @@ -776,9 +774,7 @@ static void ptlrpc_server_drop_request(struct ptlrpc_request *req) /* remove rqbd's reqs from svc's req history while * I've got the service lock */ - list_for_each(tmp, &rqbd->rqbd_reqs) { - req = list_entry(tmp, struct ptlrpc_request, - rq_list); + list_for_each_entry(req, &rqbd->rqbd_reqs, rq_list) { /* Track the highest culled req seq */ if (req->rq_history_seq > svcpt->scp_hist_seq_culled) { @@ -790,10 +786,9 @@ static void ptlrpc_server_drop_request(struct ptlrpc_request *req) spin_unlock(&svcpt->scp_lock); - list_for_each_safe(tmp, nxt, &rqbd->rqbd_reqs) { - req = list_entry(rqbd->rqbd_reqs.next, - struct ptlrpc_request, - rq_list); + while ((req = list_first_entry_or_null( + &rqbd->rqbd_reqs, + struct ptlrpc_request, rq_list))) { list_del(&req->rq_list); ptlrpc_server_free_request(req); } From neilb at suse.com Tue Feb 20 02:23:38 2018 From: neilb at suse.com (NeilBrown) Date: Tue, 20 Feb 2018 13:23:38 +1100 Subject: [lustre-devel] [PATCH 17/21] staging: lustre: fid: use wait_event_cmd() In-Reply-To: <151909308778.25814.9912662268627044409.stgit@noble> References: <151909308778.25814.9912662268627044409.stgit@noble> Message-ID: <151909341813.25814.17831255008034562097.stgit@noble> Rather than open-coding a wait event loop twice, use wait_event_cmd() to wait, dropping the spinlock over schedule(). This does require duplicating part of the wait condition, but that is just three tests on values that are in registers or in cache, so the cost is small and the increased readability is large. Signed-off-by: NeilBrown --- drivers/staging/lustre/lustre/fid/fid_request.c | 68 +++++++---------------- 1 file changed, 20 insertions(+), 48 deletions(-) diff --git a/drivers/staging/lustre/lustre/fid/fid_request.c b/drivers/staging/lustre/lustre/fid/fid_request.c index 6b9d024bd27b..ef9ee5426151 100644 --- a/drivers/staging/lustre/lustre/fid/fid_request.c +++ b/drivers/staging/lustre/lustre/fid/fid_request.c @@ -192,26 +192,6 @@ static int seq_client_alloc_seq(const struct lu_env *env, return rc; } -static int seq_fid_alloc_prep(struct lu_client_seq *seq, - wait_queue_entry_t *link) -{ - if (seq->lcs_update) { - add_wait_queue(&seq->lcs_waitq, link); - set_current_state(TASK_UNINTERRUPTIBLE); - spin_unlock(&seq->lcs_lock); - - schedule(); - - spin_lock(&seq->lcs_lock); - remove_wait_queue(&seq->lcs_waitq, link); - set_current_state(TASK_RUNNING); - return -EAGAIN; - } - ++seq->lcs_update; - spin_unlock(&seq->lcs_lock); - return 0; -} - static void seq_fid_alloc_fini(struct lu_client_seq *seq) { LASSERT(seq->lcs_update == 1); @@ -224,32 +204,34 @@ static void seq_fid_alloc_fini(struct lu_client_seq *seq) int seq_client_alloc_fid(const struct lu_env *env, struct lu_client_seq *seq, struct lu_fid *fid) { - wait_queue_entry_t link; int rc; LASSERT(seq); LASSERT(fid); - init_waitqueue_entry(&link, current); spin_lock(&seq->lcs_lock); if (OBD_FAIL_CHECK(OBD_FAIL_SEQ_EXHAUST)) seq->lcs_fid.f_oid = seq->lcs_width; - while (1) { + wait_event_cmd(seq->lcs_waitq, + (!fid_is_zero(&seq->lcs_fid) && + fid_oid(&seq->lcs_fid) < seq->lcs_width) || + !seq->lcs_update, + spin_unlock(&seq->lcs_lock), + spin_lock(&seq->lcs_lock)); + + if (!fid_is_zero(&seq->lcs_fid) && + fid_oid(&seq->lcs_fid) < seq->lcs_width) { + /* Just bump last allocated fid and return to caller. */ + seq->lcs_fid.f_oid += 1; + rc = 0; + } else { u64 seqnr; - if (!fid_is_zero(&seq->lcs_fid) && - fid_oid(&seq->lcs_fid) < seq->lcs_width) { - /* Just bump last allocated fid and return to caller. */ - seq->lcs_fid.f_oid += 1; - rc = 0; - break; - } - - rc = seq_fid_alloc_prep(seq, &link); - if (rc) - continue; + LASSERT(seq->lcs_update == 0); + ++seq->lcs_update; + spin_unlock(&seq->lcs_lock); rc = seq_client_alloc_seq(env, seq, &seqnr); if (rc) { @@ -274,7 +256,6 @@ int seq_client_alloc_fid(const struct lu_env *env, rc = 1; seq_fid_alloc_fini(seq); - break; } *fid = seq->lcs_fid; @@ -292,23 +273,14 @@ EXPORT_SYMBOL(seq_client_alloc_fid); */ void seq_client_flush(struct lu_client_seq *seq) { - wait_queue_entry_t link; LASSERT(seq); - init_waitqueue_entry(&link, current); spin_lock(&seq->lcs_lock); - while (seq->lcs_update) { - add_wait_queue(&seq->lcs_waitq, &link); - set_current_state(TASK_UNINTERRUPTIBLE); - spin_unlock(&seq->lcs_lock); - - schedule(); - - spin_lock(&seq->lcs_lock); - remove_wait_queue(&seq->lcs_waitq, &link); - set_current_state(TASK_RUNNING); - } + wait_event_cmd(seq->lcs_waitq, + !seq->lcs_update, + spin_unlock(&seq->lcs_lock), + spin_lock(&seq->lcs_lock)); fid_zero(&seq->lcs_fid); /** From neilb at suse.com Tue Feb 20 02:23:38 2018 From: neilb at suse.com (NeilBrown) Date: Tue, 20 Feb 2018 13:23:38 +1100 Subject: [lustre-devel] [PATCH 21/21] staging: lustre: socklnd: simplify ksnc_rx_iov_space In-Reply-To: <151909308778.25814.9912662268627044409.stgit@noble> References: <151909308778.25814.9912662268627044409.stgit@noble> Message-ID: <151909341826.25814.3735688359829435395.stgit@noble> ksnc_rx_iov_space is currently a union of two arrays, one of 'struct kvec', the other of 'struct bio_vec'. The 'struct bio_vec' option is never used. The array of kvec is used to read in a packet header, or to read data that needs to be skipped so as to synchronize with a packet boundary. In each case the target memory location is a virtual address, never a page, so 'struct bio_vec' is never needed. When we read into a page, different code steps up a separate array of 'struct bio_vec'. So remove the bio_vec option, and remove the union ksock_rxiovspace.. Signed-off-by: NeilBrown --- .../staging/lustre/lnet/klnds/socklnd/socklnd.h | 11 +---------- .../staging/lustre/lnet/klnds/socklnd/socklnd_cb.c | 4 ++-- 2 files changed, 3 insertions(+), 12 deletions(-) diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.h b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.h index d50ebdf863fa..570f54ed57b1 100644 --- a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.h +++ b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.h @@ -304,15 +304,6 @@ struct ksock_tx { /* transmit packet */ /* network zero copy callback descriptor embedded in struct ksock_tx */ -/* - * space for the rx frag descriptors; we either read a single contiguous - * header, or up to LNET_MAX_IOV frags of payload of either type. - */ -union ksock_rxiovspace { - struct kvec iov[LNET_MAX_IOV]; - struct bio_vec kiov[LNET_MAX_IOV]; -}; - #define SOCKNAL_RX_KSM_HEADER 1 /* reading ksock message header */ #define SOCKNAL_RX_LNET_HEADER 2 /* reading lnet message header */ #define SOCKNAL_RX_PARSE 3 /* Calling lnet_parse() */ @@ -359,7 +350,7 @@ struct ksock_conn { __u8 ksnc_rx_state; /* what is being read */ int ksnc_rx_nob_left; /* # bytes to next hdr/body */ struct iov_iter ksnc_rx_to; /* copy destination */ - union ksock_rxiovspace ksnc_rx_iov_space; /* space for frag descriptors */ + struct kvec ksnc_rx_iov_space[LNET_MAX_IOV]; /* space for frag descriptors */ __u32 ksnc_rx_csum; /* partial checksum for incoming * data */ diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c index 6ab002c006ab..036fecbcede8 100644 --- a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c +++ b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c @@ -986,7 +986,7 @@ int ksocknal_new_packet(struct ksock_conn *conn, int nob_to_skip) { static char ksocknal_slop_buffer[4096]; - struct kvec *kvec = (struct kvec *)&conn->ksnc_rx_iov_space; + struct kvec *kvec = conn->ksnc_rx_iov_space; int nob; unsigned int niov; @@ -1059,7 +1059,7 @@ ksocknal_new_packet(struct ksock_conn *conn, int nob_to_skip) static int ksocknal_process_receive(struct ksock_conn *conn) { - struct kvec *kvec = (struct kvec *)&conn->ksnc_rx_iov_space; + struct kvec *kvec = conn->ksnc_rx_iov_space; struct lnet_hdr *lhdr; struct lnet_process_id *id; int rc; From neilb at suse.com Tue Feb 20 02:23:37 2018 From: neilb at suse.com (NeilBrown) Date: Tue, 20 Feb 2018 13:23:37 +1100 Subject: [lustre-devel] [PATCH 01/21] staging: lustre: replace all CFS_CAP_* macros with CAP_* In-Reply-To: <151909308778.25814.9912662268627044409.stgit@noble> References: <151909308778.25814.9912662268627044409.stgit@noble> Message-ID: <151909341755.25814.8807958574153162982.stgit@noble> Lustre defines a few CFS_CAP_* macros which are exactly the same as the corresponding CAP_* macro, with one exception. CFS_CAP_SYS_BOOT is 23 CAP_SYS_BOOT is 22. CFS_CAP_SYS_BOOT is only used through CFS_CAP_FS_MASK and causes capability 23 (CAP_SYS_NICE) to be dropped in certain circumstances. It is probable that the intention was to drop CAP_SYS_BOOT, and this is what is now done. CFS_CAP_CHOWN_MASK and CFS_CAP_SYS_RESOURCE_MASK are never used, so they have been removed. Signed-off-by: NeilBrown --- .../staging/lustre/include/linux/libcfs/curproc.h | 28 ++++++-------------- drivers/staging/lustre/lustre/include/lustre_sec.h | 3 -- drivers/staging/lustre/lustre/llite/dir.c | 8 +++--- drivers/staging/lustre/lustre/llite/file.c | 6 ++-- drivers/staging/lustre/lustre/llite/llite_lib.c | 4 +-- drivers/staging/lustre/lustre/llite/xattr.c | 2 + .../lustre/lustre/obdclass/linux/linux-module.c | 2 + drivers/staging/lustre/lustre/obdclass/llog.c | 6 ++-- .../staging/lustre/lustre/obdecho/echo_client.c | 8 +++--- drivers/staging/lustre/lustre/osc/osc_cache.c | 4 +-- drivers/staging/lustre/lustre/osc/osc_page.c | 2 + 11 files changed, 30 insertions(+), 43 deletions(-) diff --git a/drivers/staging/lustre/include/linux/libcfs/curproc.h b/drivers/staging/lustre/include/linux/libcfs/curproc.h index 3cb3f086148e..cdc549e2979f 100644 --- a/drivers/staging/lustre/include/linux/libcfs/curproc.h +++ b/drivers/staging/lustre/include/linux/libcfs/curproc.h @@ -56,25 +56,15 @@ typedef u32 cfs_cap_t; -#define CFS_CAP_CHOWN 0 -#define CFS_CAP_DAC_OVERRIDE 1 -#define CFS_CAP_DAC_READ_SEARCH 2 -#define CFS_CAP_FOWNER 3 -#define CFS_CAP_FSETID 4 -#define CFS_CAP_LINUX_IMMUTABLE 9 -#define CFS_CAP_SYS_ADMIN 21 -#define CFS_CAP_SYS_BOOT 23 -#define CFS_CAP_SYS_RESOURCE 24 - -#define CFS_CAP_FS_MASK (BIT(CFS_CAP_CHOWN) | \ - BIT(CFS_CAP_DAC_OVERRIDE) | \ - BIT(CFS_CAP_DAC_READ_SEARCH) | \ - BIT(CFS_CAP_FOWNER) | \ - BIT(CFS_CAP_FSETID) | \ - BIT(CFS_CAP_LINUX_IMMUTABLE) | \ - BIT(CFS_CAP_SYS_ADMIN) | \ - BIT(CFS_CAP_SYS_BOOT) | \ - BIT(CFS_CAP_SYS_RESOURCE)) +#define CFS_CAP_FS_MASK (BIT(CAP_CHOWN) | \ + BIT(CAP_DAC_OVERRIDE) | \ + BIT(CAP_DAC_READ_SEARCH) | \ + BIT(CAP_FOWNER) | \ + BIT(CAP_FSETID) | \ + BIT(CAP_LINUX_IMMUTABLE) | \ + BIT(CAP_SYS_ADMIN) | \ + BIT(CAP_SYS_BOOT) | \ + BIT(CAP_SYS_RESOURCE)) void cfs_cap_raise(cfs_cap_t cap); void cfs_cap_lower(cfs_cap_t cap); diff --git a/drivers/staging/lustre/lustre/include/lustre_sec.h b/drivers/staging/lustre/lustre/include/lustre_sec.h index 64b6fd4fed8f..c5cb07acd0da 100644 --- a/drivers/staging/lustre/lustre/include/lustre_sec.h +++ b/drivers/staging/lustre/lustre/include/lustre_sec.h @@ -1058,9 +1058,6 @@ int sptlrpc_current_user_desc_size(void); int sptlrpc_pack_user_desc(struct lustre_msg *msg, int offset); int sptlrpc_unpack_user_desc(struct lustre_msg *req, int offset, int swabbed); -#define CFS_CAP_CHOWN_MASK (1 << CFS_CAP_CHOWN) -#define CFS_CAP_SYS_RESOURCE_MASK (1 << CFS_CAP_SYS_RESOURCE) - enum { LUSTRE_SEC_NONE = 0, LUSTRE_SEC_REMOTE = 1, diff --git a/drivers/staging/lustre/lustre/llite/dir.c b/drivers/staging/lustre/lustre/llite/dir.c index 99b0b77c75f5..09e3a4999079 100644 --- a/drivers/staging/lustre/lustre/llite/dir.c +++ b/drivers/staging/lustre/lustre/llite/dir.c @@ -885,7 +885,7 @@ static int quotactl_ioctl(struct ll_sb_info *sbi, struct if_quotactl *qctl) switch (cmd) { case Q_SETQUOTA: case Q_SETINFO: - if (!capable(CFS_CAP_SYS_ADMIN)) + if (!capable(CAP_SYS_ADMIN)) return -EPERM; break; case Q_GETQUOTA: @@ -893,7 +893,7 @@ static int quotactl_ioctl(struct ll_sb_info *sbi, struct if_quotactl *qctl) !uid_eq(current_euid(), make_kuid(&init_user_ns, id))) || (type == GRPQUOTA && !in_egroup_p(make_kgid(&init_user_ns, id)))) && - !capable(CFS_CAP_SYS_ADMIN)) + !capable(CAP_SYS_ADMIN)) return -EPERM; break; case Q_GETINFO: @@ -1452,7 +1452,7 @@ static long ll_dir_ioctl(struct file *file, unsigned int cmd, unsigned long arg) } case OBD_IOC_CHANGELOG_SEND: case OBD_IOC_CHANGELOG_CLEAR: - if (!capable(CFS_CAP_SYS_ADMIN)) + if (!capable(CAP_SYS_ADMIN)) return -EPERM; rc = copy_and_ioctl(cmd, sbi->ll_md_exp, (void __user *)arg, @@ -1556,7 +1556,7 @@ static long ll_dir_ioctl(struct file *file, unsigned int cmd, unsigned long arg) return rc; } case LL_IOC_HSM_CT_START: - if (!capable(CFS_CAP_SYS_ADMIN)) + if (!capable(CAP_SYS_ADMIN)) return -EPERM; rc = copy_and_ioctl(cmd, sbi->ll_md_exp, (void __user *)arg, diff --git a/drivers/staging/lustre/lustre/llite/file.c b/drivers/staging/lustre/lustre/llite/file.c index 938b859b6650..4aad2e331948 100644 --- a/drivers/staging/lustre/lustre/llite/file.c +++ b/drivers/staging/lustre/lustre/llite/file.c @@ -1315,7 +1315,7 @@ static int ll_lov_setea(struct inode *inode, struct file *file, sizeof(struct lov_user_ost_data); int rc; - if (!capable(CFS_CAP_SYS_ADMIN)) + if (!capable(CAP_SYS_ADMIN)) return -EPERM; lump = libcfs_kvzalloc(lum_size, GFP_NOFS); @@ -1570,7 +1570,7 @@ int ll_fid2path(struct inode *inode, void __user *arg) size_t outsize; int rc; - if (!capable(CFS_CAP_DAC_READ_SEARCH) && + if (!capable(CAP_DAC_READ_SEARCH) && !(ll_i2sbi(inode)->ll_flags & LL_SBI_USER_FID2PATH)) return -EPERM; @@ -1840,7 +1840,7 @@ int ll_hsm_state_set(struct inode *inode, struct hsm_state_set *hss) * NOT defined in HSM_USER_MASK. */ if (((hss->hss_setmask | hss->hss_clearmask) & ~HSM_USER_MASK) && - !capable(CFS_CAP_SYS_ADMIN)) + !capable(CAP_SYS_ADMIN)) return -EPERM; /* Detect out-of range archive id */ diff --git a/drivers/staging/lustre/lustre/llite/llite_lib.c b/drivers/staging/lustre/lustre/llite/llite_lib.c index 9e96a8ee1783..07072ab92bb6 100644 --- a/drivers/staging/lustre/lustre/llite/llite_lib.c +++ b/drivers/staging/lustre/lustre/llite/llite_lib.c @@ -1450,7 +1450,7 @@ int ll_setattr_raw(struct dentry *dentry, struct iattr *attr, bool hsm_import) /* POSIX: check before ATTR_*TIME_SET set (from setattr_prepare) */ if (attr->ia_valid & TIMES_SET_FLAGS) { if ((!uid_eq(current_fsuid(), inode->i_uid)) && - !capable(CFS_CAP_FOWNER)) + !capable(CAP_FOWNER)) return -EPERM; } @@ -2597,7 +2597,7 @@ int ll_getparent(struct file *file, struct getparent __user *arg) u32 linkno; int rc; - if (!capable(CFS_CAP_DAC_READ_SEARCH) && + if (!capable(CAP_DAC_READ_SEARCH) && !(ll_i2sbi(inode)->ll_flags & LL_SBI_USER_FID2PATH)) return -EPERM; diff --git a/drivers/staging/lustre/lustre/llite/xattr.c b/drivers/staging/lustre/lustre/llite/xattr.c index 532384c91447..a723056f7166 100644 --- a/drivers/staging/lustre/lustre/llite/xattr.c +++ b/drivers/staging/lustre/lustre/llite/xattr.c @@ -75,7 +75,7 @@ static int xattr_type_filter(struct ll_sb_info *sbi, return -EOPNOTSUPP; if (handler->flags == XATTR_TRUSTED_T && - !capable(CFS_CAP_SYS_ADMIN)) + !capable(CAP_SYS_ADMIN)) return -EPERM; return 0; diff --git a/drivers/staging/lustre/lustre/obdclass/linux/linux-module.c b/drivers/staging/lustre/lustre/obdclass/linux/linux-module.c index 57951237def2..5b1122c408fb 100644 --- a/drivers/staging/lustre/lustre/obdclass/linux/linux-module.c +++ b/drivers/staging/lustre/lustre/obdclass/linux/linux-module.c @@ -251,7 +251,7 @@ static long obd_class_ioctl(struct file *filp, unsigned int cmd, int err = 0; /* Allow non-root access for OBD_IOC_PING_TARGET - used by lfs check */ - if (!capable(CFS_CAP_SYS_ADMIN) && (cmd != OBD_IOC_PING_TARGET)) + if (!capable(CAP_SYS_ADMIN) && (cmd != OBD_IOC_PING_TARGET)) return err = -EACCES; if ((cmd & 0xffffff00) == ((int)'T') << 8) /* ignore all tty ioctls */ return err = -ENOTTY; diff --git a/drivers/staging/lustre/lustre/obdclass/llog.c b/drivers/staging/lustre/lustre/obdclass/llog.c index cd051e31233e..aa48b3d2199e 100644 --- a/drivers/staging/lustre/lustre/obdclass/llog.c +++ b/drivers/staging/lustre/lustre/obdclass/llog.c @@ -483,12 +483,12 @@ int llog_open(const struct lu_env *env, struct llog_ctxt *ctxt, (*lgh)->lgh_ctxt = ctxt; (*lgh)->lgh_logops = ctxt->loc_logops; - raised = cfs_cap_raised(CFS_CAP_SYS_RESOURCE); + raised = cfs_cap_raised(CAP_SYS_RESOURCE); if (!raised) - cfs_cap_raise(CFS_CAP_SYS_RESOURCE); + cfs_cap_raise(CAP_SYS_RESOURCE); rc = ctxt->loc_logops->lop_open(env, *lgh, logid, name, open_param); if (!raised) - cfs_cap_lower(CFS_CAP_SYS_RESOURCE); + cfs_cap_lower(CAP_SYS_RESOURCE); if (rc) { llog_free_handle(*lgh); *lgh = NULL; diff --git a/drivers/staging/lustre/lustre/obdecho/echo_client.c b/drivers/staging/lustre/lustre/obdecho/echo_client.c index 9c5ce5074b66..99a76db51ae0 100644 --- a/drivers/staging/lustre/lustre/obdecho/echo_client.c +++ b/drivers/staging/lustre/lustre/obdecho/echo_client.c @@ -1502,7 +1502,7 @@ echo_client_iocontrol(unsigned int cmd, struct obd_export *exp, int len, switch (cmd) { case OBD_IOC_CREATE: /* may create echo object */ - if (!capable(CFS_CAP_SYS_ADMIN)) { + if (!capable(CAP_SYS_ADMIN)) { rc = -EPERM; goto out; } @@ -1511,7 +1511,7 @@ echo_client_iocontrol(unsigned int cmd, struct obd_export *exp, int len, goto out; case OBD_IOC_DESTROY: - if (!capable(CFS_CAP_SYS_ADMIN)) { + if (!capable(CAP_SYS_ADMIN)) { rc = -EPERM; goto out; } @@ -1534,7 +1534,7 @@ echo_client_iocontrol(unsigned int cmd, struct obd_export *exp, int len, goto out; case OBD_IOC_SETATTR: - if (!capable(CFS_CAP_SYS_ADMIN)) { + if (!capable(CAP_SYS_ADMIN)) { rc = -EPERM; goto out; } @@ -1547,7 +1547,7 @@ echo_client_iocontrol(unsigned int cmd, struct obd_export *exp, int len, goto out; case OBD_IOC_BRW_WRITE: - if (!capable(CFS_CAP_SYS_ADMIN)) { + if (!capable(CAP_SYS_ADMIN)) { rc = -EPERM; goto out; } diff --git a/drivers/staging/lustre/lustre/osc/osc_cache.c b/drivers/staging/lustre/lustre/osc/osc_cache.c index dacfab12c501..1c70a504ee89 100644 --- a/drivers/staging/lustre/lustre/osc/osc_cache.c +++ b/drivers/staging/lustre/lustre/osc/osc_cache.c @@ -2345,7 +2345,7 @@ int osc_prep_async_page(struct osc_object *osc, struct osc_page *ops, oap->oap_obj_off = offset; LASSERT(!(offset & ~PAGE_MASK)); - if (capable(CFS_CAP_SYS_RESOURCE)) + if (capable(CAP_SYS_RESOURCE)) oap->oap_brw_flags = OBD_BRW_NOQUOTA; INIT_LIST_HEAD(&oap->oap_pending_item); @@ -2384,7 +2384,7 @@ int osc_queue_async_io(const struct lu_env *env, struct cl_io *io, /* Set the OBD_BRW_SRVLOCK before the page is queued. */ brw_flags |= ops->ops_srvlock ? OBD_BRW_SRVLOCK : 0; - if (capable(CFS_CAP_SYS_RESOURCE)) { + if (capable(CAP_SYS_RESOURCE)) { brw_flags |= OBD_BRW_NOQUOTA; cmd |= OBD_BRW_NOQUOTA; } diff --git a/drivers/staging/lustre/lustre/osc/osc_page.c b/drivers/staging/lustre/lustre/osc/osc_page.c index 6fdd521feb21..01a930dbbf64 100644 --- a/drivers/staging/lustre/lustre/osc/osc_page.c +++ b/drivers/staging/lustre/lustre/osc/osc_page.c @@ -307,7 +307,7 @@ void osc_page_submit(const struct lu_env *env, struct osc_page *opg, oap->oap_count = opg->ops_to - opg->ops_from; oap->oap_brw_flags = brw_flags | OBD_BRW_SYNC; - if (capable(CFS_CAP_SYS_RESOURCE)) { + if (capable(CAP_SYS_RESOURCE)) { oap->oap_brw_flags |= OBD_BRW_NOQUOTA; oap->oap_cmd |= OBD_BRW_NOQUOTA; } From neilb at suse.com Tue Feb 20 02:23:37 2018 From: neilb at suse.com (NeilBrown) Date: Tue, 20 Feb 2018 13:23:37 +1100 Subject: [lustre-devel] [PATCH 02/21] staging: lustre: opencode cfs_cap_{raise, lower, raised} In-Reply-To: <151909308778.25814.9912662268627044409.stgit@noble> References: <151909308778.25814.9912662268627044409.stgit@noble> Message-ID: <151909341760.25814.3463876381688003123.stgit@noble> Each of these functions is used precisely once, so having a separate exported function seems like overkill. cfs_cap_raised() is trivial - one line. cfs_cap_raise() and cfs_cap_lower() are used as a pair which is more effectively implemented with override_cred() / revert_creds(). --- .../staging/lustre/include/linux/libcfs/curproc.h | 3 -- .../lustre/lnet/libcfs/linux/linux-curproc.c | 30 -------------------- drivers/staging/lustre/lustre/obdclass/llog.c | 18 ++++++++---- 3 files changed, 12 insertions(+), 39 deletions(-) diff --git a/drivers/staging/lustre/include/linux/libcfs/curproc.h b/drivers/staging/lustre/include/linux/libcfs/curproc.h index cdc549e2979f..51f2179b7534 100644 --- a/drivers/staging/lustre/include/linux/libcfs/curproc.h +++ b/drivers/staging/lustre/include/linux/libcfs/curproc.h @@ -66,9 +66,6 @@ typedef u32 cfs_cap_t; BIT(CAP_SYS_BOOT) | \ BIT(CAP_SYS_RESOURCE)) -void cfs_cap_raise(cfs_cap_t cap); -void cfs_cap_lower(cfs_cap_t cap); -int cfs_cap_raised(cfs_cap_t cap); cfs_cap_t cfs_curproc_cap_pack(void); /* __LIBCFS_CURPROC_H__ */ diff --git a/drivers/staging/lustre/lnet/libcfs/linux/linux-curproc.c b/drivers/staging/lustre/lnet/libcfs/linux/linux-curproc.c index 1d8949f1a4fa..6b75c5cdc3ec 100644 --- a/drivers/staging/lustre/lnet/libcfs/linux/linux-curproc.c +++ b/drivers/staging/lustre/lnet/libcfs/linux/linux-curproc.c @@ -52,36 +52,6 @@ * for Linux kernel. */ -void cfs_cap_raise(cfs_cap_t cap) -{ - struct cred *cred; - - cred = prepare_creds(); - if (cred) { - cap_raise(cred->cap_effective, cap); - commit_creds(cred); - } -} -EXPORT_SYMBOL(cfs_cap_raise); - -void cfs_cap_lower(cfs_cap_t cap) -{ - struct cred *cred; - - cred = prepare_creds(); - if (cred) { - cap_lower(cred->cap_effective, cap); - commit_creds(cred); - } -} -EXPORT_SYMBOL(cfs_cap_lower); - -int cfs_cap_raised(cfs_cap_t cap) -{ - return cap_raised(current_cap(), cap); -} -EXPORT_SYMBOL(cfs_cap_raised); - static void cfs_kernel_cap_pack(kernel_cap_t kcap, cfs_cap_t *cap) { /* XXX lost high byte */ diff --git a/drivers/staging/lustre/lustre/obdclass/llog.c b/drivers/staging/lustre/lustre/obdclass/llog.c index aa48b3d2199e..934f067adb14 100644 --- a/drivers/staging/lustre/lustre/obdclass/llog.c +++ b/drivers/staging/lustre/lustre/obdclass/llog.c @@ -466,7 +466,7 @@ int llog_open(const struct lu_env *env, struct llog_ctxt *ctxt, struct llog_handle **lgh, struct llog_logid *logid, char *name, enum llog_open_param open_param) { - int raised; + const struct cred *old_cred = NULL; int rc; LASSERT(ctxt); @@ -483,12 +483,18 @@ int llog_open(const struct lu_env *env, struct llog_ctxt *ctxt, (*lgh)->lgh_ctxt = ctxt; (*lgh)->lgh_logops = ctxt->loc_logops; - raised = cfs_cap_raised(CAP_SYS_RESOURCE); - if (!raised) - cfs_cap_raise(CAP_SYS_RESOURCE); + if (cap_raised(current_cap(), CAP_SYS_RESOURCE)) { + struct cred *cred = prepare_creds(); + + if (cred) { + cap_raise(cred->cap_effective, CAP_SYS_RESOURCE); + old_cred = override_creds(cred); + } + } rc = ctxt->loc_logops->lop_open(env, *lgh, logid, name, open_param); - if (!raised) - cfs_cap_lower(CAP_SYS_RESOURCE); + if (old_cred) + revert_creds(old_cred); + if (rc) { llog_free_handle(*lgh); *lgh = NULL; From neilb at suse.com Tue Feb 20 02:23:37 2018 From: neilb at suse.com (NeilBrown) Date: Tue, 20 Feb 2018 13:23:37 +1100 Subject: [lustre-devel] [PATCH 06/21] staging: lustre: simplify linux-prim.c In-Reply-To: <151909308778.25814.9912662268627044409.stgit@noble> References: <151909308778.25814.9912662268627044409.stgit@noble> Message-ID: <151909341773.25814.3960998744576050714.stgit@noble> cfs_block_sigs() is never used. cfs_clear_sigpending() is never used. cfs_block_allsigs() is no longer used. So those three functions can go. Signed-off-by: NeilBrown --- .../staging/lustre/include/linux/libcfs/libcfs.h | 3 - .../staging/lustre/lnet/libcfs/linux/linux-prim.c | 41 -------------------- 2 files changed, 44 deletions(-) diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs.h b/drivers/staging/lustre/include/linux/libcfs/libcfs.h index ca3472cc952f..7f06b0118154 100644 --- a/drivers/staging/lustre/include/linux/libcfs/libcfs.h +++ b/drivers/staging/lustre/include/linux/libcfs/libcfs.h @@ -67,11 +67,8 @@ /* * Defined by platform */ -sigset_t cfs_block_allsigs(void); -sigset_t cfs_block_sigs(unsigned long sigs); sigset_t cfs_block_sigsinv(unsigned long sigs); void cfs_restore_sigs(sigset_t sigset); -void cfs_clear_sigpending(void); struct libcfs_ioctl_handler { struct list_head item; diff --git a/drivers/staging/lustre/lnet/libcfs/linux/linux-prim.c b/drivers/staging/lustre/lnet/libcfs/linux/linux-prim.c index 6f92ea272186..6b73b9845e3f 100644 --- a/drivers/staging/lustre/lnet/libcfs/linux/linux-prim.c +++ b/drivers/staging/lustre/lnet/libcfs/linux/linux-prim.c @@ -43,36 +43,6 @@ #include #endif -sigset_t -cfs_block_allsigs(void) -{ - unsigned long flags; - sigset_t old; - - spin_lock_irqsave(¤t->sighand->siglock, flags); - old = current->blocked; - sigfillset(¤t->blocked); - recalc_sigpending(); - spin_unlock_irqrestore(¤t->sighand->siglock, flags); - - return old; -} -EXPORT_SYMBOL(cfs_block_allsigs); - -sigset_t cfs_block_sigs(unsigned long sigs) -{ - unsigned long flags; - sigset_t old; - - spin_lock_irqsave(¤t->sighand->siglock, flags); - old = current->blocked; - sigaddsetmask(¤t->blocked, sigs); - recalc_sigpending(); - spin_unlock_irqrestore(¤t->sighand->siglock, flags); - return old; -} -EXPORT_SYMBOL(cfs_block_sigs); - /* Block all signals except for the @sigs */ sigset_t cfs_block_sigsinv(unsigned long sigs) { @@ -100,14 +70,3 @@ cfs_restore_sigs(sigset_t old) spin_unlock_irqrestore(¤t->sighand->siglock, flags); } EXPORT_SYMBOL(cfs_restore_sigs); - -void -cfs_clear_sigpending(void) -{ - unsigned long flags; - - spin_lock_irqsave(¤t->sighand->siglock, flags); - clear_tsk_thread_flag(current, TIF_SIGPENDING); - spin_unlock_irqrestore(¤t->sighand->siglock, flags); -} -EXPORT_SYMBOL(cfs_clear_sigpending); From neilb at suse.com Tue Feb 20 02:23:38 2018 From: neilb at suse.com (NeilBrown) Date: Tue, 20 Feb 2018 13:23:38 +1100 Subject: [lustre-devel] [PATCH 18/21] staging: lustre: fid: remove seq_fid_alloc_fini() and simplify In-Reply-To: <151909308778.25814.9912662268627044409.stgit@noble> References: <151909308778.25814.9912662268627044409.stgit@noble> Message-ID: <151909341816.25814.6263195320580542505.stgit@noble> seq_fid_alloc_fini() is tiny and only called from two places in the one function. We can move both those calls earlier and merge them so only one call is needed. At that point, there is no value added by having a separate function. Also instead of using ++ and -- on ->lcs_update to toggle between 0 and 1, explicitly set to 0 or 1 as appropriate. Moving the locking earlier means that the code which updates seq->lcs_fid is now protected, so ldebugfs_fid_fid_seq_show() now cannot see a torn value. Signed-off-by: NeilBrown --- drivers/staging/lustre/lustre/fid/fid_request.c | 18 ++++++------------ 1 file changed, 6 insertions(+), 12 deletions(-) diff --git a/drivers/staging/lustre/lustre/fid/fid_request.c b/drivers/staging/lustre/lustre/fid/fid_request.c index ef9ee5426151..6e0dfe18ef5f 100644 --- a/drivers/staging/lustre/lustre/fid/fid_request.c +++ b/drivers/staging/lustre/lustre/fid/fid_request.c @@ -192,14 +192,6 @@ static int seq_client_alloc_seq(const struct lu_env *env, return rc; } -static void seq_fid_alloc_fini(struct lu_client_seq *seq) -{ - LASSERT(seq->lcs_update == 1); - spin_lock(&seq->lcs_lock); - --seq->lcs_update; - wake_up(&seq->lcs_waitq); -} - /* Allocate new fid on passed client @seq and save it to @fid. */ int seq_client_alloc_fid(const struct lu_env *env, struct lu_client_seq *seq, struct lu_fid *fid) @@ -230,14 +222,18 @@ int seq_client_alloc_fid(const struct lu_env *env, u64 seqnr; LASSERT(seq->lcs_update == 0); - ++seq->lcs_update; + seq->lcs_update = 1; spin_unlock(&seq->lcs_lock); rc = seq_client_alloc_seq(env, seq, &seqnr); + + spin_lock(&seq->lcs_lock); + seq->lcs_update = 0; + wake_up(&seq->lcs_waitq); + if (rc) { CERROR("%s: Can't allocate new sequence, rc %d\n", seq->lcs_name, rc); - seq_fid_alloc_fini(seq); spin_unlock(&seq->lcs_lock); return rc; } @@ -254,8 +250,6 @@ int seq_client_alloc_fid(const struct lu_env *env, * to setup FLD for it. */ rc = 1; - - seq_fid_alloc_fini(seq); } *fid = seq->lcs_fid; From neilb at suse.com Tue Feb 20 02:23:38 2018 From: neilb at suse.com (NeilBrown) Date: Tue, 20 Feb 2018 13:23:38 +1100 Subject: [lustre-devel] [PATCH 13/21] staging: lustre: remove phantom struct cfs_crypto_hash_desc In-Reply-To: <151909308778.25814.9912662268627044409.stgit@noble> References: <151909308778.25814.9912662268627044409.stgit@noble> Message-ID: <151909341798.25814.6579149760977367071.stgit@noble> There is no "struct cfs_crypto_hash_desc" structure. There are only pointers to this structure, which are cast back and forth to struct ahash_request. So discard cfs_crypto_hash_desc, and just use ahash_request directly. Signed-off-by: NeilBrown --- .../lustre/include/linux/libcfs/libcfs_crypto.h | 11 +++----- .../lustre/lnet/libcfs/linux/linux-crypto.c | 29 +++++++++----------- drivers/staging/lustre/lustre/osc/osc_request.c | 2 + drivers/staging/lustre/lustre/ptlrpc/sec_bulk.c | 2 + 4 files changed, 19 insertions(+), 25 deletions(-) diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_crypto.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_crypto.h index e5c156e9d907..3a72117140ed 100644 --- a/drivers/staging/lustre/include/linux/libcfs/libcfs_crypto.h +++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_crypto.h @@ -189,18 +189,15 @@ int cfs_crypto_hash_digest(enum cfs_crypto_hash_alg hash_alg, unsigned char *key, unsigned int key_len, unsigned char *hash, unsigned int *hash_len); -/* cfs crypto hash descriptor */ -struct cfs_crypto_hash_desc; - -struct cfs_crypto_hash_desc * +struct ahash_request * cfs_crypto_hash_init(enum cfs_crypto_hash_alg hash_alg, unsigned char *key, unsigned int key_len); -int cfs_crypto_hash_update_page(struct cfs_crypto_hash_desc *desc, +int cfs_crypto_hash_update_page(struct ahash_request *desc, struct page *page, unsigned int offset, unsigned int len); -int cfs_crypto_hash_update(struct cfs_crypto_hash_desc *desc, const void *buf, +int cfs_crypto_hash_update(struct ahash_request *desc, const void *buf, unsigned int buf_len); -int cfs_crypto_hash_final(struct cfs_crypto_hash_desc *desc, +int cfs_crypto_hash_final(struct ahash_request *desc, unsigned char *hash, unsigned int *hash_len); int cfs_crypto_register(void); void cfs_crypto_unregister(void); diff --git a/drivers/staging/lustre/lnet/libcfs/linux/linux-crypto.c b/drivers/staging/lustre/lnet/libcfs/linux/linux-crypto.c index 80072b2a443c..b55006264155 100644 --- a/drivers/staging/lustre/lnet/libcfs/linux/linux-crypto.c +++ b/drivers/staging/lustre/lnet/libcfs/linux/linux-crypto.c @@ -42,7 +42,7 @@ static int cfs_crypto_hash_speeds[CFS_HASH_ALG_MAX]; /** * Initialize the state descriptor for the specified hash algorithm. * - * An internal routine to allocate the hash-specific state in \a hdesc for + * An internal routine to allocate the hash-specific state in \a req for * use with cfs_crypto_hash_digest() to compute the hash of a single message, * though possibly in multiple chunks. The descriptor internal state should * be freed with cfs_crypto_hash_final(). @@ -50,7 +50,7 @@ static int cfs_crypto_hash_speeds[CFS_HASH_ALG_MAX]; * \param[in] hash_alg hash algorithm id (CFS_HASH_ALG_*) * \param[out] type pointer to the hash description in hash_types[] * array - * \param[in,out] hdesc hash state descriptor to be initialized + * \param[in,out] req hash state descriptor to be initialized * \param[in] key initial hash value/state, NULL to use default * value * \param[in] key_len length of \a key @@ -194,7 +194,7 @@ EXPORT_SYMBOL(cfs_crypto_hash_digest); * \retval pointer to descriptor of hash instance * \retval ERR_PTR(errno) in case of error */ -struct cfs_crypto_hash_desc * +struct ahash_request * cfs_crypto_hash_init(enum cfs_crypto_hash_alg hash_alg, unsigned char *key, unsigned int key_len) { @@ -206,14 +206,14 @@ cfs_crypto_hash_init(enum cfs_crypto_hash_alg hash_alg, if (err) return ERR_PTR(err); - return (struct cfs_crypto_hash_desc *)req; + return req; } EXPORT_SYMBOL(cfs_crypto_hash_init); /** * Update hash digest computed on data within the given \a page * - * \param[in] hdesc hash state descriptor + * \param[in] hreq hash state descriptor * \param[in] page data page on which to compute the hash * \param[in] offset offset within \a page at which to start hash * \param[in] len length of data on which to compute hash @@ -221,11 +221,10 @@ EXPORT_SYMBOL(cfs_crypto_hash_init); * \retval 0 for success * \retval negative errno on failure */ -int cfs_crypto_hash_update_page(struct cfs_crypto_hash_desc *hdesc, +int cfs_crypto_hash_update_page(struct ahash_request *req, struct page *page, unsigned int offset, unsigned int len) { - struct ahash_request *req = (void *)hdesc; struct scatterlist sl; sg_init_table(&sl, 1); @@ -239,17 +238,16 @@ EXPORT_SYMBOL(cfs_crypto_hash_update_page); /** * Update hash digest computed on the specified data * - * \param[in] hdesc hash state descriptor + * \param[in] req hash state descriptor * \param[in] buf data buffer on which to compute the hash * \param[in] buf_len length of \buf on which to compute hash * * \retval 0 for success * \retval negative errno on failure */ -int cfs_crypto_hash_update(struct cfs_crypto_hash_desc *hdesc, +int cfs_crypto_hash_update(struct ahash_request *req, const void *buf, unsigned int buf_len) { - struct ahash_request *req = (void *)hdesc; struct scatterlist sl; sg_init_one(&sl, buf, buf_len); @@ -262,20 +260,19 @@ EXPORT_SYMBOL(cfs_crypto_hash_update); /** * Finish hash calculation, copy hash digest to buffer, clean up hash descriptor * - * \param[in] hdesc hash descriptor + * \param[in] req hash descriptor * \param[out] hash pointer to hash buffer to store hash digest - * \param[in,out] hash_len pointer to hash buffer size, if \a hdesc = NULL - * only free \a hdesc instead of computing the hash + * \param[in,out] hash_len pointer to hash buffer size, if \a req = NULL + * only free \a req instead of computing the hash * * \retval 0 for success * \retval -EOVERFLOW if hash_len is too small for the hash digest * \retval negative errno for other errors from lower layers */ -int cfs_crypto_hash_final(struct cfs_crypto_hash_desc *hdesc, +int cfs_crypto_hash_final(struct ahash_request *req, unsigned char *hash, unsigned int *hash_len) { int err; - struct ahash_request *req = (void *)hdesc; int size = crypto_ahash_digestsize(crypto_ahash_reqtfm(req)); if (!hash || !hash_len) { @@ -331,7 +328,7 @@ static void cfs_crypto_performance_test(enum cfs_crypto_hash_alg hash_alg) for (start = jiffies, end = start + msecs_to_jiffies(MSEC_PER_SEC), bcount = 0; time_before(jiffies, end); bcount++) { - struct cfs_crypto_hash_desc *hdesc; + struct ahash_request *hdesc; int i; hdesc = cfs_crypto_hash_init(hash_alg, NULL, 0); diff --git a/drivers/staging/lustre/lustre/osc/osc_request.c b/drivers/staging/lustre/lustre/osc/osc_request.c index 074b5ce6284c..1c2bbbf5d864 100644 --- a/drivers/staging/lustre/lustre/osc/osc_request.c +++ b/drivers/staging/lustre/lustre/osc/osc_request.c @@ -931,7 +931,7 @@ static u32 osc_checksum_bulk(int nob, u32 pg_count, { __u32 cksum; int i = 0; - struct cfs_crypto_hash_desc *hdesc; + struct ahash_request *hdesc; unsigned int bufsize; unsigned char cfs_alg = cksum_obd2cfs(cksum_type); diff --git a/drivers/staging/lustre/lustre/ptlrpc/sec_bulk.c b/drivers/staging/lustre/lustre/ptlrpc/sec_bulk.c index 2184022ed724..577c5822b823 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/sec_bulk.c +++ b/drivers/staging/lustre/lustre/ptlrpc/sec_bulk.c @@ -530,7 +530,7 @@ EXPORT_SYMBOL(bulk_sec_desc_unpack); int sptlrpc_get_bulk_checksum(struct ptlrpc_bulk_desc *desc, __u8 alg, void *buf, int buflen) { - struct cfs_crypto_hash_desc *hdesc; + struct ahash_request *hdesc; int hashsize; unsigned int bufsize; int i, err; From neilb at suse.com Tue Feb 20 02:23:37 2018 From: neilb at suse.com (NeilBrown) Date: Tue, 20 Feb 2018 13:23:37 +1100 Subject: [lustre-devel] [PATCH 04/21] staging: lustre: remove unnecessary cfs_block_allsigs() calls In-Reply-To: <151909308778.25814.9912662268627044409.stgit@noble> References: <151909308778.25814.9912662268627044409.stgit@noble> Message-ID: <151909341766.25814.10094434408580614776.stgit@noble> Threads started by kthread_run() ignore all signals, as kthreadd() calls ignore_signals(), and this is inherited by all children. So there is no need to call cfs_block_allsigs() in functions that are only run from kthread_run(). Signed-off-by: NeilBrown --- .../staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c | 6 ------ .../staging/lustre/lnet/klnds/socklnd/socklnd_cb.c | 6 ------ drivers/staging/lustre/lnet/lnet/acceptor.c | 2 -- drivers/staging/lustre/lnet/lnet/router.c | 2 -- drivers/staging/lustre/lnet/selftest/timer.c | 2 -- 5 files changed, 18 deletions(-) diff --git a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c index ca094555e04b..6690a6cd4e34 100644 --- a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c +++ b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c @@ -3288,8 +3288,6 @@ kiblnd_connd(void *arg) int peer_index = 0; unsigned long deadline = jiffies; - cfs_block_allsigs(); - init_waitqueue_entry(&wait, current); kiblnd_data.kib_connd = current; @@ -3542,8 +3540,6 @@ kiblnd_scheduler(void *arg) int busy_loops = 0; int rc; - cfs_block_allsigs(); - init_waitqueue_entry(&wait, current); sched = kiblnd_data.kib_scheds[KIB_THREAD_CPT(id)]; @@ -3676,8 +3672,6 @@ kiblnd_failover_thread(void *arg) LASSERT(*kiblnd_tunables.kib_dev_failover); - cfs_block_allsigs(); - init_waitqueue_entry(&wait, current); write_lock_irqsave(glock, flags); diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c index 63e452f666bf..6ab002c006ab 100644 --- a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c +++ b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c @@ -1324,8 +1324,6 @@ int ksocknal_scheduler(void *arg) info = ksocknal_data.ksnd_sched_info[KSOCK_THREAD_CPT(id)]; sched = &info->ksi_scheds[KSOCK_THREAD_SID(id)]; - cfs_block_allsigs(); - rc = cfs_cpt_bind(lnet_cpt_table(), info->ksi_cpt); if (rc) { CWARN("Can't set CPU partition affinity to %d: %d\n", @@ -2078,8 +2076,6 @@ ksocknal_connd(void *arg) int nloops = 0; int cons_retry = 0; - cfs_block_allsigs(); - init_waitqueue_entry(&wait, current); spin_lock_bh(connd_lock); @@ -2472,8 +2468,6 @@ ksocknal_reaper(void *arg) int peer_index = 0; unsigned long deadline = cfs_time_current(); - cfs_block_allsigs(); - INIT_LIST_HEAD(&enomem_conns); init_waitqueue_entry(&wait, current); diff --git a/drivers/staging/lustre/lnet/lnet/acceptor.c b/drivers/staging/lustre/lnet/lnet/acceptor.c index 6c1f4941d4ba..fb478e20e204 100644 --- a/drivers/staging/lustre/lnet/lnet/acceptor.c +++ b/drivers/staging/lustre/lnet/lnet/acceptor.c @@ -335,8 +335,6 @@ lnet_acceptor(void *arg) LASSERT(!lnet_acceptor_state.pta_sock); - cfs_block_allsigs(); - rc = lnet_sock_listen(&lnet_acceptor_state.pta_sock, 0, accept_port, accept_backlog); if (rc) { diff --git a/drivers/staging/lustre/lnet/lnet/router.c b/drivers/staging/lustre/lnet/lnet/router.c index ec1d3e58519a..a3c3f4959f46 100644 --- a/drivers/staging/lustre/lnet/lnet/router.c +++ b/drivers/staging/lustre/lnet/lnet/router.c @@ -1226,8 +1226,6 @@ lnet_router_checker(void *arg) struct lnet_peer *rtr; struct list_head *entry; - cfs_block_allsigs(); - while (the_lnet.ln_rc_state == LNET_RC_STATE_RUNNING) { __u64 version; int cpt; diff --git a/drivers/staging/lustre/lnet/selftest/timer.c b/drivers/staging/lustre/lnet/selftest/timer.c index 9716afeb3c94..1b2c5fc81358 100644 --- a/drivers/staging/lustre/lnet/selftest/timer.c +++ b/drivers/staging/lustre/lnet/selftest/timer.c @@ -170,8 +170,6 @@ stt_timer_main(void *arg) { int rc = 0; - cfs_block_allsigs(); - while (!stt_data.stt_shuttingdown) { stt_check_timers(&stt_data.stt_prev_slot); From neilb at suse.com Tue Feb 20 02:23:38 2018 From: neilb at suse.com (NeilBrown) Date: Tue, 20 Feb 2018 13:23:38 +1100 Subject: [lustre-devel] [PATCH 19/21] staging: lustre: fid: fix up debugfs access to ->lcs_space In-Reply-To: <151909308778.25814.9912662268627044409.stgit@noble> References: <151909308778.25814.9912662268627044409.stgit@noble> Message-ID: <151909341819.25814.15953004503314348403.stgit@noble> lcs_space can change while the lock is not held if an RPC in underway. This can be detected by seq->ls_update being set. In this case, reading or writing the value should return -EBUSY. Also, the D_INFO CDEBUG() which reports the lcs_space being updated never fires, as it tests the wrong value - ldebugfs_fid_write_common() returns 'count' on success. Finally, this return value should be returned from ldebugfs_fid_space_seq_write(), rather than always returning 'count', so that errors can be detected. Signed-off-by: NeilBrown --- drivers/staging/lustre/lustre/fid/lproc_fid.c | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-) diff --git a/drivers/staging/lustre/lustre/fid/lproc_fid.c b/drivers/staging/lustre/lustre/fid/lproc_fid.c index e05487662afd..a1e5bf9f36ec 100644 --- a/drivers/staging/lustre/lustre/fid/lproc_fid.c +++ b/drivers/staging/lustre/lustre/fid/lproc_fid.c @@ -106,28 +106,35 @@ ldebugfs_fid_space_seq_write(struct file *file, rc = ldebugfs_fid_write_common(buffer, count, &range); spin_lock(&seq->lcs_lock); + if (seq->lcs_update) + /* An RPC call is active to update lcs_space */ + rc = -EBUSY; if (rc > 0) seq->lcs_space = range; spin_unlock(&seq->lcs_lock); - if (rc == 0) { + if (rc > 0) { CDEBUG(D_INFO, "%s: Space: " DRANGE "\n", seq->lcs_name, PRANGE(&range)); } - return count; + return rc; } static int ldebugfs_fid_space_seq_show(struct seq_file *m, void *unused) { struct lu_client_seq *seq = (struct lu_client_seq *)m->private; + int rc = 0; spin_lock(&seq->lcs_lock); - seq_printf(m, "[%#llx - %#llx]:%x:%s\n", PRANGE(&seq->lcs_space)); + if (seq->lcs_update) + rc = -EBUSY; + else + seq_printf(m, "[%#llx - %#llx]:%x:%s\n", PRANGE(&seq->lcs_space)); spin_unlock(&seq->lcs_lock); - return 0; + return rc; } static ssize_t From neilb at suse.com Tue Feb 20 02:23:38 2018 From: neilb at suse.com (NeilBrown) Date: Tue, 20 Feb 2018 13:23:38 +1100 Subject: [lustre-devel] [PATCH 14/21] staging: lustre: fix assorted checkpatch errors In-Reply-To: <151909308778.25814.9912662268627044409.stgit@noble> References: <151909308778.25814.9912662268627044409.stgit@noble> Message-ID: <151909341801.25814.16760126077741712102.stgit@noble> Possible the most interesting is the for-loop with no body. I think it reads better as a while loop. Signed-off-by: NeilBrown --- .../lustre/lnet/klnds/socklnd/socklnd_lib.c | 2 +- .../staging/lustre/lnet/libcfs/linux/linux-cpu.c | 2 +- drivers/staging/lustre/lustre/include/obd_class.h | 8 ++++---- .../staging/lustre/lustre/include/obd_support.h | 2 +- drivers/staging/lustre/lustre/llite/dcache.c | 2 +- drivers/staging/lustre/lustre/llite/llite_lib.c | 2 +- drivers/staging/lustre/lustre/lov/lov_request.c | 2 +- drivers/staging/lustre/lustre/mdc/mdc_request.c | 6 ++++-- drivers/staging/lustre/lustre/ptlrpc/layout.c | 2 +- drivers/staging/lustre/lustre/ptlrpc/recover.c | 2 +- drivers/staging/lustre/lustre/ptlrpc/service.c | 2 +- 11 files changed, 17 insertions(+), 15 deletions(-) diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_lib.c b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_lib.c index cb28dd2baf2f..7941cfa526bc 100644 --- a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_lib.c +++ b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_lib.c @@ -189,7 +189,7 @@ ksocknal_lib_recv(struct ksock_conn *conn) if (!(conn->ksnc_rx_to.type & ITER_BVEC) && conn->ksnc_proto != &ksocknal_protocol_v2x) return rc; - + /* accumulate checksum */ conn->ksnc_msg.ksm_csum = 0; iov_iter_for_each_range(&conn->ksnc_rx_to, rc, lustre_csum, conn); diff --git a/drivers/staging/lustre/lnet/libcfs/linux/linux-cpu.c b/drivers/staging/lustre/lnet/libcfs/linux/linux-cpu.c index c07165e0ad95..388521e4e354 100644 --- a/drivers/staging/lustre/lnet/libcfs/linux/linux-cpu.c +++ b/drivers/staging/lustre/lnet/libcfs/linux/linux-cpu.c @@ -743,7 +743,7 @@ cfs_cpt_table_create(int ncpt) goto failed; } - if (!zalloc_cpumask_var(&mask, GFP_NOFS)){ + if (!zalloc_cpumask_var(&mask, GFP_NOFS)) { CERROR("Failed to allocate scratch cpumask\n"); goto failed; } diff --git a/drivers/staging/lustre/lustre/include/obd_class.h b/drivers/staging/lustre/lustre/include/obd_class.h index 531e8ddfa9e5..f24dd74ffa09 100644 --- a/drivers/staging/lustre/lustre/include/obd_class.h +++ b/drivers/staging/lustre/lustre/include/obd_class.h @@ -294,10 +294,10 @@ struct obdo; void obdo_to_ioobj(const struct obdo *oa, struct obd_ioobj *ioobj); -#define OBT(dev) (dev)->obd_type -#define OBP(dev, op) (dev)->obd_type->typ_dt_ops->op -#define MDP(dev, op) (dev)->obd_type->typ_md_ops->op -#define CTXTP(ctxt, op) (ctxt)->loc_logops->lop_##op +#define OBT(dev) ((dev)->obd_type) +#define OBP(dev, op) ((dev)->obd_type->typ_dt_ops->op) +#define MDP(dev, op) ((dev)->obd_type->typ_md_ops->op) +#define CTXTP(ctxt, op) ((ctxt)->loc_logops->lop_##op) /* * Ensure obd_setup: used for cleanup which must be called diff --git a/drivers/staging/lustre/lustre/include/obd_support.h b/drivers/staging/lustre/lustre/include/obd_support.h index 3f4fe290f6ea..8595091b8b86 100644 --- a/drivers/staging/lustre/lustre/include/obd_support.h +++ b/drivers/staging/lustre/lustre/include/obd_support.h @@ -516,7 +516,7 @@ extern char obd_jobid_var[]; #define POISON_PTR(ptr) ((void)0) #else #define POISON(ptr, c, s) memset(ptr, c, s) -#define POISON_PTR(ptr) (ptr) = (void *)0xdeadbeef +#define POISON_PTR(ptr) ((ptr) = (void *)0xdeadbeef) #endif #ifdef POISON_BULK diff --git a/drivers/staging/lustre/lustre/llite/dcache.c b/drivers/staging/lustre/lustre/llite/dcache.c index dc30b4582234..3e768f997172 100644 --- a/drivers/staging/lustre/lustre/llite/dcache.c +++ b/drivers/staging/lustre/lustre/llite/dcache.c @@ -100,7 +100,7 @@ static int ll_dcompare(const struct dentry *dentry, return 0; /* ensure exclusion against parallel lookup of the same name */ - if (d_in_lookup((struct dentry*)dentry)) + if (d_in_lookup((struct dentry *)dentry)) return 0; if (d_lustre_invalid(dentry)) diff --git a/drivers/staging/lustre/lustre/llite/llite_lib.c b/drivers/staging/lustre/lustre/llite/llite_lib.c index efbd551e7842..844182ad7dd7 100644 --- a/drivers/staging/lustre/lustre/llite/llite_lib.c +++ b/drivers/staging/lustre/lustre/llite/llite_lib.c @@ -2022,7 +2022,7 @@ void ll_umount_begin(struct super_block *sb) */ while (cnt < 10 && !may_umount(sbi->ll_mnt.mnt)) { schedule_timeout_uninterruptible(HZ); - cnt ++; + cnt++; } schedule(); diff --git a/drivers/staging/lustre/lustre/lov/lov_request.c b/drivers/staging/lustre/lustre/lov/lov_request.c index c1e58fcc30b3..051450d67524 100644 --- a/drivers/staging/lustre/lustre/lov/lov_request.c +++ b/drivers/staging/lustre/lustre/lov/lov_request.c @@ -126,7 +126,7 @@ static int lov_check_and_wait_active(struct lov_obd *lov, int ost_idx) while (cnt < obd_timeout && !lov_check_set(lov, ost_idx)) { schedule_timeout_uninterruptible(HZ); - cnt ++; + cnt++; } if (tgt->ltd_active) return 1; diff --git a/drivers/staging/lustre/lustre/mdc/mdc_request.c b/drivers/staging/lustre/lustre/mdc/mdc_request.c index ab48746ce433..bde27acb0dd3 100644 --- a/drivers/staging/lustre/lustre/mdc/mdc_request.c +++ b/drivers/staging/lustre/lustre/mdc/mdc_request.c @@ -1060,8 +1060,10 @@ static void mdc_adjust_dirpages(struct page **pages, int cfs_pgs, int lu_pgs) while (--lu_pgs > 0) { ent = lu_dirent_start(dp); - for (end_dirent = ent; ent; - end_dirent = ent, ent = lu_dirent_next(ent)); + while (ent) { + end_dirent = ent; + ent = lu_dirent_next(ent); + } /* Advance dp to next lu_dirpage. */ dp = (struct lu_dirpage *)((char *)dp + LU_PAGE_SIZE); diff --git a/drivers/staging/lustre/lustre/ptlrpc/layout.c b/drivers/staging/lustre/lustre/ptlrpc/layout.c index 18769d335751..2855f38c8190 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/layout.c +++ b/drivers/staging/lustre/lustre/ptlrpc/layout.c @@ -1555,7 +1555,7 @@ struct req_format RQF_OST_GET_INFO_FIEMAP = EXPORT_SYMBOL(RQF_OST_GET_INFO_FIEMAP); /* Convenience macro */ -#define FMT_FIELD(fmt, i, j) (fmt)->rf_fields[(i)].d[(j)] +#define FMT_FIELD(fmt, i, j) ((fmt)->rf_fields[(i)].d[(j)]) /** * Initializes the capsule abstraction by computing and setting the \a rf_idx diff --git a/drivers/staging/lustre/lustre/ptlrpc/recover.c b/drivers/staging/lustre/lustre/ptlrpc/recover.c index 7b5f2429d144..5bb9f9fe91d8 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/recover.c +++ b/drivers/staging/lustre/lustre/ptlrpc/recover.c @@ -354,7 +354,7 @@ int ptlrpc_recover_import(struct obd_import *imp, char *new_uuid, int async) obd_timeout * HZ); CDEBUG(D_HA, "%s: recovery finished\n", obd2cli_tgt(imp->imp_obd)); - rc = rc? 0 : -ETIMEDOUT; + rc = rc ? 0 : -ETIMEDOUT; } out: diff --git a/drivers/staging/lustre/lustre/ptlrpc/service.c b/drivers/staging/lustre/lustre/ptlrpc/service.c index 79d9f3860022..99aeb291f3f2 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/service.c +++ b/drivers/staging/lustre/lustre/ptlrpc/service.c @@ -2670,7 +2670,7 @@ ptlrpc_service_unlink_rqbd(struct ptlrpc_service *svc) (rc = wait_event_idle_timeout(svcpt->scp_waitq, svcpt->scp_nrqbds_posted == 0, HZ)) == 0) - cnt ++; + cnt++; if (rc == 0) { CWARN("Service %s waiting for request buffers\n", svcpt->scp_service->srv_name); From neilb at suse.com Tue Feb 20 02:23:38 2018 From: neilb at suse.com (NeilBrown) Date: Tue, 20 Feb 2018 13:23:38 +1100 Subject: [lustre-devel] [PATCH 20/21] staging: lustre: fid: perform sanity checks before commiting In-Reply-To: <151909308778.25814.9912662268627044409.stgit@noble> References: <151909308778.25814.9912662268627044409.stgit@noble> Message-ID: <151909341823.25814.12569503440862267846.stgit@noble> When fid fetches a new range from the server, it commits to it (*output = *out) *before* performing sanity checks. This looks backwards. Don't commit to a value until it has been found to be sane. Signed-off-by: NeilBrown --- drivers/staging/lustre/lustre/fid/fid_request.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/drivers/staging/lustre/lustre/fid/fid_request.c b/drivers/staging/lustre/lustre/fid/fid_request.c index 6e0dfe18ef5f..fa23423eb8b3 100644 --- a/drivers/staging/lustre/lustre/fid/fid_request.c +++ b/drivers/staging/lustre/lustre/fid/fid_request.c @@ -118,22 +118,22 @@ static int seq_client_rpc(struct lu_client_seq *seq, goto out_req; out = req_capsule_server_get(&req->rq_pill, &RMF_SEQ_RANGE); - *output = *out; - if (!lu_seq_range_is_sane(output)) { + if (!lu_seq_range_is_sane(out)) { CERROR("%s: Invalid range received from server: " - DRANGE "\n", seq->lcs_name, PRANGE(output)); + DRANGE "\n", seq->lcs_name, PRANGE(out)); rc = -EINVAL; goto out_req; } - if (lu_seq_range_is_exhausted(output)) { + if (lu_seq_range_is_exhausted(out)) { CERROR("%s: Range received from server is exhausted: " - DRANGE "]\n", seq->lcs_name, PRANGE(output)); + DRANGE "]\n", seq->lcs_name, PRANGE(out)); rc = -EINVAL; goto out_req; } + *output = *out; CDEBUG_LIMIT(debug_mask, "%s: Allocated %s-sequence " DRANGE "]\n", seq->lcs_name, opcname, PRANGE(output)); From neilb at suse.com Tue Feb 20 02:23:38 2018 From: neilb at suse.com (NeilBrown) Date: Tue, 20 Feb 2018 13:23:38 +1100 Subject: [lustre-devel] [PATCH 16/21] staging: lustre: fid: convert lcs_mutex to a spinlock In-Reply-To: <151909308778.25814.9912662268627044409.stgit@noble> References: <151909308778.25814.9912662268627044409.stgit@noble> Message-ID: <151909341810.25814.8461248924883891726.stgit@noble> There is only one place where this lock is held while the task might sleep - in ldebugfs_fid_space_seq_write() while ldebugfs_fid_write_common() is called. This call can easily be taken out of the locked region by asking it to parse the user data into a local variable, and then copying that variable into ->lcs_space while holding the lock. Note that ldebugfs_gid_write_common returns >0 on success, so use that to gate updating ->lcs_space. So make that change, and convert lcs_mutex to a spinlock named lcs_lock. spinlocks are slightly cheaper than mutexes and using one makes is clear that the lock is only held for a short time. Signed-off-by: NeilBrown --- drivers/staging/lustre/lustre/fid/fid_request.c | 24 ++++++++--------- drivers/staging/lustre/lustre/fid/lproc_fid.c | 29 +++++++++++--------- drivers/staging/lustre/lustre/include/lustre_fid.h | 2 + 3 files changed, 29 insertions(+), 26 deletions(-) diff --git a/drivers/staging/lustre/lustre/fid/fid_request.c b/drivers/staging/lustre/lustre/fid/fid_request.c index 009c2367f74e..6b9d024bd27b 100644 --- a/drivers/staging/lustre/lustre/fid/fid_request.c +++ b/drivers/staging/lustre/lustre/fid/fid_request.c @@ -198,24 +198,24 @@ static int seq_fid_alloc_prep(struct lu_client_seq *seq, if (seq->lcs_update) { add_wait_queue(&seq->lcs_waitq, link); set_current_state(TASK_UNINTERRUPTIBLE); - mutex_unlock(&seq->lcs_mutex); + spin_unlock(&seq->lcs_lock); schedule(); - mutex_lock(&seq->lcs_mutex); + spin_lock(&seq->lcs_lock); remove_wait_queue(&seq->lcs_waitq, link); set_current_state(TASK_RUNNING); return -EAGAIN; } ++seq->lcs_update; - mutex_unlock(&seq->lcs_mutex); + spin_unlock(&seq->lcs_lock); return 0; } static void seq_fid_alloc_fini(struct lu_client_seq *seq) { LASSERT(seq->lcs_update == 1); - mutex_lock(&seq->lcs_mutex); + spin_lock(&seq->lcs_lock); --seq->lcs_update; wake_up(&seq->lcs_waitq); } @@ -231,7 +231,7 @@ int seq_client_alloc_fid(const struct lu_env *env, LASSERT(fid); init_waitqueue_entry(&link, current); - mutex_lock(&seq->lcs_mutex); + spin_lock(&seq->lcs_lock); if (OBD_FAIL_CHECK(OBD_FAIL_SEQ_EXHAUST)) seq->lcs_fid.f_oid = seq->lcs_width; @@ -256,7 +256,7 @@ int seq_client_alloc_fid(const struct lu_env *env, CERROR("%s: Can't allocate new sequence, rc %d\n", seq->lcs_name, rc); seq_fid_alloc_fini(seq); - mutex_unlock(&seq->lcs_mutex); + spin_unlock(&seq->lcs_lock); return rc; } @@ -278,7 +278,7 @@ int seq_client_alloc_fid(const struct lu_env *env, } *fid = seq->lcs_fid; - mutex_unlock(&seq->lcs_mutex); + spin_unlock(&seq->lcs_lock); CDEBUG(D_INFO, "%s: Allocated FID " DFID "\n", seq->lcs_name, PFID(fid)); @@ -296,16 +296,16 @@ void seq_client_flush(struct lu_client_seq *seq) LASSERT(seq); init_waitqueue_entry(&link, current); - mutex_lock(&seq->lcs_mutex); + spin_lock(&seq->lcs_lock); while (seq->lcs_update) { add_wait_queue(&seq->lcs_waitq, &link); set_current_state(TASK_UNINTERRUPTIBLE); - mutex_unlock(&seq->lcs_mutex); + spin_unlock(&seq->lcs_lock); schedule(); - mutex_lock(&seq->lcs_mutex); + spin_lock(&seq->lcs_lock); remove_wait_queue(&seq->lcs_waitq, &link); set_current_state(TASK_RUNNING); } @@ -319,7 +319,7 @@ void seq_client_flush(struct lu_client_seq *seq) seq->lcs_space.lsr_index = -1; lu_seq_range_init(&seq->lcs_space); - mutex_unlock(&seq->lcs_mutex); + spin_unlock(&seq->lcs_lock); } EXPORT_SYMBOL(seq_client_flush); @@ -382,7 +382,7 @@ static int seq_client_init(struct lu_client_seq *seq, seq->lcs_type = type; - mutex_init(&seq->lcs_mutex); + spin_lock_init(&seq->lcs_lock); if (type == LUSTRE_SEQ_METADATA) seq->lcs_width = LUSTRE_METADATA_SEQ_MAX_WIDTH; else diff --git a/drivers/staging/lustre/lustre/fid/lproc_fid.c b/drivers/staging/lustre/lustre/fid/lproc_fid.c index 083419f77697..e05487662afd 100644 --- a/drivers/staging/lustre/lustre/fid/lproc_fid.c +++ b/drivers/staging/lustre/lustre/fid/lproc_fid.c @@ -98,20 +98,23 @@ ldebugfs_fid_space_seq_write(struct file *file, size_t count, loff_t *off) { struct lu_client_seq *seq; + struct lu_seq_range range; int rc; seq = ((struct seq_file *)file->private_data)->private; - mutex_lock(&seq->lcs_mutex); - rc = ldebugfs_fid_write_common(buffer, count, &seq->lcs_space); + rc = ldebugfs_fid_write_common(buffer, count, &range); + + spin_lock(&seq->lcs_lock); + if (rc > 0) + seq->lcs_space = range; + spin_unlock(&seq->lcs_lock); if (rc == 0) { CDEBUG(D_INFO, "%s: Space: " DRANGE "\n", - seq->lcs_name, PRANGE(&seq->lcs_space)); + seq->lcs_name, PRANGE(&range)); } - mutex_unlock(&seq->lcs_mutex); - return count; } @@ -120,9 +123,9 @@ ldebugfs_fid_space_seq_show(struct seq_file *m, void *unused) { struct lu_client_seq *seq = (struct lu_client_seq *)m->private; - mutex_lock(&seq->lcs_mutex); + spin_lock(&seq->lcs_lock); seq_printf(m, "[%#llx - %#llx]:%x:%s\n", PRANGE(&seq->lcs_space)); - mutex_unlock(&seq->lcs_mutex); + spin_unlock(&seq->lcs_lock); return 0; } @@ -142,7 +145,7 @@ ldebugfs_fid_width_seq_write(struct file *file, if (rc) return rc; - mutex_lock(&seq->lcs_mutex); + spin_lock(&seq->lcs_lock); if (seq->lcs_type == LUSTRE_SEQ_DATA) max = LUSTRE_DATA_SEQ_MAX_WIDTH; else @@ -155,7 +158,7 @@ ldebugfs_fid_width_seq_write(struct file *file, seq->lcs_width); } - mutex_unlock(&seq->lcs_mutex); + spin_unlock(&seq->lcs_lock); return count; } @@ -165,9 +168,9 @@ ldebugfs_fid_width_seq_show(struct seq_file *m, void *unused) { struct lu_client_seq *seq = (struct lu_client_seq *)m->private; - mutex_lock(&seq->lcs_mutex); + spin_lock(&seq->lcs_lock); seq_printf(m, "%llu\n", seq->lcs_width); - mutex_unlock(&seq->lcs_mutex); + spin_unlock(&seq->lcs_lock); return 0; } @@ -177,9 +180,9 @@ ldebugfs_fid_fid_seq_show(struct seq_file *m, void *unused) { struct lu_client_seq *seq = (struct lu_client_seq *)m->private; - mutex_lock(&seq->lcs_mutex); + spin_lock(&seq->lcs_lock); seq_printf(m, DFID "\n", PFID(&seq->lcs_fid)); - mutex_unlock(&seq->lcs_mutex); + spin_unlock(&seq->lcs_lock); return 0; } diff --git a/drivers/staging/lustre/lustre/include/lustre_fid.h b/drivers/staging/lustre/lustre/include/lustre_fid.h index d19c7a27ee48..094ad282de2c 100644 --- a/drivers/staging/lustre/lustre/include/lustre_fid.h +++ b/drivers/staging/lustre/lustre/include/lustre_fid.h @@ -324,7 +324,7 @@ enum lu_mgr_type { struct lu_client_seq { /* Sequence-controller export. */ struct obd_export *lcs_exp; - struct mutex lcs_mutex; + spinlock_t lcs_lock; /* * Range of allowed for allocation sequences. When using lu_client_seq on From neilb at suse.com Tue Feb 20 02:23:37 2018 From: neilb at suse.com (NeilBrown) Date: Tue, 20 Feb 2018 13:23:37 +1100 Subject: [lustre-devel] [PATCH 07/21] staging: lustre: improve API and implementation of blocking signals. In-Reply-To: <151909308778.25814.9912662268627044409.stgit@noble> References: <151909308778.25814.9912662268627044409.stgit@noble> Message-ID: <151909341776.25814.1357059430355267483.stgit@noble> According to comment for set_current_blocked() in kernel/signal.c, changing ->blocked directly is wrong. sigprocmask() should be called instead. So change cfs_block_sigsinv() and cfs_restore_sigs() to use sigprocmask(). For consistency, change them to pass the sigset_t by reference rather than by value. Also fix cfs_block_sigsinv() so that it correctly blocks signals above 32 on a 32bit host. Signed-off-by: NeilBrown --- .../staging/lustre/include/linux/libcfs/libcfs.h | 4 ++- .../staging/lustre/lnet/libcfs/linux/linux-prim.c | 24 ++++++-------------- drivers/staging/lustre/lustre/include/lustre_lib.h | 18 ++++++++------- drivers/staging/lustre/lustre/llite/llite_mmap.c | 8 +++---- 4 files changed, 22 insertions(+), 32 deletions(-) diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs.h b/drivers/staging/lustre/include/linux/libcfs/libcfs.h index 7f06b0118154..9dff050810c4 100644 --- a/drivers/staging/lustre/include/linux/libcfs/libcfs.h +++ b/drivers/staging/lustre/include/linux/libcfs/libcfs.h @@ -67,8 +67,8 @@ /* * Defined by platform */ -sigset_t cfs_block_sigsinv(unsigned long sigs); -void cfs_restore_sigs(sigset_t sigset); +void cfs_block_sigsinv(unsigned long sigs, sigset_t *sigset); +void cfs_restore_sigs(sigset_t *sigset); struct libcfs_ioctl_handler { struct list_head item; diff --git a/drivers/staging/lustre/lnet/libcfs/linux/linux-prim.c b/drivers/staging/lustre/lnet/libcfs/linux/linux-prim.c index 6b73b9845e3f..0766659ddf70 100644 --- a/drivers/staging/lustre/lnet/libcfs/linux/linux-prim.c +++ b/drivers/staging/lustre/lnet/libcfs/linux/linux-prim.c @@ -44,29 +44,19 @@ #endif /* Block all signals except for the @sigs */ -sigset_t cfs_block_sigsinv(unsigned long sigs) +void cfs_block_sigsinv(unsigned long sigs, sigset_t *old) { - unsigned long flags; - sigset_t old; + sigset_t new; - spin_lock_irqsave(¤t->sighand->siglock, flags); - old = current->blocked; - sigaddsetmask(¤t->blocked, ~sigs); - recalc_sigpending(); - spin_unlock_irqrestore(¤t->sighand->siglock, flags); - - return old; + siginitsetinv(&new, sigs); + sigorsets(&new, ¤t->blocked, &new); + sigprocmask(SIG_BLOCK, &new, old); } EXPORT_SYMBOL(cfs_block_sigsinv); void -cfs_restore_sigs(sigset_t old) +cfs_restore_sigs(sigset_t *old) { - unsigned long flags; - - spin_lock_irqsave(¤t->sighand->siglock, flags); - current->blocked = old; - recalc_sigpending(); - spin_unlock_irqrestore(¤t->sighand->siglock, flags); + sigprocmask(SIG_SETMASK, old, NULL); } EXPORT_SYMBOL(cfs_restore_sigs); diff --git a/drivers/staging/lustre/lustre/include/lustre_lib.h b/drivers/staging/lustre/lustre/include/lustre_lib.h index 1efd86f18c1f..0053eafc1c10 100644 --- a/drivers/staging/lustre/lustre/include/lustre_lib.h +++ b/drivers/staging/lustre/lustre/include/lustre_lib.h @@ -94,31 +94,31 @@ static inline int l_fatal_signal_pending(struct task_struct *p) */ #define l_wait_event_abortable(wq, condition) \ ({ \ - sigset_t __blocked; \ + sigset_t __old_blocked; \ int __ret = 0; \ - __blocked = cfs_block_sigsinv(LUSTRE_FATAL_SIGS); \ + cfs_block_sigsinv(LUSTRE_FATAL_SIGS, &__old_blocked); \ __ret = wait_event_interruptible(wq, condition); \ - cfs_restore_sigs(__blocked); \ + cfs_restore_sigs(&__old_blocked); \ __ret; \ }) #define l_wait_event_abortable_timeout(wq, condition, timeout) \ ({ \ - sigset_t __blocked; \ + sigset_t __old_blocked; \ int __ret = 0; \ - __blocked = cfs_block_sigsinv(LUSTRE_FATAL_SIGS); \ + cfs_block_sigsinv(LUSTRE_FATAL_SIGS, &__old_blocked); \ __ret = wait_event_interruptible_timeout(wq, condition, timeout);\ - cfs_restore_sigs(__blocked); \ + cfs_restore_sigs(&__old_blocked); \ __ret; \ }) #define l_wait_event_abortable_exclusive(wq, condition) \ ({ \ - sigset_t __blocked; \ + sigset_t __old_blocked; \ int __ret = 0; \ - __blocked = cfs_block_sigsinv(LUSTRE_FATAL_SIGS); \ + cfs_block_sigsinv(LUSTRE_FATAL_SIGS, &__old_blocked); \ __ret = wait_event_interruptible_exclusive(wq, condition); \ - cfs_restore_sigs(__blocked); \ + cfs_restore_sigs(&__old_blocked); \ __ret; \ }) #endif /* _LUSTRE_LIB_H */ diff --git a/drivers/staging/lustre/lustre/llite/llite_mmap.c b/drivers/staging/lustre/lustre/llite/llite_mmap.c index c0533bd6f352..214b07554e62 100644 --- a/drivers/staging/lustre/lustre/llite/llite_mmap.c +++ b/drivers/staging/lustre/lustre/llite/llite_mmap.c @@ -177,14 +177,14 @@ static int ll_page_mkwrite0(struct vm_area_struct *vma, struct page *vmpage, vio->u.fault.ft_vma = vma; vio->u.fault.ft_vmpage = vmpage; - set = cfs_block_sigsinv(sigmask(SIGKILL) | sigmask(SIGTERM)); + cfs_block_sigsinv(sigmask(SIGKILL) | sigmask(SIGTERM), &set); inode = vvp_object_inode(io->ci_obj); lli = ll_i2info(inode); result = cl_io_loop(env, io); - cfs_restore_sigs(set); + cfs_restore_sigs(&set); if (result == 0) { struct inode *inode = file_inode(vma->vm_file); @@ -334,7 +334,7 @@ static int ll_fault(struct vm_fault *vmf) * so that it can be killed by admin but not cause segfault by * other signals. */ - set = cfs_block_sigsinv(sigmask(SIGKILL) | sigmask(SIGTERM)); + cfs_block_sigsinv(sigmask(SIGKILL) | sigmask(SIGTERM), &set); restart: result = ll_fault0(vmf->vma, vmf); @@ -360,7 +360,7 @@ static int ll_fault(struct vm_fault *vmf) result = VM_FAULT_LOCKED; } - cfs_restore_sigs(set); + cfs_restore_sigs(&set); return result; } From paf at cray.com Tue Feb 20 04:18:07 2018 From: paf at cray.com (Patrick Farrell) Date: Tue, 20 Feb 2018 04:18:07 +0000 Subject: [lustre-devel] [PATCH 00/21] staging: assorted lustre clean-up In-Reply-To: <151909308778.25814.9912662268627044409.stgit@noble> References: <151909308778.25814.9912662268627044409.stgit@noble> Message-ID: Lustre networking driver. (Or so I was taught) Regards, Patrick ________________________________________ From: lustre-devel on behalf of NeilBrown Sent: Monday, February 19, 2018 8:23:37 PM To: Oleg Drokin; James Simmons; Andreas Dilger; Greg Kroah-Hartman Cc: lkml; lustre Subject: [lustre-devel] [PATCH 00/21] staging: assorted lustre clean-up Following are assorted patches which clean up parts of lustre. One or two tiny buglets are fixed, but mostly the patches just make the code more readable, often by removing distractions. The first several patches remove content from libcfs, completely removing 3 files. There are some list_for_each improvements in ptlrpc, and some code simplification in fid. Finally a small simplification in socklnd. (lnd ?? Lustre Networking Domain??) Thanks, NeilBrown --- NeilBrown (21): staging: lustre: replace all CFS_CAP_* macros with CAP_* staging: lustre: opencode cfs_cap_{raise,lower,raised} staging: lustre: remove linux-curproc.c staging: lustre: remove unnecessary cfs_block_allsigs() calls staging: lustre: lnet: remove cfs_block_allsigs calls. staging: lustre: simplify linux-prim.c staging: lustre: improve API and implementation of blocking signals. staging: lustre: make signal-blocking functions inline staging: lustre: discard libcfs_kvzalloc_cpt() staging: lustre: discard lu_buf allocation library. staging: lustre: improve some libcfs_kvzalloc calls. staging: lustre: discard libcfs_kvzalloc and linux-mem.c staging: lustre: remove phantom struct cfs_crypto_hash_desc staging: lustre: fix assorted checkpatch errors staging: lustre: ptlrpc: list_for_each improvements. staging: lustre: fid: convert lcs_mutex to a spinlock staging: lustre: fid: use wait_event_cmd() staging: lustre: fid: remove seq_fid_alloc_fini() and simplify staging: lustre: fid: fix up debugfs access to ->lcs_space staging: lustre: fid: perform sanity checks before commiting staging: lustre: socklnd: simplify ksnc_rx_iov_space .../staging/lustre/include/linux/libcfs/curproc.h | 37 ++----- .../staging/lustre/include/linux/libcfs/libcfs.h | 27 +++-- .../lustre/include/linux/libcfs/libcfs_crypto.h | 11 +- drivers/staging/lustre/include/linux/lnet/api.h | 1 .../staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c | 6 - .../staging/lustre/lnet/klnds/socklnd/socklnd.h | 11 -- .../staging/lustre/lnet/klnds/socklnd/socklnd_cb.c | 10 -- .../lustre/lnet/klnds/socklnd/socklnd_lib.c | 2 drivers/staging/lustre/lnet/libcfs/Makefile | 4 - .../staging/lustre/lnet/libcfs/linux/linux-cpu.c | 2 .../lustre/lnet/libcfs/linux/linux-crypto.c | 29 ++--- .../lustre/lnet/libcfs/linux/linux-curproc.c | 108 ------------------- .../staging/lustre/lnet/libcfs/linux/linux-mem.c | 51 --------- .../staging/lustre/lnet/libcfs/linux/linux-prim.c | 113 -------------------- drivers/staging/lustre/lnet/lnet/acceptor.c | 2 drivers/staging/lustre/lnet/lnet/api-ni.c | 15 +-- drivers/staging/lustre/lnet/lnet/lib-eq.c | 10 +- drivers/staging/lustre/lnet/lnet/router.c | 2 drivers/staging/lustre/lnet/selftest/timer.c | 2 drivers/staging/lustre/lustre/fid/fid_request.c | 106 ++++++------------- drivers/staging/lustre/lustre/fid/lproc_fid.c | 44 +++++--- drivers/staging/lustre/lustre/include/lu_object.h | 7 - drivers/staging/lustre/lustre/include/lustre_fid.h | 2 drivers/staging/lustre/lustre/include/lustre_lib.h | 18 ++- drivers/staging/lustre/lustre/include/lustre_sec.h | 3 - drivers/staging/lustre/lustre/include/obd_class.h | 8 + .../staging/lustre/lustre/include/obd_support.h | 2 drivers/staging/lustre/lustre/llite/dcache.c | 2 drivers/staging/lustre/lustre/llite/dir.c | 10 +- drivers/staging/lustre/lustre/llite/file.c | 12 +- drivers/staging/lustre/lustre/llite/llite_lib.c | 8 + drivers/staging/lustre/lustre/llite/llite_mmap.c | 8 + drivers/staging/lustre/lustre/llite/xattr.c | 2 drivers/staging/lustre/lustre/lmv/lmv_obd.c | 4 - drivers/staging/lustre/lustre/lov/lov_ea.c | 2 drivers/staging/lustre/lustre/lov/lov_io.c | 2 drivers/staging/lustre/lustre/lov/lov_lock.c | 2 drivers/staging/lustre/lustre/lov/lov_object.c | 4 - drivers/staging/lustre/lustre/lov/lov_pack.c | 2 drivers/staging/lustre/lustre/lov/lov_request.c | 2 drivers/staging/lustre/lustre/mdc/mdc_locks.c | 2 drivers/staging/lustre/lustre/mdc/mdc_request.c | 6 + drivers/staging/lustre/lustre/obdclass/linkea.c | 16 ++- .../lustre/lustre/obdclass/linux/linux-module.c | 4 - drivers/staging/lustre/lustre/obdclass/llog.c | 22 ++-- drivers/staging/lustre/lustre/obdclass/lu_object.c | 70 ------------ .../lustre/lustre/obdclass/lustre_handles.c | 4 - .../staging/lustre/lustre/obdecho/echo_client.c | 8 + drivers/staging/lustre/lustre/osc/osc_cache.c | 4 - drivers/staging/lustre/lustre/osc/osc_page.c | 2 drivers/staging/lustre/lustre/osc/osc_request.c | 2 drivers/staging/lustre/lustre/ptlrpc/client.c | 93 +++++----------- drivers/staging/lustre/lustre/ptlrpc/import.c | 34 ++---- drivers/staging/lustre/lustre/ptlrpc/layout.c | 2 drivers/staging/lustre/lustre/ptlrpc/pinger.c | 8 - drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c | 15 +-- drivers/staging/lustre/lustre/ptlrpc/recover.c | 28 ++--- drivers/staging/lustre/lustre/ptlrpc/sec.c | 4 - drivers/staging/lustre/lustre/ptlrpc/sec_bulk.c | 4 - drivers/staging/lustre/lustre/ptlrpc/sec_null.c | 8 + drivers/staging/lustre/lustre/ptlrpc/sec_plain.c | 8 + drivers/staging/lustre/lustre/ptlrpc/service.c | 27 ++--- 62 files changed, 295 insertions(+), 769 deletions(-) delete mode 100644 drivers/staging/lustre/lnet/libcfs/linux/linux-curproc.c delete mode 100644 drivers/staging/lustre/lnet/libcfs/linux/linux-mem.c delete mode 100644 drivers/staging/lustre/lnet/libcfs/linux/linux-prim.c -- Signature _______________________________________________ lustre-devel mailing list lustre-devel at lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org From dmitry.eremin at intel.com Tue Feb 20 08:07:36 2018 From: dmitry.eremin at intel.com (Eremin, Dmitry) Date: Tue, 20 Feb 2018 08:07:36 +0000 Subject: [lustre-devel] [PATCH 14/21] staging: lustre: fix assorted checkpatch errors In-Reply-To: <151909341801.25814.16760126077741712102.stgit@noble> References: <151909308778.25814.9912662268627044409.stgit@noble> <151909341801.25814.16760126077741712102.stgit@noble> Message-ID: <9FC73D3DBECE0941BD2ED069D26863425CE8FF1E@irsmsx110.ger.corp.intel.com> Hello Neil, > diff --git a/drivers/staging/lustre/lustre/mdc/mdc_request.c > b/drivers/staging/lustre/lustre/mdc/mdc_request.c > index ab48746ce433..bde27acb0dd3 100644 > --- a/drivers/staging/lustre/lustre/mdc/mdc_request.c > +++ b/drivers/staging/lustre/lustre/mdc/mdc_request.c > @@ -1060,8 +1060,10 @@ static void mdc_adjust_dirpages(struct page > **pages, int cfs_pgs, int lu_pgs) > > while (--lu_pgs > 0) { > ent = lu_dirent_start(dp); > - for (end_dirent = ent; ent; > - end_dirent = ent, ent = lu_dirent_next(ent)); > + while (ent) { > + end_dirent = ent; > + ent = lu_dirent_next(ent); > + } > > /* Advance dp to next lu_dirpage. */ > dp = (struct lu_dirpage *)((char *)dp + LU_PAGE_SIZE); I doubt this is correct replacement. In original code end_dirent is set always in the begin of outer loop (while (--lu_pgs > 0)). But in new code this is missed. Therefore in case of second iteration and (ent = lu_dirent_start(dp)) == NULL the end_dirent will contain a value from previous iteration which is not correct. Dmitry. -------------------------------------------------------------------- Joint Stock Company Intel A/O Registered legal address: Krylatsky Hills Business Park, 17 Krylatskaya Str., Bldg 4, Moscow 121614, Russian Federation This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. From neilb at suse.com Tue Feb 20 08:55:30 2018 From: neilb at suse.com (NeilBrown) Date: Tue, 20 Feb 2018 19:55:30 +1100 Subject: [lustre-devel] [PATCH 14/21] staging: lustre: fix assorted checkpatch errors In-Reply-To: <9FC73D3DBECE0941BD2ED069D26863425CE8FF1E@irsmsx110.ger.corp.intel.com> References: <151909308778.25814.9912662268627044409.stgit@noble> <151909341801.25814.16760126077741712102.stgit@noble> <9FC73D3DBECE0941BD2ED069D26863425CE8FF1E@irsmsx110.ger.corp.intel.com> Message-ID: <87lgfojb0d.fsf@notabene.neil.brown.name> On Tue, Feb 20 2018, Eremin, Dmitry wrote: > Hello Neil, > >> diff --git a/drivers/staging/lustre/lustre/mdc/mdc_request.c >> b/drivers/staging/lustre/lustre/mdc/mdc_request.c >> index ab48746ce433..bde27acb0dd3 100644 >> --- a/drivers/staging/lustre/lustre/mdc/mdc_request.c >> +++ b/drivers/staging/lustre/lustre/mdc/mdc_request.c >> @@ -1060,8 +1060,10 @@ static void mdc_adjust_dirpages(struct page >> **pages, int cfs_pgs, int lu_pgs) >> >> while (--lu_pgs > 0) { >> ent = lu_dirent_start(dp); >> - for (end_dirent = ent; ent; >> - end_dirent = ent, ent = lu_dirent_next(ent)); >> + while (ent) { >> + end_dirent = ent; >> + ent = lu_dirent_next(ent); >> + } >> >> /* Advance dp to next lu_dirpage. */ >> dp = (struct lu_dirpage *)((char *)dp + LU_PAGE_SIZE); > > I doubt this is correct replacement. In original code end_dirent is > set always in the begin of outer loop (while (--lu_pgs > 0)). But in > new code this is missed. Therefore in case of second iteration and > (ent = lu_dirent_start(dp)) == NULL the end_dirent will contain a > value from previous iteration which is not correct. Thanks for the review. Yes, you are correct. I had seen that end_dirent was initialized to NULL, and let myself believe that would make the transformation safe. In fact, that initialization to NULL is pointless as it is never used. Maybe this would be better @@ -1055,13 +1055,14 @@ static void mdc_adjust_dirpages(struct page **pages, int cfs_pgs, int lu_pgs) __u64 hash_end = le64_to_cpu(dp->ldp_hash_end); __u32 flags = le32_to_cpu(dp->ldp_flags); struct lu_dirpage *first = dp; - struct lu_dirent *end_dirent = NULL; - struct lu_dirent *ent; while (--lu_pgs > 0) { - ent = lu_dirent_start(dp); - for (end_dirent = ent; ent; - end_dirent = ent, ent = lu_dirent_next(ent)); + struct lu_dirent *end_dirent = NULL; + struct lu_dirent *ent; + + for (ent = lu_dirent_start(dp); ent; + ent = lu_dirent_next(ent)) + end_dirent = ent; /* Advance dp to next lu_dirpage. */ dp = (struct lu_dirpage *)((char *)dp + LU_PAGE_SIZE); ?? Thanks, NeilBrown -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 832 bytes Desc: not available URL: From dmitry.eremin at intel.com Tue Feb 20 10:50:25 2018 From: dmitry.eremin at intel.com (Eremin, Dmitry) Date: Tue, 20 Feb 2018 10:50:25 +0000 Subject: [lustre-devel] [PATCH 14/21] staging: lustre: fix assorted checkpatch errors In-Reply-To: <87lgfojb0d.fsf@notabene.neil.brown.name> References: <151909308778.25814.9912662268627044409.stgit@noble> <151909341801.25814.16760126077741712102.stgit@noble> <9FC73D3DBECE0941BD2ED069D26863425CE8FF1E@irsmsx110.ger.corp.intel.com> <87lgfojb0d.fsf@notabene.neil.brown.name> Message-ID: <9FC73D3DBECE0941BD2ED069D26863425CE8FF72@irsmsx110.ger.corp.intel.com> > -----Original Message----- > Thanks for the review. Yes, you are correct. > I had seen that end_dirent was initialized to NULL, and let myself > believe that would make the transformation safe. > In fact, that initialization to NULL is pointless as it is never used. > > Maybe this would be better > > @@ -1055,13 +1055,14 @@ static void mdc_adjust_dirpages(struct page > **pages, int cfs_pgs, int lu_pgs) > __u64 hash_end = le64_to_cpu(dp->ldp_hash_end); > __u32 flags = le32_to_cpu(dp->ldp_flags); > struct lu_dirpage *first = dp; > - struct lu_dirent *end_dirent = NULL; > - struct lu_dirent *ent; > > while (--lu_pgs > 0) { > - ent = lu_dirent_start(dp); > - for (end_dirent = ent; ent; > - end_dirent = ent, ent = lu_dirent_next(ent)); > + struct lu_dirent *end_dirent = NULL; > + struct lu_dirent *ent; > + > + for (ent = lu_dirent_start(dp); ent; > + ent = lu_dirent_next(ent)) > + end_dirent = ent; > > /* Advance dp to next lu_dirpage. */ > dp = (struct lu_dirpage *)((char *)dp + LU_PAGE_SIZE); > ?? Thanks, I agree with this version. Dmitry. -------------------------------------------------------------------- Joint Stock Company Intel A/O Registered legal address: Krylatsky Hills Business Park, 17 Krylatskaya Str., Bldg 4, Moscow 121614, Russian Federation This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. From neilb at suse.com Tue Feb 20 20:42:20 2018 From: neilb at suse.com (NeilBrown) Date: Wed, 21 Feb 2018 07:42:20 +1100 Subject: [lustre-devel] [PATCH 14/21 - v2] staging: lustre: fix assorted checkpatch errors In-Reply-To: <151909341801.25814.16760126077741712102.stgit@noble> References: <151909308778.25814.9912662268627044409.stgit@noble> <151909341801.25814.16760126077741712102.stgit@noble> Message-ID: <87d10zjsur.fsf@notabene.neil.brown.name> Possibly the most interesting is the for-loop with no body. Rearranging and initializing end_dirent on each iteration of the outer while, makes the intent clearer. Reviewed-by: "Eremin, Dmitry" Signed-off-by: NeilBrown --- This version has a correct conversion for that for loop with not body, thanks Dmitry! drivers/staging/lustre/lnet/klnds/socklnd/socklnd_lib.c | 2 +- drivers/staging/lustre/lnet/libcfs/linux/linux-cpu.c | 2 +- drivers/staging/lustre/lustre/include/obd_class.h | 8 ++++---- drivers/staging/lustre/lustre/include/obd_support.h | 2 +- drivers/staging/lustre/lustre/llite/dcache.c | 2 +- drivers/staging/lustre/lustre/llite/llite_lib.c | 2 +- drivers/staging/lustre/lustre/lov/lov_request.c | 2 +- drivers/staging/lustre/lustre/mdc/mdc_request.c | 11 ++++++----- drivers/staging/lustre/lustre/ptlrpc/layout.c | 2 +- drivers/staging/lustre/lustre/ptlrpc/recover.c | 2 +- drivers/staging/lustre/lustre/ptlrpc/service.c | 2 +- 11 files changed, 19 insertions(+), 18 deletions(-) diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_lib.c b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_lib.c index cb28dd2baf2f..7941cfa526bc 100644 --- a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_lib.c +++ b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_lib.c @@ -189,7 +189,7 @@ ksocknal_lib_recv(struct ksock_conn *conn) if (!(conn->ksnc_rx_to.type & ITER_BVEC) && conn->ksnc_proto != &ksocknal_protocol_v2x) return rc; - + /* accumulate checksum */ conn->ksnc_msg.ksm_csum = 0; iov_iter_for_each_range(&conn->ksnc_rx_to, rc, lustre_csum, conn); diff --git a/drivers/staging/lustre/lnet/libcfs/linux/linux-cpu.c b/drivers/staging/lustre/lnet/libcfs/linux/linux-cpu.c index c07165e0ad95..388521e4e354 100644 --- a/drivers/staging/lustre/lnet/libcfs/linux/linux-cpu.c +++ b/drivers/staging/lustre/lnet/libcfs/linux/linux-cpu.c @@ -743,7 +743,7 @@ cfs_cpt_table_create(int ncpt) goto failed; } - if (!zalloc_cpumask_var(&mask, GFP_NOFS)){ + if (!zalloc_cpumask_var(&mask, GFP_NOFS)) { CERROR("Failed to allocate scratch cpumask\n"); goto failed; } diff --git a/drivers/staging/lustre/lustre/include/obd_class.h b/drivers/staging/lustre/lustre/include/obd_class.h index 531e8ddfa9e5..f24dd74ffa09 100644 --- a/drivers/staging/lustre/lustre/include/obd_class.h +++ b/drivers/staging/lustre/lustre/include/obd_class.h @@ -294,10 +294,10 @@ struct obdo; void obdo_to_ioobj(const struct obdo *oa, struct obd_ioobj *ioobj); -#define OBT(dev) (dev)->obd_type -#define OBP(dev, op) (dev)->obd_type->typ_dt_ops->op -#define MDP(dev, op) (dev)->obd_type->typ_md_ops->op -#define CTXTP(ctxt, op) (ctxt)->loc_logops->lop_##op +#define OBT(dev) ((dev)->obd_type) +#define OBP(dev, op) ((dev)->obd_type->typ_dt_ops->op) +#define MDP(dev, op) ((dev)->obd_type->typ_md_ops->op) +#define CTXTP(ctxt, op) ((ctxt)->loc_logops->lop_##op) /* * Ensure obd_setup: used for cleanup which must be called diff --git a/drivers/staging/lustre/lustre/include/obd_support.h b/drivers/staging/lustre/lustre/include/obd_support.h index 3f4fe290f6ea..8595091b8b86 100644 --- a/drivers/staging/lustre/lustre/include/obd_support.h +++ b/drivers/staging/lustre/lustre/include/obd_support.h @@ -516,7 +516,7 @@ extern char obd_jobid_var[]; #define POISON_PTR(ptr) ((void)0) #else #define POISON(ptr, c, s) memset(ptr, c, s) -#define POISON_PTR(ptr) (ptr) = (void *)0xdeadbeef +#define POISON_PTR(ptr) ((ptr) = (void *)0xdeadbeef) #endif #ifdef POISON_BULK diff --git a/drivers/staging/lustre/lustre/llite/dcache.c b/drivers/staging/lustre/lustre/llite/dcache.c index dc30b4582234..3e768f997172 100644 --- a/drivers/staging/lustre/lustre/llite/dcache.c +++ b/drivers/staging/lustre/lustre/llite/dcache.c @@ -100,7 +100,7 @@ static int ll_dcompare(const struct dentry *dentry, return 0; /* ensure exclusion against parallel lookup of the same name */ - if (d_in_lookup((struct dentry*)dentry)) + if (d_in_lookup((struct dentry *)dentry)) return 0; if (d_lustre_invalid(dentry)) diff --git a/drivers/staging/lustre/lustre/llite/llite_lib.c b/drivers/staging/lustre/lustre/llite/llite_lib.c index efbd551e7842..844182ad7dd7 100644 --- a/drivers/staging/lustre/lustre/llite/llite_lib.c +++ b/drivers/staging/lustre/lustre/llite/llite_lib.c @@ -2022,7 +2022,7 @@ void ll_umount_begin(struct super_block *sb) */ while (cnt < 10 && !may_umount(sbi->ll_mnt.mnt)) { schedule_timeout_uninterruptible(HZ); - cnt ++; + cnt++; } schedule(); diff --git a/drivers/staging/lustre/lustre/lov/lov_request.c b/drivers/staging/lustre/lustre/lov/lov_request.c index c1e58fcc30b3..051450d67524 100644 --- a/drivers/staging/lustre/lustre/lov/lov_request.c +++ b/drivers/staging/lustre/lustre/lov/lov_request.c @@ -126,7 +126,7 @@ static int lov_check_and_wait_active(struct lov_obd *lov, int ost_idx) while (cnt < obd_timeout && !lov_check_set(lov, ost_idx)) { schedule_timeout_uninterruptible(HZ); - cnt ++; + cnt++; } if (tgt->ltd_active) return 1; diff --git a/drivers/staging/lustre/lustre/mdc/mdc_request.c b/drivers/staging/lustre/lustre/mdc/mdc_request.c index ab48746ce433..3b1c8e5a3053 100644 --- a/drivers/staging/lustre/lustre/mdc/mdc_request.c +++ b/drivers/staging/lustre/lustre/mdc/mdc_request.c @@ -1055,13 +1055,14 @@ static void mdc_adjust_dirpages(struct page **pages, int cfs_pgs, int lu_pgs) __u64 hash_end = le64_to_cpu(dp->ldp_hash_end); __u32 flags = le32_to_cpu(dp->ldp_flags); struct lu_dirpage *first = dp; - struct lu_dirent *end_dirent = NULL; - struct lu_dirent *ent; while (--lu_pgs > 0) { - ent = lu_dirent_start(dp); - for (end_dirent = ent; ent; - end_dirent = ent, ent = lu_dirent_next(ent)); + struct lu_dirent *end_dirent = NULL; + struct lu_dirent *ent; + + for (ent = lu_dirent_start(dp); ent; + ent = lu_dirent_next(ent)) + end_dirent = ent; /* Advance dp to next lu_dirpage. */ dp = (struct lu_dirpage *)((char *)dp + LU_PAGE_SIZE); diff --git a/drivers/staging/lustre/lustre/ptlrpc/layout.c b/drivers/staging/lustre/lustre/ptlrpc/layout.c index 18769d335751..2855f38c8190 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/layout.c +++ b/drivers/staging/lustre/lustre/ptlrpc/layout.c @@ -1555,7 +1555,7 @@ struct req_format RQF_OST_GET_INFO_FIEMAP = EXPORT_SYMBOL(RQF_OST_GET_INFO_FIEMAP); /* Convenience macro */ -#define FMT_FIELD(fmt, i, j) (fmt)->rf_fields[(i)].d[(j)] +#define FMT_FIELD(fmt, i, j) ((fmt)->rf_fields[(i)].d[(j)]) /** * Initializes the capsule abstraction by computing and setting the \a rf_idx diff --git a/drivers/staging/lustre/lustre/ptlrpc/recover.c b/drivers/staging/lustre/lustre/ptlrpc/recover.c index 7b5f2429d144..5bb9f9fe91d8 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/recover.c +++ b/drivers/staging/lustre/lustre/ptlrpc/recover.c @@ -354,7 +354,7 @@ int ptlrpc_recover_import(struct obd_import *imp, char *new_uuid, int async) obd_timeout * HZ); CDEBUG(D_HA, "%s: recovery finished\n", obd2cli_tgt(imp->imp_obd)); - rc = rc? 0 : -ETIMEDOUT; + rc = rc ? 0 : -ETIMEDOUT; } out: diff --git a/drivers/staging/lustre/lustre/ptlrpc/service.c b/drivers/staging/lustre/lustre/ptlrpc/service.c index 79d9f3860022..99aeb291f3f2 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/service.c +++ b/drivers/staging/lustre/lustre/ptlrpc/service.c @@ -2670,7 +2670,7 @@ ptlrpc_service_unlink_rqbd(struct ptlrpc_service *svc) (rc = wait_event_idle_timeout(svcpt->scp_waitq, svcpt->scp_nrqbds_posted == 0, HZ)) == 0) - cnt ++; + cnt++; if (rc == 0) { CWARN("Service %s waiting for request buffers\n", svcpt->scp_service->srv_name); -- 2.14.0.rc0.dirty -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 832 bytes Desc: not available URL: From neilb at suse.com Thu Feb 22 03:06:54 2018 From: neilb at suse.com (NeilBrown) Date: Thu, 22 Feb 2018 14:06:54 +1100 Subject: [lustre-devel] [PATCH 24/41] staging: lustre: clio: add CIT_DATA_VERSION and remove IOC_LOV_GETINFO In-Reply-To: <1475461717-21631-25-git-send-email-jsimmons@infradead.org> References: <1475461717-21631-1-git-send-email-jsimmons@infradead.org> <1475461717-21631-25-git-send-email-jsimmons@infradead.org> Message-ID: <871shdk9ip.fsf@notabene.neil.brown.name> Another ancient patch.... On Sun, Oct 02 2016, James Simmons wrote: > From: John L. Hammond > > During development a new api, cl_object_obd_info_get() > and cl_object_data_version() which then were later > replaced by a better solution CIT_DATA_VERSION. For > the case of the upstream client their is no point in > introducing a API to only have it removed later. Due > to the way the patches landed with their dependencies > it is not possible to separate out two patches. These > two combined patches do the following: > > * Add a new cl_io type CIT_DATA_VERSION to get file > data version. > * Remove the unused IOC_LOV_GETINFO ioctl. > * Remove ll_glimpse_ioctl() and ll_lsm_getattr(). > * Remove the OBD API method obd_getattr_async(). > > Signed-off-by: John L. Hammond > Signed-off-by: Bobi Jam > Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-5823 > Reviewed-on: http://review.whamcloud.com/12748 > Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-6356 > Reviewed-on: http://review.whamcloud.com/14649 > Reviewed-by: Henri Doreau > Reviewed-by: Jinshan Xiong > Reviewed-by: Oleg Drokin > Signed-off-by: James Simmons With so many Reviewed-by :-) > > +static void > +lov_io_data_version_end(const struct lu_env *env, const struct cl_io_slice *ios) > +{ > + struct lov_io *lio = cl2lov_io(env, ios); > + struct cl_io *parent = lio->lis_cl.cis_io; > + struct lov_io_sub *sub; > + > + list_for_each_entry(sub, &lio->lis_active, sub_linkage) { > + lov_io_end_wrapper(env, sub->sub_io); This sort of construct occurs several other times in the same file: lov_io_read_ahead: rc = cl_io_read_ahead(sub->sub_env, sub->sub_io, lov_io_submit: rc = cl_io_submit_rw(sub->sub_env, sub->sub_io, crt, queue); lio_io_commit_async: rc = cl_io_commit_async(sub->sub_env, sub->sub_io, queue, from, to, cb); lov_io_fsync_end: struct cl_io *subio = sub->sub_io; lov_io_end_wrapper(sub->sub_env, subio); Every other time, sub->sub_env is used with sub->sub_io. In this new code, 'env' is (incorrectly) used with sub->sub_io. This reliably causes my testing to crash as the LNVRNT() in cl2osc_io() fails, and I test with CONFIG_LUSTRE_DEBUG_EXPENSIVE_CHECK=y Does anyone have any idea why no other testing trips over this? lustre-release has the same bug in Commit: fcd45488711a ("LU-5683 clio: add CIT_DATA_VERSION") I'll send a patch for Linux. Someone else might like to fix lustre-release. Thanks, NeilBrown -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 832 bytes Desc: not available URL: From dan.carpenter at oracle.com Thu Feb 22 09:15:34 2018 From: dan.carpenter at oracle.com (Dan Carpenter) Date: Thu, 22 Feb 2018 12:15:34 +0300 Subject: [lustre-devel] [PATCH] staging: lustre: selftest: freeing an error pointer Message-ID: <20180222091534.GD9883@mwanda> We should just return directly if memdup_user() fails. The current code tries to free "param" which is an error pointer so it will Oops. Fixes: 2baddf262e98 ("staging: lustre: use memdup_user to allocate memory and copy from user") Signed-off-by: Dan Carpenter diff --git a/drivers/staging/lustre/lnet/selftest/conctl.c b/drivers/staging/lustre/lnet/selftest/conctl.c index 3c919a536e91..51497cf9a832 100644 --- a/drivers/staging/lustre/lnet/selftest/conctl.c +++ b/drivers/staging/lustre/lnet/selftest/conctl.c @@ -650,10 +650,8 @@ static int lst_test_add_ioctl(struct lstio_test_args *args) if (args->lstio_tes_param) { param = memdup_user(args->lstio_tes_param, args->lstio_tes_param_len); - if (IS_ERR(param)) { - rc = PTR_ERR(param); - goto out; - } + if (IS_ERR(param)) + return PTR_ERR(param); } rc = -EFAULT; From neilb at suse.com Thu Feb 22 22:09:33 2018 From: neilb at suse.com (NeilBrown) Date: Fri, 23 Feb 2018 09:09:33 +1100 Subject: [lustre-devel] [PATCH 0/3] Three lustre bugfixes Message-ID: <151933726925.16436.9827896439357574011.stgit@noble> First two patches fix bugs that have been prevented the test suite from finished for me - now it completes with about 10% of tests failing. Third patch fixes a tiny bug I noticed while reviewing another recent patch to the same file. Thanks, NeilBrown --- NeilBrown (3): staging: lustre: lov: use correct env in lov_io_data_version_end() staging: lustre: lmv: correctly iput lmo_root staging: lustre: lnet/selftest: don't ignore status from lstcon_test_add drivers/staging/lustre/lnet/selftest/conctl.c | 2 +- drivers/staging/lustre/lustre/lmv/lmv_obd.c | 2 +- drivers/staging/lustre/lustre/lov/lov_io.c | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) -- Signature From neilb at suse.com Thu Feb 22 22:09:33 2018 From: neilb at suse.com (NeilBrown) Date: Fri, 23 Feb 2018 09:09:33 +1100 Subject: [lustre-devel] [PATCH 1/3] staging: lustre: lov: use correct env in lov_io_data_version_end() In-Reply-To: <151933726925.16436.9827896439357574011.stgit@noble> References: <151933726925.16436.9827896439357574011.stgit@noble> Message-ID: <151933737380.16436.3113702074708223689.stgit@noble> lov - the logical object volume manager - is responsible for striping data across multiple volumes. So when it is given a request, it creates one or more sub-requests, one for each target volume. Each sub_io request has a sub_env environment which it operates in. When lov_io_data_version_end() calls lov_io_end_wrapper() to wait for and close off a sub_io, it passes the wrong environment. This causes an LINVRNT() to fail in cl2osc_io(), and may cause other problems. This patch changes the call to use ->sub_env, much like other code in the same file. Fixes: f0cf21abcccc ("staging: lustre: clio: add CIT_DATA_VERSION and remove IOC_LOV_GETINFO") Signed-off-by: NeilBrown --- drivers/staging/lustre/lustre/lov/lov_io.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/staging/lustre/lustre/lov/lov_io.c b/drivers/staging/lustre/lustre/lov/lov_io.c index c0dbf6cd53b4..b823f8a21856 100644 --- a/drivers/staging/lustre/lustre/lov/lov_io.c +++ b/drivers/staging/lustre/lustre/lov/lov_io.c @@ -483,7 +483,7 @@ lov_io_data_version_end(const struct lu_env *env, const struct cl_io_slice *ios) struct lov_io_sub *sub; list_for_each_entry(sub, &lio->lis_active, sub_linkage) { - lov_io_end_wrapper(env, sub->sub_io); + lov_io_end_wrapper(sub->sub_env, sub->sub_io); parent->u.ci_data_version.dv_data_version += sub->sub_io->u.ci_data_version.dv_data_version; From neilb at suse.com Thu Feb 22 22:09:33 2018 From: neilb at suse.com (NeilBrown) Date: Fri, 23 Feb 2018 09:09:33 +1100 Subject: [lustre-devel] [PATCH 2/3] staging: lustre: lmv: correctly iput lmo_root In-Reply-To: <151933726925.16436.9827896439357574011.stgit@noble> References: <151933726925.16436.9827896439357574011.stgit@noble> Message-ID: <151933737384.16436.1999360148075400466.stgit@noble> Commit 8f18c8a48b73 ("staging: lustre: lmv: separate master object with master stripe") changed how lmo_root inodes were managed, particularly when LMV_HASH_FLAG_MIGRATION is not set. Previously lsm_md_oinfo[0].lmo_root was always a borrowed inode reference and didn't need to by iput(). Since the change, that special case only applies when LMV_HASH_FLAG_MIGRATION is set In the upstream (lustre-release) version of this patch [Commit 60e07b972114 ("LU-4690 lod: separate master object with master stripe")] the for loop in the lmv_unpack_md() was changed to count from 0 and to ignore entry 0 if LMV_HASH_FLAG_MIGRATION is set. In the patch that got applied to Linux, that change was missing, so lsm_md_oinfo[0].lmo_root is never iput(). This results in a "VFS: Busy inodes" warning at unmount. Fixes: 8f18c8a48b73 ("staging: lustre: lmv: separate master object with master stripe") Signed-off-by: NeilBrown --- drivers/staging/lustre/lustre/lmv/lmv_obd.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/staging/lustre/lustre/lmv/lmv_obd.c b/drivers/staging/lustre/lustre/lmv/lmv_obd.c index 179651531862..e8a9b9902c37 100644 --- a/drivers/staging/lustre/lustre/lmv/lmv_obd.c +++ b/drivers/staging/lustre/lustre/lmv/lmv_obd.c @@ -2695,7 +2695,7 @@ static int lmv_unpackmd(struct obd_export *exp, struct lmv_stripe_md **lsmp, if (lsm && !lmm) { int i; - for (i = 1; i < lsm->lsm_md_stripe_count; i++) { + for (i = 0; i < lsm->lsm_md_stripe_count; i++) { /* * For migrating inode, the master stripe and master * object will be the same, so do not need iput, see From neilb at suse.com Thu Feb 22 22:09:33 2018 From: neilb at suse.com (NeilBrown) Date: Fri, 23 Feb 2018 09:09:33 +1100 Subject: [lustre-devel] [PATCH 3/3] staging: lustre: lnet/selftest: don't ignore status from lstcon_test_add In-Reply-To: <151933726925.16436.9827896439357574011.stgit@noble> References: <151933726925.16436.9827896439357574011.stgit@noble> Message-ID: <151933737388.16436.11922095105762928334.stgit@noble> If lstcon_test_add sets 'ret' (passed by reference) to 1, then lst_test_add_ioctl() ignores the return value. This isn't justified - the return value must be zero for 'ret' to be meaningful. Signed-off-by: NeilBrown --- drivers/staging/lustre/lnet/selftest/conctl.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/staging/lustre/lnet/selftest/conctl.c b/drivers/staging/lustre/lnet/selftest/conctl.c index 51497cf9a832..a2d8092bdeb7 100644 --- a/drivers/staging/lustre/lnet/selftest/conctl.c +++ b/drivers/staging/lustre/lnet/selftest/conctl.c @@ -670,7 +670,7 @@ static int lst_test_add_ioctl(struct lstio_test_args *args) args->lstio_tes_param_len, &ret, args->lstio_tes_resultp); - if (ret) + if (!rc && ret) rc = (copy_to_user(args->lstio_tes_retp, &ret, sizeof(ret))) ? -EFAULT : 0; out: From jsimmons at infradead.org Mon Feb 26 14:47:56 2018 From: jsimmons at infradead.org (James Simmons) Date: Mon, 26 Feb 2018 14:47:56 +0000 (GMT) Subject: [lustre-devel] [PATCH 1/3] staging: lustre: lov: use correct env in lov_io_data_version_end() In-Reply-To: <151933737380.16436.3113702074708223689.stgit@noble> References: <151933726925.16436.9827896439357574011.stgit@noble> <151933737380.16436.3113702074708223689.stgit@noble> Message-ID: > lov - the logical object volume manager - is responsible for > striping data across multiple volumes. > > So when it is given a request, it creates one or more > sub-requests, one for each target volume. Each sub_io > request has a sub_env environment which it operates in. > > When lov_io_data_version_end() calls lov_io_end_wrapper() to > wait for and close off a sub_io, it passes the wrong > environment. > > This causes an LINVRNT() to fail in cl2osc_io(), and may > cause other problems. > > This patch changes the call to use ->sub_env, much like > other code in the same file. > > Fixes: f0cf21abcccc ("staging: lustre: clio: add CIT_DATA_VERSION and remove IOC_LOV_GETINFO") > Signed-off-by: NeilBrown Reviewed-by: James Simmons > --- > drivers/staging/lustre/lustre/lov/lov_io.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/drivers/staging/lustre/lustre/lov/lov_io.c b/drivers/staging/lustre/lustre/lov/lov_io.c > index c0dbf6cd53b4..b823f8a21856 100644 > --- a/drivers/staging/lustre/lustre/lov/lov_io.c > +++ b/drivers/staging/lustre/lustre/lov/lov_io.c > @@ -483,7 +483,7 @@ lov_io_data_version_end(const struct lu_env *env, const struct cl_io_slice *ios) > struct lov_io_sub *sub; > > list_for_each_entry(sub, &lio->lis_active, sub_linkage) { > - lov_io_end_wrapper(env, sub->sub_io); > + lov_io_end_wrapper(sub->sub_env, sub->sub_io); > > parent->u.ci_data_version.dv_data_version += > sub->sub_io->u.ci_data_version.dv_data_version; > > > From jsimmons at infradead.org Mon Feb 26 14:48:25 2018 From: jsimmons at infradead.org (James Simmons) Date: Mon, 26 Feb 2018 14:48:25 +0000 (GMT) Subject: [lustre-devel] [PATCH 2/3] staging: lustre: lmv: correctly iput lmo_root In-Reply-To: <151933737384.16436.1999360148075400466.stgit@noble> References: <151933726925.16436.9827896439357574011.stgit@noble> <151933737384.16436.1999360148075400466.stgit@noble> Message-ID: > Commit 8f18c8a48b73 ("staging: lustre: lmv: separate master object > with master stripe") changed how lmo_root inodes were managed, > particularly when LMV_HASH_FLAG_MIGRATION is not set. > Previously lsm_md_oinfo[0].lmo_root was always a borrowed > inode reference and didn't need to by iput(). > Since the change, that special case only applies when > LMV_HASH_FLAG_MIGRATION is set > > In the upstream (lustre-release) version of this patch [Commit > 60e07b972114 ("LU-4690 lod: separate master object with master > stripe")] the for loop in the lmv_unpack_md() was changed to count > from 0 and to ignore entry 0 if LMV_HASH_FLAG_MIGRATION is set. > In the patch that got applied to Linux, that change was missing, > so lsm_md_oinfo[0].lmo_root is never iput(). > This results in a "VFS: Busy inodes" warning at unmount. > > Fixes: 8f18c8a48b73 ("staging: lustre: lmv: separate master object with master stripe") > Signed-off-by: NeilBrown Reviewed-by: James Simmons > --- > drivers/staging/lustre/lustre/lmv/lmv_obd.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/drivers/staging/lustre/lustre/lmv/lmv_obd.c b/drivers/staging/lustre/lustre/lmv/lmv_obd.c > index 179651531862..e8a9b9902c37 100644 > --- a/drivers/staging/lustre/lustre/lmv/lmv_obd.c > +++ b/drivers/staging/lustre/lustre/lmv/lmv_obd.c > @@ -2695,7 +2695,7 @@ static int lmv_unpackmd(struct obd_export *exp, struct lmv_stripe_md **lsmp, > if (lsm && !lmm) { > int i; > > - for (i = 1; i < lsm->lsm_md_stripe_count; i++) { > + for (i = 0; i < lsm->lsm_md_stripe_count; i++) { > /* > * For migrating inode, the master stripe and master > * object will be the same, so do not need iput, see > > > From jsimmons at infradead.org Mon Feb 26 14:48:48 2018 From: jsimmons at infradead.org (James Simmons) Date: Mon, 26 Feb 2018 14:48:48 +0000 (GMT) Subject: [lustre-devel] [PATCH 3/3] staging: lustre: lnet/selftest: don't ignore status from lstcon_test_add In-Reply-To: <151933737388.16436.11922095105762928334.stgit@noble> References: <151933726925.16436.9827896439357574011.stgit@noble> <151933737388.16436.11922095105762928334.stgit@noble> Message-ID: > If lstcon_test_add sets 'ret' (passed by reference) to 1, > then lst_test_add_ioctl() ignores the return value. > This isn't justified - the return value must be zero for 'ret' > to be meaningful. > > Signed-off-by: NeilBrown Reviewed-by: James Simmons > --- > drivers/staging/lustre/lnet/selftest/conctl.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/drivers/staging/lustre/lnet/selftest/conctl.c b/drivers/staging/lustre/lnet/selftest/conctl.c > index 51497cf9a832..a2d8092bdeb7 100644 > --- a/drivers/staging/lustre/lnet/selftest/conctl.c > +++ b/drivers/staging/lustre/lnet/selftest/conctl.c > @@ -670,7 +670,7 @@ static int lst_test_add_ioctl(struct lstio_test_args *args) > args->lstio_tes_param_len, > &ret, args->lstio_tes_resultp); > > - if (ret) > + if (!rc && ret) > rc = (copy_to_user(args->lstio_tes_retp, &ret, > sizeof(ret))) ? -EFAULT : 0; > out: > > > From darrick.wong at oracle.com Mon Feb 26 18:08:49 2018 From: darrick.wong at oracle.com (Darrick J. Wong) Date: Mon, 26 Feb 2018 10:08:49 -0800 Subject: [lustre-devel] [e2fsprogs PATCH] tune2fs: don't recover journal if device is busy. In-Reply-To: <20180224222352.GC14111@thunk.org> References: <871shrrok4.fsf@notabene.neil.brown.name> <20180212021609.GA5204@magnolia> <20180224222352.GC14111@thunk.org> Message-ID: <20180226180849.GA19295@magnolia> On Sat, Feb 24, 2018 at 05:23:52PM -0500, Theodore Ts'o wrote: > On Sun, Feb 11, 2018 at 06:16:09PM -0800, Darrick J. Wong wrote: > > > Note: it seems wrong to recover the journal *after* making > > > changes to the superblock - there is a good chance that > > > recovering the journal will over-write those changes. > > > This is what was happening that lead me to this problem. > > > Shouldn't journal recovery happen *first*?? > > > > Yes. Oops. :/ > > > > This whole hunk ought to move up to be right after > > ext2fs_check_if_mounted, I think. > > After I did that, the test t_replay_and_set started failing. The > problem is that the test deliberately corrupts all of the inode and > block bitmaps by writing bogus journal entries. When tune2fs replays > the journal, it ends up smashing the inode and block bitmaps; but then > when it tries to rewrite checksums, the fact that inode bitmap is > completely zeroed out means all of the inode entries are also cleared > out. Oops! > > This normally isn't supposed to happen because check_fsck_needed() is > not supposed to allow us to do dangerous things that require rewriting > checksums unless the file system is freshly checked. > > But the way the test was constructed the superblock's last mount time > is still 0, since the file system was never mounted. By definition, > though, if there is a journal to be replayed, the file system has been > mounted, and hence must be checked. Once fixed I this to force > s_lastcheck to be always less than s_mtimea after replaying the > journal, then the t_replay_and_set test would fail because it's not > safe to run "tune2fs -O ^metadata_csum" without running e2fsck first. > > So I'll change the test to set the file system label instead doing > something more dangerous like clear the metadata checksum feature. > Was there someone who really wanted to be able to execute "tune2fs -O > ^metadata_csum" on a file system with a dirty journal w/o running > e2fsck first? No, I didn't have a specific user in mind. Frankly I doubt there will be very many people who want to turn *off* that feature, but for those who do so to a dirty fs we could replay the journal first. (Though let's be honest, if you're going to tune2fs you really ought to e2fsck before just to make sure the fs is ok...) --D > > - Ted