From bevans at cray.com Thu Jul 2 15:18:48 2015 From: bevans at cray.com (Ben Evans) Date: Thu, 2 Jul 2015 10:18:48 -0500 Subject: [lustre-devel] [PATCH] LU-3677 mdt: Set HSM dirty open-for-write file when evicted. Message-ID: <20150702151848.GA26896@lusbld01.us.cray.com> Fix regression introduced by LU-1303 (5165cdd). Previously, MDS_CLOSE_CLEANUP was used to detect file closing, due to eviction. This flag is no more since 5165cdd. It completely removed this symbol. Signed-off-by: Aurelien Degremont Change-Id: I20e18103fe085672f499c956e831564e45bd5200 Reviewed-on: http://review.whamcloud.com/7195 Tested-by: Hudson Reviewed-by: Andreas Dilger Reviewed-by: Jinshan Xiong Tested-by: Maloo Reviewed-by: John L. Hammond Reviewed-by: Fan Yong Reviewed-by: Oleg Drokin --- .../lustre/lustre/include/lustre/lustre_idl.h | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h b/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h index 4d72d6e..c039cbc 100644 --- a/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h +++ b/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h @@ -2390,7 +2390,7 @@ enum mds_op_bias { MDS_PERM_BYPASS = 1 << 3, MDS_SOM = 1 << 4, MDS_QUOTA_IGNORE = 1 << 5, - MDS_CLOSE_CLEANUP = 1 << 6, + /* Was MDS_CLOSE_CLEANUP = (1 << 6), No more used */ MDS_KEEP_ORPHAN = 1 << 7, MDS_RECOV_OPEN = 1 << 8, MDS_DATA_MODIFIED = 1 << 9, -- 1.6.5.6 From bevans at cray.com Thu Jul 2 15:20:00 2015 From: bevans at cray.com (Ben Evans) Date: Thu, 2 Jul 2015 15:20:00 +0000 Subject: [lustre-devel] First attempt at a patch for lustre-staging Message-ID: <6E6D936CCB2E3C428716BD24C5C6CDF264486B@CFWEX01.americas.cray.com> Any feedback would be greatly welcomed. Thanks -Ben -------------- next part -------------- An HTML attachment was scrubbed... URL: From richard.henwood at intel.com Thu Jul 2 19:25:46 2015 From: richard.henwood at intel.com (Henwood, Richard) Date: Thu, 2 Jul 2015 19:25:46 +0000 Subject: [lustre-devel] DNE2 feature patch landing Message-ID: <1435865147.2548.26.camel@intel.com> All, Last week a patch landed conflicting with the DNE2 project that was preparing to land for 2.8: http://wiki.lustre.org/Projects The DNE2 feature patches were rapidly re-based, re-reviewed and prepared again for landing. However, a unrelated test failure from the continuous integration environment has delayed the landing of these three patches beyond the feature freeze date of June 30th. We anticipate that it would only take an extra day or two to get the outstanding DNE2 patches landed and recommend that we extend the feature freeze for these to allow this OpenSFS-funded project to complete. We do not expect this to have a material affect on the 2.8 release date. I'm asking the list now for a couple of supporting community voices to endorse this request. best regards, Richard -- Richard.Henwood at intel.com Tel: +1 512 410 9612 Intel High Performance Data Division From paf at cray.com Thu Jul 2 19:36:40 2015 From: paf at cray.com (Patrick Farrell) Date: Thu, 2 Jul 2015 14:36:40 -0500 Subject: [lustre-devel] DNE2 feature patch landing In-Reply-To: <1435865147.2548.26.camel@intel.com> References: <1435865147.2548.26.camel@intel.com> Message-ID: <559592C8.2090905@cray.com> Speaking for myself (IE, not on behalf of my employer), strongly endorsed. While the feature freeze is important, there's a good reason for the slight slippage here, and this is an important feature. I have no issue with slight slippage as long as a serious, good faith effort was made with a realistic plan to hit the cutoff, as I think is the case here. - Patrick On 07/02/2015 02:25 PM, Henwood, Richard wrote: > All, > > Last week a patch landed conflicting with the DNE2 project that was > preparing to land for 2.8: > http://wiki.lustre.org/Projects > > The DNE2 feature patches were rapidly re-based, re-reviewed and prepared > again for landing. However, a unrelated test failure from the continuous > integration environment has delayed the landing of these three patches > beyond the feature freeze date of June 30th. > > We anticipate that it would only take an extra day or two to get the > outstanding DNE2 patches landed and recommend that we extend the feature > freeze for these to allow this OpenSFS-funded project to complete. We > do not expect this to have a material affect on the 2.8 release date. > > I'm asking the list now for a couple of supporting community voices to > endorse this request. > > best regards, > Richard From simmonsja at ornl.gov Thu Jul 2 20:46:24 2015 From: simmonsja at ornl.gov (Simmons, James A.) Date: Thu, 2 Jul 2015 20:46:24 +0000 Subject: [lustre-devel] DNE2 feature patch landing In-Reply-To: <1435865147.2548.26.camel@intel.com> References: <1435865147.2548.26.camel@intel.com> Message-ID: >All, > >Last week a patch landed conflicting with the DNE2 project that was >preparing to land for 2.8: >http://wiki.lustre.org/Projects > >The DNE2 feature patches were rapidly re-based, re-reviewed and prepared >again for landing. However, a unrelated test failure from the continuous >integration environment has delayed the landing of these three patches >beyond the feature freeze date of June 30th. > >We anticipate that it would only take an extra day or two to get the >outstanding DNE2 patches landed and recommend that we extend the feature >freeze for these to allow this OpenSFS-funded project to complete. We >do not expect this to have a material affect on the 2.8 release date. > >I'm asking the list now for a couple of supporting community voices to >endorse this request. We had this discussion at the last OpenSFS working group which we agreed that the patch for LU-3534 should land after the feature freeze. Before the merge conflict the patch did pass all maloo test and had the needed reviews. The merge conflict was due to another change that landed that altered one of the test in the test suite. So their is no C code change between versions of the LU-3534 patch. The rest of the patches are fixes for bug so they fall into the bug fixing phase. From morrone2 at llnl.gov Thu Jul 2 20:58:11 2015 From: morrone2 at llnl.gov (Christopher J. Morrone) Date: Thu, 02 Jul 2015 13:58:11 -0700 Subject: [lustre-devel] DNE2 feature patch landing In-Reply-To: References: <1435865147.2548.26.camel@intel.com> Message-ID: <5595A5E3.2060201@llnl.gov> On 07/02/2015 01:46 PM, Simmons, James A. wrote: >> All, >> >> Last week a patch landed conflicting with the DNE2 project that was >> preparing to land for 2.8: >> http://wiki.lustre.org/Projects >> >> The DNE2 feature patches were rapidly re-based, re-reviewed and prepared >> again for landing. However, a unrelated test failure from the continuous >> integration environment has delayed the landing of these three patches >> beyond the feature freeze date of June 30th. >> >> We anticipate that it would only take an extra day or two to get the >> outstanding DNE2 patches landed and recommend that we extend the feature >> freeze for these to allow this OpenSFS-funded project to complete. We >> do not expect this to have a material affect on the 2.8 release date. >> >> I'm asking the list now for a couple of supporting community voices to >> endorse this request. > > We had this discussion at the last OpenSFS working group which we agreed > that the patch for LU-3534 should land after the feature freeze. Before the > merge conflict the patch did pass all maloo test and had the needed reviews. > The merge conflict was due to another change that landed that altered one > of the test in the test suite. So their is no C code change between versions of > the LU-3534 patch. The rest of the patches are fixes for bug so they fall into > the bug fixing phase. On the contract call today we encouraged Intel to have this conversation and make this decision more publicly. The LWG calls are only attended by a small fraction of the development community. I think it is important that we start being more open and transparent with the activities and decisions surrounding Lustre's master branch. So thanks, Richard for posting on lustre-devel. Since this feature has been developed under the old style of piecemeal landing rather than the new topic branch development style, I too agree that landing this after the feature freeze date is acceptable. Chris From simmonsja at ornl.gov Thu Jul 2 22:25:41 2015 From: simmonsja at ornl.gov (Simmons, James A.) Date: Thu, 2 Jul 2015 22:25:41 +0000 Subject: [lustre-devel] LIBCFS_ALLOC In-Reply-To: References: <1434819550-3193-1-git-send-email-Julia.Lawall@lip6.fr> <1434819550-3193-2-git-send-email-Julia.Lawall@lip6.fr> <20150628215408.GG28762@mwanda> Message-ID: <9cb85b423527448db721927a35974317@EXCHCS32.ornl.gov> >> >Yeah. You're right. Doing a vmalloc() when kmalloc() doesn't have even >> >a tiny sliver of RAM isn't going to work. It's easier to use >> >libcfs_kvzalloc() everywhere, but it's probably the wrong thing. >> >> The original reason we have the vmalloc water mark wasn't so much the >> issue of memory exhaustion but to handle the case of memory fragmentation. >> Some sites had after a extended period of time started to see failures of >> allocating even 32K using kmalloc. In our latest development branch we moved >> away from using a water mark to always try kmalloc first and if it fails then we >> try vmalloc. At ORNL we ran into severe performance issues when we entered >> vmalloc territory. It has been discussed before on what might replace vmalloc >> handling in the case of kmalloc fails but no solution has been worked out. > >OK, but if a structure contains only 4 words, would it be better to just >use kzalloc? Or does it not matter? It would only save trying vmalloc in >a case that it is guaranteed to fail, but if a structure with 4 words >can't be allocated, the system has other problems. Another argument is >that kzalloc is a well known function that people and bug-finding tools >understand, so it is better to use it whenever possible. > >Some of the other structures contain a lot more fields, as well as small >arrays. They are probably acceptable for kzalloc too, but I wouldn't know >the exact dividing line. The reason I bring this up is to discuss sorting this out. Once long ago we had just LIBCFS_ALLOC. For some reason before my time OBD_ALLOC got spawned off of that. Currently LIBCFS_ALLOC is used just by the libcfs/LNet layer. Now OBD_ALLOC in our development branch has moved to a try kmalloc first and if it fails try vmalloc for any size memory allocation. LIBCFS_ALLOC still does the original approach. So we have two possible solutions depending on if libcfs/LNet needs to ever do a vmalloc. One solution if libcfs/LNet never needs a vmalloc is remove LIBCFS_ALLOC and replace it with kzalloc everywhere. We can then move libcfs_kzvalloc to the lustre layer and port the change we did in the development branch to here of the try kmalloc then vmalloc approach. The other approach is if libcfs/LNet does in some case need to use vmalloc we could then update LIBCFS_ALLOC to first try kmalloc then vmalloc. Once this is implemented we can nuke the OBD_ALLOC system. Either way I like to see it consolidated down to one system. From andreas.dilger at intel.com Fri Jul 3 10:59:54 2015 From: andreas.dilger at intel.com (Dilger, Andreas) Date: Fri, 3 Jul 2015 10:59:54 +0000 Subject: [lustre-devel] [PATCH] LU-3677 mdt: Set HSM dirty open-for-write file when evicted. In-Reply-To: <20150702151848.GA26896@lusbld01.us.cray.com> References: <20150702151848.GA26896@lusbld01.us.cray.com> Message-ID: Hi Ben, for patches being submitted to the upstream kernel, they also need to be CC'd to the maintainers and lists of the "Linux Staging" project. Since we're pretty new at taking outside patches for upstream, I'll let Oleg comment on whether he prefers to accumulate the patches himself and pass them upstream, or if you should CC the upstream maintainers/lists directly. That said, since you are doing this for the first time it makes sense to just send the patch to lustre-devel and we can iterate until the patch is ready to go upstream. A few comments inline. On 2015/07/02, 9:18 AM, "Ben Evans" wrote: >LU-3677 mdt: Set HSM dirty open-for-write file when evicted. For patches being pushed upstream, the "LU-3677" label should be removed from the subject line to a separate tag at the end after Reviewed-on: Intel-bug-ID: http://jira.hpdd.intel.com/browse/LU-3677 >Fix regression introduced by LU-1303 (5165cdd). Previously, >MDS_CLOSE_CLEANUP was used to detect file closing, due to eviction. >This flag is no more since 5165cdd. The patch commit hash referenced here (5165cdd) is only relevant for the Lustre "master" branch, but do not make any sense for the kernel tree itself (where this patch is going), and the upstream maintainers don't like that. Since the referenced patch was landed before the Lustre client code was pushed upstream, it probably doesn't make sense to reference this at all, and just update the commit comment like: Previously, MDS_CLOSE_CLEANUP was used to detect file closing due to eviction. This flag is not used anymore. Completely remove this symbol. >Change-Id: I20e18103fe085672f499c956e831564e45bd5200 Upstream doesn't like the Change-Id, since it is confusing for other Gerrit instances if the patches are being imported. This should be flagged if checkpatch.pl is run against the patch, which should be done for every patch going upstream, even if it was landed to master previously. >Reviewed-on: http://review.whamcloud.com/7195 Reviewed-on: is OK. >Tested-by: Hudson >Tested-by: Maloo Remove Tested-by: since those don't mean anything to upstream. >Signed-off-by: Aurelien Degremont >Reviewed-by: Andreas Dilger >Reviewed-by: Jinshan Xiong >Reviewed-by: John L. Hammond >Reviewed-by: Fan Yong >Reviewed-by: Oleg Drokin The Signed-off-by: and Reviewed-by: lines should stay. As you can see, the commit message and possibly the patch itself may be slightly different between master and upstream, but at least with the Reviewed-on: and Intel-bug-id: tags we can track the patches between the two. There are similar labels needed when porting patches from upstream kernels into master. Both of these cases are already described on the wiki page: https://wiki.hpdd.intel.com/display/PUB/Commit+Comments under the "Additional commit tags" section. Cheers, Andreas >--- > .../lustre/lustre/include/lustre/lustre_idl.h | 2 +- > 1 files changed, 1 insertions(+), 1 deletions(-) > >diff --git a/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h >b/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h >index 4d72d6e..c039cbc 100644 >--- a/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h >+++ b/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h >@@ -2390,7 +2390,7 @@ enum mds_op_bias { > MDS_PERM_BYPASS = 1 << 3, > MDS_SOM = 1 << 4, > MDS_QUOTA_IGNORE = 1 << 5, >- MDS_CLOSE_CLEANUP = 1 << 6, >+ /* Was MDS_CLOSE_CLEANUP = (1 << 6), No more used */ > MDS_KEEP_ORPHAN = 1 << 7, > MDS_RECOV_OPEN = 1 << 8, > MDS_DATA_MODIFIED = 1 << 9, >-- >1.6.5.6 > >_______________________________________________ >lustre-devel mailing list >lustre-devel at lists.lustre.org >http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org > Cheers, Andreas -- Andreas Dilger Lustre Software Architect Intel High Performance Data Division From andreas.dilger at intel.com Fri Jul 3 11:52:06 2015 From: andreas.dilger at intel.com (Dilger, Andreas) Date: Fri, 3 Jul 2015 11:52:06 +0000 Subject: [lustre-devel] LIBCFS_ALLOC In-Reply-To: <9cb85b423527448db721927a35974317@EXCHCS32.ornl.gov> References: <1434819550-3193-1-git-send-email-Julia.Lawall@lip6.fr> <1434819550-3193-2-git-send-email-Julia.Lawall@lip6.fr> <20150628215408.GG28762@mwanda> <9cb85b423527448db721927a35974317@EXCHCS32.ornl.gov> Message-ID: On 2015/07/02, 4:25 PM, "Simmons, James A." wrote: > >>> >Yeah. You're right. Doing a vmalloc() when kmalloc() doesn't have >>>even >>> >a tiny sliver of RAM isn't going to work. It's easier to use >>> >libcfs_kvzalloc() everywhere, but it's probably the wrong thing. >>> >>> The original reason we have the vmalloc water mark wasn't so much the >>> issue of memory exhaustion but to handle the case of memory >>>fragmentation. >>> Some sites had after a extended period of time started to see failures >>>of >>> allocating even 32K using kmalloc. In our latest development branch >>>we moved >>> away from using a water mark to always try kmalloc first and if it >>>fails then we >>> try vmalloc. At ORNL we ran into severe performance issues when we >>>entered >>> vmalloc territory. It has been discussed before on what might replace >>>vmalloc >>> handling in the case of kmalloc fails but no solution has been worked >>>out. >> >>OK, but if a structure contains only 4 words, would it be better to just >>use kzalloc? Or does it not matter? It would only save trying vmalloc >>in >>a case that it is guaranteed to fail, but if a structure with 4 words >>can't be allocated, the system has other problems. Another argument is >>that kzalloc is a well known function that people and bug-finding tools >>understand, so it is better to use it whenever possible. >> >>Some of the other structures contain a lot more fields, as well as small >>arrays. They are probably acceptable for kzalloc too, but I wouldn't >>know >>the exact dividing line. > >The reason I bring this up is to discuss sorting this out. Once long ago >we had just LIBCFS_ALLOC. For some reason before my time OBD_ALLOC got > spawned off of that. Currently LIBCFS_ALLOC is used just by the >libcfs/LNet > layer. That is because there was (is?) interest from Cray and others to use LNet independently from Lustre (Zest and DVS, for example) so LNet should be self-contained and not depend on anything from Lustre. > Now OBD_ALLOC in our development branch has moved to a try kmalloc first >and >if it fails try vmalloc for any size memory allocation. LIBCFS_ALLOC >still > does the original approach. So we have two possible solutions >depending on if libcfs/LNet needs to ever do a vmalloc. > >One solution if libcfs/LNet never needs a vmalloc is remove LIBCFS_ALLOC >and replace it with kzalloc everywhere. We can then move libcfs_kzvalloc >to > the lustre layer and port the change we did in the development branch to > here of the try kmalloc then vmalloc approach. > >The other approach is if libcfs/LNet does in some case need to use vmalloc > we could then update LIBCFS_ALLOC to first try kmalloc then vmalloc. Once > this is implemented we can nuke the OBD_ALLOC system. I don't agree. I think there are a few places where vmalloc() makes sense to try (if the allocation may be large), but in most places LIBCFS_ALLOC() should only use kmalloc(). Unfortunately, there wasn't a separate LIBCFS_ALLOC_LARGE() like there was for OBD_ALLOC_LARGE() that made it clear which callsites are (potentially) large and which are small. The macro approach allowed the compile-time optimization of the small callsites, but that needs to be done by hand now. >Either way I like to see it consolidated down to one system. Given the proliferation of foo_kvmalloc() and foo_kvzalloc() helpers (ext4_, kvm_, dm_, apparmor, ceph_, __aa_), maybe it it is time to move these to common kernel code instead of introducing yet another new one? Cheers, Andreas -- Andreas Dilger Lustre Software Architect Intel High Performance Data Division From oleg.drokin at intel.com Sat Jul 4 02:00:45 2015 From: oleg.drokin at intel.com (Drokin, Oleg) Date: Sat, 4 Jul 2015 02:00:45 +0000 Subject: [lustre-devel] [PATCH] LU-3677 mdt: Set HSM dirty open-for-write file when evicted. In-Reply-To: References: <20150702151848.GA26896@lusbld01.us.cray.com> Message-ID: <7010D276-D817-4A22-87FE-4997221EF5A1@intel.com> Hello! On Jul 3, 2015, at 6:59 AM, Dilger, Andreas wrote: > Hi Ben, > for patches being submitted to the upstream kernel, they also need to be > CC'd to the maintainers and lists of the "Linux Staging" project. Since > we're pretty new at taking outside patches for upstream, I'll let Oleg > comment on whether he prefers to accumulate the patches himself and pass > them upstream, or if you should CC the upstream maintainers/lists directly. I think it's ok for people to send patches themselves. > On 2015/07/02, 9:18 AM, "Ben Evans" wrote: > >> LU-3677 mdt: Set HSM dirty open-for-write file when evicted. > > For patches being pushed upstream, the "LU-3677" label should be removed > from the subject line to a separate tag at the end after Reviewed-on: > > Intel-bug-ID: http://jira.hpdd.intel.com/browse/LU-3677 Additionally instead of the subsystem (mdt in this instance), I typically expand that some more to something like: staging/lustre/mdt (or jsut staging/lustre if there's no clear subsystem). also note taht mdt is not really a subsystem in upstream kernel so typically patches to it are skipped from being sent upstream or redone to only retain relevant client-side bits. >> Fix regression introduced by LU-1303 (5165cdd). Previously, >> MDS_CLOSE_CLEANUP was used to detect file closing, due to eviction. >> This flag is no more since 5165cdd. > > The patch commit hash referenced here (5165cdd) is only relevant for > the Lustre "master" branch, but do not make any sense for the kernel > tree itself (where this patch is going), and the upstream maintainers > don't like that. Since the referenced patch was landed before the > Lustre client code was pushed upstream, it probably doesn't make sense > to reference this at all, and just update the commit comment like: > > Previously, MDS_CLOSE_CLEANUP was used to detect file closing > due to eviction. This flag is not used anymore. > > Completely remove this symbol. > >> Change-Id: I20e18103fe085672f499c956e831564e45bd5200 > > Upstream doesn't like the Change-Id, since it is confusing for other > Gerrit instances if the patches are being imported. This should be > flagged if checkpatch.pl is run against the patch, which should be > done for every patch going upstream, even if it was landed to master > previously. > >> Reviewed-on: http://review.whamcloud.com/7195 > > Reviewed-on: is OK. > >> Tested-by: Hudson >> Tested-by: Maloo > > Remove Tested-by: since those don't mean anything to upstream. > >> Signed-off-by: Aurelien Degremont >> Reviewed-by: Andreas Dilger >> Reviewed-by: Jinshan Xiong >> Reviewed-by: John L. Hammond >> Reviewed-by: Fan Yong >> Reviewed-by: Oleg Drokin > > The Signed-off-by: and Reviewed-by: lines should stay. Also your own Signed-off-by: line needs to appear here at the end. > As you can see, the commit message and possibly the patch itself > may be slightly different between master and upstream, but at least > with the Reviewed-on: and Intel-bug-id: tags we can track the patches > between the two. > > There are similar labels needed when porting patches from upstream > kernels into master. > > Both of these cases are already described on the wiki page: > https://wiki.hpdd.intel.com/display/PUB/Commit+Comments under the > "Additional commit tags" section. > > Cheers, Andreas > >> --- >> .../lustre/lustre/include/lustre/lustre_idl.h | 2 +- >> 1 files changed, 1 insertions(+), 1 deletions(-) >> >> diff --git a/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h >> b/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h >> index 4d72d6e..c039cbc 100644 >> --- a/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h >> +++ b/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h >> @@ -2390,7 +2390,7 @@ enum mds_op_bias { >> MDS_PERM_BYPASS = 1 << 3, >> MDS_SOM = 1 << 4, >> MDS_QUOTA_IGNORE = 1 << 5, >> - MDS_CLOSE_CLEANUP = 1 << 6, >> + /* Was MDS_CLOSE_CLEANUP = (1 << 6), No more used */ I imagine you can also fix spelling/grammar as you go (submitting symmetric patches to master). This would likely be fixed in upstream by others creating further drift, so probably a good idea to take it under control from the get go. Thanks. Bye, Oleg From richard.henwood at intel.com Mon Jul 6 14:46:46 2015 From: richard.henwood at intel.com (Henwood, Richard) Date: Mon, 6 Jul 2015 14:46:46 +0000 Subject: [lustre-devel] DNE2 feature patch landing In-Reply-To: <1435865147.2548.26.camel@intel.com> References: <1435865147.2548.26.camel@intel.com> Message-ID: <1436194005.7255.2.camel@intel.com> All, To close this loop: the outstanding DNE2 patches landed last week on the 3rd. thanks for your support, best regards, Richard From bevans at cray.com Mon Jul 6 17:59:31 2015 From: bevans at cray.com (Ben Evans) Date: Mon, 6 Jul 2015 12:59:31 -0500 Subject: [lustre-devel] [PATCH] Remove unused MDS_CLOSE_CLEANUP define Message-ID: <20150706175930.GA2909@lusbld01.us.cray.com> Previously, MDS_CLOSE_CLEANUP was used to detect file closing, due to eviction. This flag is no longer used. Signed-off-by: Aurelien Degremont Intel-bug-ID: http://jira.hpdd.intel.com/browse/LU-3677 Reviewed-on: http://review.whamcloud.com/7195 Reviewed-by: Andreas Dilger Reviewed-by: Jinshan Xiong Reviewed-by: John L. Hammond Reviewed-by: Fan Yong Reviewed-by: Oleg Drokin Signed-off-by: Ben Evans --- .../lustre/lustre/include/lustre/lustre_idl.h | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h b/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h index 4d72d6e..14547d1 100644 --- a/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h +++ b/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h @@ -2390,7 +2390,7 @@ enum mds_op_bias { MDS_PERM_BYPASS = 1 << 3, MDS_SOM = 1 << 4, MDS_QUOTA_IGNORE = 1 << 5, - MDS_CLOSE_CLEANUP = 1 << 6, + /* MDS_UNUSED = 1 << 6, */ MDS_KEEP_ORPHAN = 1 << 7, MDS_RECOV_OPEN = 1 << 8, MDS_DATA_MODIFIED = 1 << 9, -- 1.6.5.6 From andreas.dilger at intel.com Mon Jul 6 18:12:38 2015 From: andreas.dilger at intel.com (Dilger, Andreas) Date: Mon, 6 Jul 2015 18:12:38 +0000 Subject: [lustre-devel] [PATCH] Remove unused MDS_CLOSE_CLEANUP define In-Reply-To: <20150706175930.GA2909@lusbld01.us.cray.com> References: <20150706175930.GA2909@lusbld01.us.cray.com> Message-ID: For submitting upstream, the component should be listed like: staging/lustre: remove unused MDS_CLOSE_CLEANUP define On 2015/07/06, 11:59 AM, "Ben Evans" wrote: >Previously, MDS_CLOSE_CLEANUP was used to detect file closing, due to >eviction. This flag is no longer used. > >Signed-off-by: Aurelien Degremont >Intel-bug-ID: http://jira.hpdd.intel.com/browse/LU-3677 >Reviewed-on: http://review.whamcloud.com/7195 >Reviewed-by: Andreas Dilger >Reviewed-by: Jinshan Xiong >Reviewed-by: John L. Hammond >Reviewed-by: Fan Yong >Reviewed-by: Oleg Drokin >Signed-off-by: Ben Evans >--- > .../lustre/lustre/include/lustre/lustre_idl.h | 2 +- > 1 files changed, 1 insertions(+), 1 deletions(-) > >diff --git a/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h >b/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h >index 4d72d6e..14547d1 100644 >--- a/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h >+++ b/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h >@@ -2390,7 +2390,7 @@ enum mds_op_bias { > MDS_PERM_BYPASS = 1 << 3, > MDS_SOM = 1 << 4, > MDS_QUOTA_IGNORE = 1 << 5, >- MDS_CLOSE_CLEANUP = 1 << 6, >+ /* MDS_UNUSED = 1 << 6, */ > MDS_KEEP_ORPHAN = 1 << 7, > MDS_RECOV_OPEN = 1 << 8, > MDS_DATA_MODIFIED = 1 << 9, >-- >1.6.5.6 > >_______________________________________________ >lustre-devel mailing list >lustre-devel at lists.lustre.org >http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org > Cheers, Andreas -- Andreas Dilger Lustre Software Architect Intel High Performance Data Division From bevans at cray.com Mon Jul 6 18:17:09 2015 From: bevans at cray.com (Ben Evans) Date: Mon, 6 Jul 2015 18:17:09 +0000 Subject: [lustre-devel] [PATCH] Remove unused MDS_CLOSE_CLEANUP define In-Reply-To: References: <20150706175930.GA2909@lusbld01.us.cray.com> Message-ID: <6E6D936CCB2E3C428716BD24C5C6CDF2644A76@CFWEX01.americas.cray.com> Got it, thanks for the feedback! -Ben -----Original Message----- From: Dilger, Andreas [mailto:andreas.dilger at intel.com] Sent: Monday, July 06, 2015 2:13 PM To: Ben Evans Cc: lustre-devel at lists.lustre.org Subject: Re: [lustre-devel] [PATCH] Remove unused MDS_CLOSE_CLEANUP define For submitting upstream, the component should be listed like: staging/lustre: remove unused MDS_CLOSE_CLEANUP define On 2015/07/06, 11:59 AM, "Ben Evans" wrote: >Previously, MDS_CLOSE_CLEANUP was used to detect file closing, due to >eviction. This flag is no longer used. > >Signed-off-by: Aurelien Degremont >Intel-bug-ID: http://jira.hpdd.intel.com/browse/LU-3677 >Reviewed-on: http://review.whamcloud.com/7195 >Reviewed-by: Andreas Dilger >Reviewed-by: Jinshan Xiong >Reviewed-by: John L. Hammond >Reviewed-by: Fan Yong >Reviewed-by: Oleg Drokin >Signed-off-by: Ben Evans >--- > .../lustre/lustre/include/lustre/lustre_idl.h | 2 +- > 1 files changed, 1 insertions(+), 1 deletions(-) > >diff --git a/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h >b/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h >index 4d72d6e..14547d1 100644 >--- a/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h >+++ b/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h >@@ -2390,7 +2390,7 @@ enum mds_op_bias { > MDS_PERM_BYPASS = 1 << 3, > MDS_SOM = 1 << 4, > MDS_QUOTA_IGNORE = 1 << 5, >- MDS_CLOSE_CLEANUP = 1 << 6, >+ /* MDS_UNUSED = 1 << 6, */ > MDS_KEEP_ORPHAN = 1 << 7, > MDS_RECOV_OPEN = 1 << 8, > MDS_DATA_MODIFIED = 1 << 9, >-- >1.6.5.6 > >_______________________________________________ >lustre-devel mailing list >lustre-devel at lists.lustre.org >http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org > Cheers, Andreas -- Andreas Dilger Lustre Software Architect Intel High Performance Data Division From bevans at cray.com Mon Jul 6 18:19:41 2015 From: bevans at cray.com (Ben Evans) Date: Mon, 6 Jul 2015 13:19:41 -0500 Subject: [lustre-devel] [PATCH] staging/lustre: emove unused MDS_CLOSE_CLEANUP define Message-ID: <20150706181941.GA5014@lusbld01.us.cray.com> Previously, MDS_CLOSE_CLEANUP was used to detect file closing, due to eviction. This flag is no longer used. Signed-off-by: Aurelien Degremont Intel-bug-ID: http://jira.hpdd.intel.com/browse/LU-3677 Reviewed-on: http://review.whamcloud.com/7195 Reviewed-by: Andreas Dilger Reviewed-by: Jinshan Xiong Reviewed-by: John L. Hammond Reviewed-by: Fan Yong Reviewed-by: Oleg Drokin Signed-off-by: Ben Evans --- .../lustre/lustre/include/lustre/lustre_idl.h | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h b/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h index 4d72d6e..14547d1 100644 --- a/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h +++ b/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h @@ -2390,7 +2390,7 @@ enum mds_op_bias { MDS_PERM_BYPASS = 1 << 3, MDS_SOM = 1 << 4, MDS_QUOTA_IGNORE = 1 << 5, - MDS_CLOSE_CLEANUP = 1 << 6, + /* MDS_UNUSED = 1 << 6, */ MDS_KEEP_ORPHAN = 1 << 7, MDS_RECOV_OPEN = 1 << 8, MDS_DATA_MODIFIED = 1 << 9, -- 1.6.5.6 From oleg.drokin at intel.com Mon Jul 6 18:27:04 2015 From: oleg.drokin at intel.com (Drokin, Oleg) Date: Mon, 6 Jul 2015 18:27:04 +0000 Subject: [lustre-devel] [PATCH] staging/lustre: emove unused MDS_CLOSE_CLEANUP define In-Reply-To: <20150706181941.GA5014@lusbld01.us.cray.com> References: <20150706181941.GA5014@lusbld01.us.cray.com> Message-ID: The typo in the subject? "emove" -> Remove? On Jul 6, 2015, at 2:19 PM, Ben Evans wrote: > Previously, MDS_CLOSE_CLEANUP was used to detect file closing, due to eviction. > This flag is no longer used. > > Signed-off-by: Aurelien Degremont > Intel-bug-ID: http://jira.hpdd.intel.com/browse/LU-3677 > Reviewed-on: http://review.whamcloud.com/7195 > Reviewed-by: Andreas Dilger > Reviewed-by: Jinshan Xiong > Reviewed-by: John L. Hammond > Reviewed-by: Fan Yong > Reviewed-by: Oleg Drokin > Signed-off-by: Ben Evans > --- > .../lustre/lustre/include/lustre/lustre_idl.h | 2 +- > 1 files changed, 1 insertions(+), 1 deletions(-) > > diff --git a/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h b/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h > index 4d72d6e..14547d1 100644 > --- a/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h > +++ b/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h > @@ -2390,7 +2390,7 @@ enum mds_op_bias { > MDS_PERM_BYPASS = 1 << 3, > MDS_SOM = 1 << 4, > MDS_QUOTA_IGNORE = 1 << 5, > - MDS_CLOSE_CLEANUP = 1 << 6, > + /* MDS_UNUSED = 1 << 6, */ > MDS_KEEP_ORPHAN = 1 << 7, > MDS_RECOV_OPEN = 1 << 8, > MDS_DATA_MODIFIED = 1 << 9, > -- > 1.6.5.6 > > _______________________________________________ > lustre-devel mailing list > lustre-devel at lists.lustre.org > http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org From bevans at cray.com Mon Jul 6 18:30:19 2015 From: bevans at cray.com (Ben Evans) Date: Mon, 6 Jul 2015 13:30:19 -0500 Subject: [lustre-devel] [PATCH] staging/lustre: Remove unused MDS_CLOSE_CLEANUP define Message-ID: <20150706183019.GA6127@lusbld01.us.cray.com> Previously, MDS_CLOSE_CLEANUP was used to detect file closing, due to eviction. This flag is no longer used. Signed-off-by: Aurelien Degremont Intel-bug-ID: http://jira.hpdd.intel.com/browse/LU-3677 Reviewed-on: http://review.whamcloud.com/7195 Reviewed-by: Andreas Dilger Reviewed-by: Jinshan Xiong Reviewed-by: John L. Hammond Reviewed-by: Fan Yong Reviewed-by: Oleg Drokin Signed-off-by: Ben Evans --- .../lustre/lustre/include/lustre/lustre_idl.h | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h b/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h index 4d72d6e..14547d1 100644 --- a/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h +++ b/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h @@ -2390,7 +2390,7 @@ enum mds_op_bias { MDS_PERM_BYPASS = 1 << 3, MDS_SOM = 1 << 4, MDS_QUOTA_IGNORE = 1 << 5, - MDS_CLOSE_CLEANUP = 1 << 6, + /* MDS_UNUSED = 1 << 6, */ MDS_KEEP_ORPHAN = 1 << 7, MDS_RECOV_OPEN = 1 << 8, MDS_DATA_MODIFIED = 1 << 9, -- 1.6.5.6 From andreas.dilger at intel.com Mon Jul 6 20:36:55 2015 From: andreas.dilger at intel.com (Dilger, Andreas) Date: Mon, 6 Jul 2015 20:36:55 +0000 Subject: [lustre-devel] [PATCH] staging/lustre: Remove unused MDS_CLOSE_CLEANUP define In-Reply-To: <20150706183019.GA6127@lusbld01.us.cray.com> References: <20150706183019.GA6127@lusbld01.us.cray.com> Message-ID: The character between "unused" and "MDS" in the patch summary line appears to be a tab instead of a space. On 2015/07/06, 12:30 PM, "Ben Evans" wrote: >Previously, MDS_CLOSE_CLEANUP was used to detect file closing, due to >eviction. >This flag is no longer used. > >Signed-off-by: Aurelien Degremont >Intel-bug-ID: http://jira.hpdd.intel.com/browse/LU-3677 >Reviewed-on: http://review.whamcloud.com/7195 >Reviewed-by: Andreas Dilger >Reviewed-by: Jinshan Xiong >Reviewed-by: John L. Hammond >Reviewed-by: Fan Yong >Reviewed-by: Oleg Drokin >Signed-off-by: Ben Evans >--- > .../lustre/lustre/include/lustre/lustre_idl.h | 2 +- > 1 files changed, 1 insertions(+), 1 deletions(-) > >diff --git a/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h >b/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h >index 4d72d6e..14547d1 100644 >--- a/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h >+++ b/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h >@@ -2390,7 +2390,7 @@ enum mds_op_bias { > MDS_PERM_BYPASS = 1 << 3, > MDS_SOM = 1 << 4, > MDS_QUOTA_IGNORE = 1 << 5, >- MDS_CLOSE_CLEANUP = 1 << 6, >+ /* MDS_UNUSED = 1 << 6, */ When wire protocol definess are removed, it is better to use something like: /* MDS_CLOSE_CLEANUP = 1 << 6, obsolete since 2.4.0 */ The version is from "git describe 5165cdd" from the original commit. > MDS_KEEP_ORPHAN = 1 << 7, > MDS_RECOV_OPEN = 1 << 8, > MDS_DATA_MODIFIED = 1 << 9, I think after these changes you could add the upstream maintainers to the "To:" and "CC:" so that the patch can be accepted upstream. Cheers, Andreas -- Andreas Dilger Lustre Software Architect Intel High Performance Data Division From bevans at cray.com Mon Jul 6 20:39:49 2015 From: bevans at cray.com (Ben Evans) Date: Mon, 6 Jul 2015 20:39:49 +0000 Subject: [lustre-devel] [PATCH] staging/lustre: Remove unused MDS_CLOSE_CLEANUP define In-Reply-To: References: <20150706183019.GA6127@lusbld01.us.cray.com> Message-ID: <6E6D936CCB2E3C428716BD24C5C6CDF2644B9A@CFWEX01.americas.cray.com> Ok, I'll clean it up. Thanks for all the help with the feedback -Ben -----Original Message----- From: Dilger, Andreas [mailto:andreas.dilger at intel.com] Sent: Monday, July 06, 2015 4:37 PM To: Ben Evans Cc: lustre-devel at lists.lustre.org Subject: Re: [lustre-devel] [PATCH] staging/lustre: Remove unused MDS_CLOSE_CLEANUP define The character between "unused" and "MDS" in the patch summary line appears to be a tab instead of a space. On 2015/07/06, 12:30 PM, "Ben Evans" wrote: >Previously, MDS_CLOSE_CLEANUP was used to detect file closing, due to >eviction. >This flag is no longer used. > >Signed-off-by: Aurelien Degremont >Intel-bug-ID: http://jira.hpdd.intel.com/browse/LU-3677 >Reviewed-on: http://review.whamcloud.com/7195 >Reviewed-by: Andreas Dilger >Reviewed-by: Jinshan Xiong >Reviewed-by: John L. Hammond >Reviewed-by: Fan Yong >Reviewed-by: Oleg Drokin >Signed-off-by: Ben Evans >--- > .../lustre/lustre/include/lustre/lustre_idl.h | 2 +- > 1 files changed, 1 insertions(+), 1 deletions(-) > >diff --git a/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h >b/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h >index 4d72d6e..14547d1 100644 >--- a/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h >+++ b/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h >@@ -2390,7 +2390,7 @@ enum mds_op_bias { > MDS_PERM_BYPASS = 1 << 3, > MDS_SOM = 1 << 4, > MDS_QUOTA_IGNORE = 1 << 5, >- MDS_CLOSE_CLEANUP = 1 << 6, >+ /* MDS_UNUSED = 1 << 6, */ When wire protocol definess are removed, it is better to use something like: /* MDS_CLOSE_CLEANUP = 1 << 6, obsolete since 2.4.0 */ The version is from "git describe 5165cdd" from the original commit. > MDS_KEEP_ORPHAN = 1 << 7, > MDS_RECOV_OPEN = 1 << 8, > MDS_DATA_MODIFIED = 1 << 9, I think after these changes you could add the upstream maintainers to the "To:" and "CC:" so that the patch can be accepted upstream. Cheers, Andreas -- Andreas Dilger Lustre Software Architect Intel High Performance Data Division From oleg.drokin at intel.com Mon Jul 6 22:48:19 2015 From: oleg.drokin at intel.com (Drokin, Oleg) Date: Mon, 6 Jul 2015 22:48:19 +0000 Subject: [lustre-devel] New master tag 2.5.56 Message-ID: <7AA9ADD3-B5D0-4606-9F00-37A76EC9E69E@intel.com> Hello! This is a friendly note to let you know that I just tagged 2.5.56 in the master branch of the community Lustre repo. Please file new issues found during testing of this code in the Jira tracker. In particular the new additions in DNE area. Bye, Oleg From morrone2 at llnl.gov Mon Jul 6 23:41:43 2015 From: morrone2 at llnl.gov (Christopher J. Morrone) Date: Mon, 06 Jul 2015 16:41:43 -0700 Subject: [lustre-devel] New master tag 2.7.56 In-Reply-To: <7AA9ADD3-B5D0-4606-9F00-37A76EC9E69E@intel.com> References: <7AA9ADD3-B5D0-4606-9F00-37A76EC9E69E@intel.com> Message-ID: <559B1237.9010004@llnl.gov> I believe that you meant 2.7.56. Chris On 07/06/2015 03:48 PM, Drokin, Oleg wrote: > Hello! > > This is a friendly note to let you know that I just tagged 2.5.56 in the master branch of the community Lustre repo. > Please file new issues found during testing of this code in the Jira tracker. > In particular the new additions in DNE area. > > Bye, > Oleg > _______________________________________________ > lustre-devel mailing list > lustre-devel at lists.lustre.org > http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org > From oleg.drokin at intel.com Mon Jul 6 23:50:46 2015 From: oleg.drokin at intel.com (Drokin, Oleg) Date: Mon, 6 Jul 2015 23:50:46 +0000 Subject: [lustre-devel] New master tag 2.7.56 In-Reply-To: <559B1237.9010004@llnl.gov> References: <7AA9ADD3-B5D0-4606-9F00-37A76EC9E69E@intel.com> <559B1237.9010004@llnl.gov> Message-ID: <4C762B76-570E-4DEB-9CB6-2AE0506F325A@intel.com> Ah, right! Thanks! On Jul 6, 2015, at 7:41 PM, Christopher J. Morrone wrote: > I believe that you meant 2.7.56. > > Chris > > On 07/06/2015 03:48 PM, Drokin, Oleg wrote: >> Hello! >> >> This is a friendly note to let you know that I just tagged 2.5.56 in the master branch of the community Lustre repo. >> Please file new issues found during testing of this code in the Jira tracker. >> In particular the new additions in DNE area. >> >> Bye, >> Oleg >> _______________________________________________ >> lustre-devel mailing list >> lustre-devel at lists.lustre.org >> http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org >> > > _______________________________________________ > lustre-devel mailing list > lustre-devel at lists.lustre.org > http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org From bevans at cray.com Tue Jul 7 15:32:34 2015 From: bevans at cray.com (Ben Evans) Date: Tue, 7 Jul 2015 10:32:34 -0500 Subject: [lustre-devel] [PATCH] [PATCH] staging/lustre: Remove unused MDS_CLOSE_CLEANUP define Message-ID: <20150707153234.GA1872@lusbld01.us.cray.com> Previously, MDS_CLOSE_CLEANUP was used to detect file closing, due to eviction. This flag is no longer used. Signed-off-by: Aurelien Degremont Intel-bug-ID: http://jira.hpdd.intel.com/browse/LU-3677 Reviewed-on: http://review.whamcloud.com/7195 Reviewed-by: Andreas Dilger Reviewed-by: Jinshan Xiong Reviewed-by: John L. Hammond Reviewed-by: Fan Yong Reviewed-by: Oleg Drokin Signed-off-by: Ben Evans --- .../lustre/lustre/include/lustre/lustre_idl.h | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h b/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h index 4d72d6e..bea70cb 100644 --- a/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h +++ b/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h @@ -2390,7 +2390,7 @@ enum mds_op_bias { MDS_PERM_BYPASS = 1 << 3, MDS_SOM = 1 << 4, MDS_QUOTA_IGNORE = 1 << 5, - MDS_CLOSE_CLEANUP = 1 << 6, + /* MDS_CLOSE_CLEANUP = 1 << 6, obsolete since 2.4.0 */ MDS_KEEP_ORPHAN = 1 << 7, MDS_RECOV_OPEN = 1 << 8, MDS_DATA_MODIFIED = 1 << 9, -- 1.6.5.6 From morrone2 at llnl.gov Tue Jul 7 19:57:38 2015 From: morrone2 at llnl.gov (Christopher J. Morrone) Date: Tue, 07 Jul 2015 12:57:38 -0700 Subject: [lustre-devel] [PATCH] [PATCH] staging/lustre: Remove unused MDS_CLOSE_CLEANUP define In-Reply-To: <20150707153234.GA1872@lusbld01.us.cray.com> References: <20150707153234.GA1872@lusbld01.us.cray.com> Message-ID: <559C2F32.5040104@llnl.gov> Unless there is code in this client that explicitly prevents it from connecting with a pre-2.4.0 server, perhaps this flag should not be removed. Chris On 07/07/2015 08:32 AM, Ben Evans wrote: > Previously, MDS_CLOSE_CLEANUP was used to detect file closing, > due to eviction. This flag is no longer used. > > Signed-off-by: Aurelien Degremont > Intel-bug-ID: http://jira.hpdd.intel.com/browse/LU-3677 > Reviewed-on: http://review.whamcloud.com/7195 > Reviewed-by: Andreas Dilger > Reviewed-by: Jinshan Xiong > Reviewed-by: John L. Hammond > Reviewed-by: Fan Yong > Reviewed-by: Oleg Drokin > Signed-off-by: Ben Evans > --- > .../lustre/lustre/include/lustre/lustre_idl.h | 2 +- > 1 files changed, 1 insertions(+), 1 deletions(-) > > diff --git a/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h b/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h > index 4d72d6e..bea70cb 100644 > --- a/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h > +++ b/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h > @@ -2390,7 +2390,7 @@ enum mds_op_bias { > MDS_PERM_BYPASS = 1 << 3, > MDS_SOM = 1 << 4, > MDS_QUOTA_IGNORE = 1 << 5, > - MDS_CLOSE_CLEANUP = 1 << 6, > + /* MDS_CLOSE_CLEANUP = 1 << 6, obsolete since 2.4.0 */ > MDS_KEEP_ORPHAN = 1 << 7, > MDS_RECOV_OPEN = 1 << 8, > MDS_DATA_MODIFIED = 1 << 9, > From andreas.dilger at intel.com Wed Jul 8 00:42:38 2015 From: andreas.dilger at intel.com (Dilger, Andreas) Date: Wed, 8 Jul 2015 00:42:38 +0000 Subject: [lustre-devel] [PATCH] [PATCH] staging/lustre: Remove unused MDS_CLOSE_CLEANUP define In-Reply-To: <20150707153234.GA1872@lusbld01.us.cray.com> References: <20150707153234.GA1872@lusbld01.us.cray.com> Message-ID: <88997EA7-E368-4626-A2B9-B515BF329FAC@intel.com> The patch looks good at this point. Now you need to also sent it to Greg KH and the other staging lists. Cheers, Andreas > On Jul 7, 2015, at 09:32, Ben Evans wrote: > > Previously, MDS_CLOSE_CLEANUP was used to detect file closing, > due to eviction. This flag is no longer used. > > Signed-off-by: Aurelien Degremont > Intel-bug-ID: http://jira.hpdd.intel.com/browse/LU-3677 > Reviewed-on: http://review.whamcloud.com/7195 > Reviewed-by: Andreas Dilger > Reviewed-by: Jinshan Xiong > Reviewed-by: John L. Hammond > Reviewed-by: Fan Yong > Reviewed-by: Oleg Drokin > Signed-off-by: Ben Evans > --- > .../lustre/lustre/include/lustre/lustre_idl.h | 2 +- > 1 files changed, 1 insertions(+), 1 deletions(-) > > diff --git a/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h b/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h > index 4d72d6e..bea70cb 100644 > --- a/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h > +++ b/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h > @@ -2390,7 +2390,7 @@ enum mds_op_bias { > MDS_PERM_BYPASS = 1 << 3, > MDS_SOM = 1 << 4, > MDS_QUOTA_IGNORE = 1 << 5, > - MDS_CLOSE_CLEANUP = 1 << 6, > + /* MDS_CLOSE_CLEANUP = 1 << 6, obsolete since 2.4.0 */ > MDS_KEEP_ORPHAN = 1 << 7, > MDS_RECOV_OPEN = 1 << 8, > MDS_DATA_MODIFIED = 1 << 9, > -- > 1.6.5.6 > > _______________________________________________ > lustre-devel mailing list > lustre-devel at lists.lustre.org > http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org From andreas.dilger at intel.com Wed Jul 8 00:44:24 2015 From: andreas.dilger at intel.com (Dilger, Andreas) Date: Wed, 8 Jul 2015 00:44:24 +0000 Subject: [lustre-devel] [PATCH] [PATCH] staging/lustre: Remove unused MDS_CLOSE_CLEANUP define In-Reply-To: <559C2F32.5040104@llnl.gov> References: <20150707153234.GA1872@lusbld01.us.cray.com>, <559C2F32.5040104@llnl.gov> Message-ID: The flag is no longer used in the code so there is no value to keep it around. It was only ever used for SOM, which was deleted and never supported. Cheers, Andreas > On Jul 7, 2015, at 13:57, Christopher J. Morrone wrote: > > Unless there is code in this client that explicitly prevents it from connecting with a pre-2.4.0 server, perhaps this flag should not be removed. > > Chris > >> On 07/07/2015 08:32 AM, Ben Evans wrote: >> Previously, MDS_CLOSE_CLEANUP was used to detect file closing, >> due to eviction. This flag is no longer used. >> >> Signed-off-by: Aurelien Degremont >> Intel-bug-ID: http://jira.hpdd.intel.com/browse/LU-3677 >> Reviewed-on: http://review.whamcloud.com/7195 >> Reviewed-by: Andreas Dilger >> Reviewed-by: Jinshan Xiong >> Reviewed-by: John L. Hammond >> Reviewed-by: Fan Yong >> Reviewed-by: Oleg Drokin >> Signed-off-by: Ben Evans >> --- >> .../lustre/lustre/include/lustre/lustre_idl.h | 2 +- >> 1 files changed, 1 insertions(+), 1 deletions(-) >> >> diff --git a/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h b/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h >> index 4d72d6e..bea70cb 100644 >> --- a/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h >> +++ b/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h >> @@ -2390,7 +2390,7 @@ enum mds_op_bias { >> MDS_PERM_BYPASS = 1 << 3, >> MDS_SOM = 1 << 4, >> MDS_QUOTA_IGNORE = 1 << 5, >> - MDS_CLOSE_CLEANUP = 1 << 6, >> + /* MDS_CLOSE_CLEANUP = 1 << 6, obsolete since 2.4.0 */ >> MDS_KEEP_ORPHAN = 1 << 7, >> MDS_RECOV_OPEN = 1 << 8, >> MDS_DATA_MODIFIED = 1 << 9, > > _______________________________________________ > lustre-devel mailing list > lustre-devel at lists.lustre.org > http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org From james.a.nunez at intel.com Thu Jul 9 14:06:40 2015 From: james.a.nunez at intel.com (Nunez, James A) Date: Thu, 9 Jul 2015 14:06:40 +0000 Subject: [lustre-devel] [Iudev] proposed version change for PTLRPC GSS In-Reply-To: References: <55111C81.5010009@bull.net> <551518C3.70002@bull.net> <553001AE.2070601@bull.net> Message-ID: <204419CE11160F44944331EE9BAD7D224610A775@FMSMSX114.amr.corp.intel.com> Nathan and Sebastien, I’m interested in testing the revived Kerberos support in Lustre. I’d like to understand how you tested this feature and suggested best practices on setting up the Kerberos environment for use with Lustre. I’ve looked in Jira and can’t find a test plan for Kerberos. Do you have a test plan and would you please share it? I’ve looked over all the patches for the Kerberos Revival ticket and don’t see any new tests added by these patches. Does this mean that the existing Kerberos tests in sanity-krb5 still work and test the feature thoroughly? If not, would you please submit a patch adding new tests? Thank you, James From: lustre-devel [mailto:lustre-devel-bounces at lists.lustre.org] On Behalf Of Nathan Rutman Sent: Tuesday, April 21, 2015 1:47 PM To: Sebastien Buisson; Andrew Perepechko Cc: iudev at lists.opensfs.org; lustre-devel at lists.lustre.org Subject: Re: [lustre-devel] [Iudev] proposed version change for PTLRPC GSS Hi Sebastien - thanks for pushing this forward. At this point, I want to make sure that we have the "union" of all the Kerberos fixes from Bull and Seagate on track for landing. I know you're already working with Andrew but if there's anything else that you need from Seagate please let me know. -- Nathan Rutman · Principal Systems Architect Seagate Technology · +1 503 877-9507 · GMT-8 On Thu, Apr 16, 2015 at 11:38 AM, Sebastien Buisson > wrote: Hi Nathan, All the Kerberos related patches that are not landed yet are: LU-3778: http://review.whamcloud.com/14040 LU-6356: http://review.whamcloud.com/14349 http://review.whamcloud.com/14041 http://review.whamcloud.com/14042 http://review.whamcloud.com/14404 They can all possibly impact the work done on Shared Key feature, this is why I was proposing that have them merged as soon as possible. Best regards, Sebastien. Le 16/04/2015 12:22, Nathan Rutman a écrit : Sebastien, do these patches represent all the merged Kerberos changes? Or can these be landed independently of the others? *--* *Nathan Rutman · Principal Systems Architect Seagate Technology** · *+1 503 877-9507* · *GMT-8 On Fri, Mar 27, 2015 at 1:45 AM, Sebastien Buisson >> wrote: Re-emitting because of issues with my email address. -------- Hi, As I understand the need for evolutions in the GSS code, I advocate the review and merge of all patches related to Kerberos revival as soon as possible. It would avoid painful rebase of work done by IU, or Kerberos patches, or both. The Kerberos revival patches waiting for review are: http://review.whamcloud.com/__14040 http://review.whamcloud.com/__14041 http://review.whamcloud.com/__14042 Best regards, Sebastien. Le 25/03/2015 22:25, Dilger, Andreas a écrit : Sebastien, can you please also add lustre-devel at lists.lustre.org > to the CC list for this discussion. On 2015/03/24, 3:12 AM, "Sebastien Buisson" >> wrote: Hi there, I agree we should not bother with backward compatibility. Kerberos revival patches aim at Lustre 2.8, so we are good if the modifications you propose also land in 2.8. Taking advantage of the opportunity to replace handle_nullreq with subflavor specific code is really nice, as those bits are really confusing for someone who tries to understand the code. Cheers, Sebastien. Le 24/03/2015 02:34, Jeremy Filizetti a écrit : On the phone call last week we discussed an increment of the PTLRPC_GSS_VERSION to version 2 to allow some changes changes/restructuring. No one had any objections on the phone call but I wanted to send it out for wider distribution and feedback. Changing the request format would allow us to support larger GSS token sizes which today are limited (see ticket LU-3855). From what I have looked through so far the following seems to allow for larger tokens and also allow some of these changes without having to worry about backwards compatibility since it was never really "working" anyways. Change PTLRPC_GSS_VERSION to 2 Enlarge GSS_CTX_INIT_MAX_LEN to something larger then 1024. Ideally we would support MaxTokenSize of 64k for the largest active directory ticket: (see http://blogs.technet.com/b/__shanecothran/archive/2010/07/__16/maxtokensize-a nd-kerberos-token-bloat.aspx). The purpose of enlarging this is to support larger tokens. The sizeof(struct rsi) needs to remain under PAGE_SIZE right now with rsi_request calling sunrpc_cache_pipe_upcall. Since there is only one lsvcgssd process supposed to be running maybe it would be acceptable to use larger requests and just slightly modify rsi_request to incorporate must of the functionality of sunrpc_cache_pipe_upcall. To keep things simple with the lsvcgssd and continue to use a single channel proc file interface I'd like to AND the GSS subflavor onto most significant bits of lustre_svc in struct rsi. Instead of calling the inappropriately named handle_nullreq things would be changed to handle the multiple subflavors (gssnull, sk, krb5). gssnull and sk won't have a full userspace component so gss_accept_sec_context can't be called. Thoughts welcome. I'm sure I missed something along the way here but this is just what I have looked at so far. Thanks, Jeremy _________________________________________________ Iudev mailing list Iudev at lists.opensfs.org > http://lists.opensfs.org/__listinfo.cgi/iudev-opensfs.org _________________________________________________ Iudev mailing list Iudev at lists.opensfs.org > http://lists.opensfs.org/__listinfo.cgi/iudev-opensfs.org Cheers, Andreas _________________________________________________ Iudev mailing list Iudev at lists.opensfs.org > http://lists.opensfs.org/__listinfo.cgi/iudev-opensfs.org _______________________________________________ Iudev mailing list Iudev at lists.opensfs.org http://lists.opensfs.org/listinfo.cgi/iudev-opensfs.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From sebastien.buisson at atos.net Thu Jul 9 15:34:54 2015 From: sebastien.buisson at atos.net (Sebastien Buisson) Date: Thu, 9 Jul 2015 17:34:54 +0200 Subject: [lustre-devel] [Iudev] proposed version change for PTLRPC GSS In-Reply-To: <204419CE11160F44944331EE9BAD7D224610A775@FMSMSX114.amr.corp.intel.com> References: <55111C81.5010009@bull.net> <551518C3.70002@bull.net> <553001AE.2070601@bull.net> <204419CE11160F44944331EE9BAD7D224610A775@FMSMSX114.amr.corp.intel.com> Message-ID: <559E949E.2000301@bull.net> Hello James, I just updated LU-6356 with the Kerberos Test Plan. Basically, all sanity-krb5 tests pass, with some patches that I will submit very soon in the ticket. The notable exceptions are tests related to 'lfs flushctx': I started investigating, but the thing is I cannot figure out what this command is supposed to do exactly. So it prevents me from determining which part of the code should be fixed. Cheers, Sebastien. Le 09/07/2015 16:06, Nunez, James A a écrit : > Nathan and Sebastien, > > I’m interested in testing the revived Kerberos support in Lustre. I’d > like to understand how you tested this feature and suggested best > practices on setting up the Kerberos environment for use with Lustre. > > I’ve looked in Jira and can’t find a test plan for Kerberos. Do you have > a test plan and would you please share it? > > I’ve looked over all the patches for the Kerberos Revival ticket and > don’t see any new tests added by these patches. Does this mean that the > existing Kerberos tests in sanity-krb5 still work and test the feature > thoroughly? If not, would you please submit a patch adding new tests? > > Thank you, > > James > > *From:*lustre-devel [mailto:lustre-devel-bounces at lists.lustre.org] *On > Behalf Of *Nathan Rutman > *Sent:* Tuesday, April 21, 2015 1:47 PM > *To:* Sebastien Buisson; Andrew Perepechko > *Cc:* iudev at lists.opensfs.org; lustre-devel at lists.lustre.org > *Subject:* Re: [lustre-devel] [Iudev] proposed version change for PTLRPC GSS > > Hi Sebastien - > > thanks for pushing this forward. At this point, I want to make sure that > we have the "union" of all the Kerberos fixes from Bull and Seagate on > track for landing. I know you're already working with Andrew but if > there's anything else that you need from Seagate please let me know. > > > *--* > > *Nathan Rutman · Principal Systems Architect > Seagate Technology · *+1 503 877-9507* · *GMT-8 > > On Thu, Apr 16, 2015 at 11:38 AM, Sebastien Buisson > > wrote: > > Hi Nathan, > > All the Kerberos related patches that are not landed yet are: > LU-3778: > http://review.whamcloud.com/14040 > LU-6356: > http://review.whamcloud.com/14349 > http://review.whamcloud.com/14041 > http://review.whamcloud.com/14042 > http://review.whamcloud.com/14404 > > They can all possibly impact the work done on Shared Key feature, this > is why I was proposing that have them merged as soon as possible. > > Best regards, > Sebastien. > > > Le 16/04/2015 12:22, Nathan Rutman a écrit : > > Sebastien, do these patches represent all the merged Kerberos changes? > Or can these be landed independently of the others? > > > *--* > *Nathan Rutman · Principal Systems Architect > Seagate Technology** · *+1 503 877-9507 * · > *GMT-8 > > On Fri, Mar 27, 2015 at 1:45 AM, Sebastien Buisson > > >> > wrote: > > Re-emitting because of issues with my email address. > -------- > > Hi, > > As I understand the need for evolutions in the GSS code, I advocate > the review and merge of all patches related to Kerberos revival as > soon as possible. It would avoid painful rebase of work done by IU, > or Kerberos patches, or both. > The Kerberos revival patches waiting for review are: > http://review.whamcloud.com/__14040 > http://review.whamcloud.com/__14041 > http://review.whamcloud.com/__14042 > > Best regards, > Sebastien. > > > Le 25/03/2015 22:25, Dilger, Andreas a écrit : > > Sebastien, > can you please also add lustre-devel at lists.lustre.org > > > to the CC list for > this discussion. > > On 2015/03/24, 3:12 AM, "Sebastien Buisson" > > >> > > > wrote: > > Hi there, > > I agree we should not bother with backward compatibility. > Kerberos > revival patches aim at Lustre 2.8, so we are good if the > modifications > you propose also land in 2.8. > > Taking advantage of the opportunity to replace > handle_nullreq with > subflavor specific code is really nice, as those bits are > really > confusing for someone who tries to understand the code. > > Cheers, > Sebastien. > > > Le 24/03/2015 02:34, Jeremy Filizetti a écrit : > > On the phone call last week we discussed an increment > of the > PTLRPC_GSS_VERSION to version 2 to allow some changes > changes/restructuring. No one had any objections on the > phone call but > I wanted to send it out for wider distribution and > feedback. > > Changing the request format would allow us to support > larger GSS token > sizes which today are limited (see ticket LU-3855). > From what I have > looked through so far the following seems to allow for > larger tokens and > also allow some of these changes without having to worry > about backwards > compatibility since it was never really "working" anyways. > > Change PTLRPC_GSS_VERSION to 2 > > Enlarge GSS_CTX_INIT_MAX_LEN to something larger then > 1024. Ideally we > would support MaxTokenSize of 64k for the largest active > directory > ticket: (see > > http://blogs.technet.com/b/__shanecothran/archive/2010/07/__16/maxtokensize-a > > > > > nd-kerberos-token-bloat.aspx). > The purpose of enlarging this is to support larger > tokens. The > sizeof(struct rsi) needs to remain under PAGE_SIZE right > now with > rsi_request calling sunrpc_cache_pipe_upcall. Since > there is only one > lsvcgssd process supposed to be running maybe it would > be acceptable to > use larger requests and just slightly modify rsi_request > to incorporate > must of the functionality of sunrpc_cache_pipe_upcall. > > To keep things simple with the lsvcgssd and continue to > use a single > channel proc file interface I'd like to AND the GSS > subflavor onto most > significant bits of lustre_svc in struct rsi. Instead > of calling the > inappropriately named handle_nullreq things would be > changed to handle > the multiple subflavors (gssnull, sk, krb5). gssnull > and sk won't have > a full userspace component so gss_accept_sec_context > can't be called. > > Thoughts welcome. I'm sure I missed something along the > way here but > this is just what I have looked at so far. > > Thanks, > Jeremy > > _________________________________________________ > Iudev mailing list > Iudev at lists.opensfs.org > > > http://lists.opensfs.org/__listinfo.cgi/iudev-opensfs.org > > > _________________________________________________ > Iudev mailing list > Iudev at lists.opensfs.org > > > http://lists.opensfs.org/__listinfo.cgi/iudev-opensfs.org > > > > > Cheers, Andreas > > _________________________________________________ > Iudev mailing list > Iudev at lists.opensfs.org > > > http://lists.opensfs.org/__listinfo.cgi/iudev-opensfs.org > > > > > > _______________________________________________ > Iudev mailing list > Iudev at lists.opensfs.org > http://lists.opensfs.org/listinfo.cgi/iudev-opensfs.org > From bevans at cray.com Fri Jul 10 15:09:31 2015 From: bevans at cray.com (Ben Evans) Date: Fri, 10 Jul 2015 10:09:31 -0500 Subject: [lustre-devel] [PATCH] lustre/staging: remove unused MAY_ constants Message-ID: <20150710150931.GA29330@lusbld01.us.cray.com> Remove unused MAY_ constants from lustre_idl.h Intel-bug-ID: http://jira.httpd.intel.com/browse/LU-6450 Reviewed-by: James Simmons Signed-off-by: Ben Evans --- .../lustre/lustre/include/lustre/lustre_idl.h | 24 ++++++++----------- 1 files changed, 10 insertions(+), 14 deletions(-) diff --git a/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h b/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h index bea70cb..be1cbae 100644 --- a/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h +++ b/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h @@ -2366,20 +2366,16 @@ extern void lustre_swab_mdt_rec_setattr (struct mdt_rec_setattr *sa); */ #define MDS_OPEN_RELEASE 02000000000000ULL /* Open the file for HSM release */ -/* permission for create non-directory file */ -#define MAY_CREATE (1 << 7) -/* permission for create directory file */ -#define MAY_LINK (1 << 8) -/* permission for delete from the directory */ -#define MAY_UNLINK (1 << 9) -/* source's permission for rename */ -#define MAY_RENAME_SRC (1 << 10) -/* target's permission for rename */ -#define MAY_RENAME_TAR (1 << 11) -/* part (parent's) VTX permission check */ -#define MAY_VTX_PART (1 << 12) -/* full VTX permission check */ -#define MAY_VTX_FULL (1 << 13) +/* +#define UNUSED_MAY_CREATE (1 << 7) create non-directory file +#define UNUSED_MAY_LINK (1 << 8) create directory file +#define UNUSED_MAY_UNLINK (1 << 9) delete from the directory +#define UNUSED_MAY_RENAME_SRC (1 << 10) source's rename +#define UNUSED_MAY_RENAME_TAR (1 << 11) target's rename +#define UNUSED_MAY_VTX_PART (1 << 12) part (parent's) VTX check +#define UNUSED_MAY_VTX_FULL (1 << 13) full VTX check +*/ + /* lfs rgetfacl permission check */ #define MAY_RGETFACL (1 << 14) -- 1.6.5.6 From paf at cray.com Fri Jul 10 15:11:54 2015 From: paf at cray.com (Patrick Farrell) Date: Fri, 10 Jul 2015 10:11:54 -0500 Subject: [lustre-devel] [PATCH] lustre/staging: remove unused MAY_ constants In-Reply-To: <20150710150931.GA29330@lusbld01.us.cray.com> References: <20150710150931.GA29330@lusbld01.us.cray.com> Message-ID: <559FE0BA.1060206@cray.com> Typo in the bug-ID URL. httpd instead of hpdd. On 07/10/2015 10:09 AM, Ben Evans wrote: > Remove unused MAY_ constants from lustre_idl.h > > Intel-bug-ID: http://jira.httpd.intel.com/browse/LU-6450 > Reviewed-by: James Simmons > Signed-off-by: Ben Evans > --- > .../lustre/lustre/include/lustre/lustre_idl.h | 24 ++++++++----------- > 1 files changed, 10 insertions(+), 14 deletions(-) > > diff --git a/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h b/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h > index bea70cb..be1cbae 100644 > --- a/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h > +++ b/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h > @@ -2366,20 +2366,16 @@ extern void lustre_swab_mdt_rec_setattr (struct mdt_rec_setattr *sa); > */ > #define MDS_OPEN_RELEASE 02000000000000ULL /* Open the file for HSM release */ > > -/* permission for create non-directory file */ > -#define MAY_CREATE (1 << 7) > -/* permission for create directory file */ > -#define MAY_LINK (1 << 8) > -/* permission for delete from the directory */ > -#define MAY_UNLINK (1 << 9) > -/* source's permission for rename */ > -#define MAY_RENAME_SRC (1 << 10) > -/* target's permission for rename */ > -#define MAY_RENAME_TAR (1 << 11) > -/* part (parent's) VTX permission check */ > -#define MAY_VTX_PART (1 << 12) > -/* full VTX permission check */ > -#define MAY_VTX_FULL (1 << 13) > +/* > +#define UNUSED_MAY_CREATE (1 << 7) create non-directory file > +#define UNUSED_MAY_LINK (1 << 8) create directory file > +#define UNUSED_MAY_UNLINK (1 << 9) delete from the directory > +#define UNUSED_MAY_RENAME_SRC (1 << 10) source's rename > +#define UNUSED_MAY_RENAME_TAR (1 << 11) target's rename > +#define UNUSED_MAY_VTX_PART (1 << 12) part (parent's) VTX check > +#define UNUSED_MAY_VTX_FULL (1 << 13) full VTX check > +*/ > + > /* lfs rgetfacl permission check */ > #define MAY_RGETFACL (1 << 14) > From paf at cray.com Fri Jul 10 16:14:45 2015 From: paf at cray.com (Patrick Farrell) Date: Fri, 10 Jul 2015 11:14:45 -0500 Subject: [lustre-devel] Lock ahead - Possible change Message-ID: <559FEF75.4080103@cray.com> Good morning, I am reluctant to make further changes to lock ahead at this point when it is ready and pending review, but I wanted to float an idea to see if it would likely meet acceptance. (I am primarily asking Jinshan and Andreas, who have been involved from the beginning.) Currently, lock ahead requests are asynchronous, and, because of that, non-blocking. What about the possibility of also allowing blocking, *synchronous* requests via the same API? This is intended as a way for a client to make a blocking lock request on a specific region of a file, rather than the whole file by taking a group lock as is currently the only option. This would not be used for my immediate goal of enhancing strided writing performance with lock ahead itself, but might be handy in other situations and is a pretty minor change to the API. (Just allow blocking requests and make them synchronous, rather than not allowing them as is the case now.) I still think I will push this out to a future enhancement of the feature rather than integrate it into the current patch. Thoughts? Objections? I admit that this expansion of the API is making me question 'lock ahead' as the name for the API itself, rather than a particular usage of it. Perhaps that's not a direction we want to go, or perhaps we should rename the API. (Suggestions encouraged) - Patrick Farrell From oleg.drokin at intel.com Mon Jul 27 01:19:25 2015 From: oleg.drokin at intel.com (Drokin, Oleg) Date: Mon, 27 Jul 2015 01:19:25 +0000 Subject: [lustre-devel] Extended Lustre testing Message-ID: <7F0D23EB-DEAF-42C7-A085-F9C778B84C10@intel.com> Hello! Last LUG and at other events I have been getting questions about additional testing I perform on Lustre patches before accepting them. While explaining it in general in person when asked, I finally came up with a written Howto that I placed in the wiki at http://wiki.lustre.org/Testing_Setup_To_Induce_Race_Conditions Now, I've been doing this for a couple of years, so certain details that are obvious for me and I take for granted might be less obvious for people exposed to all of this for the first time. So now might be a good time to ask away your questions so that the document could be improved. Also, I wanted some visibility for this addition in the hopes that more people would start doing stuff like this and perhaps even enhance the technique so that the Lustre becomes even more stable and robust. And additionally it would be great if others share their own cool testing techniques that might be deployed by others. Thanks. Bye, Oleg From oleg.drokin at intel.com Mon Jul 27 14:16:40 2015 From: oleg.drokin at intel.com (Drokin, Oleg) Date: Mon, 27 Jul 2015 14:16:40 +0000 Subject: [lustre-devel] Expanding Build/Test Coverage for community tree Message-ID: <8FD97F7B-BF6F-4DB9-8A87-A88BF3683EC4@intel.com> Hello! I have been thinking about ways to extend our build/test coverage beyond the capacity and composition of the HPDD test cluster. Without participating in continuous build/testing, it is easy for regressions affecting options used in the community (architectures, Linux distros or lnds, say) to creep in and only be reported after the responsible changes have landed (and perhaps even a release has gone out) In order to alleviate this somehow, I created a small howto with a handful of scripts that would allow all interested parties to insert their own builders and test scenarios into process from comfort of their own data centers and without full blown additional Jenkins instances and such. The scripts are pretty primitive and single threaded (mostly geared for build testing now), but they do their intended job. All enhancements to make them better handle testing, integrate with web servers to post better output and such are also greatly appreciated. The howto could be located at http://wiki.lustre.org/Simple_Gerrit_Builder_Howto Thanks. Bye, Oleg From aurelien.degremont at cea.fr Mon Jul 27 16:34:03 2015 From: aurelien.degremont at cea.fr (DEGREMONT Aurelien) Date: Mon, 27 Jul 2015 18:34:03 +0200 Subject: [lustre-devel] LAD'15: Call for papers extended up to August 4th Message-ID: <55B65D7B.40506@cea.fr> *LAD'15 - Lustre Administrator and Developer Workshop* September 22th - 23th, 2015 Paris Marriott Champs Elysees Hotel, Paris - France *CALL FOR PAPERS* We are extending call for papers up to August 4th, 2015. You have one more week to send your abstract! We are inviting community members to send proposals for presentations at this event. No proceeding is required, just an abstract of a 30-min (technical) presentation. Please send this to lad at eofs.eu Topics may include (but are not limited to): site updates or future projects, Lustre administration, monitoring and tools, Lustre feature overview, Lustre client performance (benefits of hardware evolution to Lustre (like SSD, many-cores…), comparison between Lustre and other parallel file systems (perf. and/or features), Lustre and Exascale I/O, tunings, etc. *REGISTRATION* Registration for the workshop is open: http://lad.eofs.org/register.php *WEB SITE* Get all details on http://www.eofs.eu/?id=lad15 *SPONSORS* We are very pleased this event is organized thanks to the following generous sponsors: ATOS, CEA, CRAY, DDN, INTEL and SEAGATE For any other information, please contact lad at eofs.eu -------------- next part -------------- An HTML attachment was scrubbed... URL: From oleg.drokin at intel.com Mon Jul 27 19:00:35 2015 From: oleg.drokin at intel.com (Drokin, Oleg) Date: Mon, 27 Jul 2015 19:00:35 +0000 Subject: [lustre-devel] New master tag 2.7.57 Message-ID: Hello! This is a friendly note that new tag 2.7.57 has been created in the master branch of the community Lustre repo. Bye, Oleg From andreas.dilger at intel.com Wed Jul 29 04:53:28 2015 From: andreas.dilger at intel.com (Dilger, Andreas) Date: Wed, 29 Jul 2015 04:53:28 +0000 Subject: [lustre-devel] Lock ahead - Possible change In-Reply-To: <559FEF75.4080103@cray.com> References: <559FEF75.4080103@cray.com> Message-ID: On 2015/07/10, 10:14 AM, "lustre-devel on behalf of Patrick Farrell" wrote: >Good morning, > >I am reluctant to make further changes to lock ahead at this point when >it is ready and pending review, but I wanted to float an idea to see if >it would likely meet acceptance. (I am primarily asking Jinshan and >Andreas, who have been involved from the beginning.) > >Currently, lock ahead requests are asynchronous, and, because of that, >non-blocking. What about the possibility of also allowing blocking, >*synchronous* requests via the same API? This seems like it could be useful in the future, if it doesn't make the code significantly more complex. >This is intended as a way for a client to make a blocking lock request >on a specific region of a file, rather than the whole file by taking a >group lock as is currently the only option. > >This would not be used for my immediate goal of enhancing strided >writing performance with lock ahead itself, but might be handy in other >situations and is a pretty minor change to the API. (Just allow >blocking requests and make them synchronous, rather than not allowing >them as is the case now.) > >I still think I will push this out to a future enhancement of the >feature rather than integrate it into the current patch. > >Thoughts? Objections? > >I admit that this expansion of the API is making me question 'lock >ahead' as the name for the API itself, rather than a particular usage of >it. Perhaps that's not a direction we want to go, or perhaps we should >rename the API. (Suggestions encouraged) What about just "lock request" or "lock enqueue"? Cheers, Andreas -- Andreas Dilger Lustre Software Architect Intel High Performance Data Division From oleg.drokin at intel.com Fri Jul 31 03:54:02 2015 From: oleg.drokin at intel.com (Drokin, Oleg) Date: Fri, 31 Jul 2015 03:54:02 +0000 Subject: [lustre-devel] Lock ahead - Possible change In-Reply-To: <559FEF75.4080103@cray.com> References: <559FEF75.4080103@cray.com> Message-ID: <7A7A49C2-E8AB-4661-8991-0A23D4188D6B@intel.com> On Jul 10, 2015, at 12:14 PM, Patrick Farrell wrote: > Good morning, > > I am reluctant to make further changes to lock ahead at this point when it is ready and pending review, but I wanted to float an idea to see if it would likely meet acceptance. (I am primarily asking Jinshan and Andreas, who have been involved from the beginning.) > > Currently, lock ahead requests are asynchronous, and, because of that, non-blocking. What about the possibility of also allowing blocking, *synchronous* requests via the same API? This is potentially deadlockprone, so definitely think it through first. There are plenty of design assumptions about how there could be just one blocking lock request from a thread. Otherwise stuff like process1: request lock1, request lock2 process2: request lock3, that conflicts with lock1 and lock2. Now if lock1 is granted then lock3 is processed next and process1 is blocked with lock1 actually held until lock2 is granted - this will not end well. > This is intended as a way for a client to make a blocking lock request on a specific region of a file, rather than the whole file by taking a group lock as is currently the only option. > > This would not be used for my immediate goal of enhancing strided writing performance with lock ahead itself, but might be handy in other situations and is a pretty minor change to the API. (Just allow blocking requests and make them synchronous, rather than not allowing them as is the case now.) > > I still think I will push this out to a future enhancement of the feature rather than integrate it into the current patch. In my view this definitely should be a separate patch. Bye, Oleg