From jsimmons at infradead.org Tue Jan 7 00:02:36 2020 From: jsimmons at infradead.org (James Simmons) Date: Tue, 7 Jan 2020 00:02:36 +0000 (GMT) Subject: [lustre-devel] Lustre upstreaming status. In-Reply-To: <87sglgg9ub.fsf@notabene.neil.brown.name> References: <87sglgg9ub.fsf@notabene.neil.brown.name> Message-ID: > Hi all, > At the LUG in Houston, I said that I hoped to submit something upstream > by the end of 2019. Clearly that isn't going to happen now. > > The main reason that caused me to not even try is IPv6 support. > It became apparent to me that LNet would not be accepted until it has > working IPv6 support, and that doesn't exist yet. > I hope to put some development time into IPv6, and to have something > that works and is worth reviewing by the end of January 2020. That would be awesome. I believe the original plan was for IPv6 support for 2.14 but USDP didn't make it in for 2.13 so everything got delayed. > The other issue is that development has progressed slowly because > there is no spare review bandwidth. James has contributed a lot, and > others have helped, but reviewing patches for two code streams (OpenSFS > and Linux-upstream) turns out to be too much to ask for. > So I've decided to take a different approach. From now on I'm not > going to wait for reviews for patches going into my linux-lustre tree. > Part of my justification for this is that historically, review hasn't > really provided much promise of correctness. Patches go missing. > Random lines from patches go missing. Errors creep in in other ways. I have been going over the patches from your backport tree to find missing patches and test for regressions. I think all regressions I saw was stomped out for everything for 2.12. I'm doing full regression right now. The only bug I see now is very unique to the linux client. 2020-01-06T16:24:58.006823-05:00 ninja81.ccs.ornl.gov kernel: RIP: 0010:ll_dcompare+0x62/0xf0 [lustre] 2020-01-06T16:24:58.006880-05:00 ninja81.ccs.ornl.gov kernel: RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000002 2020-01-06T16:24:58.006934-05:00 ninja81.ccs.ornl.gov kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: 00000000ffffffff 2020-01-06T16:24:58.006992-05:00 ninja81.ccs.ornl.gov kernel: Code: 85 c0 89 c3 75 2d f6 05 c8 c7 c8 ff 20 74 09 f6 05 c2 c7 c8 ff 80 75 2b 41 f7 04 24 00 00 01 10 75 c2 49 8b 84 24 f8 00 00 00 <0f> b6 58 0c 83 e3 01 eb b1 48 83 c4 08 bb 01 00 00 00 89 d8 5b 5 d 2020-01-06T16:24:58.007051-05:00 ninja81.ccs.ornl.gov kernel: RBP: ffffc90009137cf0 R08: 0000000000000000 R09: 0000000000000000 2020-01-06T16:24:58.007105-05:00 ninja81.ccs.ornl.gov kernel: R10: 0000000000000000 R11: 000000000000000f R12: ffff888fecf4ab40 2020-01-06T16:24:58.007157-05:00 ninja81.ccs.ornl.gov kernel: RSP: 0018:ffffc9000944b950 EFLAGS: 00010246 2020-01-06T16:24:58.007216-05:00 ninja81.ccs.ornl.gov kernel: R13: 000000137118a4ee R14: ffffc90009137cf0 R15: 0000000000000000 2020-01-06T16:24:58.007270-05:00 ninja81.ccs.ornl.gov kernel: FS: 00007fb072a3a740(0000) GS:ffff88885ec00000(0000) knlGS:00000000 00000000 2020-01-06T16:24:58.007334-05:00 ninja81.ccs.ornl.gov kernel: RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000002 2020-01-06T16:24:58.007394-05:00 ninja81.ccs.ornl.gov kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: 00000000ffffffff 2020-01-06T16:24:58.007448-05:00 ninja81.ccs.ornl.gov kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 2020-01-06T16:24:58.007505-05:00 ninja81.ccs.ornl.gov kernel: CR2: 000000000000000c CR3: 00000007cd282001 CR4: 00000000001606e0 2020-01-06T16:24:58.007562-05:00 ninja81.ccs.ornl.gov kernel: RBP: ffffc9000944bcf0 R08: 0000000000000000 R09: 0000000000000000 2020-01-06T16:24:58.007616-05:00 ninja81.ccs.ornl.gov kernel: Call Trace: 2020-01-06T16:24:58.007669-05:00 ninja81.ccs.ornl.gov kernel: R10: 0000000000000000 R11: 000000000000000f R12: ffff8887d0562640 2020-01-06T16:24:58.007727-05:00 ninja81.ccs.ornl.gov kernel: R13: 000000137118a4ee R14: ffffc9000944bcf0 R15: 0000000000000000 2020-01-06T16:24:58.007780-05:00 ninja81.ccs.ornl.gov kernel: __d_lookup_rcu+0x183/0x1e0 2020-01-06T16:24:58.007832-05:00 ninja81.ccs.ornl.gov kernel: __d_lookup_rcu+0x183/0x1e0 2020-01-06T16:24:58.007885-05:00 ninja81.ccs.ornl.gov kernel: d_alloc_parallel+0x15e/0x7c0 2020-01-06T16:24:58.007936-05:00 ninja81.ccs.ornl.gov kernel: d_alloc_parallel+0x15e/0x7c0 2020-01-06T16:24:58.007999-05:00 ninja81.ccs.ornl.gov kernel: ? __lookup_slow+0xf5/0x1d0 2020-01-06T16:24:58.008056-05:00 ninja81.ccs.ornl.gov kernel: ? __lookup_slow+0xf5/0x1d0 2020-01-06T16:24:58.008112-05:00 ninja81.ccs.ornl.gov kernel: ? wake_up_q+0x80/0x80 2020-01-06T16:24:58.008169-05:00 ninja81.ccs.ornl.gov kernel: ? _raw_spin_unlock_irq+0x34/0x50 This might be resolved with https://review.whamcloud.com/#/c/24175 I also have started working through the 2.13 release. I'm up to 2.12.54 but no heavy testing as of yet of those patches. Once I'm done testing 2.12 in depth I can push quickly through 2.13 and even sync up to OpenSFS branch. I think the back porting work can be wrapped up by the end of the month. > Instead, I am developing a tool which will compare OpenSFS lustre > and Linux-lustre and report relevant differences. I have a prototype > working, and it is helping me to find missing patches and parts of > patches in both trees. > > I will continue to submit patches to gerrit to bring OpenSFS closer to > my linux tree when that is needed, and will apply patches from OpenSFS > to my tree without extra review when that it needed. > > When the time comes to submit upstream, I plan to present the tool so > that other developers can confirm that what I am submitting is > functionally equivalent to OpenSFS, and so that we can ensure the > equivalence remains. > > Consequently my "lustre" branch will jump forward to v5.4 soon, > probably tomorrow, and will remain close to mainline. > I will also be growing my list of outstanding OpenSFS patches > (currently about 100, many of which haven't been submitted to gerrit > yet) and will hope to get those reviewed. Any changes that result from > the review will be detected by my comparison script when the patch > lands, and I'll update linux-lustre to match. > > My new goal for upstream submission is the end of Q1-2020. This is > probably a bit optimistic, but gives me a suitable focus. I believe having it ready for LUG 2020 is a reasonable goal. From adilger at whamcloud.com Tue Jan 7 01:53:59 2020 From: adilger at whamcloud.com (Andreas Dilger) Date: Tue, 7 Jan 2020 01:53:59 +0000 Subject: [lustre-devel] Lustre upstreaming status. In-Reply-To: References: <87sglgg9ub.fsf@notabene.neil.brown.name> Message-ID: <8E3FD560-7730-49D7-B5F5-0EC2BD08E149@whamcloud.com> On Jan 6, 2020, at 17:02, James Simmons > wrote: Hi all, At the LUG in Houston, I said that I hoped to submit something upstream by the end of 2019. Clearly that isn't going to happen now. The main reason that caused me to not even try is IPv6 support. It became apparent to me that LNet would not be accepted until it has working IPv6 support, and that doesn't exist yet. I hope to put some development time into IPv6, and to have something that works and is worth reviewing by the end of January 2020. That would be awesome. I believe the original plan was for IPv6 support for 2.14 but USDP didn't make it in for 2.13 so everything got delayed. The other issue is that development has progressed slowly because there is no spare review bandwidth. James has contributed a lot, and others have helped, but reviewing patches for two code streams (OpenSFS and Linux-upstream) turns out to be too much to ask for. So I've decided to take a different approach. From now on I'm not going to wait for reviews for patches going into my linux-lustre tree. Part of my justification for this is that historically, review hasn't really provided much promise of correctness. Patches go missing. Random lines from patches go missing. Errors creep in in other ways. I have been going over the patches from your backport tree to find missing patches and test for regressions. I think all regressions I saw was stomped out for everything for 2.12. I'm doing full regression right now. The only bug I see now is very unique to the linux client. [snip] I also have started working through the 2.13 release. I'm up to 2.12.54 but no heavy testing as of yet of those patches. Once I'm done testing 2.12 in depth I can push quickly through 2.13 and even sync up to OpenSFS branch. I think the back porting work can be wrapped up by the end of the month. I thought the goal was to stop at 2.12.x (following the b2_12 branch to get important fixes) and try to get that included upstream? That gives a good point-in-time to track, and ensures that the upstream code is aligned with a relatively stable version of the code. It also has the major benefit that 2.12 is an LTS branch and we will need to keep compatibility with that for a long time, which isn't always true of intermediate releases. Cheers, Andreas -- Andreas Dilger Principal Lustre Architect Whamcloud -------------- next part -------------- An HTML attachment was scrubbed... URL: From adilger at whamcloud.com Tue Jan 7 02:24:22 2020 From: adilger at whamcloud.com (Andreas Dilger) Date: Tue, 7 Jan 2020 02:24:22 +0000 Subject: [lustre-devel] Lustre upstreaming status. In-Reply-To: <8E3FD560-7730-49D7-B5F5-0EC2BD08E149@whamcloud.com> References: <87sglgg9ub.fsf@notabene.neil.brown.name> <8E3FD560-7730-49D7-B5F5-0EC2BD08E149@whamcloud.com> Message-ID: <7CDEC7E6-F64B-4014-8CAB-EA77EB9AA0EA@whamcloud.com> Replying to my own email to get email addresses correct. On Jan 6, 2020, at 18:53, Andreas Dilger > wrote: On Jan 6, 2020, at 17:02, James Simmons > wrote: Hi all, At the LUG in Houston, I said that I hoped to submit something upstream by the end of 2019. Clearly that isn't going to happen now. The main reason that caused me to not even try is IPv6 support. It became apparent to me that LNet would not be accepted until it has working IPv6 support, and that doesn't exist yet. I hope to put some development time into IPv6, and to have something that works and is worth reviewing by the end of January 2020. That would be awesome. I believe the original plan was for IPv6 support for 2.14 but USDP didn't make it in for 2.13 so everything got delayed. The other issue is that development has progressed slowly because there is no spare review bandwidth. James has contributed a lot, and others have helped, but reviewing patches for two code streams (OpenSFS and Linux-upstream) turns out to be too much to ask for. So I've decided to take a different approach. From now on I'm not going to wait for reviews for patches going into my linux-lustre tree. Part of my justification for this is that historically, review hasn't really provided much promise of correctness. Patches go missing. Random lines from patches go missing. Errors creep in in other ways. I have been going over the patches from your backport tree to find missing patches and test for regressions. I think all regressions I saw was stomped out for everything for 2.12. I'm doing full regression right now. The only bug I see now is very unique to the linux client. [snip] I also have started working through the 2.13 release. I'm up to 2.12.54 but no heavy testing as of yet of those patches. Once I'm done testing 2.12 in depth I can push quickly through 2.13 and even sync up to OpenSFS branch. I think the back porting work can be wrapped up by the end of the month. I thought the goal was to stop at 2.12.x (following the b2_12 branch to get important fixes) and try to get that included upstream? That gives a good point-in-time to track, and ensures that the upstream code is aligned with a relatively stable version of the code. It also has the major benefit that 2.12 is an LTS branch and we will need to keep compatibility with that for a long time, which isn't always true of intermediate releases. Cheers, Andreas -- Andreas Dilger Principal Lustre Architect Whamcloud _______________________________________________ lustre-devel mailing list lustre-devel at lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org Cheers, Andreas -- Andreas Dilger Principal Lustre Architect Whamcloud -------------- next part -------------- An HTML attachment was scrubbed... URL: From neilb at suse.de Tue Jan 7 04:05:47 2020 From: neilb at suse.de (NeilBrown) Date: Tue, 07 Jan 2020 15:05:47 +1100 Subject: [lustre-devel] Lustre upstreaming status. In-Reply-To: References: <87sglgg9ub.fsf@notabene.neil.brown.name> Message-ID: <87zhezdi78.fsf@notabene.neil.brown.name> On Tue, Jan 07 2020, James Simmons wrote: > > I have been going over the patches from your backport tree to find > missing patches and test for regressions. I think all regressions I > saw was stomped out for everything for 2.12. I'm doing full regression > right now. The only bug I see now is very unique to the linux client. > > 2020-01-06T16:24:58.006823-05:00 ninja81.ccs.ornl.gov kernel: RIP: > 0010:ll_dcompare+0x62/0xf0 [lustre] > 2020-01-06T16:24:58.006880-05:00 ninja81.ccs.ornl.gov kernel: RAX: > 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000002 > 2020-01-06T16:24:58.006934-05:00 ninja81.ccs.ornl.gov kernel: RDX: > 0000000000000001 RSI: 0000000000000001 RDI: 00000000ffffffff > 2020-01-06T16:24:58.006992-05:00 ninja81.ccs.ornl.gov kernel: Code: 85 c0 > 89 c3 75 2d f6 05 c8 c7 c8 ff 20 74 09 f6 05 c2 c7 c8 ff > 80 75 2b 41 f7 04 24 00 00 01 10 75 c2 49 8b 84 24 f8 00 00 00 <0f> b6 58 > 0c 83 e3 01 eb b1 48 83 c4 08 bb 01 00 00 00 89 d8 5b 5 > d The crashing code here is 22: 49 8b 84 24 f8 00 00 mov 0xf8(%r12),%rax 29: 00 2a:* 0f b6 58 0c movzbl 0xc(%rax),%ebx <-- trapping instruction 2e: 83 e3 01 and $0x1,%ebx 31: eb b1 jmp 0xffffffffffffffe4 The only place that could happen in ll_dcompare is the if (d_lustre_invalid(dentry)) return 1; call at the end. ll_d2d(dentry) must be NULL. in OpenSFS lustre, d_lustre_invalid() protects against that being NULL. Linux/lustre lost that protection in Commit 7126bc2e8d60 ("lustre: switch to use of ->d_init()") because it really shouldn't need it. Have you reported this to me before? It seems awfully familiar. > > This might be resolved with > > https://review.whamcloud.com/#/c/24175 That would certainly resolve this particular symptom. I don't know if it is correct though ... maybe. > > I believe having it ready for LUG 2020 is a reasonable goal. Sounds like a nice goal. Thanks, NeilBrown -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 832 bytes Desc: not available URL: From neilb at suse.de Tue Jan 7 04:32:20 2020 From: neilb at suse.de (NeilBrown) Date: Tue, 07 Jan 2020 15:32:20 +1100 Subject: [lustre-devel] Lustre upstreaming status. In-Reply-To: <8E3FD560-7730-49D7-B5F5-0EC2BD08E149@whamcloud.com> References: <87sglgg9ub.fsf@notabene.neil.brown.name> <8E3FD560-7730-49D7-B5F5-0EC2BD08E149@whamcloud.com> Message-ID: <87woa3dgyz.fsf@notabene.neil.brown.name> (sorry for including the intel addresses in the original :-( ) On Tue, Jan 07 2020, Andreas Dilger wrote: > > I thought the goal was to stop at 2.12.x (following the b2_12 branch to get > important fixes) and try to get that included upstream? That gives a good > point-in-time to track, and ensures that the upstream code is aligned with > a relatively stable version of the code. It also has the major benefit that > 2.12 is an LTS branch and we will need to keep compatibility with that for > a long time, which isn't always true of intermediate releases. That was suggested at Houston, but I was against it. I think I said I would considered it going all the way to 'master' looked like too much work - but it didn't. Upstream Linux is not a place for old code. It is a place for developing new code. If we don't submit the latest code to Linux, people will want to know why. Bug fixes will flow into the 'stable' releases with little or no effort from the lustre team. For the "community" face of lustre, this is really all you need. Compatability is import, but there should be no need to break it. There are feature flags (or similar) in the protocol, and a modest amount of care with using them should avoid obvious breakage. You don't need to test every single combination - you can assume that people who care are doing that testing. If someone reports a compatability regression, we need to fix it. But if no-one reports one, then no fix is needed. Obviously you will test the combinations that you support for your customers - just as you do now. We must have a long term goal of doing all (kernel) development against upstream, and have it appear upstream first. The lustre-release from whamcloud will eventually contain no stand-alone kernel code. Just user-space code and a selection of backported patches (maybe). So think of upstream-linux much the same way that you currently think about "master". My focus is to keep the two in-sync, so that thinking this way will be natural. Thanks, NeilBrown -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 832 bytes Desc: not available URL: From slas+lustre at pdc.kth.se Tue Jan 7 17:48:02 2020 From: slas+lustre at pdc.kth.se (Mattias Claesson) Date: Tue, 7 Jan 2020 18:48:02 +0100 Subject: [lustre-devel] LU-12759 not in 2.12 In-Reply-To: <94C86C98-64D5-445C-98E8-F695C4AD163D@ddn.com> References: <0377f0de-25e8-db29-e59a-ef885bcd1e7d@pdc.kth.se> <94C86C98-64D5-445C-98E8-F695C4AD163D@ddn.com> Message-ID: Hi Peter, Sorry for not answering earlier. > The short answer is because nobody has flagged it for inclusion. Do > you consider that there would still be value to include it now that a > fix for LU-12120 has landed? Ok. I'm not too familiar with what should go into which version stream, but since 2.12 is the current LTS version, and I would consider this (set parameters are "forgotten") a separate bug in itself, it seems natural to me to include it there also. The fewer patches we need to remember to apply when using the LTS version the better, I would say. And I assume there are other cases also when one would want to turn off 'grant_shrink', even when not trying to mitigate LU-12120, since the feature was already (mostly) implemented :) Mattias PDC-staff From neilb at suse.de Wed Jan 8 21:18:35 2020 From: neilb at suse.de (NeilBrown) Date: Thu, 09 Jan 2020 08:18:35 +1100 Subject: [lustre-devel] Lustre upstreaming status. In-Reply-To: <87zhezdi78.fsf@notabene.neil.brown.name> References: <87sglgg9ub.fsf@notabene.neil.brown.name> <87zhezdi78.fsf@notabene.neil.brown.name> Message-ID: <87blrdd4us.fsf@notabene.neil.brown.name> On Tue, Jan 07 2020, NeilBrown wrote: > On Tue, Jan 07 2020, James Simmons wrote: >> >> I have been going over the patches from your backport tree to find >> missing patches and test for regressions. I think all regressions I >> saw was stomped out for everything for 2.12. I'm doing full regression >> right now. The only bug I see now is very unique to the linux client. >> >> 2020-01-06T16:24:58.006823-05:00 ninja81.ccs.ornl.gov kernel: RIP: >> 0010:ll_dcompare+0x62/0xf0 [lustre] >> 2020-01-06T16:24:58.006880-05:00 ninja81.ccs.ornl.gov kernel: RAX: >> 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000002 >> 2020-01-06T16:24:58.006934-05:00 ninja81.ccs.ornl.gov kernel: RDX: >> 0000000000000001 RSI: 0000000000000001 RDI: 00000000ffffffff >> 2020-01-06T16:24:58.006992-05:00 ninja81.ccs.ornl.gov kernel: Code: 85 c0 >> 89 c3 75 2d f6 05 c8 c7 c8 ff 20 74 09 f6 05 c2 c7 c8 ff >> 80 75 2b 41 f7 04 24 00 00 01 10 75 c2 49 8b 84 24 f8 00 00 00 <0f> b6 58 >> 0c 83 e3 01 eb b1 48 83 c4 08 bb 01 00 00 00 89 d8 5b 5 >> d > > The crashing code here is > > 22: 49 8b 84 24 f8 00 00 mov 0xf8(%r12),%rax > 29: 00 > 2a:* 0f b6 58 0c movzbl 0xc(%rax),%ebx <-- trapping instruction > 2e: 83 e3 01 and $0x1,%ebx > 31: eb b1 jmp 0xffffffffffffffe4 > > The only place that could happen in ll_dcompare is the > > if (d_lustre_invalid(dentry)) > return 1; > call at the end. > ll_d2d(dentry) must be NULL. > > in OpenSFS lustre, d_lustre_invalid() protects against that being NULL. > Linux/lustre lost that protection in > > Commit 7126bc2e8d60 ("lustre: switch to use of ->d_init()") > > because it really shouldn't need it. > > Have you reported this to me before? It seems awfully familiar. I remember now. You have reported it. The problem is de->d_fsdata = NULL; in ll_release(). I'll remove that. Thanks, NeilBrown -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 832 bytes Desc: not available URL: From quentin.bouget at cea.fr Wed Jan 22 11:44:49 2020 From: quentin.bouget at cea.fr (quentin.bouget at cea.fr) Date: Wed, 22 Jan 2020 12:44:49 +0100 Subject: [lustre-devel] upgrade Gerrit's UI? Message-ID: Hi all, I was wondering if Whamcloud had any plan to upgrade its Gerrit's UI. I am only asking because when you compare two different versions of the same patch, it often happens that some of the differences are not part of the patch itself, but rather introduced by other commits that have landed upstream between the two versions. The current UI makes those indistinguishable from "actual" changes. The latest UI -- which can be seen in action on GerritHub -- uses a different color to highlight changes that are not directly introduced by the patch being reviewed. There are other perks to using the latest versions (such as reviewing files directly from a patch's "main page"), and of course, there are also a few things that some might not find to their liking, but I think the pros outweigh the cons. Cheers, Quentin -------------- next part -------------- An HTML attachment was scrubbed... URL: From quentin.bouget at cea.fr Wed Jan 22 11:49:43 2020 From: quentin.bouget at cea.fr (quentin.bouget at cea.fr) Date: Wed, 22 Jan 2020 12:49:43 +0100 Subject: [lustre-devel] [PROVENANCE INTERNET] upgrade Gerrit's UI? In-Reply-To: <20616_1579693506_5E2835C1_20616_2766_1_b5147dfa-ca7a-5fea-623c-dc2cfd407bfb@cea.fr> References: <20616_1579693506_5E2835C1_20616_2766_1_b5147dfa-ca7a-5fea-623c-dc2cfd407bfb@cea.fr> Message-ID: <432dbbab-523f-de60-4f19-88c9bc9cff83@cea.fr> On 22/01/2020 12:44, quentin.bouget at cea.fr wrote: > > Hi all, > > I was wondering if Whamcloud had any plan to upgrade its Gerrit's UI. > > I am only asking because when you compare two different versions of > the same patch, it often happens that some of the differences are not > part of the patch itself, but rather introduced by other commits that > have landed upstream between the two versions. The current UI makes > those indistinguishable from "actual" changes. > > The latest UI -- which can be seen in action on GerritHub > -- uses a different color to highlight changes > that are not directly introduced by the patch being reviewed. > > There are other perks to using the latest versions (such as reviewing > files directly from a patch's "main page"), and of course, there are > also a few things that some might not find to their liking, but I > think the pros outweigh the cons. > > Cheers, > Quentin > The link above is broken (I must have typed it wrong) : GerritHub -------------- next part -------------- An HTML attachment was scrubbed... URL: From adilger at whamcloud.com Thu Jan 23 22:05:50 2020 From: adilger at whamcloud.com (Andreas Dilger) Date: Thu, 23 Jan 2020 22:05:50 +0000 Subject: [lustre-devel] upgrade Gerrit's UI? In-Reply-To: <432dbbab-523f-de60-4f19-88c9bc9cff83@cea.fr> References: <20616_1579693506_5E2835C1_20616_2766_1_b5147dfa-ca7a-5fea-623c-dc2cfd407bfb@cea.fr> <432dbbab-523f-de60-4f19-88c9bc9cff83@cea.fr> Message-ID: <22D61ADD-B986-4C39-ACC5-58489A587263@whamcloud.com> I don't think anyone is _against_ updating Gerrit, but as with all changes this has to be weighed against other tasks. Since Gerrit is key to all of the Lustre developers, we try to avoid changes there unless needed for stability or security. That said, I agree the "external changes visible during patch comparison" issue you mention is definitely something I've found annoying as well, so having that fixed would be useful. I can't guarantee anything, but will bring this up in our next meeting. On Jan 22, 2020, at 04:49, quentin.bouget at cea.fr wrote: On 22/01/2020 12:44, quentin.bouget at cea.fr wrote: Hi all, I was wondering if Whamcloud had any plan to upgrade its Gerrit's UI. I am only asking because when you compare two different versions of the same patch, it often happens that some of the differences are not part of the patch itself, but rather introduced by other commits that have landed upstream between the two versions. The current UI makes those indistinguishable from "actual" changes. The latest UI -- which can be seen in action on GerritHub -- uses a different color to highlight changes that are not directly introduced by the patch being reviewed. There are other perks to using the latest versions (such as reviewing files directly from a patch's "main page"), and of course, there are also a few things that some might not find to their liking, but I think the pros outweigh the cons. Cheers, Quentin The link above is broken (I must have typed it wrong) : GerritHub _______________________________________________ lustre-devel mailing list lustre-devel at lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org Cheers, Andreas -- Andreas Dilger Principal Lustre Architect Whamcloud -------------- next part -------------- An HTML attachment was scrubbed... URL: From quentin.bouget at cea.fr Fri Jan 24 08:47:07 2020 From: quentin.bouget at cea.fr (quentin.bouget at cea.fr) Date: Fri, 24 Jan 2020 09:47:07 +0100 Subject: [lustre-devel] upgrade Gerrit's UI? In-Reply-To: <22D61ADD-B986-4C39-ACC5-58489A587263@whamcloud.com> References: <20616_1579693506_5E2835C1_20616_2766_1_b5147dfa-ca7a-5fea-623c-dc2cfd407bfb@cea.fr> <432dbbab-523f-de60-4f19-88c9bc9cff83@cea.fr> <22D61ADD-B986-4C39-ACC5-58489A587263@whamcloud.com> Message-ID: On 23/01/2020 23:05, Andreas Dilger wrote: > I don't think anyone is _against_ updating Gerrit, but as with all > changes this has to be weighed against other tasks.  Since Gerrit is > key to all of the Lustre developers, we try to avoid changes there > unless needed for stability or security. > > That said, I agree the "external changes visible during patch > comparison" issue you mention is definitely something I've found > annoying as well, so having that fixed would be useful. > > I can't guarantee anything, but will bring this up in our next meeting. Thanks ! I understand. Quentin From nangelinas at cray.com Tue Jan 28 18:23:18 2020 From: nangelinas at cray.com (Nikitas Angelinas) Date: Tue, 28 Jan 2020 18:23:18 +0000 Subject: [lustre-devel] New slack chat room for linux client work In-Reply-To: References: <88A13941-47CB-4D08-8CA8-4FD041487322@cray.com> Message-ID: Hi James, Could you please allow access to the cray.com and hpe.com domains for this channel? Cheers, Nikitas On 7/6/18, 4:18 PM, "lustre-devel on behalf of James Simmons" wrote: > James, Slack tells me: > "Don't have an @whamcloud.com email address? > Contact your Workspace Administrator for an invitation." Yeah slack doesn't seem to be a open to anyone platform. You have to ask the admin (me in this case) if you can join. I can let people in the same domain like whamcloud.com or cray.com to join. > > -Cory > > -- > > On 7/6/18, 9:34 AM, "lustre-devel on behalf of James Simmons" wrote: > > > Some people have asked for a chat room since it can be difficult > to filter between the mixture of patches and technical discusssions > going on here. If people are interested to join the site is below > > https://lustreupstreamclient.slack.com/messages/CAV4QF0ER/ > _______________________________________________ > lustre-devel mailing list > lustre-devel at lists.lustre.org > http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org > > > From anna.fuchs at informatik.uni-hamburg.de Tue Jan 28 19:37:34 2020 From: anna.fuchs at informatik.uni-hamburg.de (Anna Fuchs) Date: Tue, 28 Jan 2020 20:37:34 +0100 Subject: [lustre-devel] Gerrit option - build only one backend Message-ID: <1580240254.12827.0@informatik.uni-hamburg.de> Dear all, I am wondering why the Gerrit option "fstype=zfs" in my commit does not influence the build process - both targets, ldiskfs and zfs are built: checking whether to build ldiskfs... yes checking whether to enable zfs... yes The option seems to affect only the following tests. Is this behavior intended? Is there any option to build and test only one backend? Regards Anna -------------- next part -------------- An HTML attachment was scrubbed... URL: From adilger at whamcloud.com Tue Jan 28 20:36:24 2020 From: adilger at whamcloud.com (Andreas Dilger) Date: Tue, 28 Jan 2020 20:36:24 +0000 Subject: [lustre-devel] Gerrit option - build only one backend In-Reply-To: <1580240254.12827.0@informatik.uni-hamburg.de> References: <1580240254.12827.0@informatik.uni-hamburg.de> Message-ID: <102CE8C0-6E14-44C1-9189-FF8A0C50EE56@whamcloud.com> The fstype option affects the testing, because this is "Test-Parameters:", not "Build-Parameters:". :-) The zfs vs. ldiskfs option mostly only affects the osd-ldiskfs and oss-zfs, and the matching mount.lustre helpers. All of the rest of the code should be independent of the backing filesystem. Cheers, Andreas > On Jan 28, 2020, at 12:37, Anna Fuchs wrote: > >  > Dear all, > > I am wondering why the Gerrit option "fstype=zfs" in my commit does not influence the build process - both targets, ldiskfs and zfs are built: > > checking whether to build ldiskfs... yes > checking whether to enable zfs... yes > > The option seems to affect only the following tests. Is this behavior intended? > Is there any option to build and test only one backend? > > Regards > Anna > _______________________________________________ > lustre-devel mailing list > lustre-devel at lists.lustre.org > http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org From jsimmons at infradead.org Wed Jan 29 00:49:26 2020 From: jsimmons at infradead.org (James Simmons) Date: Wed, 29 Jan 2020 00:49:26 +0000 (GMT) Subject: [lustre-devel] New slack chat room for linux client work In-Reply-To: References: <88A13941-47CB-4D08-8CA8-4FD041487322@cray.com> Message-ID: > Hi James, > > Could you please allow access to the cray.com and hpe.com domains for this channel? I tried cray.com in the past but it conflicted with an internal slack channel cray uses. The hpe domain might be okay. Cory would that be fine? > Cheers, > Nikitas > > On 7/6/18, 4:18 PM, "lustre-devel on behalf of James Simmons" wrote: > > > > James, Slack tells me: > > "Don't have an @whamcloud.com email address? > > Contact your Workspace Administrator for an invitation." > > Yeah slack doesn't seem to be a open to anyone platform. You have to ask > the admin (me in this case) if you can join. I can let people in the same > domain like whamcloud.com or cray.com to join. > > > > > -Cory > > > > -- > > > > On 7/6/18, 9:34 AM, "lustre-devel on behalf of James Simmons" wrote: > > > > > > Some people have asked for a chat room since it can be difficult > > to filter between the mixture of patches and technical discusssions > > going on here. If people are interested to join the site is below > > > > https://lustreupstreamclient.slack.com/messages/CAV4QF0ER/ > > _______________________________________________ > > lustre-devel mailing list > > lustre-devel at lists.lustre.org > > http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org > > > > > > > > From anna.fuchs at informatik.uni-hamburg.de Wed Jan 29 14:18:26 2020 From: anna.fuchs at informatik.uni-hamburg.de (Anna Fuchs) Date: Wed, 29 Jan 2020 15:18:26 +0100 Subject: [lustre-devel] Gerrit option - build only one backend In-Reply-To: <102CE8C0-6E14-44C1-9189-FF8A0C50EE56@whamcloud.com> References: <1580240254.12827.0@informatik.uni-hamburg.de> <102CE8C0-6E14-44C1-9189-FF8A0C50EE56@whamcloud.com> Message-ID: I see, but are there any "build-parameters" I can set? I haven't seen anything like that in another commits. I want to prevent the osd-ldiskfs to be built since my stuff involves experimental interface changes for the backend and this build will fail at the moment. None of the tests will run if any build fails, am I right? I wanted to have first some reviews before I change everything. So either I (currently unnecessarily) adjust the ldiskfs path just for the build to pass or.. is there any or? Thanks, Anna Am 28.01.2020 um 21:36 schrieb Andreas Dilger: > The fstype option affects the testing, because this is "Test-Parameters:", not "Build-Parameters:". :-) > > The zfs vs. ldiskfs option mostly only affects the osd-ldiskfs and oss-zfs, and the matching mount.lustre helpers. All of the rest of the code should be independent of the backing filesystem. > > Cheers, Andreas > >> On Jan 28, 2020, at 12:37, Anna Fuchs wrote: >> >>  >> Dear all, >> >> I am wondering why the Gerrit option "fstype=zfs" in my commit does not influence the build process - both targets, ldiskfs and zfs are built: >> >> checking whether to build ldiskfs... yes >> checking whether to enable zfs... yes >> >> The option seems to affect only the following tests. Is this behavior intended? >> Is there any option to build and test only one backend? >> >> Regards >> Anna >> _______________________________________________ >> lustre-devel mailing list >> lustre-devel at lists.lustre.org >> http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org -- Anna Fuchs Universität Hamburg https://wr.informatik.uni-hamburg.de anna.fuchs at informatik.uni-hamburg.de https://wr.informatik.uni-hamburg.de/people/anna_fuchs From adilger at whamcloud.com Wed Jan 29 17:51:06 2020 From: adilger at whamcloud.com (Andreas Dilger) Date: Wed, 29 Jan 2020 17:51:06 +0000 Subject: [lustre-devel] Gerrit option - build only one backend In-Reply-To: References: <1580240254.12827.0@informatik.uni-hamburg.de> <102CE8C0-6E14-44C1-9189-FF8A0C50EE56@whamcloud.com>, Message-ID: You could potentially include a change in your patch series to always disable ldiskfs building, probably in config/lustre-build-ldiskfs.m4. It would be best if that was a separate patch with a description like "osd-ldiskfs: permanently disable building" and tagged "forbuildonly" so that it cannot be landed. Cheers, Andreas > On Jan 29, 2020, at 07:18, Anna Fuchs wrote: > > I see, but are there any "build-parameters" I can set? I haven't seen anything like that in another commits. > > I want to prevent the osd-ldiskfs to be built since my stuff involves experimental interface changes for the backend and this build will fail at the moment. > > None of the tests will run if any build fails, am I right? I wanted to have first some reviews before I change everything. > > So either I (currently unnecessarily) adjust the ldiskfs path just for the build to pass or.. is there any or? > > Thanks, > > Anna > >> Am 28.01.2020 um 21:36 schrieb Andreas Dilger: >> The fstype option affects the testing, because this is "Test-Parameters:", not "Build-Parameters:". :-) >> >> The zfs vs. ldiskfs option mostly only affects the osd-ldiskfs and oss-zfs, and the matching mount.lustre helpers. All of the rest of the code should be independent of the backing filesystem. >> >> Cheers, Andreas >> >>>> On Jan 28, 2020, at 12:37, Anna Fuchs wrote: >>> >>>  >>> Dear all, >>> >>> I am wondering why the Gerrit option "fstype=zfs" in my commit does not influence the build process - both targets, ldiskfs and zfs are built: >>> >>> checking whether to build ldiskfs... yes >>> checking whether to enable zfs... yes >>> >>> The option seems to affect only the following tests. Is this behavior intended? >>> Is there any option to build and test only one backend? >>> >>> Regards >>> Anna >>> _______________________________________________ >>> lustre-devel mailing list >>> lustre-devel at lists.lustre.org >>> http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org > > -- > Anna Fuchs > Universität Hamburg > https://wr.informatik.uni-hamburg.de > > anna.fuchs at informatik.uni-hamburg.de > https://wr.informatik.uni-hamburg.de/people/anna_fuchs > From cory.spitz at hpe.com Thu Jan 30 23:04:11 2020 From: cory.spitz at hpe.com (Spitz, Cory James) Date: Thu, 30 Jan 2020 23:04:11 +0000 Subject: [lustre-devel] New slack chat room for linux client work In-Reply-To: References: <88A13941-47CB-4D08-8CA8-4FD041487322@cray.com> Message-ID: <4E2F0E72-5D9D-46F7-8A8F-3B1B446D4CE5@hpe.com> Hi, James. Cray folks are transitioning over to @hpe.com addresses. So, yeah, it'd be great if you can allow hpe.com. -Cory On 1/28/20, 6:49 PM, "James Simmons" wrote: > Hi James, > > Could you please allow access to the cray.com and hpe.com domains for this channel? I tried cray.com in the past but it conflicted with an internal slack channel cray uses. The hpe domain might be okay. Cory would that be fine? > Cheers, > Nikitas > > On 7/6/18, 4:18 PM, "lustre-devel on behalf of James Simmons" wrote: > > > > James, Slack tells me: > > "Don't have an @whamcloud.com email address? > > Contact your Workspace Administrator for an invitation." > > Yeah slack doesn't seem to be a open to anyone platform. You have to ask > the admin (me in this case) if you can join. I can let people in the same > domain like whamcloud.com or cray.com to join. > > > > > -Cory > > > > -- > > > > On 7/6/18, 9:34 AM, "lustre-devel on behalf of James Simmons" wrote: > > > > > > Some people have asked for a chat room since it can be difficult > > to filter between the mixture of patches and technical discusssions > > going on here. If people are interested to join the site is below > > > > https://lustreupstreamclient.slack.com/messages/CAV4QF0ER/ > > _______________________________________________ > > lustre-devel mailing list > > lustre-devel at lists.lustre.org > > http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org > > > > > > > > From pjones at whamcloud.com Fri Jan 31 12:53:21 2020 From: pjones at whamcloud.com (Peter Jones) Date: Fri, 31 Jan 2020 12:53:21 +0000 Subject: [lustre-devel] LU-12759 not in 2.12 In-Reply-To: References: <0377f0de-25e8-db29-e59a-ef885bcd1e7d@pdc.kth.se> <94C86C98-64D5-445C-98E8-F695C4AD163D@ddn.com> Message-ID: <4965B65D-E55A-4285-921D-BCEDC57099B0@ddn.com> Ok, well, this issue is included in the 2.12.4 RC1 build that is presently undergoing release testing. On 2020-01-07, 9:48 AM, "Mattias Claesson" wrote: Hi Peter, Sorry for not answering earlier. > The short answer is because nobody has flagged it for inclusion. Do > you consider that there would still be value to include it now that a > fix for LU-12120 has landed? Ok. I'm not too familiar with what should go into which version stream, but since 2.12 is the current LTS version, and I would consider this (set parameters are "forgotten") a separate bug in itself, it seems natural to me to include it there also. The fewer patches we need to remember to apply when using the LTS version the better, I would say. And I assume there are other cases also when one would want to turn off 'grant_shrink', even when not trying to mitigate LU-12120, since the feature was already (mostly) implemented :) Mattias PDC-staff From anna.fuchs at informatik.uni-hamburg.de Fri Jan 31 17:29:35 2020 From: anna.fuchs at informatik.uni-hamburg.de (Anna Fuchs) Date: Fri, 31 Jan 2020 18:29:35 +0100 Subject: [lustre-devel] Gerrit option - build only one backend In-Reply-To: References: <1580240254.12827.0@informatik.uni-hamburg.de> <102CE8C0-6E14-44C1-9189-FF8A0C50EE56@whamcloud.com> Message-ID: <1580491775.5117.0@informatik.uni-hamburg.de> Ok, I will see which way is better, thanks a lot. Regards Anna On Wed, Jan 29, 2020 at 6:51 PM, Andreas Dilger wrote: > You could potentially include a change in your patch series to always > disable ldiskfs building, probably in config/lustre-build-ldiskfs.m4. > > It would be best if that was a separate patch with a description like > "osd-ldiskfs: permanently disable building" and tagged "forbuildonly" > so that it cannot be landed. > > Cheers, Andreas > >> On Jan 29, 2020, at 07:18, Anna Fuchs >> > > wrote: >> >> I see, but are there any "build-parameters" I can set? I haven't >> seen anything like that in another commits. >> >> I want to prevent the osd-ldiskfs to be built since my stuff >> involves experimental interface changes for the backend and this >> build will fail at the moment. >> >> None of the tests will run if any build fails, am I right? I wanted >> to have first some reviews before I change everything. >> >> So either I (currently unnecessarily) adjust the ldiskfs path just >> for the build to pass or.. is there any or? >> >> Thanks, >> >> Anna >> >>> Am 28.01.2020 um 21:36 schrieb Andreas Dilger: >>> The fstype option affects the testing, because this is >>> "Test-Parameters:", not "Build-Parameters:". :-) >>> >>> The zfs vs. ldiskfs option mostly only affects the osd-ldiskfs and >>> oss-zfs, and the matching mount.lustre helpers. All of the rest of >>> the code should be independent of the backing filesystem. >>> >>> Cheers, Andreas >>> >>>>> On Jan 28, 2020, at 12:37, Anna Fuchs >>>>> >>>> > wrote: >>>> >>>>  >>>> Dear all, >>>> >>>> I am wondering why the Gerrit option "fstype=zfs" in my commit >>>> does not influence the build process - both targets, ldiskfs and >>>> zfs are built: >>>> >>>> checking whether to build ldiskfs... yes >>>> checking whether to enable zfs... yes >>>> >>>> The option seems to affect only the following tests. Is this >>>> behavior intended? >>>> Is there any option to build and test only one backend? >>>> >>>> Regards >>>> Anna >>>> _______________________________________________ >>>> lustre-devel mailing list >>>> lustre-devel at lists.lustre.org >>>> >>>> >> >> -- >> Anna Fuchs >> Universität Hamburg >> https://wr.informatik.uni-hamburg.de >> >> >> anna.fuchs at informatik.uni-hamburg.de >> >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: