From pjones at whamcloud.com Thu Dec 5 15:30:13 2019 From: pjones at whamcloud.com (Peter Jones) Date: Thu, 5 Dec 2019 15:30:13 +0000 Subject: [lustre-devel] Lustre 2.13.0 released Message-ID: <11EBD557-E6ED-4E4B-90F2-88086EAFE1DC@ddn.com> We are pleased to announce that the Lustre 2.13.0 Release has been declared GA and is available for download . You can also grab the source from git This major release includes new features: Persistent Client Cache allows Lustre to utilize client-local storage such as NVMe or NVRAM as part of the filesystem namespace. Clients can cache the data of newly created or existing files in a locally-mounted cache filesystem (e.g. ext4) for exclusive access at local filesystem speeds, and data migrates transparently into the global filesystem when accessed by other clients (LU-10092) Multi-Rail Routing enhances multi-rail support for LNet routers, allowing auto discovery of routes and improved resiliency in configurations with multiple interfaces per node (LU-11297) Overstriping allows files to have multiple stripes of a single file on each OST. This allows configurations with relatively few clients/threads or very large OSTs to better utilize the full storage performance (LU-9846) Self-Extending Layouts improves the flexibility of Progressive File Layouts (PFL) to better handle filesystems with imbalanced OST capacity, such as smaller flash OST pools combined with larger disk OST pools. Files using Self-Extending Layouts can write to the flash pool as much as possible, but will automatically change over to the disk pool if there is not enough space (LU-10070) Fuller details can be found in the 2.13 wiki page (including the change log and test matrix) Please log any issues found in the issue tracking system Thanks to all those who have contributed to the creation of this release. -------------- next part -------------- An HTML attachment was scrubbed... URL: From neilb at suse.de Thu Dec 19 05:31:24 2019 From: neilb at suse.de (NeilBrown) Date: Thu, 19 Dec 2019 16:31:24 +1100 Subject: [lustre-devel] Lustre upstreaming status. References: Message-ID: <87sglgg9ub.fsf@notabene.neil.brown.name> Hi all, At the LUG in Houston, I said that I hoped to submit something upstream by the end of 2019. Clearly that isn't going to happen now. The main reason that caused me to not even try is IPv6 support. It became apparent to me that LNet would not be accepted until it has working IPv6 support, and that doesn't exist yet. I hope to put some development time into IPv6, and to have something that works and is worth reviewing by the end of January 2020. The other issue is that development has progressed slowly because there is no spare review bandwidth. James has contributed a lot, and others have helped, but reviewing patches for two code streams (OpenSFS and Linux-upstream) turns out to be too much to ask for. So I've decided to take a different approach. From now on I'm not going to wait for reviews for patches going into my linux-lustre tree. Part of my justification for this is that historically, review hasn't really provided much promise of correctness. Patches go missing. Random lines from patches go missing. Errors creep in in other ways. Instead, I am developing a tool which will compare OpenSFS lustre and Linux-lustre and report relevant differences. I have a prototype working, and it is helping me to find missing patches and parts of patches in both trees. I will continue to submit patches to gerrit to bring OpenSFS closer to my linux tree when that is needed, and will apply patches from OpenSFS to my tree without extra review when that it needed. When the time comes to submit upstream, I plan to present the tool so that other developers can confirm that what I am submitting is functionally equivalent to OpenSFS, and so that we can ensure the equivalence remains. Consequently my "lustre" branch will jump forward to v5.4 soon, probably tomorrow, and will remain close to mainline. I will also be growing my list of outstanding OpenSFS patches (currently about 100, many of which haven't been submitted to gerrit yet) and will hope to get those reviewed. Any changes that result from the review will be detected by my comparison script when the patch lands, and I'll update linux-lustre to match. My new goal for upstream submission is the end of Q1-2020. This is probably a bit optimistic, but gives me a suitable focus. NeilBrown -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 832 bytes Desc: not available URL: From slas+lustre at pdc.kth.se Fri Dec 20 13:40:14 2019 From: slas+lustre at pdc.kth.se (Mattias Claesson) Date: Fri, 20 Dec 2019 14:40:14 +0100 Subject: [lustre-devel] LU-12759 not in 2.12 Message-ID: <0377f0de-25e8-db29-e59a-ef885bcd1e7d@pdc.kth.se> Hi, Just wanted to check why the patch for LU-12759 hasn't landed for 2.12(.4) also? Not knowing the Lustre code well, but it seems like a fairly simple patch, without obvious side effects at least. And it would help with the mitigation of LU-12120. Cheers, Mattias Claesson PDC-staff From pjones at whamcloud.com Fri Dec 20 15:07:17 2019 From: pjones at whamcloud.com (Peter Jones) Date: Fri, 20 Dec 2019 15:07:17 +0000 Subject: [lustre-devel] LU-12759 not in 2.12 In-Reply-To: <0377f0de-25e8-db29-e59a-ef885bcd1e7d@pdc.kth.se> References: <0377f0de-25e8-db29-e59a-ef885bcd1e7d@pdc.kth.se> Message-ID: <94C86C98-64D5-445C-98E8-F695C4AD163D@ddn.com> Hi Mattias The short answer is because nobody has flagged it for inclusion. Do you consider that there would still be value to include it now that a fix for LU-12120 has landed? Peter On 2019-12-20, 5:40 AM, "lustre-devel on behalf of Mattias Claesson" wrote: Hi, Just wanted to check why the patch for LU-12759 hasn't landed for 2.12(.4) also? Not knowing the Lustre code well, but it seems like a fairly simple patch, without obvious side effects at least. And it would help with the mitigation of LU-12120. Cheers, Mattias Claesson PDC-staff _______________________________________________ lustre-devel mailing list lustre-devel at lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org From degremoa at amazon.com Fri Dec 27 16:04:25 2019 From: degremoa at amazon.com (Degremont, Aurelien) Date: Fri, 27 Dec 2019 16:04:25 +0000 Subject: [lustre-devel] Lustre upstreaming status. In-Reply-To: <87sglgg9ub.fsf@notabene.neil.brown.name> References: <87sglgg9ub.fsf@notabene.neil.brown.name> Message-ID: <844DFA02-C483-4708-AAD8-6647F68A48EB@amazon.com> Thanks for that. My understanding is that the only sustainable plan at long term is for linux lustre client to be the same code that OpenSFS master branch - without backward compat ifdef and similar - without server specific code. We need to avoid managing a kind of fork because it will too difficult to maintain both branches. IPV6 support in LNET should be at least half accepted in OpenSFS branch before going too far on that front. Aurélien Le 19/12/2019 06:32, « lustre-devel au nom de NeilBrown » a écrit : Hi all, At the LUG in Houston, I said that I hoped to submit something upstream by the end of 2019. Clearly that isn't going to happen now. The main reason that caused me to not even try is IPv6 support. It became apparent to me that LNet would not be accepted until it has working IPv6 support, and that doesn't exist yet. I hope to put some development time into IPv6, and to have something that works and is worth reviewing by the end of January 2020. The other issue is that development has progressed slowly because there is no spare review bandwidth. James has contributed a lot, and others have helped, but reviewing patches for two code streams (OpenSFS and Linux-upstream) turns out to be too much to ask for. So I've decided to take a different approach. From now on I'm not going to wait for reviews for patches going into my linux-lustre tree. Part of my justification for this is that historically, review hasn't really provided much promise of correctness. Patches go missing. Random lines from patches go missing. Errors creep in in other ways. Instead, I am developing a tool which will compare OpenSFS lustre and Linux-lustre and report relevant differences. I have a prototype working, and it is helping me to find missing patches and parts of patches in both trees. I will continue to submit patches to gerrit to bring OpenSFS closer to my linux tree when that is needed, and will apply patches from OpenSFS to my tree without extra review when that it needed. When the time comes to submit upstream, I plan to present the tool so that other developers can confirm that what I am submitting is functionally equivalent to OpenSFS, and so that we can ensure the equivalence remains. Consequently my "lustre" branch will jump forward to v5.4 soon, probably tomorrow, and will remain close to mainline. I will also be growing my list of outstanding OpenSFS patches (currently about 100, many of which haven't been submitted to gerrit yet) and will hope to get those reviewed. Any changes that result from the review will be detected by my comparison script when the patch lands, and I'll update linux-lustre to match. My new goal for upstream submission is the end of Q1-2020. This is probably a bit optimistic, but gives me a suitable focus. NeilBrown