<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;
mso-ligatures:standardcontextual;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style>
</head>
<body lang="EN-US" link="#0563C1" vlink="#954F72" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal">The following is a draft topic for the upcoming LSF/MM conference.<o:p></o:p></p>
<p class="MsoNormal">I wanted to solicit feedback from the wider Lustre development<o:p></o:p></p>
<p class="MsoNormal">community before submitting this to fsdevel. If I’ve omitted anything,<o:p></o:p></p>
<p class="MsoNormal">something doesn’t seem right, or you know of something that strengthens<o:p></o:p></p>
<p class="MsoNormal">the argument, please let me know!<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">----------------------------------------------------<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Lustre is a high-performance parallel filesystem used for HPC and AI/ML<o:p></o:p></p>
<p class="MsoNormal">compute clusters available under GPLv2. Lustre has achieved widespread<o:p></o:p></p>
<p class="MsoNormal">adoption in the HPC and AI/ML and is commercially supported by numerous<o:p></o:p></p>
<p class="MsoNormal">vendors and cloud service providers [1].<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">After 21 years and an ill-fated stint in staging, Lustre is still maintained as<o:p></o:p></p>
<p class="MsoNormal">an out-of-tree module [6]. The previous upstreaming effort suffered from a<o:p></o:p></p>
<p class="MsoNormal">lack of developer focus and user adoption, which eventually led to Lustre<o:p></o:p></p>
<p class="MsoNormal">being removed from staging altogether [2].<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">However, the work to improve Lustre has not stopped. In the intervening<o:p></o:p></p>
<p class="MsoNormal">years, the code improvements that would preempt a return to mainline<o:p></o:p></p>
<p class="MsoNormal">have been steadily progressing. At least 25% of patches accepted for<o:p></o:p></p>
<p class="MsoNormal">Lustre 2.16 were related to the upstreaming effort [3]. And all of the<o:p></o:p></p>
<p class="MsoNormal">remaining work is in-flight [4][5]. Our eventual goal is to a get a minimal<o:p></o:p></p>
<p class="MsoNormal">TCP/IP-only Lustre client to an acceptable quality before submitting to<o:p></o:p></p>
<p class="MsoNormal">mainline.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">I propose to discuss:<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">- Expectations for a new filesystem to be accepted to mainline<o:p></o:p></p>
<p class="MsoNormal">- Weaknesses in the previous upstreaming effort in staging<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Lustre has already received a plethora of feedback in the past. While much<o:p></o:p></p>
<p class="MsoNormal">of that has been addressed since - the kernel is a moving target. Several<o:p></o:p></p>
<p class="MsoNormal">filesystems have been merged (and removed) since Lustre left staging. We're<o:p></o:p></p>
<p class="MsoNormal">aiming to avoid the mistakes of the past and hope to address as many<o:p></o:p></p>
<p class="MsoNormal">concerns as possible before submitting for inclusion.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Thanks!<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Timothy Day (Amazon Web Services - AWS)<o:p></o:p></p>
<p class="MsoNormal">James Simmons (Oak Ridge National Labs - ORNL)<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">[1] Lustre Community Update: https://youtu.be/BE--ySVQb2M?si=YMHitJfcE4ASWQcE&t=960<o:p></o:p></p>
<p class="MsoNormal">[2] Kicked out of staging: https://lwn.net/Articles/756565/<o:p></o:p></p>
<p class="MsoNormal">[3] ORNL, Aeon, SuSe, AWS, and more: https://youtu.be/BE--ySVQb2M?si=YMHitJfcE4ASWQcE&t=960<o:p></o:p></p>
<p class="MsoNormal">[4] LUG24 Upstreaming Update: https://www.depts.ttu.edu/hpcc/events/LUG24/slides/Day1/LUG_2024_Talk_02-Native_Linux_client_status.pdf<o:p></o:p></p>
<p class="MsoNormal">[5] Lustre Jira Upstream Progress: https://jira.whamcloud.com/browse/LU-12511<o:p></o:p></p>
<p class="MsoNormal">[6] Out-of-tree codebase: https://git.whamcloud.com/?p=fs/lustre-release.git;a=tree<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</body>
</html>