<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:Helvetica;
panose-1:0 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Aptos;
panose-1:2 11 0 4 2 2 2 2 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:12.0pt;
font-family:"Aptos",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
span.apple-converted-space
{mso-style-name:apple-converted-space;}
span.EmailStyle21
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;
mso-ligatures:none;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple" style="word-wrap:break-word;overflow-wrap: break-word;-webkit-nbsp-mode: space;line-break:after-white-space">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">Lustre itself going upstream might be strong enough justification for<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">ext4 to accept the needed ldiskfs patches. I agree that we shouldn\u2019t<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">block upstreaming the client on upstreaming the server. But I think<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">we should advocate upstreaming both client and server together,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">and then see what feedback we get. We should at least document<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">the objections. I don\u2019t think we have to commit to one path or another<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">right now.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">Regarding ZFS, I think the OSD will eventually have to live in the<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">normal openZFS repo. And it will have to live with whatever<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">interface ldiskfs/ext4 gets. But I don\u2019t think we have to prioritize<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">that work until we have more confidence that Lustre is on a path<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">to upstream.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">Tim Day<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><o:p> </o:p></span></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal" style="margin-left:.5in"><b><span style="font-family:"Calibri",sans-serif;color:black">From:
</span></b><span style="font-family:"Calibri",sans-serif;color:black">Andreas Dilger <adilger@ddn.com><br>
<b>Date: </b>Wednesday, January 22, 2025 at 1:58\u202fAM<br>
<b>To: </b>Patrick Farrell <pfarrell@ddn.com><br>
<b>Cc: </b>"Day, Timothy" <timday@amazon.com>, NeilBrown <neilb@suse.de>, "lustre-devel@lists.lustre.org" <lustre-devel@lists.lustre.org><br>
<b>Subject: </b>RE: [EXTERNAL] [lustre-devel] [LSF/MM/BPF TOPIC] [DRAFT] Lustre client upstreaming<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
</div>
<div>
<table class="MsoNormalTable" border="0" cellspacing="0" cellpadding="0" width="622" style="width:466.5pt;margin-left:.5in;border-collapse:collapse">
<tbody>
<tr style="height:15.25pt">
<td width="622" valign="top" style="width:466.5pt;border:solid #ED7D31 1.5pt;padding:0in 5.4pt 0in 5.4pt;height:15.25pt">
<p><strong><span style="font-family:"Aptos",sans-serif;color:black;background:#FFFF99">CAUTION</span></strong><span style="color:black;background:#FFFF99">: This email originated from outside of the organization. Do not click links or open attachments unless
you can confirm the sender and know the content is safe.</span><o:p></o:p></p>
</td>
</tr>
</tbody>
</table>
</div>
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
<div>
<p class="MsoNormal" style="margin-left:.5in">IMHO, there would be objections to Lustre changes to ext4 to allow it to be used
<o:p></o:p></p>
<div>
<p class="MsoNormal" style="margin-left:.5in">like ldiskfs. <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">We cannot use the VFS interface as-is, since Lustre needs to have<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">compound journaled transactions that are atomically committed. Also, there are<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">some operations (e.g. DNE namespace operations) which do not have VFS<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">equivalents, so they would require poking through the VFS, and in general the<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">VFS does a lot of things we *don't* want it to do for Lustre.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">If there are objections to patching ext4 to allow osd-ldiskfs to access transactions<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">directly, then the alternative would be to copy it to ldiskfs and patch it as we do<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">today, but I suspect that would also be frowned upon.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">Don't get me wrong, it's not that I *want* to maintain ldiskfs forever out of tree,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">but pretty much every patch we try to upstream to ext4 is rejected for one<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">reason or another, so I've stopped holding my breath that this will move forward.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">Running osd-zfs doesn't need any kernel/ext4 patches, but that is an even<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">larger can of worms, and will never fly in a million years.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">I think upstreaming the client is a realistic goal, but I think tying this to the<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">upstreaming of the server with ldiskfs support will derail the whole project.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">Cheers, Andreas<o:p></o:p></p>
</div>
<div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><br>
<br>
<o:p></o:p></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal" style="margin-left:.5in">On Jan 21, 2025, at 10:02, Patrick Farrell <pfarrell@ddn.com> wrote:<o:p></o:p></p>
</div>
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal" style="margin-left:.5in">I agree strongly here, and I think going upstream with both makes some things much easier. It forces us to deal with ldiskfs but there's all of that shared code reorg, etc, which this can let us partially skip.
While there's probably some value in fully separating client and server code, it would be a fair bit of work and then the keeping in sync, etc... All at once seems nicer to me.<o:p></o:p></p>
</div>
<div class="MsoNormal" align="center" style="margin-left:.5in;text-align:center">
<hr size="2" width="1440" style="width:15.0in" align="center">
</div>
<div id="divRplyFwdMsg">
<p class="MsoNormal" style="margin-left:.5in"><b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span class="apple-converted-space"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"> </span></span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">lustre-devel
<lustre-devel-bounces@lists.lustre.org> on behalf of Day, Timothy <timday@amazon.com><br>
<b>Sent:</b><span class="apple-converted-space"> </span>Sunday, January 19, 2025 9:57 PM<br>
<b>To:</b><span class="apple-converted-space"> </span>NeilBrown <neilb@suse.de><br>
<b>Cc:</b><span class="apple-converted-space"> </span>lustre-devel@lists.lustre.org <lustre-devel@lists.lustre.org><br>
<b>Subject:</b><span class="apple-converted-space"> </span>Re: [lustre-devel] [LSF/MM/BPF TOPIC] [DRAFT] Lustre client upstreaming</span><span style="font-size:9.0pt;font-family:Helvetica">
<o:p></o:p></span></p>
<div>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:9.0pt;font-family:Helvetica"> <o:p></o:p></span></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:11.0pt;font-family:Helvetica"><br>
<br>
> On 1/18/25, 5:21 PM, "NeilBrown" <neilb@suse.de <<a href="mailto:neilb@suse.de">mailto:neilb@suse.de</a>>> wrote:<br>
> > On Sun, 19 Jan 2025, Day, Timothy wrote:<br>
> ><br>
> > On the other hand, I wonder if we upstream the whole thing all at once. Beside<br>
> > the code being a bit nicer, the client isn't really that much closer to being upstream<br>
> > than the server is. And no one else can test the client without having a Lustre<br>
> > server on-hand. So no-one can easily run xfstests or similar. And doing everything<br>
> > all at once would preempt questions of client/server split or the server upstreaming<br>
> > timeline. But upstreaming so much all at once is probably more unrealistic.<br>
><br>
><br>
> The main difference I see between server and client in upstreaming terms<br>
> is the storage backend. It would need to use un-patched ext4 - ideally<br>
> using VFS interfaces though we might be able to negotiate with the ext4<br>
> team to get some exports. I don't know much about the delta between<br>
> ldiskfs and ext4 and understand it is much smaller than it once was, but<br>
> it would need to be zero. I'm working towards getting the pdirop patch<br>
> upstreamable. Andreas would know what else is needed better than I.<br>
<br>
I've been working on a third storage backend [1]. It'll likely be done<br>
well before we submit anything upstream. It's a just memory-only<br>
target. That might be justification enough to keep the OSD APIs.<br>
<br>
[1]<span class="apple-converted-space"> </span><a href="https://review.whamcloud.com/c/fs/lustre-release/+/55594">https://review.whamcloud.com/c/fs/lustre-release/+/55594</a><br>
<br>
> The other difference is that a lot of the "revise code to match upstream<br>
> style" work has focused on client and ignored server-only code.<br>
><br>
><br>
> It might be sensible to set the goal as "client and server" including<br>
> only the ext4 backend and possibly only the socklnd network interface.<br>
> It will be a big code drop either way. People aren't going to go over<br>
> every line with a fine-tooth-comb. They will mostly look at whichever<br>
> bit particularly interests them, and look at the process and community<br>
> behind the code.<br>
><br>
><br>
> Being able to build a pure upstream kernel, add a user-space tools<br>
> package, and test would certainly be a plus. That would be something<br>
> worth canvassing at LSF - is there any value in landing the client<br>
> without the server?<br>
<br>
Yeah, I'm leaning towards setting the goal as both client/server and<br>
gathering opinions from LSF. The client and server are still pretty<br>
intertwined. I think having the client go upstream and then basing<br>
the server on top an in-tree client would make server development<br>
noticeably more difficult. Thinking on it more - I don't think<br>
upstreaming the server is more ambitious than the client. We<br>
have more of a process problem than a code problem. And I don't<br>
think the server is in particularly bad shape.<br>
<br>
><br>
> NeilBrown<br>
><br>
<br>
Tim Day<br>
<br>
_______________________________________________<br>
lustre-devel mailing list<br>
lustre-devel@lists.lustre.org<br>
<a href="http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org">http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org</a><o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:9.0pt;font-family:Helvetica">_______________________________________________<br>
lustre-devel mailing list<br>
</span><a href="mailto:lustre-devel@lists.lustre.org"><span style="font-size:9.0pt;font-family:Helvetica">lustre-devel@lists.lustre.org</span></a><span style="font-size:9.0pt;font-family:Helvetica"><br>
</span><a href="http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org"><span style="font-size:9.0pt;font-family:Helvetica">http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org</span></a><o:p></o:p></p>
</div>
</blockquote>
</div>
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
<div>
<div>
<div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><span style="color:black">Cheers, Andreas<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><span style="color:black">\u2014<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><span style="color:black">Andreas Dilger<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><span style="color:black">Lustre Principal Architect<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><span style="color:black">Whamcloud/DDN<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal" style="margin-left:.5in"><span style="color:black"><o:p> </o:p></span></p>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:0in;margin-right:0in;margin-bottom:12.0pt;margin-left:.5in">
<o:p> </o:p></p>
</div>
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
</div>
</div>
</div>
</body>
</html>