[lustre-devel] [LSF/MM/BPF TOPIC] [DRAFT] Lustre client upstreaming

[Patrick Farrell]

I agree strongly here, and I think going upstream with both makes some things much easier.  It forces us to deal with ldiskfs but there's all of that shared code reorg, etc, which this can let us partially skip.  While there's probably some value in fully separating client and server code, it would be a fair bit of work and then the keeping in sync, etc...  All at once seems nicer to me.
________________________________
From: lustre-devel <lustre-devel-bounces at lists.lustre.org> on behalf of Day, Timothy <timday at amazon.com>
Sent: Sunday, January 19, 2025 9:57 PM
To: NeilBrown <neilb at suse.de>
Cc: lustre-devel at lists.lustre.org <lustre-devel at lists.lustre.org>
Subject: Re: [lustre-devel] [LSF/MM/BPF TOPIC] [DRAFT] Lustre client upstreaming



> On 1/18/25, 5:21 PM, "NeilBrown" <neilb at suse.de <mailto:neilb at suse.de>> wrote:
> > On Sun, 19 Jan 2025, Day, Timothy wrote:
> >
> > On the other hand, I wonder if we upstream the whole thing all at once. Beside
> > the code being a bit nicer, the client isn't really that much closer to being upstream
> > than the server is. And no one else can test the client without having a Lustre
> > server on-hand. So no-one can easily run xfstests or similar. And doing everything
> > all at once would preempt questions of client/server split or the server upstreaming
> > timeline. But upstreaming so much all at once is probably more unrealistic.
>
>
> The main difference I see between server and client in upstreaming terms
> is the storage backend. It would need to use un-patched ext4 - ideally
> using VFS interfaces though we might be able to negotiate with the ext4
> team to get some exports. I don't know much about the delta between
> ldiskfs and ext4 and understand it is much smaller than it once was, but
> it would need to be zero. I'm working towards getting the pdirop patch
> upstreamable. Andreas would know what else is needed better than I.

I've been working on a third storage backend [1]. It'll likely be done
well before we submit anything upstream. It's a just memory-only
target. That might be justification enough to keep the OSD APIs.

[1] https://review.whamcloud.com/c/fs/lustre-release/+/55594

> The other difference is that a lot of the "revise code to match upstream
> style" work has focused on client and ignored server-only code.
>
>
> It might be sensible to set the goal as "client and server" including
> only the ext4 backend and possibly only the socklnd network interface.
> It will be a big code drop either way. People aren't going to go over
> every line with a fine-tooth-comb. They will mostly look at whichever
> bit particularly interests them, and look at the process and community
> behind the code.
>
>
> Being able to build a pure upstream kernel, add a user-space tools
> package, and test would certainly be a plus. That would be something
> worth canvassing at LSF - is there any value in landing the client
> without the server?

Yeah, I'm leaning towards setting the goal as both client/server and
gathering opinions from LSF. The client and server are still pretty
intertwined. I think having the client go upstream and then basing
the server on top an in-tree client would make server development
noticeably more difficult. Thinking on it more - I don't think
upstreaming the server is more ambitious than the client. We
have more of a process problem than a code problem. And I don't
think the server is in particularly bad shape.

>
> NeilBrown
>

Tim Day

_______________________________________________
lustre-devel mailing list
lustre-devel at lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel-lustre.org/attachments/20250121/ebcbfc92/attachment.htm>