[lustre-devel] [LSF/MM/BPF TOPIC] [DRAFT] Lustre client upstreaming

[Day, Timothy]

> On 1/18/25, 5:21 PM, "NeilBrown" <neilb at suse.de <mailto:neilb at suse.de>> wrote:
> > On Sun, 19 Jan 2025, Day, Timothy wrote:
> >
> > On the other hand, I wonder if we upstream the whole thing all at once. Beside
> > the code being a bit nicer, the client isn't really that much closer to being upstream
> > than the server is. And no one else can test the client without having a Lustre
> > server on-hand. So no-one can easily run xfstests or similar. And doing everything
> > all at once would preempt questions of client/server split or the server upstreaming
> > timeline. But upstreaming so much all at once is probably more unrealistic.
>
>
> The main difference I see between server and client in upstreaming terms
> is the storage backend. It would need to use un-patched ext4 - ideally
> using VFS interfaces though we might be able to negotiate with the ext4
> team to get some exports. I don't know much about the delta between
> ldiskfs and ext4 and understand it is much smaller than it once was, but
> it would need to be zero. I'm working towards getting the pdirop patch
> upstreamable. Andreas would know what else is needed better than I.

I've been working on a third storage backend [1]. It'll likely be done
well before we submit anything upstream. It's a just memory-only
target. That might be justification enough to keep the OSD APIs.

[1] https://review.whamcloud.com/c/fs/lustre-release/+/55594

> The other difference is that a lot of the "revise code to match upstream
> style" work has focused on client and ignored server-only code.
>
>
> It might be sensible to set the goal as "client and server" including
> only the ext4 backend and possibly only the socklnd network interface.
> It will be a big code drop either way. People aren't going to go over
> every line with a fine-tooth-comb. They will mostly look at whichever
> bit particularly interests them, and look at the process and community
> behind the code.
>
>
> Being able to build a pure upstream kernel, add a user-space tools
> package, and test would certainly be a plus. That would be something
> worth canvassing at LSF - is there any value in landing the client
> without the server?

Yeah, I'm leaning towards setting the goal as both client/server and
gathering opinions from LSF. The client and server are still pretty
intertwined. I think having the client go upstream and then basing
the server on top an in-tree client would make server development
noticeably more difficult. Thinking on it more - I don't think
upstreaming the server is more ambitious than the client. We
have more of a process problem than a code problem. And I don't
think the server is in particularly bad shape.

>
> NeilBrown
>

Tim Day