[lustre-devel] [LSF/MM/BPF TOPIC] [DRAFT] Lustre client upstreaming

[NeilBrown]

On Sun, 02 Feb 2025, NeilBrown wrote:
> 
> Maybe it would be good for me to paint a more details picture of what I
> imagine would happen - assuming we do take the path of landing all of
> lustre, both client and server, upstream.
> 
> - we would change the kernel code in lustre-release so that it was
>   exactly what we plan to submit upstream.
> - we submit it and once accepted we have identical code in upstream
>   linux and lustre-release
> - we fork lustre-release to a new package called (e.g.) lustre-tools and 
>   remove all kernel code leaving just utils and documentation and 
>   test code.  The kinode.c kernel module that is in lustre/tests/kernel/
>   would need to go upstream with the rest of the kernel code I think.
>   lustre-tools would be easily accessible and buildable by anyone who
>   wants to test lustre
> - we fork lustre-release to another new package lustre-backports
>   and remove all non-kernel code from there.  We configure it to build
>   out-of-tree modules with names like "backport-lustre" "backport-lnet"
>   and provide modprobe.conf files that alias the standard names to
>   these.  That should allow to over-ride the distributed modules (if
>   any) when people choose to use backports.
> - upstream commits which touch lustre or lnet are automatically add to
>   lustre-backports and someone is notified to help when they don't apply
>   
> With this:
>  Anyone who wants to test or use the lustre included with a particular
>  kernel can do with with only the lustre-tools package.  Anyone who
>  wants to use the latest lustre code with an older kernel can build and
>  use lustre-backports.
> 

Sorry, I completely forgot the other part of the picture.

We would have a git tree with upstream linux and branches called
"lustre-next" and "lustre-fixes" and "lustre-testing" or whatever the
lead maintainers preferred.
Developer would be encouraged to work against lustre-next (or one of the
others when necessary) but submit patches or pull-requests against that.
Patches destined for the next upstream merge window would be be pulled
into lustre-next once that have enough reviews and verification.  Bug
fixes that need to go before the merge window would go to lustre-fixes.
Any patches that are reasonably mature might be merged into
lustre-testing which is where the more heavy-weight testing might focus.
lustre-testing would likely be rebased often.

Obviously if people want to develop features against exactly the kernel
their paying customer is using that would be fine.  But the features
would need to go to lustre-next and then upstream before they could land
in the lustre-backports release.

One consequence of all this that is worth highlighting is that when a
feature needs changes to the kernel and to tools, it will need separate
patches for separate packages.  I think we already aim to ensure new
tools work with old kernels and vice-versa.  Having this split would
help keep the focus on that.

NeilBrown