[lustre-discuss] Adding lustre clients into the Debian

[Andreas Dilger]

On Oct 1, 2023, at 05:54, Arman Khalatyan via lustre-discuss <lustre-discuss at lists.lustre.org> wrote:
> 
> Hello everyone,
> 
> We are in the process of integrating the Lustre client into Debian. Are there any legal concerns or significant obstacles to this? We're curious why it hasn't been included in the official Debian repository so far. There used to be an old, unmaintained Lustre client branch dating back to around 2013.

I don't think there is any particular barrier to add a Debian Lustre package today.  As you wrote, there was some effort put into that in the past, but I think there wasn't someone with the time and Debian-specific experience maintain it and get it included into a release.  Lustre is GPLv2 licensed, with some LGPL user library API code, so there aren't any legal issues.

The Lustre CI system builds Ubuntu clients, and I'd be happy to see patches to improve the Debian packaging in the Lustre tree.  Most Lustre users are using RHEL derivatives, and secondarily Ubuntu, so there hasn't been anyone to work on specific Debian packaging in some time.  Thomas (CC'd) was most active on the Debian front, and might be able to comment on this in more detail, and hopefully can also help review the patches.

> You can check our wishlist on Debian here: https://bugs.debian.org/1053214
> 
> At AIP, one of our colleagues is responsible for maintaining Astropy, so we have some experience with Debian.
> 
> I've also set up a CI system in our GitLab, which includes a simple build and push to a public S3 bucket. This is primarily for testing purposes to see if it functions correctly for others...

There is an automated build farm for Lustre, with patches submitted via Gerrit (https://wiki.lustre.org/Using_Gerrit), and while there aren't Debian build/test nodes to test the correctness of changes for Debian, this will at least avoid regressions with Ubuntu or, though unlikely, with RHEL.  My assumption would be that any Debian-specific changes would continue to work with Ubuntu?

Depending on how actively you want to build/test Lustre, it is also possible to configure your CI system to asynchronously follow patches under development in Gerrit to provide build and/or test feedback on the patches before they land.  The contrib/scripts/gerrit_checkpatch.py script follows new patch submissions and runs code style reviews on patches after they are pushed, and posts comments back to Gerrit.

A step beyond this, once your CI system is working reliably, it is possible to also post review comments directly into the patches in Gerrit, as the Gerrit Janitor does (https://github.com/verygreen/lustretester/).  The gerrit_build-and-test-new.py script is derived from gerrit_checkpatch.py, but implements a more complex set of operations on each patch - static code analysis, build, test.  It will add both general patch comments on success/failure of the build/test, or comments on specific lines in the patch.  In some cases, Gerrit Janitor will also give negative code reviews in cases when a newly-added regression test added by a patch is failing regularly in its testing.

Cheers, Andreas
--
Andreas Dilger
Lustre Principal Architect
Whamcloud