[lustre-devel] Lustre Server on Arm64 Progress

[Andreas Dilger]

On Sep 19, 2022, at 07:49, Kevin Zhao via lustre-devel <lustre-devel at lists.lustre.org<mailto:lustre-devel at lists.lustre.org>> wrote:

In June we enabled the Lustre Arm64 Server and client CI. And we have built the Lustre Master branch code RPMs daily and publish the Lustre/e2progsfs at https://uk.linaro.cloud/repo/
The CI Jenkins Link: http://213.146.155.72:8080/
The CI Management script: https://github.com/Linaro/lustretest
The jobs are daily to grab the code, build RPMs and run the whole test suite. Now we cover all the tests and recorded at https://github.com/Linaro/lustretest/blob/main/lustretest/cfg/test-groups.yaml

Hi Kevin,
thanks for the status update.  It looks like you are making good progress.

Now Lustre test cases fail on Arm64 is less than 20, we are still WIP to fix all the test failure.

The Arm64 External CI has already been running for about 3 months, and it’s time for us to think further and do more to make Arm64 official support for Lustre.
Next step:

1. Gerrit Patch Landing Triagger Arm64 Jobs: Considering our limited resources and the expectation about the Lustre code robust on Arm64, we’d planned to implement the Lustre patch landing CI. The idea is to run the Arm64 RPM build and the whole test suite when the patch in gerrit is +2 and cherry pick to the master branch.

One problem with waiting until a patch is +2 is that this will make it more difficult for developers to notice that there is a problem with aarch46 testing.  This also adds significant overhead to the development process, since any aarch64-specific bug found in the patch that needs to be fixed will invalidate all of the previous review and testing efforts.

My preference would be to "opportunistically" run tests against patches when they are pushed, as test nodes become available, and then post results on the patch in case of test failure, maybe also in case of success if there are only a few lines.  This is what we do with "optional" test sessions in autotest when we add new/experimental test configurations (new distro or architecture).  The "optional" sessions do not mark a patch with Verified: -1, and do not need to be re-run if they fail, but at least allow tracking pass/fail results in Maloo, and allow eventually enforcing those sessions when they are passing consistently.

While "optional" sessions may not run on every patch, they at least run as many tests as possible (keeping test nodes busy) and with good patch monitoring it is possible to prioritize running sessions on new patches (that have never had a previous test run), and new versions of patches that previously failed testing.  A lower priority would be patches that have already run aarch64 testing and passed.

The contrib/scripts/gerrit_checkpatch.py script can be used as an example of monitoring Gerrit for new patches/versions, and tracking which patches have already been processed.

Cheers, Andreas

2. Build and store the Lustre packages for special versions (Both for 4K and 64K page size). Now we just build and store the development release.

Any feedback/advice/help are really appreciated. Look forward to hearing from you soon. Thanks for the upstream help.

--
Best Regards
Kevin Zhao
Tech Lead, LDCG Cloud Infra & Storage
Linaro Vertical Technologies
IRC(freenode): kevinz
Slack(kubernetes.slack.com<http://kubernetes.slack.com/>): kevinz
kevin.zhao at linaro.org<mailto:kevin.zhao at linaro.org> | Mobile/Direct/Wechat:  +86 18818270915

_______________________________________________
lustre-devel mailing list
lustre-devel at lists.lustre.org<mailto:lustre-devel at lists.lustre.org>
http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org

Cheers, Andreas
--
Andreas Dilger
Lustre Principal Architect
Whamcloud







-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel-lustre.org/attachments/20220921/de2ac275/attachment-0001.htm>