<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
<span class="x_elementToProof ContentPasted0" data-ogsc="rgb(0, 0, 0)" style="font-size: 12pt; margin: 0px; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">Hi,</span></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
<div class="x_elementToProof" data-ogsc="rgb(0, 0, 0)" style="font-size: 12pt; margin: 0px; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
<br class="ContentPasted0">
</div>
<div class="x_elementToProof ContentPasted0" data-ogsc="rgb(0, 0, 0)" style="font-size: 12pt; margin: 0px; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
I'm having a bit of a nightmare trying to build server packages for 2.15.1 - I feel like I've tried quite a few different approaches and am getting stumped, it is most likely that I am missing something incredibly obvious so I'd appreciate any pointers. I'd
like to point out that I am by no means an expert in any of this though I have had about 4 years maintaining various Lustre builds - I am hugely grateful for all the work you guys do on Lustre, and I hope I don't come across as anything other than frustrated
by my inability to get this version to install!</div>
<div class="x_elementToProof" data-ogsc="rgb(0, 0, 0)" style="font-size: 12pt; margin: 0px; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
<br class="ContentPasted0">
</div>
<div class="x_elementToProof x_ContentPasted0 x_ContentPasted1 ContentPasted0" data-ogsc="rgb(0, 0, 0)" style="font-size: 12pt; margin: 0px; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
To start, we are running Rocky 8.6, with the 4.18.0-372.9.1.el8 kernel. Initially, as with previous Lustre builds I've done, I've installed the Whamcloud provided el8_lustre kernel, along with headers, then installed the Mellanox OFED stack from their repos,
making sure to use the version that appears to be being used for 2.15.1 builds (in this case 5.6-2.0.9.0 - incidentally I can't find the specific OFEDs used in any compatibility matrices or changelogs, where this information used to be provided, so I've back
formulated from the Whamcloud repos). </div>
<div class="x_elementToProof x_ContentPasted0 x_ContentPasted1" data-ogsc="rgb(0, 0, 0)" style="font-size: 12pt; margin: 0px; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
<br class="ContentPasted0">
</div>
<div class="x_elementToProof x_ContentPasted0 x_ContentPasted1 ContentPasted0" data-ogsc="rgb(0, 0, 0)" style="font-size: 12pt; margin: 0px; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
In the past, I've then simply yum/dnf installed the lustre server packages. With this release however, I immediately run in to "Nothing provides _ksym" errors, all of which appear to be for ZFS symbols. A quick check on the Whamcloud Jira throws up<span class="ContentPasted0"> </span><a href="https://jira.whamcloud.com/browse/LU-16109" target="_blank" rel="noopener noreferrer" data-auth="NotApplicable" title="https://jira.whamcloud.com/browse/LU-16109" data-loopstyle="linkonly" data-ogsc="" data-linkindex="0" style="margin:0px" class="ContentPasted0">this</a><span class="ContentPasted0"> </span>issue,
which is marked closed but references<span class="ContentPasted0"> </span><a href="https://jira.whamcloud.com/browse/LU-16059" target="_blank" rel="noopener noreferrer" data-auth="NotApplicable" title="https://jira.whamcloud.com/browse/LU-16059" data-loopstyle="linkonly" data-ogsc="" data-linkindex="1" style="margin:0px" class="ContentPasted0">this</a><span class="ContentPasted0"> </span>issue,
which says that the issue is fixed but not for 2.15.1, instead for 2.15.2.</div>
<div class="x_elementToProof x_ContentPasted0 x_ContentPasted1" data-ogsc="rgb(0, 0, 0)" style="font-size: 12pt; margin: 0px; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
<br class="ContentPasted0">
</div>
<div class="x_elementToProof x_ContentPasted0 x_ContentPasted1 ContentPasted0" data-ogsc="rgb(0, 0, 0)" style="font-size: 12pt; margin: 0px; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
I'm intending to build these servers with kickstart and puppet, and I'd much rather use the official repos rather than compile it myself but this is not a hard requirement. A quick check on the Whamcloud repos and it appears that 2.15.2 only supports 8.7, rather
than 8.6. This is a bit of a problem as I'd like to keep the same kernel version as the rest of the machines we're running where possible, but again, not a hard requirement. I spun up a new build for 8.7 on the same hardware, updated the Mellanox repos to
point to the new correct version and immediately got _ksym errors but now it appears they're for the OFEDs instead.</div>
<div class="x_elementToProof x_ContentPasted0 x_ContentPasted1" data-ogsc="rgb(0, 0, 0)" style="font-size: 12pt; margin: 0px; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
<br class="ContentPasted0">
</div>
<div class="x_elementToProof x_ContentPasted0 x_ContentPasted1 ContentPasted0" data-ogsc="rgb(0, 0, 0)" style="font-size: 12pt; margin: 0px; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
In the meantime, I've seen an email in the mailing list suggesting that the symbols are in fact provided by the package kmod-zfs, which is not provided by the OpenZFS repos, but that can be built manually, so I thought I'd have another crack at getting 2.15.1
working. I download the tar, built ZFS and installed the resulting rpms, making sure to install devel and debugsource and debuginfo. I attempted to build lustre against this and it all appears to go ok - I get some rpms out! However, installing them results
in the exact same ksym errors. The thing is the ksyms appear to be present by name in /proc/kallsyms, just not matching exactly.</div>
<div class="x_elementToProof x_ContentPasted0 x_ContentPasted1" data-ogsc="rgb(0, 0, 0)" style="font-size: 12pt; margin: 0px; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
<br class="ContentPasted0">
</div>
<div class="x_elementToProof x_ContentPasted0 x_ContentPasted1 ContentPasted0" data-ogsc="rgb(0, 0, 0)" style="font-size: 12pt; margin: 0px; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
The main point I guess is that LU-16059 appears to have been closed erroneously, as on a fresh install the issue is 100% reproducible. I also note that the packages hosted<span class="ContentPasted0"> </span><a href="https://downloads.whamcloud.com/public/lustre/lustre-2.15.1/el8.6/server/RPMS/x86_64/" target="_blank" rel="noopener noreferrer" data-auth="NotApplicable" title="https://downloads.whamcloud.com/public/lustre/lustre-2.15.1/el8.6/server/RPMS/x86_64/" data-loopstyle="linkonly" data-ogsc="" data-linkindex="2" style="margin:0px" class="ContentPasted0">here</a> have
timestamps from 2022-08-10 but the issue was created and closed after this. I'm happy to re-open the bug and provide as much detail as necessary but thought I'd check to see if anyone else has experienced this issue or if I am indeed missing something trivial.</div>
<div class="x_elementToProof x_ContentPasted0 x_ContentPasted1" data-ogsc="rgb(0, 0, 0)" style="font-size: 12pt; margin: 0px; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
<br class="ContentPasted0">
</div>
<div class="x_elementToProof x_ContentPasted0 x_ContentPasted1 ContentPasted0" data-ogsc="rgb(0, 0, 0)" style="font-size: 12pt; margin: 0px; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
Thanks in advance</div>
<span class="x_elementToProof x_ContentPasted0 x_ContentPasted1 ContentPasted0" data-ogsc="rgb(0, 0, 0)" style="font-size: 12pt; margin: 0px; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">Jon</span><br>
</div>
<div class="elementToProof">
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div id="Signature">
<div>
<div></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<p style="font-size:11pt; font-family:Calibri,sans-serif; margin:0"><span style="color: rgb(89, 89, 89);">Jon Marshall</span></p>
<p style="font-size:11pt; font-family:Calibri,sans-serif; margin:0"><span style="font-size: 10pt; color: rgb(89, 89, 89);">High Performance Computing Specialist<br>
</span></p>
<p style="font-size:11pt; font-family:Calibri,sans-serif; margin:0"><span style="font-size: 10pt; color: rgb(89, 89, 89);"> </span></p>
<p style="font-size:11pt; font-family:Calibri,sans-serif; margin:0"><b><span style="color: rgb(89, 89, 89);">IT and Scientific Computing Team</span></b><span style="font-size: 10pt; color: rgb(89, 89, 89);"></span></p>
<p style="font-size:11pt; font-family:Calibri,sans-serif; margin:0"><span style="font-size: 10pt; color: rgb(89, 89, 89);"> </span></p>
<p style="font-size:11pt; font-family:Calibri,sans-serif; margin:0"><span style="font-size: 10pt; color: rgb(89, 89, 89);">Cancer Research UK Cambridge Institute</span></p>
<p style="font-size:11pt; font-family:Calibri,sans-serif; margin:0"><span style="font-size: 10pt; color: rgb(89, 89, 89);">Li Ka Shing Centre | Robinson Way | Cambridge | CB2 0RE</span></p>
<p style="font-size:11pt; font-family:Calibri,sans-serif; margin:0"><span><a href="http://www.cruk.cam.ac.uk/" target="_blank" rel="noopener noreferrer" data-loopstyle="linkonly"><span style="font-size: 10pt; color: rgb(0, 112, 192);">Web</span></a></span><span style="font-size: 10pt; color: rgb(0, 112, 192);">
| </span><span><a href="http://www.facebook.com/cancerresearchuk" target="_blank" rel="noopener noreferrer" data-loopstyle="linkonly"><span style="font-size: 10pt; color: rgb(0, 112, 192);">Facebook</span></a></span><span style="font-size: 10pt; color: rgb(0, 112, 192);">
| </span><span><a href="http://twitter.com/CR_UK" target="_blank" rel="noopener noreferrer" data-loopstyle="linkonly"><span style="font-size: 10pt; color: rgb(0, 112, 192);">Twitter</span></a></span></p>
<p style="font-size:11pt; font-family:Calibri,sans-serif; margin:0"><span style="font-size: 10pt; color: rgb(0, 112, 192);"> </span></p>
<p style="font-size:11pt; font-family:Calibri,sans-serif; margin:0"><a href="http://www.cruk.cam.ac.uk/" target="_blank" rel="noopener noreferrer"><span style="font-size: 10.5pt; font-family: "Times New Roman", serif, serif, EmojiFont; color: blue;"><img class="EmojiInsert" alt="Description: CRI Logo" style="width:176.99pt; height:38.99pt" data-outlook-trace="F:0|T:1" src="cid:image001.jpg@01D4CCEA.783A5790"></span></a></p>
<br>
</div>
</div>
</div>
</div>
</body>
</html>