<div dir="ltr">Dear Peter,<div><br></div><div>Thanks very much for pointing out my error. I mean: Lustre-2.17.0, i.e., the newest release.</div><div><br></div><div>Sorry for my uncareful typo.</div><div><br></div><div>Best Regards,</div><div>T.H.Hsieh</div></div><br><div class="gmail_quote gmail_quote_container"><div dir="ltr" class="gmail_attr">Peter Jones <<a href="mailto:pjones@thelustrecollective.com">pjones@thelustrecollective.com</a>> 於 2026年3月30日週一 下午9:41寫道：<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>
<div style="direction:ltr;font-family:Aptos,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
There is a mix of Lustre versions - 2.7.0 and 2.17.0 - referenced below. The former is ~ 11 years old and it would be no surprise if there were issues trying to build with current versions of other software. Was this just an error during the writeup?</div>
<div style="direction:ltr;font-family:Aptos,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<br>
</div>
<div id="m_8011546441458516286mail-editor-reference-message-container">
<div style="direction:ltr">
</div>
<div style="text-align:left;padding:3pt 0in 0in;border-width:1pt medium medium;border-style:solid none none;border-color:rgb(181,196,223) currentcolor currentcolor;font-family:Aptos;font-size:12pt;color:black">
<b>From: </b>lustre-discuss <<a href="mailto:lustre-discuss-bounces@lists.lustre.org" target="_blank">lustre-discuss-bounces@lists.lustre.org</a>> on behalf of Tung-Han Hsieh via lustre-discuss <<a href="mailto:lustre-discuss@lists.lustre.org" target="_blank">lustre-discuss@lists.lustre.org</a>><br>
<b>Date: </b>Sunday, March 29, 2026 at 9:21 PM<br>
<b>To: </b>lustre <<a href="mailto:lustre-discuss@lists.lustre.org" target="_blank">lustre-discuss@lists.lustre.org</a>><br>
<b>Subject: </b>[lustre-discuss] Issues of building Lustre-2.7.0 for vanilla Linux kernel<br>
<br>
</div>
<div style="direction:ltr">
Dear All,</div>
<div style="direction:ltr">
<br>
</div>
<div style="direction:ltr">
Recently we tried to build Lustre-2.7.0 server for vanilla Linux kernel version 5, plus zfs-2.3.4, under Debian-12.13 Linux system with Mellanox Infiniband enabled. There are some issues to report. The building options are fixed to:</div>
<div style="direction:ltr">
<br>
</div>
<div style="direction:ltr">
- for zfs-2.3.4:</div>
<div style="direction:ltr">
./configure --prefix=/opt/lustre --with-linux=/usr/src/linux-<version> </div>
<div style="direction:ltr">
- for lustre-2.7.0:</div>
<div style="direction:ltr">
export KBUILD_EXTRA_SYMBOLS=/usr/src/zfs-2.3.4/module/Module.symvers</div>
<div style="direction:ltr">
./configure --prefix=/opt/lustre --with-linux=/usr/src/linux-<version> \</div>
<div style="direction:ltr">
--with-zfs=/usr/src/zfs-2.3.4 --with-o2ib=yes --disable-ldiskfs \</div>
<div style="direction:ltr">
--disable-tests --enable-mpitests=no</div>
<div style="direction:ltr">
<br>
</div>
<div style="direction:ltr">
1. For vanilla Linux kernel 5.4.279:</div>
<div style="direction:ltr">
During compilation, we encounter the following errors:</div>
<div style="direction:ltr">
=====================================================================</div>
<div style="direction:ltr">
CC [M] /usr/src/lustre-2.17.0/libcfs/libcfs/crypto/keyring.o<br>
/usr/src/lustre-2.17.0/libcfs/libcfs/crypto/keyring.c: In function free_master_key:<br>
/usr/src/lustre-2.17.0/libcfs/libcfs/crypto/keyring.c:55:9: error: implicit declaration of function kfree_sensitive; did you mean kvfree_sensitive? [-Werror=implicit-function-declaration]<br>
55 | kfree_sensitive(mk);<br>
| ^~~~~~~~~~~~~~~<br>
| kvfree_sensitive<br>
cc1: all warnings being treated as errors</div>
<div style="direction:ltr">
=====================================================================</div>
<div style="direction:ltr">
<br>
</div>
<div style="direction:ltr">
After debug, we have to add the following code:</div>
<div style="direction:ltr">
=====================================================================</div>
<div style="direction:ltr">
// In the beginning of the source file, and after all the #include files.</div>
<div style="direction:ltr">
#ifndef HAVE_KFREE_SENSITIVE<br>
#define kfree_sensitive(x) kzfree(x)<br>
#endif</div>
<div style="direction:ltr">
=====================================================================</div>
<div style="direction:ltr">
<br>
</div>
<div style="direction:ltr">
into the following source files:</div>
<div style="direction:ltr">
<br>
</div>
<div style="direction:ltr">
- lustre-2.7.0/libcfs/libcfs/crypto/keyring.c</div>
<div style="direction:ltr">
- lustre-2.7.0/libcfs/libcfs/crypto/keysetup_v1.c</div>
<div style="direction:ltr">
<br>
</div>
<div style="direction:ltr">
to work around this problem, since kfree_sensitive() seems not available in vanilla Linux kernel 5.4.X.</div>
<div style="direction:ltr">
<br>
</div>
<div style="direction:ltr">
2. For vanilla Linux kernel 5.10.252:</div>
<div style="direction:ltr">
During compilation, we encounter the following errors:</div>
<div style="direction:ltr">
=====================================================================</div>
<div style="direction:ltr">
CC [M] /usr/src/lustre-2.17.0/lustre/llite/pcc.o<br>
/usr/src/lustre-2.17.0/lustre/llite/pcc.c: In function __pcc_file_reset_projid:<br>
/usr/src/lustre-2.17.0/lustre/llite/pcc.c:3491:18: error: implicit declaration of function get_fs; did you mean sget_fc? [-Werror=implicit-function-declaration]<br>
3491 | old_fs = get_fs();<br>
| ^~~~~~<br>
| sget_fc<br>
/usr/src/lustre-2.17.0/lustre/llite/pcc.c:3491:18: error: incompatible types when assigning to type mm_segment_t from type int<br>
/usr/src/lustre-2.17.0/lustre/llite/pcc.c:3492:9: error: implicit declaration of function set_fs; did you mean sget_fc? [-Werror=implicit-function-declaration]<br>
3492 | set_fs(KERNEL_DS);<br>
| ^~~~~~<br>
| sget_fc<br>
/usr/src/lustre-2.17.0/lustre/llite/pcc.c:3492:16: error: KERNEL_DS undeclared (first use in this function); did you mean KERNFS_NS?<br>
3492 | set_fs(KERNEL_DS);<br>
| ^~~~~~~~~<br>
| KERNFS_NS<br>
/usr/src/lustre-2.17.0/lustre/llite/pcc.c:3492:16: note: each undeclared identifier is reported only once for each function it appears in<br>
cc1: all warnings being treated as errors</div>
<div style="direction:ltr">
=====================================================================</div>
<div style="direction:ltr">
<br>
</div>
<div style="direction:ltr">
When looking into this problem, the static int __pcc_file_reset_projid() function of the file: lustre-2.17.0/lustre/lllite/pcc.c reads:</div>
<div style="direction:ltr">
=====================================================================</div>
<div style="direction:ltr">
#ifdef HAVE_FILEATTR_GET<br>
....<br>
rc = inode->i_op->fileattr_set(&nop_mnt_idmap, dentry, &fa);<br>
#else<br>
....<br>
old_fs = get_fs();<br>
set_fs(KERNEL_DS);<br>
....</div>
<div style="direction:ltr">
#endif</div>
<div style="direction:ltr">
=====================================================================</div>
<div style="direction:ltr">
It is awkward that, for vanilla Linux kernel 5.10.X, the old interface get_fs() and set_fs() were removed. However, the new interface fileattr_set() is not available. As a result, Lustre-2.17.0 cannot be built under vanilla Linux kernel 5.10.X, unless there
is a specific patch.</div>
<div style="direction:ltr">
<br>
</div>
<div style="direction:ltr">
3. For vanilla Linux kernel 5.15.202:</div>
<div style="direction:ltr">
Lustre-2.17.0 can be built successfully. However, when loading lustre modules:</div>
<div style="direction:ltr">
modprobe lustre</div>
<div style="direction:ltr">
modprobe lnet</div>
<div style="direction:ltr">
<br>
</div>
<div style="direction:ltr">
Both command encountered the following error message in dmesg:</div>
<div style="direction:ltr">
<br>
</div>
<div style="direction:ltr">
=====================================================================</div>
<div style="direction:ltr">
lustre_symbols_init: error -38<br>
LNetError: 85487:0:(module.c:664:libcfs_init()) cfs_arch_init: error -38</div>
<div style="direction:ltr">
=====================================================================</div>
<div style="direction:ltr">
<br>
</div>
<div style="direction:ltr">
I did not debug it carefully. But I suspect that the mechanism of loading modules in kernel 5.15.X might be changed. So Lustre cannot start in this environment.</div>
<div style="direction:ltr">
<br>
</div>
<div style="direction:ltr">
So, currently Lustre-2.17.0 server with zfs-2.3.4 can only work in vanilla kernel 5.4.X (I only tested 5.4.279). I hope that this report can be considered to add patches for vanilla kernel 5.15.X, or even 5.10.X, since newer releases of Linux kernel is needed
for new hardware as the file servers.</div>
<div style="direction:ltr">
<br>
</div>
<div style="direction:ltr">
Best Regards,</div>
<div style="direction:ltr">
T.H.Hsieh</div>
</div>
</div>
</blockquote></div>