<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
Hello Tung-Han,
<div class=""><br class="">
</div>
<div class="">ZFS 0.7.11 doesn’t compile cleanly with Lustre, yet.</div>
<div class=""><br class="">
</div>
<div class="">There’s a ticket for adding ZFS 0.7.11 support to lustre: </div>
<div class=""><a href="https://jira.whamcloud.com/browse/LU-11393" class="">https://jira.whamcloud.com/browse/LU-11393</a></div>
<div class=""><br class="">
</div>
<div class="">It has patches for master (pre-2.12) and a separate patch for 2.10.</div>
<div class=""><br class="">
</div>
<div class="">—</div>
<div class="">Nathaniel Clark <<a href="mailto:nclark@whamcloud.com" class="">nclark@whamcloud.com</a>></div>
<div class="">Senior Engineer</div>
<div class="">Whamcloud / DDN</div>
<div class="">
<div><br class="">
<blockquote type="cite" class="">
<div class="">On Sep 24, 2018, at 2:15 PM, Tung-Han Hsieh <<a href="mailto:thhsieh@twcp1.phys.ntu.edu.tw" class="">thhsieh@twcp1.phys.ntu.edu.tw</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div class="">Dear All,<br class="">
<br class="">
I am trying to install Lustre version 2.10.5 with ZFS-0.7.11<br class="">
from source code. After compilation and installation, I tried<br class="">
to load the "lustre" module, but encountered the following<br class="">
error:<br class="">
<br class="">
# modprobe lustre<br class="">
could not load module 'lustre': no such device<br class="">
<br class="">
My procedure of installation is the following:<br class="">
<br class="">
1. Compile vanilla kernel 3.12.72 downloaded from:<br class="">
<a href="https://mirrors.edge.kernel.org/pub/linux/kernel/v3.x/linux-3.12.72.tar.gz" class="">https://mirrors.edge.kernel.org/pub/linux/kernel/v3.x/linux-3.12.72.tar.gz</a><br class="">
<br class="">
2. Compile spl-0.7.11 downloaded from:<br class="">
<a href="https://github.com/zfsonlinux/zfs/releases/download/zfs-0.7.11/spl-0.7.11.tar.gz" class="">https://github.com/zfsonlinux/zfs/releases/download/zfs-0.7.11/spl-0.7.11.tar.gz</a><br class="">
<br class="">
with the following steps:<br class="">
# ./configure --prefix=/opt/lustre --with-linux=/path/to/linux-3.12.72<br class="">
# make<br class="">
# make install<br class="">
<br class="">
3. Compile zfs-0.7.11 downloaded from:<br class="">
<a href="https://github.com/zfsonlinux/zfs/releases/download/zfs-0.7.11/zfs-0.7.11.tar.gz" class="">https://github.com/zfsonlinux/zfs/releases/download/zfs-0.7.11/zfs-0.7.11.tar.gz</a><br class="">
<br class="">
with the following steps:<br class="">
# ./configure --prefix=/opt/lustre \<br class="">
--with-linux=/path/to/linux-3.12.72 \<br class="">
--with-spl=/path/to/spl-0.7.11<br class="">
# make<br class="">
# make install<br class="">
<br class="">
4. Compile lustre downloaded from:<br class="">
<a href="https://downloads.whamcloud.com/public/lustre/lustre-2.10.5/sles12sp3/client/SRPMS/lustre-2.10.5-1.src.rpm" class="">https://downloads.whamcloud.com/public/lustre/lustre-2.10.5/sles12sp3/client/SRPMS/lustre-2.10.5-1.src.rpm</a><br class="">
<br class="">
Then I unpack the SRPM by the command:<br class="">
# rpm2cpio lustre-2.10.5-1.src.rpm | cpio --extract --make-directories<br class="">
<br class="">
and compile it by the following:<br class="">
# ./configure --prefix=/opt/lustre \<br class="">
--with-linux=/path/to/linux-3.12.72 \<br class="">
--with-spl=/path/to/spl-0.7.11 \<br class="">
--with-zfs=/path/to/zfs-0.7.11 \<br class="">
--with-o2ib=no \<br class="">
--disable-ldiskfs<br class="">
# make<br class="">
# make install<br class="">
<br class="">
5. I have made sure the following settings and utilities are correct:<br class="">
- PATH contains /opt/lustre/bin and /opt/lustre/sbin<br class="">
- /sbin/mount.lustre exists.<br class="">
- /sbin/mount.zfs exists.<br class="">
- /usr/sbin/l_getidentity exists.<br class="">
- /usr/sbin/ko2iblnd-probe exists.<br class="">
- /etc/modprobe.d/lustre.conf contains:<br class="">
options lnet networks=tcp<br class="">
- /etc/modprobe.d/ko2iblnd.conf contains:<br class="">
alias ko2iblnd-opa ko2iblnd<br class="">
options ko2iblnd-opa peer_credits=128 peer_credits_hiw=64 credits=1024 concurrent_sends=256 ntx=2048 map_on_demand=32 fmr_pool_size=2048 fmr_flush_trigger=512 fmr_cache=1<br class="">
install ko2iblnd /usr/sbin/ko2iblnd-probe<br class="">
<br class="">
Then I tried to run "modprobe lustre", it says "no such device" error.<br class="">
<br class="">
I tried to replace Lustre-2.10.5 by Lustre-2.9 downloaded from:<br class="">
<br class="">
<a href="https://downloads.whamcloud.com/public/lustre/lustre-2.9.0/sles12sp1/client/SRPMS/lustre-2.9.0-1.src.rpm" class="">https://downloads.whamcloud.com/public/lustre/lustre-2.9.0/sles12sp1/client/SRPMS/lustre-2.9.0-1.src.rpm</a><br class="">
<br class="">
and proceed exactly the same installation steps. Everything works fine.<br class="">
<br class="">
Could anyone suggest me what have I missed for lustre-2.10.5 ? Or suggest<br class="">
me how to debug.<br class="">
<br class="">
Thanks very much.<br class="">
<br class="">
<br class="">
T.H.Hsieh<br class="">
_______________________________________________<br class="">
lustre-discuss mailing list<br class="">
lustre-discuss@lists.lustre.org<br class="">
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org<br class="">
</div>
</div>
</blockquote>
</div>
<br class="">
</div>
</body>
</html>