<div dir="ltr">Hello, Christopher, <div><br></div><div>Sorry to bother you again, I seems couldn't find the <span style="font-size:14px">kmod-zfs-devel kmod-spl-devel package on the zfsonlinux. I supposed I should use the "</span><span style="color:rgb(51,51,51);font-family:'Helvetica Neue',Helvetica,'Segoe UI',Arial,freesans,sans-serif,'Apple Color Emoji','Segoe UI Emoji','Segoe UI Symbol';font-size:16px">kABI-tracking k</span><font color="#333333" face="Helvetica Neue, Helvetica, Segoe UI, Arial, freesans, sans-serif, Apple Color Emoji, Segoe UI Emoji, Segoe UI Symbol" size="3">mod" package, right? So I changed zfs.repo to use kmod(</font><span style="font-family:Menlo;font-size:18px">baseurl=<a href="http://archive.zfsonlinux.org/epel/7/kmod/$basearch/">http://archive.zfsonlinux.org/epel/7/kmod/$basearch/</a>)</span><span style="color:rgb(51,51,51);font-family:'Helvetica Neue',Helvetica,'Segoe UI',Arial,freesans,sans-serif,'Apple Color Emoji','Segoe UI Emoji','Segoe UI Symbol';font-size:medium">, but "yum install zfs" still says it's going to install "dkms, zfs-dkms, spl-dkms". Wondering if you or anybody could help me to figure this out. </span></div><div><span style="color:rgb(51,51,51);font-family:'Helvetica Neue',Helvetica,'Segoe UI',Arial,freesans,sans-serif,'Apple Color Emoji','Segoe UI Emoji','Segoe UI Symbol';font-size:medium"><br></span></div><div><span style="color:rgb(51,51,51);font-family:'Helvetica Neue',Helvetica,'Segoe UI',Arial,freesans,sans-serif,'Apple Color Emoji','Segoe UI Emoji','Segoe UI Symbol';font-size:medium">Thanks in advance for any advices!</span></div><div><span style="color:rgb(51,51,51);font-family:'Helvetica Neue',Helvetica,'Segoe UI',Arial,freesans,sans-serif,'Apple Color Emoji','Segoe UI Emoji','Segoe UI Symbol';font-size:medium"><br></span></div><div><span style="color:rgb(51,51,51);font-family:'Helvetica Neue',Helvetica,'Segoe UI',Arial,freesans,sans-serif,'Apple Color Emoji','Segoe UI Emoji','Segoe UI Symbol';font-size:medium">Chen</span></div><div><span style="color:rgb(51,51,51);font-family:'Helvetica Neue',Helvetica,'Segoe UI',Arial,freesans,sans-serif,'Apple Color Emoji','Segoe UI Emoji','Segoe UI Symbol';font-size:16px"><br></span></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Jun 24, 2016 at 4:15 PM, Christopher J. Morrone <span dir="ltr"><<a href="mailto:morrone2@llnl.gov" target="_blank">morrone2@llnl.gov</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Yes, it is all a lot harder than it should be at this point. We're<br>
slowly getting in packaging changes that should make it a little easier<br>
in future releases. But fear not, it is doable.<br>
<br>
There are other ways, but here is how I would do it with 2.8:<br>
<br>
Skip the installation of the lustre-patched kernel. It is only actually<br>
only required for ldiskfs testing these days. (If all goes well, there<br>
will be no lustre-patched included with the Lustre 2.9 release packages)<br>
Remove that and just use the stock CentOS 7 kernel.<br>
<br>
Next install zfs (and its associate spl packages). You can find<br>
pre-build binary or DKMS packages here:<br>
<br>
<a href="https://github.com/zfsonlinux/zfs/wiki/RHEL-%26-CentOS" rel="noreferrer" target="_blank">https://github.com/zfsonlinux/zfs/wiki/RHEL-%26-CentOS</a><br>
<br>
Or you can compile them yourself using the instructions here:<br>
<br>
<a href="https://github.com/zfsonlinux/zfs/wiki/Custom-Packages" rel="noreferrer" target="_blank">https://github.com/zfsonlinux/zfs/wiki/Custom-Packages</a><br>
<br>
We are going to compile lustre next, so you will also need to install<br>
the following zfs development packages:<br>
<br>
libzfs2-devel kmod-zfs-devel kmod-spl-devel.<br>
<br>
Next download the lustre source rpm from here:<br>
<br>
<br>
<a href="https://downloads.hpdd.intel.com/public/lustre/lustre-2.8.0/el7/server/SRPMS/" rel="noreferrer" target="_blank">https://downloads.hpdd.intel.com/public/lustre/lustre-2.8.0/el7/server/SRPMS/</a><br>
<br>
The package name is<br>
"lustre-2.8.0-3.10.0_327.3.1.el7_lustre.x86_64.src.rpm", which makes it<br>
seem like you have to have that specific kernel. But don't worry, that<br>
was just an artifact of the old, broken packaging system. We are hoping<br>
to fix that before the Lustre 2.9 is released.<br>
<br>
Next you should be able to generate binary lustre rpms with this command:<br>
<br>
rpmbuild --rebuild --with zfs --without ldiskfs<br>
lustre-2.8.0-3.10.0_327.3.1.el7_lustre.x86_64.src.rpm<br>
<br>
I am just guessing that you don't need ldiskfs since you are talking<br>
about zfs. You can decide if you want to compile it or not.<br>
<br>
If you hit any problems let us know. Maybe we'll use this discussion to<br>
work up a new page of documentation in the wiki.<br>
<br>
Chris<br>
<span class=""><br>
On 06/24/2016 07:48 AM, Yu Chen wrote:<br>
> Hello,<br>
><br>
> I am new to lustre system. Was trying to setup a lustre on Centos 7 with<br>
> zfs.<br>
><br>
> When came to the step to start lustre.service from systemctl, got the<br>
> errors:<br>
> ....<br>
><br>
> -------<br>
><br>
> osd_zfs: disagrees about version of symbol dmu_objset_space<br>
><br>
> osd_zfs: Unknown symbol dmu_objset_space (err -22)<br>
><br>
> ------<br>
><br>
> LustreError: 158-c: Can't load module 'osd-zfs'<br>
><br>
> LustreError: 48041:0:(genops.c:318:class_newdev()) OBD: unknown type:<br>
> osd-zfs<br>
><br>
> .....<br>
><br>
><br>
><br>
> Here is what I have done:<br>
><br>
</span>> * I installed kernel-3.10.0-327.3.1.el7_lustre.x86_64,<br>
> * Then configured and installed zfs from <a href="http://archive.zfsonlinux.org" rel="noreferrer" target="_blank">archive.zfsonlinux.org</a><br>
> <<a href="http://archive.zfsonlinux.org" rel="noreferrer" target="_blank">http://archive.zfsonlinux.org</a>>(0.6.5.7-1)<br>
> * Then downloaded and installed lustre rpms from el7.2.1511 from:<br>
> <a href="https://downloads.hpdd.intel.com/public/lustre/latest-feature-release/" rel="noreferrer" target="_blank">https://downloads.hpdd.intel.com/public/lustre/latest-feature-release/</a><br>
> * Then *zpool*create mdt01-pool /dev/sdb<br>
> * mkfs.lustre --reformat --fsname=lustre01 --mgs --mdt<br>
<span class="">> --backfstype=zfs --index=0 --mgsnode=10.1.12.1@o2ib<br>
> --servicenode=10.1.12.1@o2ib mdt01-pool/mgsmdt01<br>
</span>> * echo "mds1 - lustre01-MDT0000 zfs:mdt01-pool/mgsmdt01" > /etc/ldev.conf<br>
> * echo "options lnet networks=o2ib(ib0)" > /etc/modprobe.d/lustre.conf<br>
> * systemctl start lustre.service<br>
<span class="">><br>
> This is where it failed.<br>
><br>
> Then I think it might be because of the zfs version. So downloaded<br>
> lustre-2.8.0-3.10.0_327.3.1.el7_lustre.x86_64.src.rpm, and tried to<br>
> rebuild from it. But after rebuilding, there were no lustre-osd-zfs and<br>
> lustre-osd-zfs-mount rpms?<br>
><br>
><br>
> Would greatly appreciate any suggestions!<br>
><br>
><br>
> Thanks in advance!<br>
><br>
><br>
> yc<br>
><br>
><br>
</span>> _______________________________________________<br>
> lustre-discuss mailing list<br>
> <a href="mailto:lustre-discuss@lists.lustre.org">lustre-discuss@lists.lustre.org</a><br>
> <a href="http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org" rel="noreferrer" target="_blank">http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org</a><br>
><br>
<br>
_______________________________________________<br>
lustre-discuss mailing list<br>
<a href="mailto:lustre-discuss@lists.lustre.org">lustre-discuss@lists.lustre.org</a><br>
<a href="http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org" rel="noreferrer" target="_blank">http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org</a><br>
</blockquote></div><br></div>