[lustre-discuss] ksym errors on kmod-lustre RPM after 2.10.4 build against MOFED 4.3
Matt Rásó-Barnett
matt at rasobarnett.com
Tue Aug 7 15:42:23 PDT 2018
Hello,
I've been trying to build a fresh set of RPMs to reproduce the publicly
available build of Lustre v2_10_4 against RHEL7.5
kernel-3.10.0-862.2.3.el7 with MOFED 4.3-3.0.2.1:
https://downloads.whamcloud.com/public/lustre/lustre-2.10.4-ib/MOFED-4.3-3.0.2.1/
To do this I've been closely following the instructions at
http://wiki.lustre.org/Compiling_Lustre
The build process itself is successful, however I am running into an
error when I try to install the lustre RPMs, and specifically get the
following:
[root at build ~]# yum --enablerepo=lustre-server install lustre
kmod-lustre kmod-lustre-osd-ldiskfs lustre-osd-ldiskfs-mount
Loaded plugins: product-id, search-disabled-repos, subscription-manager
...
Resolving Dependencies
--> Running transaction check
---> Package kmod-lustre.x86_64 0:2.10.4-1.el7 will be installed
--> Processing Dependency: ksym(ib_get_dma_mr) = 0x9add141e for package: kmod-lustre-2.10.4-1.el7.x86_64
--> Processing Dependency: ksym(backport_dependency_symbol) = 0xb43a926b for package: kmod-lustre-2.10.4-1.el7.x86_64
--> Processing Dependency: ksym(__rdma_create_id) = 0x4f20f8d2 for package: kmod-lustre-2.10.4-1.el7.x86_64
--> Processing Dependency: ksym(__rdma_accept) = 0x2962f069 for package: kmod-lustre-2.10.4-1.el7.x86_64
---> Package kmod-lustre-osd-ldiskfs.x86_64 0:2.10.4-1.el7 will be installed
---> Package lustre.x86_64 0:2.10.4-1.el7 will be installed
---> Package lustre-osd-ldiskfs-mount.x86_64 0:2.10.4-1.el7 will be installed
--> Running transaction check
---> Package kmod-lustre.x86_64 0:2.10.4-1.el7 will be installed
--> Processing Dependency: ksym(ib_get_dma_mr) = 0x9add141e for package: kmod-lustre-2.10.4-1.el7.x86_64
--> Processing Dependency: ksym(__rdma_create_id) = 0x4f20f8d2 for package: kmod-lustre-2.10.4-1.el7.x86_64
--> Processing Dependency: ksym(__rdma_accept) = 0x2962f069 for package: kmod-lustre-2.10.4-1.el7.x86_64
---> Package kmod-mlnx-ofa_kernel.x86_64 0:4.3-OFED.4.3.3.0.2.1.gcf60532.rhel7u5 will be installed
--> Finished Dependency Resolution
Error: Package: kmod-lustre-2.10.4-1.el7.x86_64 (lustre-server)
Requires: ksym(__rdma_create_id) = 0x4f20f8d2
Installing: kmod-mlnx-ofa_kernel-4.3-OFED.4.3.3.0.2.1.gcf60532.rhel7u5.x86_64 (mlnx-ofed)
ksym(__rdma_create_id) = 0x9b63e879
Error: Package: kmod-lustre-2.10.4-1.el7.x86_64 (lustre-server)
Requires: ksym(ib_get_dma_mr) = 0x9add141e
Installing: kmod-mlnx-ofa_kernel-4.3-OFED.4.3.3.0.2.1.gcf60532.rhel7u5.x86_64 (mlnx-ofed)
ksym(ib_get_dma_mr) = 0x87f4f63d
Error: Package: kmod-lustre-2.10.4-1.el7.x86_64 (lustre-server)
Requires: ksym(__rdma_accept) = 0x2962f069
Installing: kmod-mlnx-ofa_kernel-4.3-OFED.4.3.3.0.2.1.gcf60532.rhel7u5.x86_64 (mlnx-ofed)
ksym(__rdma_accept) = 0x530ea333
Error: Package: kmod-lustre-2.10.4-1.el7.x86_64 (lustre-server)
Requires: ksym(ib_get_dma_mr) = 0x9add141e
Available: kmod-mlnx-ofa_kernel-4.3-OFED.4.3.3.0.2.1.gcf60532.rhel7u5.x86_64 (mlnx-ofed)
ksym(ib_get_dma_mr) = 0x87f4f63d
**********************************************************************
yum can be configured to try to resolve such errors by temporarily enabling
disabled repos and searching for missing dependencies.
To enable this functionality please set 'notify_only=0' in /etc/yum/pluginconf.d/search-disabled-repos.conf
**********************************************************************
Error: Package: kmod-lustre-2.10.4-1.el7.x86_64 (lustre-server)
Requires: ksym(__rdma_create_id) = 0x4f20f8d2
Installing: kmod-mlnx-ofa_kernel-4.3-OFED.4.3.3.0.2.1.gcf60532.rhel7u5.x86_64 (mlnx-ofed)
ksym(__rdma_create_id) = 0x9b63e879
Error: Package: kmod-lustre-2.10.4-1.el7.x86_64 (lustre-server)
Requires: ksym(ib_get_dma_mr) = 0x9add141e
Installing: kmod-mlnx-ofa_kernel-4.3-OFED.4.3.3.0.2.1.gcf60532.rhel7u5.x86_64 (mlnx-ofed)
ksym(ib_get_dma_mr) = 0x87f4f63d
Error: Package: kmod-lustre-2.10.4-1.el7.x86_64 (lustre-server)
Requires: ksym(__rdma_accept) = 0x2962f069
Installing: kmod-mlnx-ofa_kernel-4.3-OFED.4.3.3.0.2.1.gcf60532.rhel7u5.x86_64 (mlnx-ofed)
ksym(__rdma_accept) = 0x530ea333
Error: Package: kmod-lustre-2.10.4-1.el7.x86_64 (lustre-server)
Requires: ksym(ib_get_dma_mr) = 0x9add141e
Available: kmod-mlnx-ofa_kernel-4.3-OFED.4.3.3.0.2.1.gcf60532.rhel7u5.x86_64 (mlnx-ofed)
ksym(ib_get_dma_mr) = 0x87f4f63d
You could try using --skip-broken to work around the problem
You could try running: rpm -Va --nofiles --nodigest
My build process has roughly been:
* Download lustre-src and checkout tag v2_10_4
* Build patched kernel RPMs against kernel-3.10.0-862.2.3.el7 as
described in lustre wiki:
rpmbuild -ba --with firmware --with baseonly --without kabichk
--without perf --define "buildid _lustre" --target x86_64
SPECS/kernel.spec
* Install patched kernel RPMs and reboot into this kernel (also remove
all other kernels on the system)
* Download MLNX_OFED_LINUX-4.3-3.0.2.1-rhel7.5-x86_64.tgz
* Add kernel support for the above patched kernel with:
./mlnx_add_kernel_support.sh --make-tgz -m
/root/MLNX_OFED_LINUX-4.3-3.0.2.1-rhel7.5-x86_64
* Take produced RPMs from the above and install them with:
yum --enablerepo=mlnx-ofed install
mlnx-ofed-all-3.10.0-862.2.3.el7_lustre.x86_64
* Then I build Lustre RPMs with:
./configure --enable-server
--with-linux=/usr/src/kernels/3.10.0-862.2.3.el7_lustre.x86_64
--with-o2ib=/usr/src/ofa_kernel/default
make rpms
However when trying to install the RPMs produced I get the ksym errors
as above.
One other thing - I've installed the lustre rpms outside of yum with:
rpm -ivh --nodeps lustre-2.10.4-1.el7.x86_64.rpm
kmod-lustre-2.10.4-1.el7.x86_64.rpm
kmod-lustre-osd-ldiskfs-2.10.4-1.el7.x86_64.rpm
lustre-osd-ldiskfs-mount-2.10.4-1.el7.x86_64.rpm
which gets me past the ksym error above. With these installed I'm able
to load the lnet/ko2iblnd/lustre modules successfully (with no errors in
dmesg), and I'm able to setup the infiniband LNET NID successfully, so
it feels like the build has been successful but perhaps I've made an
error somewhere that affects the generated RPM dependencies perhaps? Or
there is an error to do with kABI compatibility that I don't understand?
Does anyone have any advice on whether the above build steps,
particularly when building with MOFED, look like how this is normally
done? Any ideas on what I'm doing wrong here?
Many thanks,
Matt
--
Matt Rásó-Barnett
Research Computing Platforms
University Information Services
High Performance Computing Service
University of Cambridge
More information about the lustre-discuss
mailing list