[lustre-discuss] lustre-discuss Digest, Vol 149, Issue 5

Gin Tan gin.tan at monash.edu
Thu Aug 9 05:05:12 PDT 2018


Had the same issue, it was because I still had the kernel modules loaded
while building the RPMs.

Uninstalled kmod-lustre and re-compiled fixed the ksym issues.

Also, there's a page allocation issue with MoFED 4.3, use MoFED 4.4.


-- 

*Gin Tan*
MASSIVE support and consulting services

*Monash eResearch Centre*
Monash University

Room G63B, 15 Innovation Walk
Clayton Campus
Wellington Road
Clayton VIC 3800
Australia

T: +61 3 9902 0245
E: gin.tan at monash.edu
Z: https://monash.zoom.us/my/gintan
www.monash.edu.au/eresearch


On Thu, 9 Aug 2018 at 06:21, <lustre-discuss-request at lists.lustre.org>
wrote:

> Send lustre-discuss mailing list submissions to
>         lustre-discuss at lists.lustre.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
>         http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
> or, via email, send a message with subject or body 'help' to
>         lustre-discuss-request at lists.lustre.org
>
> You can reach the person managing the list at
>         lustre-discuss-owner at lists.lustre.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of lustre-discuss digest..."
>
>
> Today's Topics:
>
>    1. ksym errors on kmod-lustre RPM after 2.10.4 build against
>       MOFED 4.3 (Matt R?s?-Barnett)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Tue, 7 Aug 2018 23:42:23 +0100
> From: Matt R?s?-Barnett <matt at rasobarnett.com>
> To: "lustre-discuss at lists.lustre.org"
>         <lustre-discuss at lists.lustre.org>
> Subject: [lustre-discuss] ksym errors on kmod-lustre RPM after 2.10.4
>         build against MOFED 4.3
> Message-ID: <20180807224223.rhags26mq2vlp6tt at yoshimo.localdomain>
> Content-Type: text/plain; charset=iso-8859-1; format=flowed
>
> Hello,
>
> I've been trying to build a fresh set of RPMs to reproduce the publicly
> available build of Lustre v2_10_4 against RHEL7.5
> kernel-3.10.0-862.2.3.el7 with MOFED 4.3-3.0.2.1:
>
> https://downloads.whamcloud.com/public/lustre/lustre-2.10.4-ib/MOFED-4.3-3.0.2.1/
>
> To do this I've been closely following the instructions at
> http://wiki.lustre.org/Compiling_Lustre
>
> The build process itself is successful, however I am running into an
> error when I try to install the lustre RPMs, and specifically get the
> following:
>
> [root at build ~]# yum --enablerepo=lustre-server install lustre
> kmod-lustre kmod-lustre-osd-ldiskfs lustre-osd-ldiskfs-mount
> Loaded plugins: product-id, search-disabled-repos, subscription-manager
> ...
> Resolving Dependencies
> --> Running transaction check
> ---> Package kmod-lustre.x86_64 0:2.10.4-1.el7 will be installed
> --> Processing Dependency: ksym(ib_get_dma_mr) = 0x9add141e for package:
> kmod-lustre-2.10.4-1.el7.x86_64
> --> Processing Dependency: ksym(backport_dependency_symbol) = 0xb43a926b
> for package: kmod-lustre-2.10.4-1.el7.x86_64
> --> Processing Dependency: ksym(__rdma_create_id) = 0x4f20f8d2 for
> package: kmod-lustre-2.10.4-1.el7.x86_64
> --> Processing Dependency: ksym(__rdma_accept) = 0x2962f069 for package:
> kmod-lustre-2.10.4-1.el7.x86_64
> ---> Package kmod-lustre-osd-ldiskfs.x86_64 0:2.10.4-1.el7 will be
> installed
> ---> Package lustre.x86_64 0:2.10.4-1.el7 will be installed
> ---> Package lustre-osd-ldiskfs-mount.x86_64 0:2.10.4-1.el7 will be
> installed
> --> Running transaction check
> ---> Package kmod-lustre.x86_64 0:2.10.4-1.el7 will be installed
> --> Processing Dependency: ksym(ib_get_dma_mr) = 0x9add141e for package:
> kmod-lustre-2.10.4-1.el7.x86_64
> --> Processing Dependency: ksym(__rdma_create_id) = 0x4f20f8d2 for
> package: kmod-lustre-2.10.4-1.el7.x86_64
> --> Processing Dependency: ksym(__rdma_accept) = 0x2962f069 for package:
> kmod-lustre-2.10.4-1.el7.x86_64
> ---> Package kmod-mlnx-ofa_kernel.x86_64
> 0:4.3-OFED.4.3.3.0.2.1.gcf60532.rhel7u5 will be installed
> --> Finished Dependency Resolution
> Error: Package: kmod-lustre-2.10.4-1.el7.x86_64 (lustre-server)
>            Requires: ksym(__rdma_create_id) = 0x4f20f8d2
>            Installing:
> kmod-mlnx-ofa_kernel-4.3-OFED.4.3.3.0.2.1.gcf60532.rhel7u5.x86_64
> (mlnx-ofed)
>                ksym(__rdma_create_id) = 0x9b63e879
> Error: Package: kmod-lustre-2.10.4-1.el7.x86_64 (lustre-server)
>            Requires: ksym(ib_get_dma_mr) = 0x9add141e
>            Installing:
> kmod-mlnx-ofa_kernel-4.3-OFED.4.3.3.0.2.1.gcf60532.rhel7u5.x86_64
> (mlnx-ofed)
>                ksym(ib_get_dma_mr) = 0x87f4f63d
> Error: Package: kmod-lustre-2.10.4-1.el7.x86_64 (lustre-server)
>            Requires: ksym(__rdma_accept) = 0x2962f069
>            Installing:
> kmod-mlnx-ofa_kernel-4.3-OFED.4.3.3.0.2.1.gcf60532.rhel7u5.x86_64
> (mlnx-ofed)
>                ksym(__rdma_accept) = 0x530ea333
> Error: Package: kmod-lustre-2.10.4-1.el7.x86_64 (lustre-server)
>            Requires: ksym(ib_get_dma_mr) = 0x9add141e
>            Available:
> kmod-mlnx-ofa_kernel-4.3-OFED.4.3.3.0.2.1.gcf60532.rhel7u5.x86_64
> (mlnx-ofed)
>                ksym(ib_get_dma_mr) = 0x87f4f63d
> **********************************************************************
> yum can be configured to try to resolve such errors by temporarily enabling
> disabled repos and searching for missing dependencies.
> To enable this functionality please set 'notify_only=0' in
> /etc/yum/pluginconf.d/search-disabled-repos.conf
> **********************************************************************
>
> Error: Package: kmod-lustre-2.10.4-1.el7.x86_64 (lustre-server)
>            Requires: ksym(__rdma_create_id) = 0x4f20f8d2
>            Installing:
> kmod-mlnx-ofa_kernel-4.3-OFED.4.3.3.0.2.1.gcf60532.rhel7u5.x86_64
> (mlnx-ofed)
>                ksym(__rdma_create_id) = 0x9b63e879
> Error: Package: kmod-lustre-2.10.4-1.el7.x86_64 (lustre-server)
>            Requires: ksym(ib_get_dma_mr) = 0x9add141e
>            Installing:
> kmod-mlnx-ofa_kernel-4.3-OFED.4.3.3.0.2.1.gcf60532.rhel7u5.x86_64
> (mlnx-ofed)
>                ksym(ib_get_dma_mr) = 0x87f4f63d
> Error: Package: kmod-lustre-2.10.4-1.el7.x86_64 (lustre-server)
>            Requires: ksym(__rdma_accept) = 0x2962f069
>            Installing:
> kmod-mlnx-ofa_kernel-4.3-OFED.4.3.3.0.2.1.gcf60532.rhel7u5.x86_64
> (mlnx-ofed)
>                ksym(__rdma_accept) = 0x530ea333
> Error: Package: kmod-lustre-2.10.4-1.el7.x86_64 (lustre-server)
>            Requires: ksym(ib_get_dma_mr) = 0x9add141e
>            Available:
> kmod-mlnx-ofa_kernel-4.3-OFED.4.3.3.0.2.1.gcf60532.rhel7u5.x86_64
> (mlnx-ofed)
>                ksym(ib_get_dma_mr) = 0x87f4f63d
>  You could try using --skip-broken to work around the problem
>  You could try running: rpm -Va --nofiles --nodigest
>
> My build process has roughly been:
>
> * Download lustre-src and checkout tag v2_10_4
> * Build patched kernel RPMs against kernel-3.10.0-862.2.3.el7 as
> described in lustre wiki:
>
>   rpmbuild -ba --with firmware --with baseonly --without kabichk
>   --without perf --define "buildid _lustre" --target x86_64
>   SPECS/kernel.spec
>
> * Install patched kernel RPMs and reboot into this kernel (also remove
> all other kernels on the system)
> * Download MLNX_OFED_LINUX-4.3-3.0.2.1-rhel7.5-x86_64.tgz
> * Add kernel support for the above patched kernel with:
>
>   ./mlnx_add_kernel_support.sh --make-tgz -m
>   /root/MLNX_OFED_LINUX-4.3-3.0.2.1-rhel7.5-x86_64
>
> * Take produced RPMs from the above and install them with:
>
>   yum --enablerepo=mlnx-ofed install
>   mlnx-ofed-all-3.10.0-862.2.3.el7_lustre.x86_64
>
> * Then I build Lustre RPMs with:
>
>   ./configure --enable-server
>   --with-linux=/usr/src/kernels/3.10.0-862.2.3.el7_lustre.x86_64
>   --with-o2ib=/usr/src/ofa_kernel/default
>
>   make rpms
>
> However when trying to install the RPMs produced I get the ksym errors
> as above.
>
> One other thing - I've installed the lustre rpms outside of yum with:
>
>   rpm -ivh --nodeps lustre-2.10.4-1.el7.x86_64.rpm
>   kmod-lustre-2.10.4-1.el7.x86_64.rpm
>   kmod-lustre-osd-ldiskfs-2.10.4-1.el7.x86_64.rpm
>   lustre-osd-ldiskfs-mount-2.10.4-1.el7.x86_64.rpm
>
> which gets me past the ksym error above. With these installed I'm able
> to load the lnet/ko2iblnd/lustre modules successfully (with no errors in
> dmesg), and I'm able to setup the infiniband LNET NID successfully, so
> it feels like the build has been successful but perhaps I've made an
> error somewhere that affects the generated RPM dependencies perhaps? Or
> there is an error to do with kABI compatibility that I don't understand?
>
> Does anyone have any advice on whether the above build steps,
> particularly when building with MOFED, look like how this is normally
> done? Any ideas on what I'm doing wrong here?
>
> Many thanks,
> Matt
>
> --
> Matt R?s?-Barnett
> Research Computing Platforms
> University Information Services
> High Performance Computing Service
> University of Cambridge
>
>
> ------------------------------
>
> Subject: Digest Footer
>
> _______________________________________________
> lustre-discuss mailing list
> lustre-discuss at lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>
>
> ------------------------------
>
> End of lustre-discuss Digest, Vol 149, Issue 5
> **********************************************
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20180809/9445d8d0/attachment.html>


More information about the lustre-discuss mailing list