[lustre-discuss] Compiling from sources with Debian 8

Jérôme BECOT jerome.becot at inserm.fr
Mon Nov 30 09:18:53 PST 2015


So,

I could move on with building the modules. They are successfully build 
and installed but I had to change the module destination directory in 
config/lustre-build-linux.m4 to target /lib/modules/`uname -r`/extra 
directory (else it goes in "kernel" subdirectory and they never get loaded)

Then it won't load anyway. I'm getting [440444.832446] lnet: no symbol 
version for module_layout in dmesg
If i copy the Module.symvers generated into the lustre-release folder to 
/usr/src/linux, then i get


[438311.953707] lnet: disagrees about version of symbol 
libcfs_deregister_ioctl
[438311.953710] lnet: Unknown symbol libcfs_deregister_ioctl (err -22)
[438311.953725] lnet: Unknown symbol cfs_str2num_check (err 0)
[438311.953760] lnet: Unknown symbol cfs_gettok (err 0)
[438311.953782] lnet: Unknown symbol lprocfs_call_handler (err 0)

(this is a sample of various symbol error)

Any clue ? I'm close to make it now (i guess)


Le 28/11/2015 18:07, Dilger, Andreas a écrit :
> The 2.3.64 version means you are using the in-kernel Lustre client (confirmed by the waning messages about "staging"), and not the 2.7.x version from the Lustre master branch.
>
> It looks like Ubuntu is building the in-kernel client, and your modules are not being loaded.
>
> Cheers, Andreas
>
> On Nov 28, 2015, at 03:26, Jérôme BECOT <jerome.becot at inserm.fr<mailto:jerome.becot at inserm.fr>> wrote:
>
> Hi there,
>
> We run lustre 2.6/2.7 on our Centos 6.6 (servers) and 7 (clients) cluster. We have a few webservers running Debian that need to access the storage. I followed the procedure given by Thomas Stibor about Ubuntu 14 last year.
>
> I could successfully compile the binaries and modules after some digging. He also left an already compiled lustre 2.7.63 and modules for kernel 3.16.0-4 online.
>
> If I install his binaries, it works well. If I install the one generated by the procedure, the modules don't load and a weird thing happen. Running dmesg warns me about one surprising thing :
>
>   > With his packages
> [212417.535369] LNet: HW CPU cores: 1, npartitions: 1
> [212417.538430] alg: No test for adler32 (adler32-zlib)
> [212417.538456] alg: No test for crc32 (crc32-table)
> [212425.548907] Lustre: Lustre: Build Version: v2_7_60_0-ge686e57-CHANGED-3.16.0-4-amd64
> [212425.565330] LNet: Added LNI 172.27.7.118 at tcp1 [8/256/0/180]
> [212425.565354] LNet: Accept secure, port 988
> [212425.595531] Lustre: Mounted lustre-client
>
>> With mine
> [209942.090874] LNet: HW CPU cores: 1, npartitions: 1
> [209942.092902] alg: No test for adler32 (adler32-zlib)
> [209950.092501] lnet: module is from the staging directory, the quality is unknown, you have been warned.
> [209950.093589] lvfs: module is from the staging directory, the quality is unknown, you have been warned.
> [209950.094634] obdclass: module is from the staging directory, the quality is unknown, you have been warned.
> [209950.098595] Lustre: Lustre: Build Version: v2_3_64_0-g6e62c21-CHANGED-3.9.0
> [209950.099999] ptlrpc: module is from the staging directory, the quality is unknown, you have been warned.
> [209950.104615] ksocklnd: module is from the staging directory, the quality is unknown, you have been warned.
> [209950.105237] LNetError: 845:0:(linux-tcpip.c:82:libcfs_ipif_query()) Can't get flags for interface eth0
> [209950.105862] LNetError: 845:0:(socklnd.c:2824:ksocknal_startup()) Can't get interface eth0 info: -515
> [209951.104194] LNetError: 105-4: Error -100 starting up LNI tcp
> [209951.104852] LustreError: 845:0:(events.c:566:ptlrpc_init_portals()) network initialisation failed
> [209990.787541] ptlrpc: module is from the staging directory, the quality is unknown, you have been warned.
>
> I pulled the master git branch, and coul obtain
> linux-patch-lustre_2.7.63.0-16-g8524994_all.deb  lustre-client-modules-3.16.7-ckt11-lustre-my-build_2.7.63.0-16-g8524994_amd64.deb  lustre-tests_2.7.63.0-16-g8524994_amd64.deb
> lustre_2.7.63.0-16-g8524994_amd64.changes        lustre-dev_2.7.63.0-16-g8524994_amd64.deb                                          lustre-utils_2.7.63.0-16-g8524994_amd64.deb
> lustre_2.7.63.0-16-g8524994.dsc                  lustre-release
> lustre_2.7.63.0-16-g8524994.tar.gz               lustre-source_2.7.63.0-16-g8524994_all.deb
>
> I just don't get it. Why the shown version of the module is 2.3 ?
> I tried to compile from the 2.7 branch but the 2.7.0 version doesn't compile with kernel 3.16, as suggested in LU-7042
>
> I probably miss something. Sorry I'm not familiar with compilers (usually the ./configure && make && make install works or warns you about missing dependencies and i don't go farther).
>
> Thank you
>
> Jerome
>
> PS : Here is the procedure followed to compile
>
>
> apt-get install linux-headers-3.16.0-4-amd64 libtool automake linux-source-3.16
> git clone git://git.whamcloud.com/fs/lustre-release.git
> cd /usr/src/
> tar xf linux-source-3.16.tar.xz
> cd /home/build/lustre-release/
> vi debian/rules (edit so it doesn't stop with autogen because it doesn't find the kernel sources)
> (modify --with-linux= option with current path --with-linux=/usr/src/linux-3.16 in kdist_config section)
> aptitude install module-assistant libreadline-dev debhelper  dpatch libsnmp-dev quilt devscripts
>   vi debian/control
> (modify the dependency for unsatisfied linux-headers called linux-headers-something in jessie)
> vi compile.sh
> #!/bin/bash
> unset DEBEMAIL
> unset EMAIL
> unset DEBFULLNAME
> unset NAME
>
> export DEBFULLNAME="Niemand Nobody"
> export EMAIL="npcomplete at example.com<http://example.com>"
>
> # Extract lustre version, replace "_" by "." and remove leading letter "v".
> LUSTRE_VERSION=$(echo `git describe` | sed -e "s/_/\./g" | cut -c2-)
>
> # Add entry into debian/changelog such that packages have proper version names.
> dch --newversion $LUSTRE_VERSION --distribution unstable --nomultimaint -t "Build from official master upstream."
>
> #
> sh ./autogen.sh
>
> # Build debian packages.
> dpkg-buildpackage
>
> # Build modules.
> #export MODULE_LOC=${PWD}
> #cd /usr/src/linux
> #make-kpkg modules_image --append-to-version -lustre-my-build --revision `date +"%Y%m%d"`
>
> Then i got stuck on "checking for external module build target" for a while, fixed by running "make scripts" in the kernel source folder
> It finally compiled !
>
> _______________________________________________
> lustre-discuss mailing list
> lustre-discuss at lists.lustre.org<mailto:lustre-discuss at lists.lustre.org>
> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

-- 
Jérome BECOT

Administrateur Systèmes et Réseaux

Molécules à visée Thérapeutique par des approches in Silico (MTi)
Univ Paris Diderot, UMRS973 Inserm
Case 013
Bât. Lamarck A, porte 412
35, rue Hélène Brion 75205 Paris Cedex 13
France

Tel : 01 57 27 83 82



More information about the lustre-discuss mailing list