[lustre-discuss] Compiling from sources with Debian 8

Wed Dec 2 02:26:02 PST 2015

Well,

Finally I started from scratch with a colleague and following the old 
method is working well

./configure --with-o2ib=no --with-kmp-moddir=/lib/modules/$(uname -r)/extra
make
make debs

The process ends with an error because there's a glitch somewhere on the 
generated filename
mv: cannot stat ‘../linux-patch-lustre__all.deb’: No such file or directory
...

The file name expected lacks the version number
But my packages are here and I can mount my lustre share

(still have to fix debian/control* for headers package name, and fix 
version number in the kernel-source makefile provided by debian and run 
make modules_prepare make scripts)

Le 01/12/2015 11:11, Jérôme BECOT a écrit :
> thanks
>
> They load .. huh .. at least, the kernel tries to load them first 
> without this. Still the symbol problem remains.
>
> Le 01/12/2015 11:02, Thomas Stibor a écrit :
>> Looks like that still staging Lustre modules are loaded first,
>> and then the remaining newer compiled modules. To make sure that ONLY
>> Lustre kernel modules from "extra" directory are loaded one can do 
>> the following:
>>
>> DEPMOD_DIR='/etc/depmod.d'
>> mkdir -p ${DEPMOD_DIR}
>> echo "search extra built-in" > ${DEPMOD_DIR}/lustre.conf
>> depmod -a
>>
>> This sets the search order in directory "extra" first and thus your 
>> compiled modules
>> in /lib/modules/3.16.0-4-amd64/extra are loaded (not the staging one).
>>
>> Cheers
>>  Thomas
>>
>> On 11/30/2015 06:18 PM, Jérôme BECOT wrote:
>>> So,
>>>
>>> I could move on with building the modules. They are successfully 
>>> build and installed but I had to change the module destination 
>>> directory in config/lustre-build-linux.m4 to target 
>>> /lib/modules/`uname -r`/extra directory (else it goes in "kernel" 
>>> subdirectory and they never get loaded)
>>>
>>> Then it won't load anyway. I'm getting [440444.832446] lnet: no 
>>> symbol version for module_layout in dmesg
>>> If i copy the Module.symvers generated into the lustre-release 
>>> folder to /usr/src/linux, then i get
>>>
>>>
>>> [438311.953707] lnet: disagrees about version of symbol 
>>> libcfs_deregister_ioctl
>>> [438311.953710] lnet: Unknown symbol libcfs_deregister_ioctl (err -22)
>>> [438311.953725] lnet: Unknown symbol cfs_str2num_check (err 0)
>>> [438311.953760] lnet: Unknown symbol cfs_gettok (err 0)
>>> [438311.953782] lnet: Unknown symbol lprocfs_call_handler (err 0)
>>>
>>> (this is a sample of various symbol error)
>>>
>>> Any clue ? I'm close to make it now (i guess)
>>>
>>>
>>> Le 28/11/2015 18:07, Dilger, Andreas a écrit :
>>>> The 2.3.64 version means you are using the in-kernel Lustre client 
>>>> (confirmed by the waning messages about "staging"), and not the 
>>>> 2.7.x version from the Lustre master branch.
>>>>
>>>> It looks like Ubuntu is building the in-kernel client, and your 
>>>> modules are not being loaded.
>>>>
>>>> Cheers, Andreas
>>>>
>>>> On Nov 28, 2015, at 03:26, Jérôme BECOT 
>>>> <jerome.becot at inserm.fr<mailto:jerome.becot at inserm.fr>> wrote:
>>>>
>>>> Hi there,
>>>>
>>>> We run lustre 2.6/2.7 on our Centos 6.6 (servers) and 7 (clients) 
>>>> cluster. We have a few webservers running Debian that need to 
>>>> access the storage. I followed the procedure given by Thomas Stibor 
>>>> about Ubuntu 14 last year.
>>>>
>>>> I could successfully compile the binaries and modules after some 
>>>> digging. He also left an already compiled lustre 2.7.63 and modules 
>>>> for kernel 3.16.0-4 online.
>>>>
>>>> If I install his binaries, it works well. If I install the one 
>>>> generated by the procedure, the modules don't load and a weird 
>>>> thing happen. Running dmesg warns me about one surprising thing :
>>>>
>>>>   > With his packages
>>>> [212417.535369] LNet: HW CPU cores: 1, npartitions: 1
>>>> [212417.538430] alg: No test for adler32 (adler32-zlib)
>>>> [212417.538456] alg: No test for crc32 (crc32-table)
>>>> [212425.548907] Lustre: Lustre: Build Version: 
>>>> v2_7_60_0-ge686e57-CHANGED-3.16.0-4-amd64
>>>> [212425.565330] LNet: Added LNI 172.27.7.118 at tcp1 [8/256/0/180]
>>>> [212425.565354] LNet: Accept secure, port 988
>>>> [212425.595531] Lustre: Mounted lustre-client
>>>>
>>>>> With mine
>>>> [209942.090874] LNet: HW CPU cores: 1, npartitions: 1
>>>> [209942.092902] alg: No test for adler32 (adler32-zlib)
>>>> [209950.092501] lnet: module is from the staging directory, the 
>>>> quality is unknown, you have been warned.
>>>> [209950.093589] lvfs: module is from the staging directory, the 
>>>> quality is unknown, you have been warned.
>>>> [209950.094634] obdclass: module is from the staging directory, the 
>>>> quality is unknown, you have been warned.
>>>> [209950.098595] Lustre: Lustre: Build Version: 
>>>> v2_3_64_0-g6e62c21-CHANGED-3.9.0
>>>> [209950.099999] ptlrpc: module is from the staging directory, the 
>>>> quality is unknown, you have been warned.
>>>> [209950.104615] ksocklnd: module is from the staging directory, the 
>>>> quality is unknown, you have been warned.
>>>> [209950.105237] LNetError: 
>>>> 845:0:(linux-tcpip.c:82:libcfs_ipif_query()) Can't get flags for 
>>>> interface eth0
>>>> [209950.105862] LNetError: 
>>>> 845:0:(socklnd.c:2824:ksocknal_startup()) Can't get interface eth0 
>>>> info: -515
>>>> [209951.104194] LNetError: 105-4: Error -100 starting up LNI tcp
>>>> [209951.104852] LustreError: 
>>>> 845:0:(events.c:566:ptlrpc_init_portals()) network initialisation 
>>>> failed
>>>> [209990.787541] ptlrpc: module is from the staging directory, the 
>>>> quality is unknown, you have been warned.
>>>>
>>>> I pulled the master git branch, and coul obtain
>>>> linux-patch-lustre_2.7.63.0-16-g8524994_all.deb 
>>>> lustre-client-modules-3.16.7-ckt11-lustre-my-build_2.7.63.0-16-g8524994_amd64.deb 
>>>> lustre-tests_2.7.63.0-16-g8524994_amd64.deb
>>>> lustre_2.7.63.0-16-g8524994_amd64.changes 
>>>> lustre-dev_2.7.63.0-16-g8524994_amd64.deb 
>>>> lustre-utils_2.7.63.0-16-g8524994_amd64.deb
>>>> lustre_2.7.63.0-16-g8524994.dsc lustre-release
>>>> lustre_2.7.63.0-16-g8524994.tar.gz 
>>>> lustre-source_2.7.63.0-16-g8524994_all.deb
>>>>
>>>> I just don't get it. Why the shown version of the module is 2.3 ?
>>>> I tried to compile from the 2.7 branch but the 2.7.0 version 
>>>> doesn't compile with kernel 3.16, as suggested in LU-7042
>>>>
>>>> I probably miss something. Sorry I'm not familiar with compilers 
>>>> (usually the ./configure && make && make install works or warns you 
>>>> about missing dependencies and i don't go farther).
>>>>
>>>> Thank you
>>>>
>>>> Jerome
>>>>
>>>> PS : Here is the procedure followed to compile
>>>>
>>>>
>>>> apt-get install linux-headers-3.16.0-4-amd64 libtool automake 
>>>> linux-source-3.16
>>>> git clone git://git.whamcloud.com/fs/lustre-release.git
>>>> cd /usr/src/
>>>> tar xf linux-source-3.16.tar.xz
>>>> cd /home/build/lustre-release/
>>>> vi debian/rules (edit so it doesn't stop with autogen because it 
>>>> doesn't find the kernel sources)
>>>> (modify --with-linux= option with current path 
>>>> --with-linux=/usr/src/linux-3.16 in kdist_config section)
>>>> aptitude install module-assistant libreadline-dev debhelper dpatch 
>>>> libsnmp-dev quilt devscripts
>>>>   vi debian/control
>>>> (modify the dependency for unsatisfied linux-headers called 
>>>> linux-headers-something in jessie)
>>>> vi compile.sh
>>>> #!/bin/bash
>>>> unset DEBEMAIL
>>>> unset EMAIL
>>>> unset DEBFULLNAME
>>>> unset NAME
>>>>
>>>> export DEBFULLNAME="Niemand Nobody"
>>>> export EMAIL="npcomplete at example.com<http://example.com>"
>>>>
>>>> # Extract lustre version, replace "_" by "." and remove leading 
>>>> letter "v".
>>>> LUSTRE_VERSION=$(echo `git describe` | sed -e "s/_/\./g" | cut -c2-)
>>>>
>>>> # Add entry into debian/changelog such that packages have proper 
>>>> version names.
>>>> dch --newversion $LUSTRE_VERSION --distribution unstable 
>>>> --nomultimaint -t "Build from official master upstream."
>>>>
>>>> #
>>>> sh ./autogen.sh
>>>>
>>>> # Build debian packages.
>>>> dpkg-buildpackage
>>>>
>>>> # Build modules.
>>>> #export MODULE_LOC=${PWD}
>>>> #cd /usr/src/linux
>>>> #make-kpkg modules_image --append-to-version -lustre-my-build 
>>>> --revision `date +"%Y%m%d"`
>>>>
>>>> Then i got stuck on "checking for external module build target" for 
>>>> a while, fixed by running "make scripts" in the kernel source folder
>>>> It finally compiled !
>>>>
>>>> _______________________________________________
>>>> lustre-discuss mailing list
>>>> lustre-discuss at lists.lustre.org<mailto:lustre-discuss at lists.lustre.org> 
>>>>
>>>> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>>>
>>
>

-- 
Jérome BECOT

Administrateur Systèmes et Réseaux

Molécules à visée Thérapeutique par des approches in Silico (MTi)
Univ Paris Diderot, UMRS973 Inserm
Case 013
Bât. Lamarck A, porte 412
35, rue Hélène Brion 75205 Paris Cedex 13
France

Tel : 01 57 27 83 82