[lustre-discuss] Compiling from sources with Debian 8

Dilger, Andreas andreas.dilger at intel.com
Wed Dec 2 15:05:28 PST 2015 

On 2015/12/02, 03:26, "Jérôme BECOT" <jerome.becot at inserm.fr> wrote:

>Well,
>
>Finally I started from scratch with a colleague and following the old
>method is working well
>
>./configure --with-o2ib=no --with-kmp-moddir=/lib/modules/$(uname
>-r)/extra
>make
>make debs
>
>The process ends with an error because there's a glitch somewhere on the
>generated filename
>mv: cannot stat Œ../linux-patch-lustre__all.deb¹: No such file or
>directory
>...
>
>The file name expected lacks the version number
>But my packages are here and I can mount my lustre share
>
>(still have to fix debian/control* for headers package name, and fix
>version number in the kernel-source makefile provided by debian and run
>make modules_prepare make scripts)

Jérôme,
if there are issues with the makefile and/or documentation, it would be
great if you could use your new-found expertise to provide a patch to fix
this, so that others don't have to go through the same ordeal?

Thanks, Andreas

>
>Le 01/12/2015 11:11, Jérôme BECOT a écrit :
>> thanks
>>
>> They load .. huh .. at least, the kernel tries to load them first
>> without this. Still the symbol problem remains.
>>
>> Le 01/12/2015 11:02, Thomas Stibor a écrit :
>>> Looks like that still staging Lustre modules are loaded first,
>>> and then the remaining newer compiled modules. To make sure that ONLY
>>> Lustre kernel modules from "extra" directory are loaded one can do
>>> the following:
>>>
>>> DEPMOD_DIR='/etc/depmod.d'
>>> mkdir -p ${DEPMOD_DIR}
>>> echo "search extra built-in" > ${DEPMOD_DIR}/lustre.conf
>>> depmod -a
>>>
>>> This sets the search order in directory "extra" first and thus your
>>> compiled modules
>>> in /lib/modules/3.16.0-4-amd64/extra are loaded (not the staging one).
>>>
>>> Cheers
>>>  Thomas
>>>
>>> On 11/30/2015 06:18 PM, Jérôme BECOT wrote:
>>>> So,
>>>>
>>>> I could move on with building the modules. They are successfully
>>>> build and installed but I had to change the module destination
>>>> directory in config/lustre-build-linux.m4 to target
>>>> /lib/modules/`uname -r`/extra directory (else it goes in "kernel"
>>>> subdirectory and they never get loaded)
>>>>
>>>> Then it won't load anyway. I'm getting [440444.832446] lnet: no
>>>> symbol version for module_layout in dmesg
>>>> If i copy the Module.symvers generated into the lustre-release
>>>> folder to /usr/src/linux, then i get
>>>>
>>>>
>>>> [438311.953707] lnet: disagrees about version of symbol
>>>> libcfs_deregister_ioctl
>>>> [438311.953710] lnet: Unknown symbol libcfs_deregister_ioctl (err -22)
>>>> [438311.953725] lnet: Unknown symbol cfs_str2num_check (err 0)
>>>> [438311.953760] lnet: Unknown symbol cfs_gettok (err 0)
>>>> [438311.953782] lnet: Unknown symbol lprocfs_call_handler (err 0)
>>>>
>>>> (this is a sample of various symbol error)
>>>>
>>>> Any clue ? I'm close to make it now (i guess)
>>>>
>>>>
>>>> Le 28/11/2015 18:07, Dilger, Andreas a écrit :
>>>>> The 2.3.64 version means you are using the in-kernel Lustre client
>>>>> (confirmed by the waning messages about "staging"), and not the
>>>>> 2.7.x version from the Lustre master branch.
>>>>>
>>>>> It looks like Ubuntu is building the in-kernel client, and your
>>>>> modules are not being loaded.
>>>>>
>>>>> Cheers, Andreas
>>>>>
>>>>> On Nov 28, 2015, at 03:26, Jérôme BECOT
>>>>> <jerome.becot at inserm.fr<mailto:jerome.becot at inserm.fr>> wrote:
>>>>>
>>>>> Hi there,
>>>>>
>>>>> We run lustre 2.6/2.7 on our Centos 6.6 (servers) and 7 (clients)
>>>>> cluster. We have a few webservers running Debian that need to
>>>>> access the storage. I followed the procedure given by Thomas Stibor
>>>>> about Ubuntu 14 last year.
>>>>>
>>>>> I could successfully compile the binaries and modules after some
>>>>> digging. He also left an already compiled lustre 2.7.63 and modules
>>>>> for kernel 3.16.0-4 online.
>>>>>
>>>>> If I install his binaries, it works well. If I install the one
>>>>> generated by the procedure, the modules don't load and a weird
>>>>> thing happen. Running dmesg warns me about one surprising thing :
>>>>>
>>>>>   > With his packages
>>>>> [212417.535369] LNet: HW CPU cores: 1, npartitions: 1
>>>>> [212417.538430] alg: No test for adler32 (adler32-zlib)
>>>>> [212417.538456] alg: No test for crc32 (crc32-table)
>>>>> [212425.548907] Lustre: Lustre: Build Version:
>>>>> v2_7_60_0-ge686e57-CHANGED-3.16.0-4-amd64
>>>>> [212425.565330] LNet: Added LNI 172.27.7.118 at tcp1 [8/256/0/180]
>>>>> [212425.565354] LNet: Accept secure, port 988
>>>>> [212425.595531] Lustre: Mounted lustre-client
>>>>>
>>>>>> With mine
>>>>> [209942.090874] LNet: HW CPU cores: 1, npartitions: 1
>>>>> [209942.092902] alg: No test for adler32 (adler32-zlib)
>>>>> [209950.092501] lnet: module is from the staging directory, the
>>>>> quality is unknown, you have been warned.
>>>>> [209950.093589] lvfs: module is from the staging directory, the
>>>>> quality is unknown, you have been warned.
>>>>> [209950.094634] obdclass: module is from the staging directory, the
>>>>> quality is unknown, you have been warned.
>>>>> [209950.098595] Lustre: Lustre: Build Version:
>>>>> v2_3_64_0-g6e62c21-CHANGED-3.9.0
>>>>> [209950.099999] ptlrpc: module is from the staging directory, the
>>>>> quality is unknown, you have been warned.
>>>>> [209950.104615] ksocklnd: module is from the staging directory, the
>>>>> quality is unknown, you have been warned.
>>>>> [209950.105237] LNetError:
>>>>> 845:0:(linux-tcpip.c:82:libcfs_ipif_query()) Can't get flags for
>>>>> interface eth0
>>>>> [209950.105862] LNetError:
>>>>> 845:0:(socklnd.c:2824:ksocknal_startup()) Can't get interface eth0
>>>>> info: -515
>>>>> [209951.104194] LNetError: 105-4: Error -100 starting up LNI tcp
>>>>> [209951.104852] LustreError:
>>>>> 845:0:(events.c:566:ptlrpc_init_portals()) network initialisation
>>>>> failed
>>>>> [209990.787541] ptlrpc: module is from the staging directory, the
>>>>> quality is unknown, you have been warned.
>>>>>
>>>>> I pulled the master git branch, and coul obtain
>>>>> linux-patch-lustre_2.7.63.0-16-g8524994_all.deb
>>>>> 
>>>>>lustre-client-modules-3.16.7-ckt11-lustre-my-build_2.7.63.0-16-g852499
>>>>>4_amd64.deb 
>>>>> lustre-tests_2.7.63.0-16-g8524994_amd64.deb
>>>>> lustre_2.7.63.0-16-g8524994_amd64.changes
>>>>> lustre-dev_2.7.63.0-16-g8524994_amd64.deb
>>>>> lustre-utils_2.7.63.0-16-g8524994_amd64.deb
>>>>> lustre_2.7.63.0-16-g8524994.dsc lustre-release
>>>>> lustre_2.7.63.0-16-g8524994.tar.gz
>>>>> lustre-source_2.7.63.0-16-g8524994_all.deb
>>>>>
>>>>> I just don't get it. Why the shown version of the module is 2.3 ?
>>>>> I tried to compile from the 2.7 branch but the 2.7.0 version
>>>>> doesn't compile with kernel 3.16, as suggested in LU-7042
>>>>>
>>>>> I probably miss something. Sorry I'm not familiar with compilers
>>>>> (usually the ./configure && make && make install works or warns you
>>>>> about missing dependencies and i don't go farther).
>>>>>
>>>>> Thank you
>>>>>
>>>>> Jerome
>>>>>
>>>>> PS : Here is the procedure followed to compile
>>>>>
>>>>>
>>>>> apt-get install linux-headers-3.16.0-4-amd64 libtool automake
>>>>> linux-source-3.16
>>>>> git clone git://git.whamcloud.com/fs/lustre-release.git
>>>>> cd /usr/src/
>>>>> tar xf linux-source-3.16.tar.xz
>>>>> cd /home/build/lustre-release/
>>>>> vi debian/rules (edit so it doesn't stop with autogen because it
>>>>> doesn't find the kernel sources)
>>>>> (modify --with-linux= option with current path
>>>>> --with-linux=/usr/src/linux-3.16 in kdist_config section)
>>>>> aptitude install module-assistant libreadline-dev debhelper dpatch
>>>>> libsnmp-dev quilt devscripts
>>>>>   vi debian/control
>>>>> (modify the dependency for unsatisfied linux-headers called
>>>>> linux-headers-something in jessie)
>>>>> vi compile.sh
>>>>> #!/bin/bash
>>>>> unset DEBEMAIL
>>>>> unset EMAIL
>>>>> unset DEBFULLNAME
>>>>> unset NAME
>>>>>
>>>>> export DEBFULLNAME="Niemand Nobody"
>>>>> export EMAIL="npcomplete at example.com<http://example.com>"
>>>>>
>>>>> # Extract lustre version, replace "_" by "." and remove leading
>>>>> letter "v".
>>>>> LUSTRE_VERSION=$(echo `git describe` | sed -e "s/_/\./g" | cut -c2-)
>>>>>
>>>>> # Add entry into debian/changelog such that packages have proper
>>>>> version names.
>>>>> dch --newversion $LUSTRE_VERSION --distribution unstable
>>>>> --nomultimaint -t "Build from official master upstream."
>>>>>
>>>>> #
>>>>> sh ./autogen.sh
>>>>>
>>>>> # Build debian packages.
>>>>> dpkg-buildpackage
>>>>>
>>>>> # Build modules.
>>>>> #export MODULE_LOC=${PWD}
>>>>> #cd /usr/src/linux
>>>>> #make-kpkg modules_image --append-to-version -lustre-my-build
>>>>> --revision `date +"%Y%m%d"`
>>>>>
>>>>> Then i got stuck on "checking for external module build target" for
>>>>> a while, fixed by running "make scripts" in the kernel source folder
>>>>> It finally compiled !
>>>>>
>>>>> _______________________________________________
>>>>> lustre-discuss mailing list
>>>>> 
>>>>>lustre-discuss at lists.lustre.org<mailto:lustre-discuss at lists.lustre.org
>>>>>> 
>>>>>
>>>>> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>>>>
>>>
>>
>
>-- 
>Jérome BECOT
>
>Administrateur Systèmes et Réseaux
>
>Molécules à visée Thérapeutique par des approches in Silico (MTi)
>Univ Paris Diderot, UMRS973 Inserm
>Case 013
>Bât. Lamarck A, porte 412
>35, rue Hélène Brion 75205 Paris Cedex 13
>France
>
>Tel : 01 57 27 83 82
>
>_______________________________________________
>lustre-discuss mailing list
>lustre-discuss at lists.lustre.org
>http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>


Cheers, Andreas
-- 
Andreas Dilger

Lustre Principal Architect
Intel High Performance Data Division

More information about the lustre-discuss mailing list