[lustre-discuss] Compiling from sources with Debian 8

Tue Dec 1 02:02:14 PST 2015

Looks like that still staging Lustre modules are loaded first,
and then the remaining newer compiled modules. To make sure that ONLY
Lustre kernel modules from "extra" directory are loaded one can do the 
following:

DEPMOD_DIR='/etc/depmod.d'
mkdir -p ${DEPMOD_DIR}
echo "search extra built-in" > ${DEPMOD_DIR}/lustre.conf
depmod -a

This sets the search order in directory "extra" first and thus your 
compiled modules
in /lib/modules/3.16.0-4-amd64/extra are loaded (not the staging one).

Cheers
  Thomas

On 11/30/2015 06:18 PM, Jérôme BECOT wrote:
> So,
>
> I could move on with building the modules. They are successfully build 
> and installed but I had to change the module destination directory in 
> config/lustre-build-linux.m4 to target /lib/modules/`uname -r`/extra 
> directory (else it goes in "kernel" subdirectory and they never get 
> loaded)
>
> Then it won't load anyway. I'm getting [440444.832446] lnet: no symbol 
> version for module_layout in dmesg
> If i copy the Module.symvers generated into the lustre-release folder 
> to /usr/src/linux, then i get
>
>
> [438311.953707] lnet: disagrees about version of symbol 
> libcfs_deregister_ioctl
> [438311.953710] lnet: Unknown symbol libcfs_deregister_ioctl (err -22)
> [438311.953725] lnet: Unknown symbol cfs_str2num_check (err 0)
> [438311.953760] lnet: Unknown symbol cfs_gettok (err 0)
> [438311.953782] lnet: Unknown symbol lprocfs_call_handler (err 0)
>
> (this is a sample of various symbol error)
>
> Any clue ? I'm close to make it now (i guess)
>
>
> Le 28/11/2015 18:07, Dilger, Andreas a écrit :
>> The 2.3.64 version means you are using the in-kernel Lustre client 
>> (confirmed by the waning messages about "staging"), and not the 2.7.x 
>> version from the Lustre master branch.
>>
>> It looks like Ubuntu is building the in-kernel client, and your 
>> modules are not being loaded.
>>
>> Cheers, Andreas
>>
>> On Nov 28, 2015, at 03:26, Jérôme BECOT 
>> <jerome.becot at inserm.fr<mailto:jerome.becot at inserm.fr>> wrote:
>>
>> Hi there,
>>
>> We run lustre 2.6/2.7 on our Centos 6.6 (servers) and 7 (clients) 
>> cluster. We have a few webservers running Debian that need to access 
>> the storage. I followed the procedure given by Thomas Stibor about 
>> Ubuntu 14 last year.
>>
>> I could successfully compile the binaries and modules after some 
>> digging. He also left an already compiled lustre 2.7.63 and modules 
>> for kernel 3.16.0-4 online.
>>
>> If I install his binaries, it works well. If I install the one 
>> generated by the procedure, the modules don't load and a weird thing 
>> happen. Running dmesg warns me about one surprising thing :
>>
>>   > With his packages
>> [212417.535369] LNet: HW CPU cores: 1, npartitions: 1
>> [212417.538430] alg: No test for adler32 (adler32-zlib)
>> [212417.538456] alg: No test for crc32 (crc32-table)
>> [212425.548907] Lustre: Lustre: Build Version: 
>> v2_7_60_0-ge686e57-CHANGED-3.16.0-4-amd64
>> [212425.565330] LNet: Added LNI 172.27.7.118 at tcp1 [8/256/0/180]
>> [212425.565354] LNet: Accept secure, port 988
>> [212425.595531] Lustre: Mounted lustre-client
>>
>>> With mine
>> [209942.090874] LNet: HW CPU cores: 1, npartitions: 1
>> [209942.092902] alg: No test for adler32 (adler32-zlib)
>> [209950.092501] lnet: module is from the staging directory, the 
>> quality is unknown, you have been warned.
>> [209950.093589] lvfs: module is from the staging directory, the 
>> quality is unknown, you have been warned.
>> [209950.094634] obdclass: module is from the staging directory, the 
>> quality is unknown, you have been warned.
>> [209950.098595] Lustre: Lustre: Build Version: 
>> v2_3_64_0-g6e62c21-CHANGED-3.9.0
>> [209950.099999] ptlrpc: module is from the staging directory, the 
>> quality is unknown, you have been warned.
>> [209950.104615] ksocklnd: module is from the staging directory, the 
>> quality is unknown, you have been warned.
>> [209950.105237] LNetError: 
>> 845:0:(linux-tcpip.c:82:libcfs_ipif_query()) Can't get flags for 
>> interface eth0
>> [209950.105862] LNetError: 845:0:(socklnd.c:2824:ksocknal_startup()) 
>> Can't get interface eth0 info: -515
>> [209951.104194] LNetError: 105-4: Error -100 starting up LNI tcp
>> [209951.104852] LustreError: 
>> 845:0:(events.c:566:ptlrpc_init_portals()) network initialisation failed
>> [209990.787541] ptlrpc: module is from the staging directory, the 
>> quality is unknown, you have been warned.
>>
>> I pulled the master git branch, and coul obtain
>> linux-patch-lustre_2.7.63.0-16-g8524994_all.deb 
>> lustre-client-modules-3.16.7-ckt11-lustre-my-build_2.7.63.0-16-g8524994_amd64.deb 
>> lustre-tests_2.7.63.0-16-g8524994_amd64.deb
>> lustre_2.7.63.0-16-g8524994_amd64.changes 
>> lustre-dev_2.7.63.0-16-g8524994_amd64.deb 
>> lustre-utils_2.7.63.0-16-g8524994_amd64.deb
>> lustre_2.7.63.0-16-g8524994.dsc                  lustre-release
>> lustre_2.7.63.0-16-g8524994.tar.gz 
>> lustre-source_2.7.63.0-16-g8524994_all.deb
>>
>> I just don't get it. Why the shown version of the module is 2.3 ?
>> I tried to compile from the 2.7 branch but the 2.7.0 version doesn't 
>> compile with kernel 3.16, as suggested in LU-7042
>>
>> I probably miss something. Sorry I'm not familiar with compilers 
>> (usually the ./configure && make && make install works or warns you 
>> about missing dependencies and i don't go farther).
>>
>> Thank you
>>
>> Jerome
>>
>> PS : Here is the procedure followed to compile
>>
>>
>> apt-get install linux-headers-3.16.0-4-amd64 libtool automake 
>> linux-source-3.16
>> git clone git://git.whamcloud.com/fs/lustre-release.git
>> cd /usr/src/
>> tar xf linux-source-3.16.tar.xz
>> cd /home/build/lustre-release/
>> vi debian/rules (edit so it doesn't stop with autogen because it 
>> doesn't find the kernel sources)
>> (modify --with-linux= option with current path 
>> --with-linux=/usr/src/linux-3.16 in kdist_config section)
>> aptitude install module-assistant libreadline-dev debhelper dpatch 
>> libsnmp-dev quilt devscripts
>>   vi debian/control
>> (modify the dependency for unsatisfied linux-headers called 
>> linux-headers-something in jessie)
>> vi compile.sh
>> #!/bin/bash
>> unset DEBEMAIL
>> unset EMAIL
>> unset DEBFULLNAME
>> unset NAME
>>
>> export DEBFULLNAME="Niemand Nobody"
>> export EMAIL="npcomplete at example.com<http://example.com>"
>>
>> # Extract lustre version, replace "_" by "." and remove leading 
>> letter "v".
>> LUSTRE_VERSION=$(echo `git describe` | sed -e "s/_/\./g" | cut -c2-)
>>
>> # Add entry into debian/changelog such that packages have proper 
>> version names.
>> dch --newversion $LUSTRE_VERSION --distribution unstable 
>> --nomultimaint -t "Build from official master upstream."
>>
>> #
>> sh ./autogen.sh
>>
>> # Build debian packages.
>> dpkg-buildpackage
>>
>> # Build modules.
>> #export MODULE_LOC=${PWD}
>> #cd /usr/src/linux
>> #make-kpkg modules_image --append-to-version -lustre-my-build 
>> --revision `date +"%Y%m%d"`
>>
>> Then i got stuck on "checking for external module build target" for a 
>> while, fixed by running "make scripts" in the kernel source folder
>> It finally compiled !
>>
>> _______________________________________________
>> lustre-discuss mailing list
>> lustre-discuss at lists.lustre.org<mailto:lustre-discuss at lists.lustre.org>
>> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>