[lustre-discuss] Lustre 2.7 deployment issues

Chris Hunter chris.hunter at yale.edu
Fri Dec 4 15:08:50 PST 2015

Hi Ray,

I'll throw my 0.02 into the ring:

There are known issues with the Truescale kernel driver in 
redhat/centos/scilinux 6.6. You should try kernel 2.6.32-504.23.4 or 
newer. Some details of the bug are in LU-6698 and RHSA-2015-1081.

Further, lustre 2.7+ now applies performance tuning parameters when 
installed with the ko2iblnd kernel driver (LU-6735). The tuning 
parameters are likely incompatible with older infiniband HCA adapters. 
Our experience is they don't work for Truescale IB adapters.

If you are running lustre 2.5 servers and lustre 2.7+ clients these 
tuning parameters likely will prevent your clients from mounting.

Fortunately you can disable the tuning parameters by modifying the 
ko2iblnd modprobe config file. On our RHEL6 clients, we removed the line 
"install ko2iblnd /usr/sbin/ko2iblnd-probe" from file 

If you are running 2.7+ servers and 2.7+ clients then I believe this is 
not an issue (LU-3322).

chris hunter
chris.hunter at yale.edu

> Le 03-12-2015 16:13, Ray Muno a ?crit :
>> I am trying to set up a test deployment of Lustre 2.7.
>> I pulled RPMS from https://urldefense.proofpoint.com/v2/url?u=http-3A__lustre.org_download_&d=AwICAg&c=-dg2m7zWuuDZ0MUcV7Sdqw&r=d_G2h_sZYG4xtHMeKo8QgjDmOcMVdQvYgM-5Dri1AOY&m=rDHVyPCaAhbdHPi5Gl5kTqo8s7e7Z84-RWqvgtBRrWI&s=3CPtgQCox02zqp-KhoiiSt5NJV0SoIiSLCIqRQHONdk&e=  and installed them
>> on a
>> set of server running Scientific Linux 6.6 which seems to be a proper
>> OS for deployment.  Everything installs and I can format the
>> filesystems on the MDS (1) and OSS (2) servers. When I try and mount
>> the OST files systems, I get communication errors. I can "lctl ping"
>> the servers from each other, but cannot establish communication
>> between the MDS and OSS.
>> The installation is on servers connected over Infiniband (Qlogic DDR
>> 4X).

More information about the lustre-discuss mailing list