[lustre-discuss] Lustre 2.7 deployment issues

[Peter Kjellström]

On Fri, 4 Dec 2015 18:08:50 -0500
Chris Hunter <chris.hunter at yale.edu> wrote:

> Hi Ray,
> 
> I'll throw my 0.02 into the ring:
> 
> There are known issues with the Truescale kernel driver in 
> redhat/centos/scilinux 6.6. You should try kernel 2.6.32-504.23.4 or 
> newer. Some details of the bug are in LU-6698 and RHSA-2015-1081.

Good summary of the problems except that IB was generally broken not
just for Truescale (multicast stuff in ipoib -> islands of
connectivity). This was however a different bug compared to the ib_qib
driver bug...

/Peter K
 
> Further, lustre 2.7+ now applies performance tuning parameters when 
> installed with the ko2iblnd kernel driver (LU-6735). The tuning 
> parameters are likely incompatible with older infiniband HCA
> adapters. Our experience is they don't work for Truescale IB adapters.
> 
> If you are running lustre 2.5 servers and lustre 2.7+ clients these 
> tuning parameters likely will prevent your clients from mounting.
> 
> Fortunately you can disable the tuning parameters by modifying the 
> ko2iblnd modprobe config file. On our RHEL6 clients, we removed the
> line "install ko2iblnd /usr/sbin/ko2iblnd-probe" from file 
> /etc/modprobe.d/ko2iblnd.conf
> 
> If you are running 2.7+ servers and 2.7+ clients then I believe this
> is not an issue (LU-3322).
> 
> regards,
> chris hunter
> chris.hunter at yale.edu
> 
> 
> 
> 
> > Le 03-12-2015 16:13, Ray Muno a ?crit :
> >> I am trying to set up a test deployment of Lustre 2.7.
> >>
> >> I pulled RPMS from
> >> https://urldefense.proofpoint.com/v2/url?u=http-3A__lustre.org_download_&d=AwICAg&c=-dg2m7zWuuDZ0MUcV7Sdqw&r=d_G2h_sZYG4xtHMeKo8QgjDmOcMVdQvYgM-5Dri1AOY&m=rDHVyPCaAhbdHPi5Gl5kTqo8s7e7Z84-RWqvgtBRrWI&s=3CPtgQCox02zqp-KhoiiSt5NJV0SoIiSLCIqRQHONdk&e=
> >> and installed them on a set of server running Scientific Linux 6.6
> >> which seems to be a proper OS for deployment.  Everything installs
> >> and I can format the filesystems on the MDS (1) and OSS (2)
> >> servers. When I try and mount the OST files systems, I get
> >> communication errors. I can "lctl ping" the servers from each
> >> other, but cannot establish communication between the MDS and OSS.
> >>
> >> The installation is on servers connected over Infiniband (Qlogic
> >> DDR 4X).
> 
> _______________________________________________
> lustre-discuss mailing list
> lustre-discuss at lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org