[Lustre-discuss] CentOS 5.4 (Rocks 5.3) and Lustre 1.8.2 client against HP SFS 3.2

[Finn Andersen]

Hi!

I'm trying to get Rocks 5.3 (Based on CentOS 5.4) to work with Lustre
over Infiniband. I'm almost done now... :)
>From what I have read on different mailinglist etc. the best option is
to use stock rpm's as much as possible.

>From the Lustre homepage I've used these 2 packages:
lustre-client-1.8.2-2.6.18_164.11.1.el5_lustre.1.8.2.x86_64.rpm
lustre-client-modules-1.8.2-2.6.18_164.11.1.el5_lustre.1.8.2.x86_64.rpm

I have used this kernel from CentOS: kernel-2.6.18_164.11.1.el.x86_64.rpm
My ofed install is the one that RHEL/CentOS ships with it's 5.5
release. (1.4.1-5)
All this seems to be within the support matrix on lustre.org

/etc/modprobe.conf has this entry:
options lnet networks=o2ib0

Why do I have to enter "modprobe lustre" manually before I can mount
the lustre filesystem? How can this be fixed, so that it will come up
automatically on reboot?

My /etc/fstab entry is:
10.129.2.1 at o2ib0:10.129.2.2 at o2ib0:/scratch  /scratch  lustre
_netdev,rw,flock 0 0

When I try to reboot one of the compute nodes, I get this error
message on the console:
---
failed to unload rdma_cm
failed to unload ib_cm
failed to unload iw_cm
LustreError: 131-3: Received notification of device removal
Please shutdown LNET to allow this to procceed
---
The compute node has to be reset in order to get it back up again.
Why? And how can it be fixed?

-
Finn Andersen
Norway