[Lustre-discuss] Installing Lustre 1.8.2 on CentOS 5.4 with OFED 1.4.2

[Marco Aurelio L Gomes]

Hi all,

I'm trying to setup my Lustre environment using CentOS 5.4
(kernel-2.6.18-164.11.1.el5-x86_64 from updates repository) Lustre 1.8.2
and OFED 1.4.2. I saw at Lustre Matrix Support that 1.8.2 works at most
with OFED 1.4.2, but when I tried to compile with this release, I saw a
lot of errors complaining about variables redefinition at ofa_kernel
compiling process (see attached ofa_kernel_rpmbuild.log. but i think
this is not related to this list). In this case, i tried to compile with
OFED 1.5, the latest stable release of OFED, that compile fine; but when
i boot a client, and run:

modprobe lustre

i got the following errors:

[root at masternode1 ~]# modprobe lustre
WARNING: Error inserting osc
(/lib/modules/2.6.18-164.11.1.el5/kernel/fs/lustre/osc.ko): Input/output
error
WARNING: Error inserting mdc
(/lib/modules/2.6.18-164.11.1.el5/kernel/fs/lustre/mdc.ko): Input/output
error
WARNING: Error inserting lov
(/lib/modules/2.6.18-164.11.1.el5/kernel/fs/lustre/lov.ko): Input/output
error
FATAL: Error inserting lustre
(/lib/modules/2.6.18-164.11.1.el5/kernel/fs/lustre/lustre.ko):
Input/output error

and at dmesg:

Lustre: OBD class driver, http://www.lustre.org/
Lustre:     Lustre Version: 1.8.2
Lustre:     Build Version:
1.8.2-20100122201357-PRISTINE-2.6.18-164.11.1.el5
ko2iblnd: disagrees about version of symbol ib_fmr_pool_unmap
ko2iblnd: Unknown symbol ib_fmr_pool_unmap
ko2iblnd: disagrees about version of symbol ib_create_cq
ko2iblnd: Unknown symbol ib_create_cq
ko2iblnd: disagrees about version of symbol rdma_resolve_addr
ko2iblnd: Unknown symbol rdma_resolve_addr
ko2iblnd: disagrees about version of symbol ib_reg_phys_mr
ko2iblnd: Unknown symbol ib_reg_phys_mr
ko2iblnd: disagrees about version of symbol ib_create_fmr_pool
ko2iblnd: Unknown symbol ib_create_fmr_pool
ko2iblnd: disagrees about version of symbol ib_dereg_mr
ko2iblnd: Unknown symbol ib_dereg_mr
ko2iblnd: disagrees about version of symbol rdma_reject
ko2iblnd: Unknown symbol rdma_reject
ko2iblnd: disagrees about version of symbol rdma_disconnect
ko2iblnd: Unknown symbol rdma_disconnect
ko2iblnd: disagrees about version of symbol rdma_resolve_route
ko2iblnd: Unknown symbol rdma_resolve_route
ko2iblnd: disagrees about version of symbol rdma_bind_addr
ko2iblnd: Unknown symbol rdma_bind_addr
ko2iblnd: disagrees about version of symbol rdma_create_qp
ko2iblnd: Unknown symbol rdma_create_qp
ko2iblnd: disagrees about version of symbol ib_destroy_cq
ko2iblnd: Unknown symbol ib_destroy_cq
ko2iblnd: disagrees about version of symbol rdma_create_id
ko2iblnd: Unknown symbol rdma_create_id
ko2iblnd: disagrees about version of symbol rdma_listen
ko2iblnd: Unknown symbol rdma_listen
ko2iblnd: disagrees about version of symbol rdma_destroy_qp
ko2iblnd: Unknown symbol rdma_destroy_qp
ko2iblnd: disagrees about version of symbol ib_query_device
ko2iblnd: Unknown symbol ib_query_device
ko2iblnd: disagrees about version of symbol ib_get_dma_mr
ko2iblnd: Unknown symbol ib_get_dma_mr
ko2iblnd: disagrees about version of symbol ib_alloc_pd
ko2iblnd: Unknown symbol ib_alloc_pd
ko2iblnd: disagrees about version of symbol rdma_connect
ko2iblnd: Unknown symbol rdma_connect
ko2iblnd: disagrees about version of symbol ib_modify_qp
ko2iblnd: Unknown symbol ib_modify_qp
ko2iblnd: disagrees about version of symbol rdma_destroy_id
ko2iblnd: Unknown symbol rdma_destroy_id
ko2iblnd: disagrees about version of symbol rdma_accept
ko2iblnd: Unknown symbol rdma_accept
ko2iblnd: disagrees about version of symbol ib_dealloc_pd
ko2iblnd: Unknown symbol ib_dealloc_pd
ko2iblnd: disagrees about version of symbol ib_fmr_pool_map_phys
ko2iblnd: Unknown symbol ib_fmr_pool_map_phys
LustreError: 4572:0:(api-ni.c:1043:lnet_startup_lndnis()) Can't load LND
o2ib, module ko2iblnd, rc=256
LustreError: 4572:0:(events.c:729:ptlrpc_init_portals()) network
initialisation failed

At this point, i thought that is related to OFED compilation. I only
compile OFED because I didn't found kernel-ib package at lustre download
site. I would like to know if someone had the same problem on their
setup and if there are some workaround to get it working.

In following I'll give more information about the lustre environment.

client:
CentOS 5.4 (2.6.18-164.11.1.el5)
Lustre 1.8.2
lustre-client-modules-1.8.2-2.6.18_164.11.1.el5_lustre.1.8.2
lustre-client-1.8.2-2.6.18_164.11.1.el5_lustre.1.8.2
OFED 1.5

Thanks in advance for the help and sorry for my bad english.

Regards,

Marco Gomes
Systems/HPC-Cluster
Numerical Offshore Tank
(11) 3777-4142 #250
(11) 3091-5350 #250

-------------- next part --------------
A non-text attachment was scrubbed...
Name: ofa_kernel.rpmbuild.log
Type: text/x-log
Size: 717707 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20100319/c27a4b60/attachment.bin>