[Lustre-discuss] problem with installing lustre and ofed

Jason Brooks brookjas at ohsu.edu
Fri Dec 28 15:54:57 PST 2012


Hello,

I am having trouble installing the server modules for  lustre 2.1.4 and use mellanox's OFED distribution so we may use infiniband.  Would you folks look at my procedure and results below and let me know what you think?  Thanks very much!

The mellanox ofed installation builds and installs some kernel modules too, so I used this method to ensure OFED compiled against the correct kernel.  This is on centos 6.3.

 1.  download all lustre rpms from whamcloud
 2.  install kernel, kernel-firmware, kernel-headers, and kernel-devel
    *   in this case, it's the rpm files with "2.6.32-279.14.1.el6_lustre.x86_64" in their name
 3.  reboot into this lustre kernel
 4.  install the remaining rpms
 5.  download ofed from mellanox "MLNX_OFED_LINUX-1.5.3-3.1.0-rhel6.3-x86_64.iso"
    *   build mellanox ofed bits using the lustre kernel and kernel-devel info
    *   install mellanox ofed
 6.  reboot
 7.  upon reboot, if I do NOT have o2ib3 in my lnet networks parameters, I can modprobe lnet and lustre.
 8.  if I DO have o2ib3 present in the lnet parameters, running modprobe lustre gets me:

ib/modules/2.6.32-279.14.1.el6_lustre.x86_64/updates/kernel/fs/lustre/fld.ko): Input/output error
WARNING: Error inserting fid (/lib/modules/2.6.32-279.14.1.el6_lustre.x86_64/updates/kernel/fs/lustre/fid.ko): Input/output error
WARNING: Error inserting mdc (/lib/modules/2.6.32-279.14.1.el6_lustre.x86_64/updates/kernel/fs/lustre/mdc.ko): Input/output error
WARNING: Error inserting osc (/lib/modules/2.6.32-279.14.1.el6_lustre.x86_64/updates/kernel/fs/lustre/osc.ko): Input/output error
WARNING: Error inserting lov (/lib/modules/2.6.32-279.14.1.el6_lustre.x86_64/updates/kernel/fs/lustre/lov.ko): Input/output error
FATAL: Error inserting lustre (/lib/modules/2.6.32-279.14.1.el6_lustre.x86_64/updates/kernel/fs/lustre/lustre.ko): Input/output error


dmesg shows:
ko2iblnd: disagrees about version of symbol ib_fmr_pool_unmap
ko2iblnd: Unknown symbol ib_fmr_pool_unmap
ko2iblnd: disagrees about version of symbol ib_create_cq
ko2iblnd: Unknown symbol ib_create_cq
…





-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20121228/e51e56c6/attachment.htm>


More information about the lustre-discuss mailing list