[Lustre-discuss] issue: OFED 1.3 compilation issues with Kernel from CFS

[Brian J. Murrell]

On Sat, 2008-06-21 at 17:57 -0700, Anand Bisen wrote:
> Hi,

Hello,

> I have been trying to build OFED 1.3 (esp. ofa_kernel: kernel-ib)
> package against kernel "2.6.18-53.1.14.el5_lustre.1.6.5smp", which is
> distributed by CFS. The error that i am encountering is during the end
> of the rpmbuild phase where the build fails trying to find some
> directories. The same build process finishes successfully if i use the
> CentOS default kernel.

The difference between the two kernels that is likely to be at cause
here is that in our .config we set:

# CONFIG_CHELSIO_T3 is not set

because we don't want to have two versions of the respective drivers in
the modules directory causing confusion.  In our build, we disable the
vendor kernel's build of those and get those drivers from the OFED
release.  We specifically force them to build in the ofa_kernel build by
passing the options: --with-mlx4-mod --with-cxgb3-mod

I suppose perhaps the stock OFED installer is picking up the values from
the kernel's .config and [not] building those drivers based on that
specification.  Pity if so.

I think during the install.pl you an select which drivers to include.
Try enabling those two drivers specifically.

> I also noticed some binaries missing in the kernel-source provided by
> CFS in the directory scripts/ subdirectory, (ex. genksyms/genksyms). I
> copied those binaries from the stock CentOS kernel.

Yes, we don't ship a built tree in the /usr/src/<version> tree.  You
should be able to find a built tree in /usr/src/<version>-obj/.../...
though.

> ------------------------------
> RPMBUILD ERROR:
> ------------------------------
> 
> + /usr/lib/rpm/redhat/brp-strip-comment-note /usr/bin/strip /usr/bin/objdump
> + /usr/lib/rpm/brp-python-bytecompile
> + /usr/lib/rpm/redhat/brp-java-repack-jars
> Processing files: kernel-ib-1.3-2.6.18_53.1.14.el5_lustre.1.6.5smp
> error: File not found:
> /var/tmp/OFED/lib/modules/2.6.18-53.1.14.el5_lustre.1.6.5smp/updates/kernel/drivers/net/cxgb3
> error: File not found:
> /var/tmp/OFED/lib/modules/2.6.18-53.1.14.el5_lustre.1.6.5smp/updates/kernel/drivers/net/mlx4

These are strange.  The ofa_kernel RPM specfile is not supposed to try
to include those modules unless they were selected in the install.pl
process:

%if %{build_cxgb3}
%{LIB_MOD_DIR_NET}/cxgb3
%{LIB_MOD_DIR_INF}/hw/cxgb3
%endif
%if %{build_nes}
%{LIB_MOD_DIR_INF}/hw/nes
%endif
%if %{build_mlx4}
%{LIB_MOD_DIR_NET}/mlx4
%{LIB_MOD_DIR_INF}/hw/mlx4
%endif

Strange you are only getting an error from the net modules and not the
hw modules too.

What's in your <ofed_source_dir>/ofed.conf file?

b.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20080624/f65e3d68/attachment.pgp>