[Lustre-discuss] Latest RHEL kernel: won't make mgs module, and ko2iblnd not built for right OFED modules

Canon, Richard Shane canonrs at ornl.gov
Fri Feb 29 12:49:17 PST 2008


Chris,

Try using /usr/local/ofed/current/src/ofa_kernel/ instead of the version
specific one.  We were seeing Oops when compiling against the version
specific tree.  I've been meaning to post to the list about this.


--Shane

-----Original Message-----
From: lustre-discuss-bounces at lists.lustre.org
[mailto:lustre-discuss-bounces at lists.lustre.org] On Behalf Of Chris
Worley
Sent: Friday, February 29, 2008 1:32 PM
To: lustre-discuss
Subject: Re: [Lustre-discuss] Latest RHEL kernel: won't make mgs
module,and ko2iblnd not built for right OFED modules

On Fri, Feb 29, 2008 at 8:50 AM, Canon, Richard Shane <canonrs at ornl.gov>
wrote:
>
>
>  Yes.  We also had to fix that.  Sorry I left that off.  In our case,
we
>  created a modified Module.symvers by grep'ing out anything with
>  drivers/infiniband.  Also, OFED will create a Module.symvers for its
>  symbols.  You can append that to the (fixed) Module.symvers so that
the
>  symbols will still be checked.

Okay, I found it:

/usr/share/doc/ofed-docs-1.2.5.5/create_Module.symvers.sh

I've also changed my config line in Lustre to point to the proper
modules:

./configure --with-o2ib=/usr/local/ofed/current/src/ofa_kernel-1.2.5.5/
--with-ldiskfs

... that caused a few other Lustre build errors that were easy to
fix/hack.

To fix the mgs.ko issue, I'm just manually copying the module into
/lib/modules.

... now it just panics when I "modprobe mgs" ;)

Chris
>
>
>  --Shane
>
>  -----Original Message-----
>  From: lustre-discuss-bounces at lists.lustre.org
>
> [mailto:lustre-discuss-bounces at lists.lustre.org] On Behalf Of Craig
>  Prescott
>  Sent: Friday, February 29, 2008 10:41 AM
>  To: Chris Worley
>  Cc: lustre-discuss at lists.lustre.org
>  Subject: Re: [Lustre-discuss] Latest RHEL kernel: won't make mgs
module,
>  and ko2iblnd not built for right OFED modules
>
>
>
>
> Nod - we ran into this issue as well.  Your Module.symvers
>  file for probably has entries in it for the IB modules
>  included with the RHEL kernel (and not from OFED).
>
>  Here's what we do to avoid this issue (and others) when
>  building a patched Lustre kernel with OFED:
>
>  1) Get the source for that kernel (rpmbuild -bp the
>     src.rpm and copy it somewhere).
>  2) Configure that kernel *without* infiniband
>     (edit the config for that kernel to say
>     "#CONFIG_INFINIBAND is not set" and get rid
>     of all the other IB stuff).
>  3) patch that kernel for Lustre
>  4) build and install that kernel
>  5) build ofa-kernel against that kernel and install it
>  6) cat <ofa-kernel-dir>/Module.symvers >>
>     <linux-kernel-dir>/Module.symvers
>  7) Build and install Lustre against this kernel and
>     ofa-kernel.
>
>  Hope that helps,
>  Craig Prescott
>  UF HPC Center
>
>  Chris Worley wrote:
>  > Lustre 1.6.4.2, OFED 1.2.5.5, RHEL kernel 2.6.9.67.0.4.
>  >
>  > Configured with: --with-o2ib --with-ldiskfs
>  >
>  > mgs.ko is built but not wrapped into modules rpm, with an undefined
>  > external that should be from the lvfs module (lvfs_check_rdonly).
>  >
>  > When ko2iblnd loads, it gets lots of "disagrees about version of
>  > symbol <some OFED symbol>" messages.  I'm guessing this is building
>  > wrt the ib modules in the kernel rather than the OFED modules I
have
>  > loaded.
>  >
>  > There were other patches I needed to make for this kernel (to
resolve
>  > mutex_lock/unlock, __d_rehash, and __d_move... but those were easy
to
>  > fix).
>  >
>  > What am I doing wrong?
>  >
>  > Thanks,
>  >
>  > Chris
>  > _______________________________________________
>  > Lustre-discuss mailing list
>  > Lustre-discuss at lists.lustre.org
>  > http://lists.lustre.org/mailman/listinfo/lustre-discuss
>
>  _______________________________________________
>  Lustre-discuss mailing list
>  Lustre-discuss at lists.lustre.org
>  http://lists.lustre.org/mailman/listinfo/lustre-discuss
>
_______________________________________________
Lustre-discuss mailing list
Lustre-discuss at lists.lustre.org
http://lists.lustre.org/mailman/listinfo/lustre-discuss



More information about the lustre-discuss mailing list