[Lustre-discuss] Latest RHEL kernel: won't make mgs module, and ko2iblnd not built for right OFED modules
Canon, Richard Shane
canonrs at ornl.gov
Fri Feb 29 12:49:17 PST 2008
Chris,
Try using /usr/local/ofed/current/src/ofa_kernel/ instead of the version
specific one. We were seeing Oops when compiling against the version
specific tree. I've been meaning to post to the list about this.
--Shane
-----Original Message-----
From: lustre-discuss-bounces at lists.lustre.org
[mailto:lustre-discuss-bounces at lists.lustre.org] On Behalf Of Chris
Worley
Sent: Friday, February 29, 2008 1:32 PM
To: lustre-discuss
Subject: Re: [Lustre-discuss] Latest RHEL kernel: won't make mgs
module,and ko2iblnd not built for right OFED modules
On Fri, Feb 29, 2008 at 8:50 AM, Canon, Richard Shane <canonrs at ornl.gov>
wrote:
>
>
> Yes. We also had to fix that. Sorry I left that off. In our case,
we
> created a modified Module.symvers by grep'ing out anything with
> drivers/infiniband. Also, OFED will create a Module.symvers for its
> symbols. You can append that to the (fixed) Module.symvers so that
the
> symbols will still be checked.
Okay, I found it:
/usr/share/doc/ofed-docs-1.2.5.5/create_Module.symvers.sh
I've also changed my config line in Lustre to point to the proper
modules:
./configure --with-o2ib=/usr/local/ofed/current/src/ofa_kernel-1.2.5.5/
--with-ldiskfs
... that caused a few other Lustre build errors that were easy to
fix/hack.
To fix the mgs.ko issue, I'm just manually copying the module into
/lib/modules.
... now it just panics when I "modprobe mgs" ;)
Chris
>
>
> --Shane
>
> -----Original Message-----
> From: lustre-discuss-bounces at lists.lustre.org
>
> [mailto:lustre-discuss-bounces at lists.lustre.org] On Behalf Of Craig
> Prescott
> Sent: Friday, February 29, 2008 10:41 AM
> To: Chris Worley
> Cc: lustre-discuss at lists.lustre.org
> Subject: Re: [Lustre-discuss] Latest RHEL kernel: won't make mgs
module,
> and ko2iblnd not built for right OFED modules
>
>
>
>
> Nod - we ran into this issue as well. Your Module.symvers
> file for probably has entries in it for the IB modules
> included with the RHEL kernel (and not from OFED).
>
> Here's what we do to avoid this issue (and others) when
> building a patched Lustre kernel with OFED:
>
> 1) Get the source for that kernel (rpmbuild -bp the
> src.rpm and copy it somewhere).
> 2) Configure that kernel *without* infiniband
> (edit the config for that kernel to say
> "#CONFIG_INFINIBAND is not set" and get rid
> of all the other IB stuff).
> 3) patch that kernel for Lustre
> 4) build and install that kernel
> 5) build ofa-kernel against that kernel and install it
> 6) cat <ofa-kernel-dir>/Module.symvers >>
> <linux-kernel-dir>/Module.symvers
> 7) Build and install Lustre against this kernel and
> ofa-kernel.
>
> Hope that helps,
> Craig Prescott
> UF HPC Center
>
> Chris Worley wrote:
> > Lustre 1.6.4.2, OFED 1.2.5.5, RHEL kernel 2.6.9.67.0.4.
> >
> > Configured with: --with-o2ib --with-ldiskfs
> >
> > mgs.ko is built but not wrapped into modules rpm, with an undefined
> > external that should be from the lvfs module (lvfs_check_rdonly).
> >
> > When ko2iblnd loads, it gets lots of "disagrees about version of
> > symbol <some OFED symbol>" messages. I'm guessing this is building
> > wrt the ib modules in the kernel rather than the OFED modules I
have
> > loaded.
> >
> > There were other patches I needed to make for this kernel (to
resolve
> > mutex_lock/unlock, __d_rehash, and __d_move... but those were easy
to
> > fix).
> >
> > What am I doing wrong?
> >
> > Thanks,
> >
> > Chris
> > _______________________________________________
> > Lustre-discuss mailing list
> > Lustre-discuss at lists.lustre.org
> > http://lists.lustre.org/mailman/listinfo/lustre-discuss
>
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-discuss
>
_______________________________________________
Lustre-discuss mailing list
Lustre-discuss at lists.lustre.org
http://lists.lustre.org/mailman/listinfo/lustre-discuss
More information about the lustre-discuss
mailing list