Hi, Oleg<div><br></div><div>I just rebuilt the SUSE Lustre 2.1 client under xen kernel environment, no problem with the rebuild process, but I installed the package on the build machine got the following errors: </div><div>
<br></div><div>
<div style="font-family:Arial;font-size:medium">
<div>ip-10-32-62-48://usr/src/packages/RPMS/x86_64 # modprobe lnet<div>FATAL: Error inserting lnet (/lib/modules/2.6.32.46-0.3-ec2/updates/kernel/net/lustre/lnet.ko): Invalid module format</div><div>ip-10-32-62-48://usr/src/packages/RPMS/x86_64 # uname -a</div>
<div>Linux ip-10-32-62-48 2.6.32.46-0.3-ec2 #1 SMP 2011-09-29 17:49:31 +0200 x86_64 x86_64 x86_64 GNU/Linux</div><div>ip-10-32-62-48://usr/src/packages/RPMS/x86_64 # modinfo /lib/modules/2.6.32.46-0.3-ec2/updates/kernel/net/lustre/lnet.ko</div>
<div>filename: /lib/modules/2.6.32.46-0.3-ec2/updates/kernel/net/lustre/lnet.ko</div><div>license: GPL</div><div>description: Portals v3.1</div><div>author: Peter J. Braam <<a href="mailto:braam@clusterfs.com">braam@clusterfs.com</a>></div>
<div>srcversion: D64519A3761B3BB8EDF41F1</div><div>depends: libcfs</div><div>vermagic: 2.6.32.46-0.3-ec2 SMP mod_unload modversions Xen</div><div>parm: accept:Accept connections (secure|all|none) (charp)</div>
<div>parm: accept_port:Acceptor's port (same on all nodes) (int)</div><div>parm: accept_backlog:Acceptor's listen backlog (int)</div><div>parm: accept_timeout:Acceptor's timeout (seconds) (int)</div>
<div>parm: forwarding:Explicitly enable/disable forwarding between networks (charp)</div><div>parm: tiny_router_buffers:# of 0 payload messages to buffer in the router (int)</div><div>parm: small_router_buffers:# of small (1 page) messages to buffer in the router (int)</div>
<div>parm: large_router_buffers:# of large messages to buffer in the router (int)</div><div>parm: peer_buffer_credits:# router buffer credits per peer (int)</div><div>parm: auto_down:Automatically mark peers down on comms error (int)</div>
<div>parm: check_routers_before_use:Assume routers are down and ping them before use (int)</div><div>parm: avoid_asym_router_failure:Avoid asymmetrical failures: reserved, use at your own risk (int)</div>
<div>parm: dead_router_check_interval:Seconds between dead router health checks (<= 0 to disable) (int)</div><div>parm: live_router_check_interval:Seconds between live router health checks (<= 0 to disable) (int)</div>
<div>parm: router_ping_timeout:Seconds to wait for the reply to a router health query (int)</div><div>parm: config_on_load:configure network at module load (int)</div><div>parm: local_nid_dist_zero:Reserved (int)</div>
<div>parm: ip2nets:LNET network <- IP table (charp)</div><div>parm: networks:local networks (charp)</div><div>parm: routes:routes to non-local networks (charp)</div></div><div><br></div></div>
Any idea on this? I do see some error message in system log as well: </div><div>
<span class="Apple-style-span" style="font-family:Arial;font-size:medium">Dec 17 01:51:41 ip-10-32-62-48 kernel: [ 3744.865201] libcfs: no symbol version for module_layout</span></div><div><font class="Apple-style-span" face="Arial" size="3"><br>
</font></div><div><font class="Apple-style-span" face="Arial" size="3">Thanks,</font></div><div><font class="Apple-style-span" face="Arial" size="3">-Jon.<br></font>
<br><br><div class="gmail_quote">On Wed, Dec 7, 2011 at 1:24 AM, Oleg Drokin <span dir="ltr"><<a href="mailto:green@whamcloud.com">green@whamcloud.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Hello!<br>
<br>
Yes, we do plan to have 3.x support. The 2.6.38 support is almost finished and we will start looking at 3.x next.<br>
<br>
Bye,<br>
Oleg<br>
<div class="HOEnZb"><div class="h5">On Nov 29, 2011, at 3:11 PM, Jon Zhu wrote:<br>
<br>
> Hi, Oleg<br>
><br>
> Does Lustre 2.1 support Ubuntu client? From the following build map I can see only ubuntu 10.04 client is supported but kernel needs to be patched. Is there any plan to support patchless kernel client on Ubuntu? The new Ubuntu Oneiric 11.10 release kernel 3.0 is being used, any plan to support that in the near future?<br>
> <a href="http://build.whamcloud.com/job/lustre-b2_1/" target="_blank">http://build.whamcloud.com/job/lustre-b2_1/</a>?<br>
><br>
> Thanks,<br>
> -Jon.<br>
> <a href="mailto:jon.zhu@gmail.com">jon.zhu@gmail.com</a><br>
><br>
> On Sun, Oct 2, 2011 at 4:31 PM, Jon Zhu <<a href="mailto:jon.zhu@gmail.com">jon.zhu@gmail.com</a>> wrote:<br>
> Thanks a lot, the work around works.<br>
><br>
> -Jon.<br>
><br>
><br>
><br>
><br>
> On Sun, Oct 2, 2011 at 3:47 PM, Oleg Drokin <<a href="mailto:green@whamcloud.com">green@whamcloud.com</a>> wrote:<br>
> Hello!<br>
><br>
> Last time I hit this (some years ago), a simple touch ldiskfs/Module.symvers helped. I don't remember what the issue was or how it was properly fixed, though.<br>
><br>
> Bye,<br>
> Oleg<br>
> On Oct 2, 2011, at 1:57 PM, Jon Zhu wrote:<br>
><br>
> > Hi, Oleg<br>
> ><br>
> > I encountered the following error while building Lustre 2.1 release on Redhat 2.1, do you have any idea on this error?<br>
> ><br>
> > make[1]: *** No rule to make target `/build/lustre-release/ldiskfs/Module.symvers', needed by `Module.symvers'. Stop.<br>
> ><br>
> ><br>
> > Full build log :<br>
> > ....<br>
> > + /usr/lib/rpm/redhat/brp-java-repack-jars<br>
> > Processing files: lustre-iokit-1.2-201110021351.noarch<br>
> > Executing(%doc): /bin/sh -e /var/tmp/rpm-tmp.IOpHyP<br>
> > + umask 022<br>
> > + cd /build/kernel/rpmbuild/BUILD<br>
> > + cd lustre-iokit-1.2<br>
> > + DOCDIR=/build/kernel/rpmbuild/BUILDROOT/lustre-iokit-1.2-201110021351.x86_64/usr/share/doc/lustre-iokit-1.2<br>
> > + export DOCDIR<br>
> > + rm -rf /build/kernel/rpmbuild/BUILDROOT/lustre-iokit-1.2-201110021351.x86_64/usr/share/doc/lustre-iokit-1.2<br>
> > + /bin/mkdir -p /build/kernel/rpmbuild/BUILDROOT/lustre-iokit-1.2-201110021351.x86_64/usr/share/doc/lustre-iokit-1.2<br>
> > + cp -pr obdfilter-survey/README.obdfilter-survey /build/kernel/rpmbuild/BUILDROOT/lustre-iokit-1.2-201110021351.x86_64/usr/share/doc/lustre-iokit-1.2<br>
> > + cp -pr ior-survey/README.ior-survey /build/kernel/rpmbuild/BUILDROOT/lustre-iokit-1.2-201110021351.x86_64/usr/share/doc/lustre-iokit-1.2<br>
> > + cp -pr ost-survey/README.ost-survey /build/kernel/rpmbuild/BUILDROOT/lustre-iokit-1.2-201110021351.x86_64/usr/share/doc/lustre-iokit-1.2<br>
> > + cp -pr sgpdd-survey/README.sgpdd-survey /build/kernel/rpmbuild/BUILDROOT/lustre-iokit-1.2-201110021351.x86_64/usr/share/doc/lustre-iokit-1.2<br>
> > + cp -pr stats-collect/<a href="http://README.lstats.sh" target="_blank">README.lstats.sh</a> /build/kernel/rpmbuild/BUILDROOT/lustre-iokit-1.2-201110021351.x86_64/usr/share/doc/lustre-iokit-1.2<br>
> > + exit 0<br>
> > Provides: lustre-iokit = 1.2<br>
> > Requires(rpmlib): rpmlib(CompressedFileNames) <= 3.0.4-1 rpmlib(FileDigests) <= 4.6.0-1 rpmlib(PayloadFilesHavePrefix) <= 4.0-1 rpmlib(VersionedDependencies) <= 3.0.3-1<br>
> > Requires: /bin/bash /bin/sh /usr/bin/perl perl(File::Path) perl(Getopt::Long) perl(Getopt::Std) perl(POSIX)<br>
> > Checking for unpackaged file(s): /usr/lib/rpm/check-files /build/kernel/rpmbuild/BUILDROOT/lustre-iokit-1.2-201110021351.x86_64<br>
> > Wrote: /build/kernel/rpmbuild/SRPMS/lustre-iokit-1.2-201110021351.src.rpm<br>
> > Wrote: /build/kernel/rpmbuild/RPMS/noarch/lustre-iokit-1.2-201110021351.noarch.rpm<br>
> > Executing(%clean): /bin/sh -e /var/tmp/rpm-tmp.0eXePd<br>
> > + umask 022<br>
> > + cd /build/kernel/rpmbuild/BUILD<br>
> > + cd lustre-iokit-1.2<br>
> > + /bin/rm -rf /build/kernel/rpmbuild/BUILDROOT/lustre-iokit-1.2-201110021351.x86_64<br>
> > + exit 0<br>
> > make[1]: Leaving directory `/build/lustre-release/lustre-iokit'<br>
> > Finished rpms in lustre-iokit<br>
> > make[1]: Entering directory `/build/lustre-release'<br>
> > make[1]: *** No rule to make target `/build/lustre-release/ldiskfs/Module.symvers', needed by `Module.symvers'. Stop.<br>
> > make[1]: Leaving directory `/build/lustre-release'<br>
> > make: *** [rpms] Error 2<br>
> ><br>
> ><br>
> > Thanks,<br>
> > -Jon.<br>
> ><br>
> ><br>
> > On Thu, Sep 29, 2011 at 11:34 PM, Oleg Drokin <<a href="mailto:green@whamcloud.com">green@whamcloud.com</a>> wrote:<br>
> > Hello!<br>
> ><br>
> > There is nothing special, same as rhel6.1:<br>
> > unpack the lustre source, run autogen.sh, run configure and provide the path to the linux kernel source for your distro (need to patch first too), make.<br>
> ><br>
> > Bye,<br>
> > Oleg<br>
> > On Sep 29, 2011, at 11:21 PM, Jon Zhu wrote:<br>
> ><br>
> > > Hi, Oleg<br>
> > ><br>
> > > Do we have a procedure on how to build v2.1 GA code on CentOS 5.6 (xen)? On whamcloud wiki I can only find build v2.1 on RHEL 6.1 or build v1.8 on CentOS 5.6.<br>
> > ><br>
> > > BTW, congratulations on the 2.1 release!<br>
> > ><br>
> > > Regards,<br>
> > ><br>
> > > Jon Zhu<br>
> > > Sent from Google Mail<br>
> > ><br>
> > ><br>
> > ><br>
> > ><br>
> > > On Fri, Jun 24, 2011 at 2:43 PM, Oleg Drokin <<a href="mailto:green@whamcloud.com">green@whamcloud.com</a>> wrote:<br>
> > > Hwllo~<br>
> > ><br>
> > > On Jun 23, 2011, at 9:51 PM, Jon Zhu wrote:<br>
> > ><br>
> > > > I still got some crash when further run some I/O test with the build, here's some system message containing call stack info maybe be useful to you to find the bug:<br>
> > ><br>
> > > > Jun 23 21:46:12 ip-10-112-59-173 kernel: ------------[ cut here ]------------<br>
> > > > Jun 23 21:46:12 ip-10-112-59-173 kernel: WARNING: at kernel/sched.c:7087 __cond_resched_lock+0x8e/0xb0() (Not tainted)<br>
> > > > Jun 23 21:46:12 ip-10-112-59-173 kernel: Modules linked in: lustre(U) lov(U) osc(U) lquota(U) mdc(U) fid(U) fld(U) ksocklnd(U) ptlrpc(U) obdclass(U) lnet(U) lvfs(U) libcfs(U) ldiskfs(U) sha256_generic cryptd aes_x86_64 aes_generic cbc dm_crypt autofs4 ipv6 microcode xen_netfront ext4 mbcache jbd2 xen_blkfront dm_mod [last unloaded: scsi_wait_scan]<br>
> > > > Jun 23 21:46:12 ip-10-112-59-173 kernel: Pid: 1421, comm: mount.lustre Not tainted 2.6.32.lustre21 #6<br>
> > > > Jun 23 21:46:12 ip-10-112-59-173 kernel: Call Trace:<br>
> > > > Jun 23 21:46:12 ip-10-112-59-173 kernel: [<ffffffff81069c37>] ? warn_slowpath_common+0x87/0xc0<br>
> > > > Jun 23 21:46:12 ip-10-112-59-173 kernel: [<ffffffff81007671>] ? __raw_callee_save_xen_save_fl+0x11/0x1e<br>
> > > > Jun 23 21:46:12 ip-10-112-59-173 kernel: [<ffffffff81069c8a>] ? warn_slowpath_null+0x1a/0x20<br>
> > > > Jun 23 21:46:12 ip-10-112-59-173 kernel: [<ffffffff810654fe>] ? __cond_resched_lock+0x8e/0xb0<br>
> > > > Jun 23 21:46:12 ip-10-112-59-173 kernel: [<ffffffff811a53b7>] ? shrink_dcache_for_umount_subtree+0x187/0x340<br>
> > > > Jun 23 21:46:12 ip-10-112-59-173 kernel: [<ffffffff811a55a6>] ? shrink_dcache_for_umount+0x36/0x60<br>
> > > > Jun 23 21:46:12 ip-10-112-59-173 kernel: [<ffffffff8118f4ff>] ? generic_shutdown_super+0x1f/0xe0<br>
> > > > Jun 23 21:46:12 ip-10-112-59-173 kernel: [<ffffffff8118f5f1>] ? kill_block_super+0x31/0x50<br>
> > > > Jun 23 21:46:12 ip-10-112-59-173 kernel: [<ffffffff811906b5>] ? deactivate_super+0x85/0xa0<br>
> > > > Jun 23 21:46:12 ip-10-112-59-173 kernel: [<ffffffff811ac5af>] ? mntput_no_expire+0xbf/0x110<br>
> > > > Jun 23 21:46:12 ip-10-112-59-173 kernel: [<ffffffffa0273f8e>] ? unlock_mntput+0x3e/0x60 [obdclass]<br>
> > > > Jun 23 21:46:12 ip-10-112-59-173 kernel: [<ffffffffa0277a98>] ? server_kernel_mount+0x268/0xe80 [obdclass]<br>
> > > > Jun 23 21:46:12 ip-10-112-59-173 kernel: [<ffffffffa0280d40>] ? lustre_fill_super+0x0/0x1290 [obdclass]<br>
> > > > Jun 23 21:46:12 ip-10-112-59-173 kernel: [<ffffffffa0279070>] ? lustre_init_lsi+0xd0/0x5b0 [obdclass]<br>
> > > > Jun 23 21:46:12 ip-10-112-59-173 kernel: [<ffffffff810ac71d>] ? lock_release+0xed/0x220<br>
> > > > Jun 23 21:46:12 ip-10-112-59-173 kernel: [<ffffffffa0280fd0>] ? lustre_fill_super+0x290/0x1290 [obdclass]<br>
> > > > Jun 23 21:46:12 ip-10-112-59-173 kernel: [<ffffffff8118ee20>] ? set_anon_super+0x0/0x110<br>
> > > > Jun 23 21:46:12 ip-10-112-59-173 kernel: [<ffffffffa0280d40>] ? lustre_fill_super+0x0/0x1290 [obdclass]<br>
> > > > Jun 23 21:46:12 ip-10-112-59-173 kernel: [<ffffffff8119035f>] ? get_sb_nodev+0x5f/0xa0<br>
> > > > Jun 23 21:46:12 ip-10-112-59-173 kernel: [<ffffffffa0272885>] ? lustre_get_sb+0x25/0x30 [obdclass]<br>
> > > > Jun 23 21:46:12 ip-10-112-59-173 kernel: [<ffffffff8118ffbb>] ? vfs_kern_mount+0x7b/0x1b0<br>
> > > > Jun 23 21:46:12 ip-10-112-59-173 kernel: [<ffffffff81190162>] ? do_kern_mount+0x52/0x130<br>
> > > > Jun 23 21:46:12 ip-10-112-59-173 kernel: [<ffffffff811ae647>] ? do_mount+0x2e7/0x870<br>
> > > > Jun 23 21:46:12 ip-10-112-59-173 kernel: [<ffffffff811aec60>] ? sys_mount+0x90/0xe0<br>
> > > > Jun 23 21:46:12 ip-10-112-59-173 kernel: [<ffffffff8100b132>] ? system_call_fastpath+0x16/0x1b<br>
> > > > Jun 23 21:46:12 ip-10-112-59-173 kernel: ---[ end trace a8fb737c71bfba13 ]---<br>
> > ><br>
> > > This is not a crash, it's just a warning about scheduling in inappropriate context I guess, but the kernel will continue to work.<br>
> > > Interesting that I have never seen anything like that in rhel5 xen kernels, perhaps it's something with rhel6.1 xen?<br>
> > ><br>
> > > Bye,<br>
> > > Oleg<br>
> > > --<br>
> > > Oleg Drokin<br>
> > > Senior Software Engineer<br>
> > > Whamcloud, Inc.<br>
> > ><br>
> > ><br>
> ><br>
> > --<br>
> > Oleg Drokin<br>
> > Senior Software Engineer<br>
> > Whamcloud, Inc.<br>
> ><br>
> ><br>
><br>
> --<br>
> Oleg Drokin<br>
> Senior Software Engineer<br>
> Whamcloud, Inc.<br>
><br>
><br>
><br>
<br>
--<br>
Oleg Drokin<br>
Senior Software Engineer<br>
Whamcloud, Inc.<br>
<br>
</div></div></blockquote></div><br></div>