Guys,<br><br>Thanks a lot for all the help.. <br>I was able to build a patchless client from source. The basic verification tests (unix commands) were successful. <br><br>I had an issue with latest CentOS kernel - 2.6.18-128.el5 though. Since I started with minimum install (withou gcc) and then installed gcc thru yum, which had dependency on kernel-headers package. By default CentOS 5.2 selects package from updates repo. So one may end up with 2.6.18-92.el5 for kernel and 2.6.18-128.el5 for kernel-headers. I also tried building it against latest 2.6.18-128.el5 kernel, however it had an issue as pointed out here: <a href="http://lists.lustre.org/pipermail/lustre-discuss/2009-May/010560.html">http://lists.lustre.org/pipermail/lustre-discuss/2009-May/010560.html</a> (bug fixed: <a href="https://bugzilla.lustre.org/show_bug.cgi?id=19024">https://bugzilla.lustre.org/show_bug.cgi?id=19024</a> ). <br>
<br>Thank you everone. <br>Excited to get started with lustre..<br><br>
-<br>CS. <br><br><br><div class="gmail_quote">On Wed, Jun 17, 2009 at 7:36 PM, Arden Wiebe <span dir="ltr"><<a href="mailto:albert682@yahoo.com" target="_blank">albert682@yahoo.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<br>
Carlos:<br>
<br>
This client of mine works. Matter of fact on all my clients it works.<br>
<br>
[root@lustreone]# rpm -qa | grep -i lustre<br>
lustre-ldiskfs-3.0.8-2.6.18_92.1.17.el5_lustre.1.8.0smp<br>
<div>lustre-1.8.0-2.6.18_92.1.17.el5_lustre.1.8.0smp<br>
</div><div>lustre-modules-1.8.0-2.6.18_92.1.17.el5_lustre.1.8.0smp<br>
</div>kernel-lustre-smp-2.6.18-92.1.17.el5_lustre.1.8.0<br>
<br>
Otherwise your output for the same command lists only 2 packages installed so you are missing some packages - those being the client packages if you don't want to use the patched kernel method of making a client as I have done above. If you issue the rpm commands I mentioned in the very first response of this thread you will have a working client.<br>
<br>
Arden<br>
<br>
--- On Wed, 6/17/09, Carlos Santana <<a href="mailto:neubyr@gmail.com" target="_blank">neubyr@gmail.com</a>> wrote:<br>
<br>
> From: Carlos Santana <<a href="mailto:neubyr@gmail.com" target="_blank">neubyr@gmail.com</a>><br>
<div>> Subject: Re: [Lustre-discuss] Lustre installation and configuration problems<br>
</div>> To: "Jerome, Ron" <<a href="mailto:Ron.Jerome@nrc-cnrc.gc.ca" target="_blank">Ron.Jerome@nrc-cnrc.gc.ca</a>><br>
> Cc: <a href="mailto:lustre-discuss@lists.lustre.org" target="_blank">lustre-discuss@lists.lustre.org</a><br>
> Date: Wednesday, June 17, 2009, 5:10 PM<br>
<div>> Folks,<br>
><br>
> It been unsuccessful till now..<br>
><br>
> I made a fresh CentOS 5.2 minimum install (2.6.18-92.el5).<br>
> Later, I<br>
> updated kernel to 2.6.18-92.1.17 version. Here is a output<br>
> from uname<br>
> and rpm query:<br>
><br>
> [root@localhost ~]# rpm -qa | grep lustre<br>
> lustre-1.8.0-2.6.18_92.1.17.el5_lustre.1.8.0smp<br>
> lustre-modules-1.8.0-2.6.18_92.1.17.el5_lustre.1.8.0smp<br>
> [root@localhost ~]# uname -a<br>
> Linux localhost.localdomain 2.6.18-92.1.17.el5 #1 SMP Tue<br>
> Nov 4<br>
> 13:45:01 EST 2008 i686 i686 i386 GNU/Linux<br>
><br>
</div><div><div></div><div>> Other details:<br>
> --- --- ---<br>
> [root@localhost ~]# ls -l /lib/modules | grep 2.6<br>
> drwxr-xr-x 6 root root 4096 Jun 17 18:47<br>
> 2.6.18-92.1.17.el5<br>
> drwxr-xr-x 6 root root 4096 Jun 17 17:38 2.6.18-92.el5<br>
><br>
><br>
> [root@localhost modules]# find . | grep lustre<br>
> ./2.6.18-92.1.17.el5/kernel/net/lustre<br>
> ./2.6.18-92.1.17.el5/kernel/net/lustre/libcfs.ko<br>
> ./2.6.18-92.1.17.el5/kernel/net/lustre/lnet.ko<br>
> ./2.6.18-92.1.17.el5/kernel/net/lustre/ksocklnd.ko<br>
> ./2.6.18-92.1.17.el5/kernel/net/lustre/ko2iblnd.ko<br>
> ./2.6.18-92.1.17.el5/kernel/net/lustre/lnet_selftest.ko<br>
> ./2.6.18-92.1.17.el5/kernel/fs/lustre<br>
> ./2.6.18-92.1.17.el5/kernel/fs/lustre/osc.ko<br>
> ./2.6.18-92.1.17.el5/kernel/fs/lustre/ptlrpc.ko<br>
> ./2.6.18-92.1.17.el5/kernel/fs/lustre/obdecho.ko<br>
> ./2.6.18-92.1.17.el5/kernel/fs/lustre/lvfs.ko<br>
> ./2.6.18-92.1.17.el5/kernel/fs/lustre/mgc.ko<br>
> ./2.6.18-92.1.17.el5/kernel/fs/lustre/llite_lloop.ko<br>
> ./2.6.18-92.1.17.el5/kernel/fs/lustre/lov.ko<br>
> ./2.6.18-92.1.17.el5/kernel/fs/lustre/mdc.ko<br>
> ./2.6.18-92.1.17.el5/kernel/fs/lustre/lquota.ko<br>
> ./2.6.18-92.1.17.el5/kernel/fs/lustre/lustre.ko<br>
> ./2.6.18-92.1.17.el5/kernel/fs/lustre/obdclass.ko<br>
> --- --- ---<br>
><br>
><br>
> I am still having same problem. I seriously doubt, am I<br>
> missing anything?<br>
> I also tried a source install for 'patchless client',<br>
> however I have<br>
> been consistent in its results too.<br>
><br>
> Are there any configuration steps needed after rpm (or<br>
> source)<br>
> installation? The one that I know of is restricting<br>
> interfaces in<br>
> modeprobe.conf, however I have tried it on-n-off with no<br>
> success.<br>
> Could anyone please suggest any debugging and tests for the<br>
> same? How<br>
> can I provide you more valuable output to help me? Any<br>
> insights?<br>
><br>
> Also, I have a suggestion here. It might be good idea to<br>
> check for<br>
> 'uname -r' check in RPM installation to check for matching<br>
> kernel<br>
> version and if not suggest for source install.<br>
><br>
> Thanks for the help. I really appreciate your patience..<br>
><br>
> -<br>
> Thanks,<br>
> CS.<br>
><br>
><br>
> On Wed, Jun 17, 2009 at 10:40 AM, Jerome, Ron<<a href="mailto:Ron.Jerome@nrc-cnrc.gc.ca" target="_blank">Ron.Jerome@nrc-cnrc.gc.ca</a>><br>
> wrote:<br>
> > I think the problem you have, as Cliff alluded to, is<br>
> a mismatch between<br>
> > your kernel version and the Luster kernel version<br>
> modules.<br>
> ><br>
> ><br>
> ><br>
> > You have kernel “2.6.18-92.el5” and are installing<br>
> Lustre<br>
> > “2.6.18_92.1.17.el5” Note the “.1.17” is<br>
> significant as the modules will<br>
> > end up in the wrong directory. There is an update to<br>
> CentOS to bring the<br>
> > kernel to the matching 2.6.18_92.1.17.el5 version you<br>
> can pull it off the<br>
> > CentOS mirror site in the updates directory.<br>
> ><br>
> ><br>
> ><br>
> ><br>
> ><br>
> > Ron.<br>
> ><br>
> ><br>
> ><br>
> > From: <a href="mailto:lustre-discuss-bounces@lists.lustre.org" target="_blank">lustre-discuss-bounces@lists.lustre.org</a><br>
> > [mailto:<a href="mailto:lustre-discuss-bounces@lists.lustre.org" target="_blank">lustre-discuss-bounces@lists.lustre.org</a>]<br>
> On Behalf Of Carlos Santana<br>
> > Sent: June 17, 2009 11:21 AM<br>
> > To: <a href="mailto:lustre-discuss@lists.lustre.org" target="_blank">lustre-discuss@lists.lustre.org</a><br>
> > Subject: Re: [Lustre-discuss] Lustre installation and<br>
> configuration problems<br>
> ><br>
> ><br>
> ><br>
> > And is there any specific installation order for<br>
> patchless client? Could<br>
> > someone please share it with me?<br>
> ><br>
> > -<br>
> > CS.<br>
> ><br>
> > On Wed, Jun 17, 2009 at 10:18 AM, Carlos Santana<br>
> <<a href="mailto:neubyr@gmail.com" target="_blank">neubyr@gmail.com</a>><br>
> wrote:<br>
> ><br>
> > Huh... :( Sorry to bug you guys again...<br>
> ><br>
> > I am planning to make a fresh start now as nothing<br>
> seems to have worked for<br>
> > me. If you have any comments/feedback please share<br>
> them.<br>
> ><br>
> > I would like to confirm installation order before I<br>
> make a fresh start. From<br>
> > Arden's experience:<br>
> > <a href="http://lists.lustre.org/pipermail/lustre-discuss/2009-June/010710.html" target="_blank">http://lists.lustre.org/pipermail/lustre-discuss/2009-June/010710.html</a><br>
> , the<br>
> > lusre-module is installed last. As I was installing<br>
> Lustre 1.8, I was<br>
> > referring 1.8 operations manual<br>
> > <a href="http://manual.lustre.org/index.php?title=Main_Page" target="_blank">http://manual.lustre.org/index.php?title=Main_Page</a> .<br>
> The installation order<br>
> > in the manual is different than what Arden has<br>
> suggested.<br>
> ><br>
> > Will it make a difference in configuration at later<br>
> stage? Which one should<br>
> > I follow now?<br>
> > Any comments?<br>
> ><br>
> > Thanks,<br>
> > CS.<br>
> ><br>
> ><br>
> ><br>
> > On Wed, Jun 17, 2009 at 12:35 AM, Carlos Santana<br>
> <<a href="mailto:neubyr@gmail.com" target="_blank">neubyr@gmail.com</a>><br>
> wrote:<br>
> ><br>
> > Thanks Cliff.<br>
> ><br>
> > The depmod -a was successful before as well. I am<br>
> using CentOS 5.2<br>
> > box. Following are the packages installed:<br>
> > [root@localhost tmp]# rpm -qa | grep -i lustre<br>
> ><br>
> lustre-modules-1.8.0-2.6.18_92.1.17.el5_lustre.1.8.0smp<br>
> ><br>
> > lustre-1.8.0-2.6.18_92.1.17.el5_lustre.1.8.0smp<br>
> ><br>
> > [root@localhost tmp]# uname -a<br>
> ><br>
> > Linux localhost.localdomain 2.6.18-92.el5 #1 SMP Tue<br>
> Jun 10 18:49:47<br>
> > EDT 2008 i686 i686 i386 GNU/Linux<br>
> ><br>
> > And here is a output from strace for mount:<br>
> > <a href="http://www.heypasteit.com/clip/8WT" target="_blank">http://www.heypasteit.com/clip/8WT</a><br>
> ><br>
> > Any further debugging hints?<br>
> ><br>
> > Thanks,<br>
> > CS.<br>
> ><br>
> > On 6/16/09, Cliff White <<a href="mailto:Cliff.White@sun.com" target="_blank">Cliff.White@sun.com</a>><br>
> wrote:<br>
> >> Carlos Santana wrote:<br>
> >>> The '$ modprobe -l lustre*' did not show any<br>
> module on a patchless<br>
> >>> client. modprobe -v returns 'FATAL: Module<br>
> lustre not found'.<br>
> >>><br>
> >>> How do I install a patchless client?<br>
> >>> I have tried lustre-client-modules and<br>
> lustre-client-ver rpm packages in<br>
> >>> both sequences. Am I missing anything?<br>
> >>><br>
> >><br>
> >> Make sure the lustre-client-modules package<br>
> matches your running kernel.<br>
> >> Run depmod -a to be sure<br>
> >> cliffw<br>
> >><br>
> >>> Thanks,<br>
> >>> CS.<br>
> >>><br>
> >>><br>
> >>><br>
> >>> On Tue, Jun 16, 2009 at 2:28 PM, Cliff White<br>
> <<a href="mailto:Cliff.White@sun.com" target="_blank">Cliff.White@sun.com</a><br>
> >>> <mailto:<a href="mailto:Cliff.White@sun.com" target="_blank">Cliff.White@sun.com</a>>><br>
> wrote:<br>
> >>><br>
> >>> Carlos Santana wrote:<br>
> >>><br>
> >>> The lctlt ping and 'net up' failed<br>
> with the following messages:<br>
> >>> --- ---<br>
> >>> [root@localhost ~]# lctl ping<br>
> 10.0.0.42<br>
> >>> opening /dev/lnet failed: No such<br>
> device<br>
> >>> hint: the kernel modules may not<br>
> be loaded<br>
> >>> failed to ping 10.0.0.42@tcp: No<br>
> such device<br>
> >>><br>
> >>> [root@localhost ~]# lctl network<br>
> up<br>
> >>> opening /dev/lnet failed: No such<br>
> device<br>
> >>> hint: the kernel modules may not<br>
> be loaded<br>
> >>> LNET configure error 19: No such<br>
> device<br>
> >>><br>
> >>><br>
> >>> Make sure modules are unloaded, then try<br>
> modprobe -v.<br>
> >>> Looks like you have lnet mis-configured,<br>
> if your module options are<br>
> >>> wrong, you will see an error during the<br>
> modprobe.<br>
> >>> cliffw<br>
> >>><br>
> >>> --- ---<br>
> >>><br>
> >>><br>
> >>> I tried lustre_rmmod and depmod<br>
> commands and it did not return<br>
> >>> any error messages. Any further<br>
> clues? Reinstall patchless<br>
> >>> client again?<br>
> >>><br>
> >>> -<br>
> >>> CS.<br>
> >>><br>
> >>><br>
> >>> On Tue, Jun 16, 2009 at 1:32 PM,<br>
> Cliff White<br>
> >>> <<a href="mailto:Cliff.White@sun.com" target="_blank">Cliff.White@sun.com</a><br>
> <mailto:<a href="mailto:Cliff.White@sun.com" target="_blank">Cliff.White@sun.com</a>><br>
> >>> <mailto:<a href="mailto:Cliff.White@sun.com" target="_blank">Cliff.White@sun.com</a><br>
> <mailto:<a href="mailto:Cliff.White@sun.com" target="_blank">Cliff.White@sun.com</a>>>><br>
> wrote:<br>
> >>><br>
> >>> Carlos Santana wrote:<br>
> >>><br>
> >>> I was able to run<br>
> lustre_rmmod and depmod successfully.<br>
> >>> The<br>
> >>> '$lctl list_nids'<br>
> returned the server ip address and<br>
> >>> interface<br>
> >>> (tcp0).<br>
> >>><br>
> >>> I tried to mount the<br>
> file system on a remote client, but<br>
> >>> it<br>
> >>> failed with the<br>
> following message.<br>
> >>> --- ---<br>
> >>> [root@localhost ~]#<br>
> mount -t lustre 10.0.0.42@tcp0:/lustre<br>
> >>> /mnt/lustre<br>
> >>> mount.lustre: mount<br>
> 10.0.0.42@tcp0:/lustre at /mnt/lustre<br>
> >>> failed: No such device<br>
> >>> Are the lustre modules<br>
> loaded?<br>
> >>> Check<br>
> /etc/modprobe.conf and /proc/filesystems<br>
> >>> Note 'alias lustre<br>
> llite' should be removed from<br>
> >>> modprobe.conf<br>
> >>> --- ---<br>
> >>><br>
> >>> However, the mounting<br>
> is successful on a single node<br>
> >>> configuration - with<br>
> client on the same machine as MDS<br>
> >>> and OST.<br>
> >>> Any clues? Where to<br>
> look for logs and debug messages?<br>
> >>><br>
> >>><br>
> >>> Syslog || /var/log/messages<br>
> is the normal place.<br>
> >>><br>
> >>> You can use 'lctl ping' to<br>
> verify that the client can reach<br>
> >>> the server.<br>
> >>> Usually in these cases, it's<br>
> a network/name misconfiguration.<br>
> >>><br>
> >>> Run 'tunefs.lustre --print'<br>
> on your servers, and verify that<br>
> >>> mgsnode=<br>
> >>> is correct.<br>
> >>><br>
> >>> cliffw<br>
> >>><br>
> >>><br>
> >>> Thanks,<br>
> >>> CS.<br>
> >>><br>
> >>><br>
> >>><br>
> >>><br>
> >>><br>
> >>> On Tue, Jun 16, 2009 at<br>
> 12:16 PM, Cliff White<br>
> >>> <<a href="mailto:Cliff.White@sun.com" target="_blank">Cliff.White@sun.com</a><br>
> <mailto:<a href="mailto:Cliff.White@sun.com" target="_blank">Cliff.White@sun.com</a>><br>
> >>> <mailto:<a href="mailto:Cliff.White@sun.com" target="_blank">Cliff.White@sun.com</a><br>
> <mailto:<a href="mailto:Cliff.White@sun.com" target="_blank">Cliff.White@sun.com</a>>><br>
> >>> <mailto:<a href="mailto:Cliff.White@sun.com" target="_blank">Cliff.White@sun.com</a><br>
> <mailto:<a href="mailto:Cliff.White@sun.com" target="_blank">Cliff.White@sun.com</a>><br>
> >>> <mailto:<a href="mailto:Cliff.White@sun.com" target="_blank">Cliff.White@sun.com</a><br>
> <mailto:<a href="mailto:Cliff.White@sun.com" target="_blank">Cliff.White@sun.com</a>>>>><br>
> >>> wrote:<br>
> >>><br>
> >>> Carlos Santana<br>
> wrote:<br>
> >>><br>
> >>> Thanks<br>
> Kevin..<br>
> >>><br>
> >>> Please read:<br>
> >>><br>
> >>><br>
> >>><br>
> >>> <a href="http://manual.lustre.org/manual/LustreManual16_HTML/ConfiguringLustre.html#50401328_pgfId-1289529" target="_blank">http://manual.lustre.org/manual/LustreManual16_HTML/ConfiguringLustre.html#50401328_pgfId-1289529</a><br>
> >>><br>
> >>> Those instructions<br>
> are identical for 1.6 and 1.8.<br>
> >>><br>
> >>> For current lustre,<br>
> only two commands are used for<br>
> >>> configuration.<br>
> >>> mkfs.lustre and<br>
> mount.<br>
> >>><br>
> >>><br>
> >>> Usually when<br>
> lustre_rmmod returns that error, you run<br>
> >>> it a second<br>
> >>> time, and it will<br>
> clear things. Unless you have live<br>
> >>> mounts or<br>
> >>> network<br>
> connections.<br>
> >>><br>
> >>> cliffw<br>
> >>><br>
> >>><br>
> >>> I am<br>
> referring to 1.8 manual, but I was also<br>
> >>> referring to<br>
> >>> HowTo<br>
> >>> page on wiki<br>
> which seems to be for 1.6. The HowTo<br>
> >>> page<br>
> >>><br>
> >>><br>
> >>><br>
> >>> <a href="http://wiki.lustre.org/index.php/Lustre_Howto#Using_Supplied_Configuration_Tools" target="_blank">http://wiki.lustre.org/index.php/Lustre_Howto#Using_Supplied_Configuration_Tools</a><br>
> >>> mentions abt<br>
> lmc, lconf, and lctl.<br>
> >>><br>
> >>> The modules<br>
> are installed in the right place. The<br>
> >>> '$<br>
> >>> lustre_rmmod'<br>
> resulted in following o/p:<br>
> >>> <br>
> [root@localhost<br>
> >>> 2.6.18-92.1.17.el5_lustre.1.8.0smp]#<br>
> >>> lustre_rmmod<br>
> >>> ERROR: Module<br>
> obdfilter is in use<br>
> >>> ERROR: Module<br>
> ost is in use<br>
> >>> ERROR: Module<br>
> mds is in use<br>
> >>> ERROR: Module<br>
> fsfilt_ldiskfs is in use<br>
> >>> ERROR: Module<br>
> mgs is in use<br>
> >>> ERROR: Module<br>
> mgc is in use by mgs<br>
> >>> ERROR: Module<br>
> ldiskfs is in use by fsfilt_ldiskfs<br>
> >>> ERROR: Module<br>
> lov is in use<br>
> >>> ERROR: Module<br>
> lquota is in use by obdfilter,mds<br>
> >>> ERROR: Module<br>
> osc is in use<br>
> >>> ERROR: Module<br>
> ksocklnd is in use<br>
> >>> ERROR: Module<br>
> ptlrpc is in use by<br>
> >>> <br>
> obdfilter,ost,mds,mgs,mgc,lov,lquota,osc<br>
> >>> ERROR: Module<br>
> obdclass is in use by<br>
> >>><br>
> >>> <br>
> obdfilter,ost,mds,fsfilt_ldiskfs,mgs,mgc,lov,lquota,osc,ptlrpc<br>
> >>> ERROR: Module<br>
> lnet is in use by<br>
> >>> ksocklnd,ptlrpc,obdclass<br>
> >>> ERROR: Module<br>
> lvfs is in use by<br>
> >>><br>
> >>><br>
> >>><br>
> obdfilter,ost,mds,fsfilt_ldiskfs,mgs,mgc,lov,lquota,osc,ptlrpc,obdclass<br>
> >>> ERROR: Module<br>
> libcfs is in use by<br>
> >>><br>
> >>><br>
> >>><br>
> >>><br>
> obdfilter,ost,mds,fsfilt_ldiskfs,mgs,mgc,lov,lquota,osc,ksocklnd,ptlrpc,obdclass,lnet,lvfs<br>
> >>><br>
> >>> Do I need to<br>
> shutdown these services? How can I do<br>
> >>> that?<br>
> >>><br>
> >>> Thanks,<br>
> >>> CS.<br>
> >>><br>
> _______________________________________________<br>
> Lustre-discuss mailing list<br>
> <a href="mailto:Lustre-discuss@lists.lustre.org" target="_blank">Lustre-discuss@lists.lustre.org</a><br>
> <a href="http://lists.lustre.org/mailman/listinfo/lustre-discuss" target="_blank">http://lists.lustre.org/mailman/listinfo/lustre-discuss</a><br>
><br>
<br>
<br>
<br>
</div></div></blockquote></div><br>