[Lustre-discuss] Lustre installation and configuration problems
Arden Wiebe
albert682 at yahoo.com
Wed Jun 17 13:21:31 PDT 2009
Cliff:
I have some questions about the client packages. I am not sure why the roadmap or lustre users require separate client packages but stating the obvious some people must need separate client packages is that correct?
Otherwise the server packages contain the client anyhow correct? If the later are the client packages for linux somewhat redundant? When will the real client .exe for windows become available?
Arden
--- On Wed, 6/17/09, Sheila Barthel <Sheila.Barthel at Sun.COM> wrote:
> From: Sheila Barthel <Sheila.Barthel at Sun.COM>
> Subject: Re: [Lustre-discuss] Lustre installation and configuration problems
> To: "Carlos Santana" <neubyr at gmail.com>
> Cc: "Cliff White" <Cliff.White at Sun.COM>, lustre-discuss at lists.lustre.org
> Date: Wednesday, June 17, 2009, 1:08 PM
> Carlos -
>
> The installation procedures for Lustre 1.6 and 1.8 are the
> same. The manual's installation procedure includes a table
> that shows which packages to install on servers and clients
> (I've attached a PDF of the table). The procedure also
> describes the installation order for packages (kernel,
> modules, ldiskfs, then utilities/userspace, then
> e2fsprogs).
>
> http://manual.lustre.org/manual/LustreManual16_HTML/LustreInstallation.html#50401389_pgfId-1291574
>
> Sheila
>
> Cliff White wrote:
> > Carlos Santana wrote:
> >
> >> Huh... :( Sorry to bug you guys again...
> >>
> >> I am planning to make a fresh start now as nothing
> seems to have worked for me. If you have any
> comments/feedback please share them.
> >>
> >> I would like to confirm installation order before
> I make a fresh start. From Arden's experience: http://lists.lustre.org/pipermail/lustre-discuss/2009-June/010710.html
> , the lusre-module is installed last. As I was installing
> Lustre 1.8, I was referring 1.8 operations manual http://manual.lustre.org/index.php?title=Main_Page .
> The installation order in the manual is different than what
> Arden has suggested.
> >>
> >> Will it make a difference in configuration at
> later stage? Which one should I follow now?
> >> Any comments?
> >>
> >
> > RPM installation order really doesn't matter. If you
> install in the 'wrong' order you will get a lot of warnings
> from RPM due to the relationship of the various RPMs. But
> these are harmless - whatever order you install in, it
> should work fine.
> > cliffw
> >
> >> Thanks,
> >> CS.
> >>
> >>
> >> On Wed, Jun 17, 2009 at 12:35 AM, Carlos Santana
> <neubyr at gmail.com
> <mailto:neubyr at gmail.com>>
> wrote:
> >>
> >> Thanks Cliff.
> >>
> >> The depmod -a was
> successful before as well. I am using CentOS 5.2
> >> box. Following are the
> packages installed:
> >> [root at localhost tmp]# rpm
> -qa | grep -i lustre
> >>
> lustre-modules-1.8.0-2.6.18_92.1.17.el5_lustre.1.8.0smp
> >>
> lustre-1.8.0-2.6.18_92.1.17.el5_lustre.1.8.0smp
> >>
> >> [root at localhost tmp]#
> uname -a
> >> Linux
> localhost.localdomain 2.6.18-92.el5 #1 SMP Tue Jun 10
> 18:49:47
> >> EDT 2008 i686 i686 i386
> GNU/Linux
> >>
> >> And here is a output from
> strace for mount:
> >> http://www.heypasteit.com/clip/8WT
> >>
> >> Any further debugging
> hints?
> >>
> >> Thanks,
> >> CS.
> >>
> >> On 6/16/09, Cliff White
> <Cliff.White at sun.com
> >> <mailto:Cliff.White at sun.com>>
> wrote:
> >> > Carlos Santana wrote:
> >> >> The '$ modprobe -l
> lustre*' did not show any module on a patchless
> >> >> client. modprobe -v
> returns 'FATAL: Module lustre not found'.
> >> >>
> >> >> How do I install a
> patchless client?
> >> >> I have tried
> lustre-client-modules and lustre-client-ver rpm
> >> packages in
> >> >> both sequences. Am I
> missing anything?
> >> >>
> >> >
> >> > Make sure the
> lustre-client-modules package matches your running
> >> kernel.
> >> > Run depmod -a to be sure
> >> > cliffw
> >> >
> >> >> Thanks,
> >> >> CS.
> >> >>
> >> >>
> >> >>
> >> >> On Tue, Jun 16, 2009
> at 2:28 PM, Cliff White
> >> <Cliff.White at sun.com
> <mailto:Cliff.White at sun.com>
> >> >> <mailto:Cliff.White at sun.com
> <mailto:Cliff.White at sun.com>>>
> wrote:
> >> >>
> >> >>
> Carlos Santana wrote:
> >> >>
> >> >>
> The lctlt ping and 'net up' failed with
> the following
> >> messages:
> >> >>
> --- ---
> >> >>
> [root at localhost ~]# lctl ping 10.0.0.42
> >> >>
> opening /dev/lnet failed: No such device
> >> >>
> hint: the kernel modules may not be
> loaded
> >> >>
> failed to ping 10.0.0.42 at tcp: No such
> device
> >> >>
> >> >>
> [root at localhost ~]# lctl network up
> >> >>
> opening /dev/lnet failed: No such device
> >> >>
> hint: the kernel modules may not be
> loaded
> >> >>
> LNET configure error 19: No such device
> >> >>
> >> >>
> >> >>
> Make sure modules are unloaded, then try
> modprobe -v.
> >> >>
> Looks like you have lnet mis-configured,
> if your module
> >> options are
> >> >>
> wrong, you will see an error during the
> modprobe.
> >> >>
> cliffw
> >> >>
> >> >>
> --- ---
> >> >>
> >> >>
> >> >>
> I tried lustre_rmmod and depmod commands
> and it did not
> >> return
> >> >>
> any error messages. Any further clues?
> Reinstall patchless
> >> >>
> client again?
> >> >>
> >> >>
> -
> >> >>
> CS.
> >> >>
> >> >>
> >> >>
> On Tue, Jun 16, 2009 at 1:32 PM, Cliff
> White
> >> >>
> <Cliff.White at sun.com
> <mailto:Cliff.White at sun.com>
> >> <mailto:Cliff.White at sun.com
> <mailto:Cliff.White at sun.com>>
> >> >>
> <mailto:Cliff.White at sun.com
> <mailto:Cliff.White at sun.com>
> >> <mailto:Cliff.White at sun.com
> <mailto:Cliff.White at sun.com>>>>
> wrote:
> >> >>
> >> >>
> Carlos Santana wrote:
> >> >>
> >> >>
> I was able to run
> lustre_rmmod and depmod
> >> successfully. The
> >> >>
> '$lctl list_nids'
> returned the server ip address and
> >> >>
> interface
> >> >>
> (tcp0).
> >> >>
> >> >>
> I tried to mount the file
> system on a remote
> >> client, but it
> >> >>
> failed with the following
> message.
> >> >>
> --- ---
> >> >>
> [root at localhost ~]# mount
> -t lustre
> >> 10.0.0.42 at tcp0:/lustre
> >> >>
> /mnt/lustre
> >> >>
> mount.lustre: mount
> 10.0.0.42 at tcp0:/lustre at
> >> /mnt/lustre
> >> >>
> failed: No such device
> >> >>
> Are the lustre modules
> loaded?
> >> >>
> Check /etc/modprobe.conf
> and /proc/filesystems
> >> >>
> Note 'alias lustre llite'
> should be removed from
> >> >>
> modprobe.conf
> >> >>
> --- ---
> >> >>
> >> >>
> However, the mounting is
> successful on a single node
> >> >>
> configuration - with
> client on the same machine
> >> as MDS
> >> >>
> and OST.
> >> >>
> Any clues? Where to look
> for logs and debug messages?
> >> >>
> >> >>
> >> >>
> Syslog || /var/log/messages is the
> normal place.
> >> >>
> >> >>
> You can use 'lctl ping' to verify that
> the client can
> >> reach
> >> >>
> the server.
> >> >>
> Usually in these cases, it's a
> network/name
> >> misconfiguration.
> >> >>
> >> >>
> Run 'tunefs.lustre --print' on your
> servers, and
> >> verify that
> >> >>
> mgsnode=
> >> >>
> is correct.
> >> >>
> >> >>
> cliffw
> >> >>
> >> >>
> >> >>
> Thanks,
> >> >>
> CS.
> >> >>
> >> >>
> >> >>
> >> >>
> >> >>
> >> >>
> On Tue, Jun 16, 2009 at
> 12:16 PM, Cliff White
> >> >>
> <Cliff.White at sun.com
> <mailto:Cliff.White at sun.com>
> >> <mailto:Cliff.White at sun.com
> <mailto:Cliff.White at sun.com>>
> >> >>
> <mailto:Cliff.White at sun.com
> <mailto:Cliff.White at sun.com>
> >> <mailto:Cliff.White at sun.com
> <mailto:Cliff.White at sun.com>>>
> >> >>
> <mailto:Cliff.White at sun.com
> >> <mailto:Cliff.White at sun.com>
> <mailto:Cliff.White at sun.com
> >> <mailto:Cliff.White at sun.com>>
> >> >>
> <mailto:Cliff.White at sun.com
> <mailto:Cliff.White at sun.com>
> >> <mailto:Cliff.White at sun.com
> <mailto:Cliff.White at sun.com>>>>>
> wrote:
> >> >>
> >> >>
> Carlos
> Santana wrote:
> >> >>
> >> >>
>
> Thanks Kevin..
> >> >>
> >> >>
> Please
> read:
> >> >>
> >> >>
> >> >>
> >> http://manual.lustre.org/manual/LustreManual16_HTML/ConfiguringLustre.html#50401328_pgfId-1289529
> >> >>
> >> >>
> Those
> instructions are identical for 1.6 and 1.8.
> >> >>
> >> >>
> For
> current lustre, only two commands are used for
> >> >>
> configuration.
> >> >>
>
> mkfs.lustre and mount.
> >> >>
> >> >>
> >> >>
> Usually
> when lustre_rmmod returns that error,
> >> you run
> >> >>
> it a second
> >> >>
> time,
> and it will clear things. Unless you
> >> have live
> >> >>
> mounts or
> >> >>
> network
> connections.
> >> >>
> >> >>
> cliffw
> >> >>
> >> >>
> >> >>
>
> I am referring to 1.8 manual, but I was
> also
> >> >>
> referring to
> >> >>
> HowTo
> >> >>
>
> page on wiki which seems to be for 1.6.
> >> The HowTo
> >> >> page
> >> >>
> >> >>
> >> >>
> >> http://wiki.lustre.org/index.php/Lustre_Howto#Using_Supplied_Configuration_Tools
> >> >>
>
> mentions abt lmc, lconf, and lctl.
> >> >>
> >> >>
>
> The modules are installed in the right
> >> place. The '$
> >> >>
>
> lustre_rmmod' resulted in following o/p:
> >> >>
>
> [root at localhost
> >>
> 2.6.18-92.1.17.el5_lustre.1.8.0smp]#
> >> >>
> lustre_rmmod
> >> >>
>
> ERROR: Module obdfilter is in use
> >> >>
>
> ERROR: Module ost is in use
> >> >>
>
> ERROR: Module mds is in use
> >> >>
>
> ERROR: Module fsfilt_ldiskfs is in use
> >> >>
>
> ERROR: Module mgs is in use
> >> >>
>
> ERROR: Module mgc is in use by mgs
> >> >>
>
> ERROR: Module ldiskfs is in use by
> >> fsfilt_ldiskfs
> >> >>
>
> ERROR: Module lov is in use
> >> >>
>
> ERROR: Module lquota is in use by
> >> obdfilter,mds
> >> >>
>
> ERROR: Module osc is in use
> >> >>
>
> ERROR: Module ksocklnd is in use
> >> >>
>
> ERROR: Module ptlrpc is in use by
> >> >>
>
> obdfilter,ost,mds,mgs,mgc,lov,lquota,osc
> >> >>
>
> ERROR: Module obdclass is in use by
> >> >>
> >> >>
>
> obdfilter,ost,mds,fsfilt_ldiskfs,mgs,mgc,lov,lquota,osc,ptlrpc
> >> >>
>
> ERROR: Module lnet is in use by
> >> >>
> ksocklnd,ptlrpc,obdclass
> >> >>
>
> ERROR: Module lvfs is in use by
> >> >>
> >> >>
> >> >>
> >>
> obdfilter,ost,mds,fsfilt_ldiskfs,mgs,mgc,lov,lquota,osc,ptlrpc,obdclass
> >> >>
>
> ERROR: Module libcfs is in use by
> >> >>
> >> >>
> >> >>
> >>
> obdfilter,ost,mds,fsfilt_ldiskfs,mgs,mgc,lov,lquota,osc,ksocklnd,ptlrpc,obdclass,lnet,lvfs
> >> >>
> >> >>
>
> Do I need to shutdown these services? How
> >> can I do
> >> >>
> that?
> >> >>
> >> >>
>
> Thanks,
> >> >>
>
> CS.
> >> >>
> >> >>
> >> >>
>
> On Tue, Jun 16, 2009 at 11:36 AM, Kevin
> >> Van Maren
> >> >>
>
> <Kevin.Vanmaren at sun.com
> >> <mailto:Kevin.Vanmaren at sun.com>
> >> >>
> <mailto:Kevin.Vanmaren at sun.com
> >> <mailto:Kevin.Vanmaren at sun.com>>
> <mailto:Kevin.Vanmaren at sun.com
> >> <mailto:Kevin.Vanmaren at sun.com>
> >> >>
> <mailto:Kevin.Vanmaren at sun.com
> >> <mailto:Kevin.Vanmaren at sun.com>>>
> >> >>
> <mailto:Kevin.Vanmaren at sun.com
> >> <mailto:Kevin.Vanmaren at sun.com>
> >> >>
> <mailto:Kevin.Vanmaren at sun.com
> >> <mailto:Kevin.Vanmaren at sun.com>>
> <mailto:Kevin.Vanmaren at sun.com
> >> <mailto:Kevin.Vanmaren at sun.com>
> >> >>
> <mailto:Kevin.Vanmaren at sun.com
> >> <mailto:Kevin.Vanmaren at sun.com>>>>
> >> >>
>
> <mailto:Kevin.Vanmaren at sun.com
> >> <mailto:Kevin.Vanmaren at sun.com>
> >> >>
> <mailto:Kevin.Vanmaren at sun.com
> >> <mailto:Kevin.Vanmaren at sun.com>>
> >> >>
> <mailto:Kevin.Vanmaren at sun.com
> >> <mailto:Kevin.Vanmaren at sun.com>
> >> >>
> <mailto:Kevin.Vanmaren at sun.com
> >> <mailto:Kevin.Vanmaren at sun.com>>>
> <mailto:Kevin.Vanmaren at sun.com
> >> <mailto:Kevin.Vanmaren at sun.com>
> >> >>
> <mailto:Kevin.Vanmaren at sun.com
> >> <mailto:Kevin.Vanmaren at sun.com>>
> >> >>
> <mailto:Kevin.Vanmaren at sun.com
> >> <mailto:Kevin.Vanmaren at sun.com>
> >> >>
> <mailto:Kevin.Vanmaren at sun.com
> >> <mailto:Kevin.Vanmaren at sun.com>>>>>>
> >> >>
> >> >>
>
> wrote:
> >> >>
> >> >>
>
> I think lconf and lmc went away with Lustre
> >> >>
> 1.6. Are you
> >> >>
>
> sure you
> >> >>
>
> are looking at the 1.8 manual, and not
> >> >>
> directions for 1.4?
> >> >>
> >> >>
>
> /usr/sbin/lctl should be in the
> >> >>
> lustre-<version> RPM.
> >> >>
> Do a:
> >> >>
>
> # rpm -q -l
> >> >>
>
> lustre-1.8.0-2.6.18_92.1.17.el5_lustre.1.8.0smp
> >> >>
> >> >>
> >> >>
>
> Do make sure the modules are installed
> >> in the
> >> >>
> right place:
> >> >>
>
> # cd /lib/modules/`uname -r`
> >> >>
>
> # find . | grep lustre.ko
> >> >>
> >> >>
>
> If it shows up, then do:
> >> >>
>
> # lustre_rmmod
> >> >>
>
> # depmod
> >> >>
>
> and try again.
> >> >>
> >> >>
>
> Otherwise, figure out where your
> >> modules are
> >> >>
> installed:
> >> >>
>
> # uname -r
> >> >>
>
> # cd /lib/modules
> >> >>
>
> # find . | grep lustre.ko
> >> >>
> >> >>
> >> >>
>
> You can also double-check the NID. On
> >> the MSD
> >> >>
> server, do
> >> >>
>
> # lctl list_nids
> >> >>
> >> >>
>
> Should show 10.0.0.42 at tcp0
> >> >>
> >> >>
>
> Kevin
> >> >>
> >> >>
> >> >>
> >> >>
> >> >>
> >> >>
> >>
> ------------------------------------------------------------------------
> >> >>
> >> >>
>
> _______________________________________________
> >> >>
> Lustre-discuss mailing
> list
> >> >>
> Lustre-discuss at lists.lustre.org
> >> <mailto:Lustre-discuss at lists.lustre.org>
> >> >>
> <mailto:Lustre-discuss at lists.lustre.org
> >> <mailto:Lustre-discuss at lists.lustre.org>>
> >> >>
> <mailto:Lustre-discuss at lists.lustre.org
> >> <mailto:Lustre-discuss at lists.lustre.org>
> >> >>
> <mailto:Lustre-discuss at lists.lustre.org
> >> <mailto:Lustre-discuss at lists.lustre.org>>>
> >> >>
> >> >>
> http://lists.lustre.org/mailman/listinfo/lustre-discuss
> >> >>
> >> >>
> >> >>
> >> >>
> >> >>
> >> >>
> >>
> ------------------------------------------------------------------------
> >> >>
> >> >>
> _______________________________________________
> >> >>
> Lustre-discuss mailing list
> >> >>
> Lustre-discuss at lists.lustre.org
> >> <mailto:Lustre-discuss at lists.lustre.org>
> >> >>
> <mailto:Lustre-discuss at lists.lustre.org
> >> <mailto:Lustre-discuss at lists.lustre.org>>
> >> >>
> http://lists.lustre.org/mailman/listinfo/lustre-discuss
> >> >>
> >> >>
> >> >>
> >> >>
> >> >>
> >>
> ------------------------------------------------------------------------
> >> >>
> >> >>
> _______________________________________________
> >> >> Lustre-discuss
> mailing list
> >> >> Lustre-discuss at lists.lustre.org
> >> <mailto:Lustre-discuss at lists.lustre.org>
> >> >> http://lists.lustre.org/mailman/listinfo/lustre-discuss
> >> >
> >> >
> >>
> >>
> >>
> >>
> ------------------------------------------------------------------------
> >>
> >> _______________________________________________
> >> Lustre-discuss mailing list
> >> Lustre-discuss at lists.lustre.org
> >> http://lists.lustre.org/mailman/listinfo/lustre-discuss
> >>
> >
> > _______________________________________________
> > Lustre-discuss mailing list
> > Lustre-discuss at lists.lustre.org
> > http://lists.lustre.org/mailman/listinfo/lustre-discuss
> >
>
>
> -----Inline Attachment Follows-----
>
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-discuss
>
More information about the lustre-discuss
mailing list