[Lustre-discuss] Lustre installation and configuration problems
Sheila Barthel
Sheila.Barthel at Sun.COM
Wed Jun 17 13:08:36 PDT 2009
Carlos -
The installation procedures for Lustre 1.6 and 1.8 are the same. The
manual's installation procedure includes a table that shows which
packages to install on servers and clients (I've attached a PDF of the
table). The procedure also describes the installation order for packages
(kernel, modules, ldiskfs, then utilities/userspace, then e2fsprogs).
http://manual.lustre.org/manual/LustreManual16_HTML/LustreInstallation.html#50401389_pgfId-1291574
Sheila
Cliff White wrote:
> Carlos Santana wrote:
>
>> Huh... :( Sorry to bug you guys again...
>>
>> I am planning to make a fresh start now as nothing seems to have worked
>> for me. If you have any comments/feedback please share them.
>>
>> I would like to confirm installation order before I make a fresh start.
>> From Arden's experience:
>> http://lists.lustre.org/pipermail/lustre-discuss/2009-June/010710.html ,
>> the lusre-module is installed last. As I was installing Lustre 1.8, I
>> was referring 1.8 operations manual
>> http://manual.lustre.org/index.php?title=Main_Page . The installation
>> order in the manual is different than what Arden has suggested.
>>
>> Will it make a difference in configuration at later stage? Which one
>> should I follow now?
>> Any comments?
>>
>
> RPM installation order really doesn't matter. If you install in the
> 'wrong' order you will get a lot of warnings from RPM due to the
> relationship of the various RPMs. But these are harmless - whatever
> order you install in, it should work fine.
> cliffw
>
>> Thanks,
>> CS.
>>
>>
>> On Wed, Jun 17, 2009 at 12:35 AM, Carlos Santana <neubyr at gmail.com
>> <mailto:neubyr at gmail.com>> wrote:
>>
>> Thanks Cliff.
>>
>> The depmod -a was successful before as well. I am using CentOS 5.2
>> box. Following are the packages installed:
>> [root at localhost tmp]# rpm -qa | grep -i lustre
>> lustre-modules-1.8.0-2.6.18_92.1.17.el5_lustre.1.8.0smp
>> lustre-1.8.0-2.6.18_92.1.17.el5_lustre.1.8.0smp
>>
>> [root at localhost tmp]# uname -a
>> Linux localhost.localdomain 2.6.18-92.el5 #1 SMP Tue Jun 10 18:49:47
>> EDT 2008 i686 i686 i386 GNU/Linux
>>
>> And here is a output from strace for mount:
>> http://www.heypasteit.com/clip/8WT
>>
>> Any further debugging hints?
>>
>> Thanks,
>> CS.
>>
>> On 6/16/09, Cliff White <Cliff.White at sun.com
>> <mailto:Cliff.White at sun.com>> wrote:
>> > Carlos Santana wrote:
>> >> The '$ modprobe -l lustre*' did not show any module on a patchless
>> >> client. modprobe -v returns 'FATAL: Module lustre not found'.
>> >>
>> >> How do I install a patchless client?
>> >> I have tried lustre-client-modules and lustre-client-ver rpm
>> packages in
>> >> both sequences. Am I missing anything?
>> >>
>> >
>> > Make sure the lustre-client-modules package matches your running
>> kernel.
>> > Run depmod -a to be sure
>> > cliffw
>> >
>> >> Thanks,
>> >> CS.
>> >>
>> >>
>> >>
>> >> On Tue, Jun 16, 2009 at 2:28 PM, Cliff White
>> <Cliff.White at sun.com <mailto:Cliff.White at sun.com>
>> >> <mailto:Cliff.White at sun.com <mailto:Cliff.White at sun.com>>> wrote:
>> >>
>> >> Carlos Santana wrote:
>> >>
>> >> The lctlt ping and 'net up' failed with the following
>> messages:
>> >> --- ---
>> >> [root at localhost ~]# lctl ping 10.0.0.42
>> >> opening /dev/lnet failed: No such device
>> >> hint: the kernel modules may not be loaded
>> >> failed to ping 10.0.0.42 at tcp: No such device
>> >>
>> >> [root at localhost ~]# lctl network up
>> >> opening /dev/lnet failed: No such device
>> >> hint: the kernel modules may not be loaded
>> >> LNET configure error 19: No such device
>> >>
>> >>
>> >> Make sure modules are unloaded, then try modprobe -v.
>> >> Looks like you have lnet mis-configured, if your module
>> options are
>> >> wrong, you will see an error during the modprobe.
>> >> cliffw
>> >>
>> >> --- ---
>> >>
>> >>
>> >> I tried lustre_rmmod and depmod commands and it did not
>> return
>> >> any error messages. Any further clues? Reinstall patchless
>> >> client again?
>> >>
>> >> -
>> >> CS.
>> >>
>> >>
>> >> On Tue, Jun 16, 2009 at 1:32 PM, Cliff White
>> >> <Cliff.White at sun.com <mailto:Cliff.White at sun.com>
>> <mailto:Cliff.White at sun.com <mailto:Cliff.White at sun.com>>
>> >> <mailto:Cliff.White at sun.com <mailto:Cliff.White at sun.com>
>> <mailto:Cliff.White at sun.com <mailto:Cliff.White at sun.com>>>> wrote:
>> >>
>> >> Carlos Santana wrote:
>> >>
>> >> I was able to run lustre_rmmod and depmod
>> successfully. The
>> >> '$lctl list_nids' returned the server ip address and
>> >> interface
>> >> (tcp0).
>> >>
>> >> I tried to mount the file system on a remote
>> client, but it
>> >> failed with the following message.
>> >> --- ---
>> >> [root at localhost ~]# mount -t lustre
>> 10.0.0.42 at tcp0:/lustre
>> >> /mnt/lustre
>> >> mount.lustre: mount 10.0.0.42 at tcp0:/lustre at
>> /mnt/lustre
>> >> failed: No such device
>> >> Are the lustre modules loaded?
>> >> Check /etc/modprobe.conf and /proc/filesystems
>> >> Note 'alias lustre llite' should be removed from
>> >> modprobe.conf
>> >> --- ---
>> >>
>> >> However, the mounting is successful on a single node
>> >> configuration - with client on the same machine
>> as MDS
>> >> and OST.
>> >> Any clues? Where to look for logs and debug messages?
>> >>
>> >>
>> >> Syslog || /var/log/messages is the normal place.
>> >>
>> >> You can use 'lctl ping' to verify that the client can
>> reach
>> >> the server.
>> >> Usually in these cases, it's a network/name
>> misconfiguration.
>> >>
>> >> Run 'tunefs.lustre --print' on your servers, and
>> verify that
>> >> mgsnode=
>> >> is correct.
>> >>
>> >> cliffw
>> >>
>> >>
>> >> Thanks,
>> >> CS.
>> >>
>> >>
>> >>
>> >>
>> >>
>> >> On Tue, Jun 16, 2009 at 12:16 PM, Cliff White
>> >> <Cliff.White at sun.com <mailto:Cliff.White at sun.com>
>> <mailto:Cliff.White at sun.com <mailto:Cliff.White at sun.com>>
>> >> <mailto:Cliff.White at sun.com <mailto:Cliff.White at sun.com>
>> <mailto:Cliff.White at sun.com <mailto:Cliff.White at sun.com>>>
>> >> <mailto:Cliff.White at sun.com
>> <mailto:Cliff.White at sun.com> <mailto:Cliff.White at sun.com
>> <mailto:Cliff.White at sun.com>>
>> >> <mailto:Cliff.White at sun.com <mailto:Cliff.White at sun.com>
>> <mailto:Cliff.White at sun.com <mailto:Cliff.White at sun.com>>>>> wrote:
>> >>
>> >> Carlos Santana wrote:
>> >>
>> >> Thanks Kevin..
>> >>
>> >> Please read:
>> >>
>> >>
>> >>
>> http://manual.lustre.org/manual/LustreManual16_HTML/ConfiguringLustre.html#50401328_pgfId-1289529
>> >>
>> >> Those instructions are identical for 1.6 and 1.8.
>> >>
>> >> For current lustre, only two commands are used for
>> >> configuration.
>> >> mkfs.lustre and mount.
>> >>
>> >>
>> >> Usually when lustre_rmmod returns that error,
>> you run
>> >> it a second
>> >> time, and it will clear things. Unless you
>> have live
>> >> mounts or
>> >> network connections.
>> >>
>> >> cliffw
>> >>
>> >>
>> >> I am referring to 1.8 manual, but I was also
>> >> referring to
>> >> HowTo
>> >> page on wiki which seems to be for 1.6.
>> The HowTo
>> >> page
>> >>
>> >>
>> >>
>> http://wiki.lustre.org/index.php/Lustre_Howto#Using_Supplied_Configuration_Tools
>> >> mentions abt lmc, lconf, and lctl.
>> >>
>> >> The modules are installed in the right
>> place. The '$
>> >> lustre_rmmod' resulted in following o/p:
>> >> [root at localhost
>> 2.6.18-92.1.17.el5_lustre.1.8.0smp]#
>> >> lustre_rmmod
>> >> ERROR: Module obdfilter is in use
>> >> ERROR: Module ost is in use
>> >> ERROR: Module mds is in use
>> >> ERROR: Module fsfilt_ldiskfs is in use
>> >> ERROR: Module mgs is in use
>> >> ERROR: Module mgc is in use by mgs
>> >> ERROR: Module ldiskfs is in use by
>> fsfilt_ldiskfs
>> >> ERROR: Module lov is in use
>> >> ERROR: Module lquota is in use by
>> obdfilter,mds
>> >> ERROR: Module osc is in use
>> >> ERROR: Module ksocklnd is in use
>> >> ERROR: Module ptlrpc is in use by
>> >> obdfilter,ost,mds,mgs,mgc,lov,lquota,osc
>> >> ERROR: Module obdclass is in use by
>> >>
>> >>
>> obdfilter,ost,mds,fsfilt_ldiskfs,mgs,mgc,lov,lquota,osc,ptlrpc
>> >> ERROR: Module lnet is in use by
>> >> ksocklnd,ptlrpc,obdclass
>> >> ERROR: Module lvfs is in use by
>> >>
>> >>
>> >>
>> obdfilter,ost,mds,fsfilt_ldiskfs,mgs,mgc,lov,lquota,osc,ptlrpc,obdclass
>> >> ERROR: Module libcfs is in use by
>> >>
>> >>
>> >>
>> obdfilter,ost,mds,fsfilt_ldiskfs,mgs,mgc,lov,lquota,osc,ksocklnd,ptlrpc,obdclass,lnet,lvfs
>> >>
>> >> Do I need to shutdown these services? How
>> can I do
>> >> that?
>> >>
>> >> Thanks,
>> >> CS.
>> >>
>> >>
>> >> On Tue, Jun 16, 2009 at 11:36 AM, Kevin
>> Van Maren
>> >> <Kevin.Vanmaren at sun.com
>> <mailto:Kevin.Vanmaren at sun.com>
>> >> <mailto:Kevin.Vanmaren at sun.com
>> <mailto:Kevin.Vanmaren at sun.com>> <mailto:Kevin.Vanmaren at sun.com
>> <mailto:Kevin.Vanmaren at sun.com>
>> >> <mailto:Kevin.Vanmaren at sun.com
>> <mailto:Kevin.Vanmaren at sun.com>>>
>> >> <mailto:Kevin.Vanmaren at sun.com
>> <mailto:Kevin.Vanmaren at sun.com>
>> >> <mailto:Kevin.Vanmaren at sun.com
>> <mailto:Kevin.Vanmaren at sun.com>> <mailto:Kevin.Vanmaren at sun.com
>> <mailto:Kevin.Vanmaren at sun.com>
>> >> <mailto:Kevin.Vanmaren at sun.com
>> <mailto:Kevin.Vanmaren at sun.com>>>>
>> >> <mailto:Kevin.Vanmaren at sun.com
>> <mailto:Kevin.Vanmaren at sun.com>
>> >> <mailto:Kevin.Vanmaren at sun.com
>> <mailto:Kevin.Vanmaren at sun.com>>
>> >> <mailto:Kevin.Vanmaren at sun.com
>> <mailto:Kevin.Vanmaren at sun.com>
>> >> <mailto:Kevin.Vanmaren at sun.com
>> <mailto:Kevin.Vanmaren at sun.com>>> <mailto:Kevin.Vanmaren at sun.com
>> <mailto:Kevin.Vanmaren at sun.com>
>> >> <mailto:Kevin.Vanmaren at sun.com
>> <mailto:Kevin.Vanmaren at sun.com>>
>> >> <mailto:Kevin.Vanmaren at sun.com
>> <mailto:Kevin.Vanmaren at sun.com>
>> >> <mailto:Kevin.Vanmaren at sun.com
>> <mailto:Kevin.Vanmaren at sun.com>>>>>>
>> >>
>> >> wrote:
>> >>
>> >> I think lconf and lmc went away with Lustre
>> >> 1.6. Are you
>> >> sure you
>> >> are looking at the 1.8 manual, and not
>> >> directions for 1.4?
>> >>
>> >> /usr/sbin/lctl should be in the
>> >> lustre-<version> RPM.
>> >> Do a:
>> >> # rpm -q -l
>> >> lustre-1.8.0-2.6.18_92.1.17.el5_lustre.1.8.0smp
>> >>
>> >>
>> >> Do make sure the modules are installed
>> in the
>> >> right place:
>> >> # cd /lib/modules/`uname -r`
>> >> # find . | grep lustre.ko
>> >>
>> >> If it shows up, then do:
>> >> # lustre_rmmod
>> >> # depmod
>> >> and try again.
>> >>
>> >> Otherwise, figure out where your
>> modules are
>> >> installed:
>> >> # uname -r
>> >> # cd /lib/modules
>> >> # find . | grep lustre.ko
>> >>
>> >>
>> >> You can also double-check the NID. On
>> the MSD
>> >> server, do
>> >> # lctl list_nids
>> >>
>> >> Should show 10.0.0.42 at tcp0
>> >>
>> >> Kevin
>> >>
>> >>
>> >>
>> >>
>> >>
>> >>
>> ------------------------------------------------------------------------
>> >>
>> >> _______________________________________________
>> >> Lustre-discuss mailing list
>> >> Lustre-discuss at lists.lustre.org
>> <mailto:Lustre-discuss at lists.lustre.org>
>> >> <mailto:Lustre-discuss at lists.lustre.org
>> <mailto:Lustre-discuss at lists.lustre.org>>
>> >> <mailto:Lustre-discuss at lists.lustre.org
>> <mailto:Lustre-discuss at lists.lustre.org>
>> >> <mailto:Lustre-discuss at lists.lustre.org
>> <mailto:Lustre-discuss at lists.lustre.org>>>
>> >>
>> >>
>> http://lists.lustre.org/mailman/listinfo/lustre-discuss
>> >>
>> >>
>> >>
>> >>
>> >>
>> >>
>> ------------------------------------------------------------------------
>> >>
>> >> _______________________________________________
>> >> Lustre-discuss mailing list
>> >> Lustre-discuss at lists.lustre.org
>> <mailto:Lustre-discuss at lists.lustre.org>
>> >> <mailto:Lustre-discuss at lists.lustre.org
>> <mailto:Lustre-discuss at lists.lustre.org>>
>> >> http://lists.lustre.org/mailman/listinfo/lustre-discuss
>> >>
>> >>
>> >>
>> >>
>> >>
>> ------------------------------------------------------------------------
>> >>
>> >> _______________________________________________
>> >> Lustre-discuss mailing list
>> >> Lustre-discuss at lists.lustre.org
>> <mailto:Lustre-discuss at lists.lustre.org>
>> >> http://lists.lustre.org/mailman/listinfo/lustre-discuss
>> >
>> >
>>
>>
>>
>> ------------------------------------------------------------------------
>>
>> _______________________________________________
>> Lustre-discuss mailing list
>> Lustre-discuss at lists.lustre.org
>> http://lists.lustre.org/mailman/listinfo/lustre-discuss
>>
>
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-discuss
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: LustreInstallTable.pdf
Type: application/pdf
Size: 24227 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20090617/cddaa210/attachment.pdf>
More information about the lustre-discuss
mailing list