[Lustre-discuss] Lustre installation and configuration problems
Cliff White
Cliff.White at Sun.COM
Wed Jun 17 12:31:00 PDT 2009
Carlos Santana wrote:
> Thanks Cliff.
>
> The depmod -a was successful before as well. I am using CentOS 5.2
> box. Following are the packages installed:
> [root at localhost tmp]# rpm -qa | grep -i lustre
> lustre-modules-1.8.0-2.6.18_92.1.17.el5_lustre.1.8.0smp
> lustre-1.8.0-2.6.18_92.1.17.el5_lustre.1.8.0smp
Those are server modules. You would need to add lustre-kernel-smp for
that to work
For a client, you install the matching vendor kernel, then:
lustre-client-modules
lustre-client
For a server, you need
lustre-kernel-smp
lustre-modules
lustre-
ldiskfs-
And as others have mentioned in this thread, kernel version must match
exactly. Check /lib/modules - if you have a mis-match, there will be an
extra directory there.
cliffw
>
> [root at localhost tmp]# uname -a
> Linux localhost.localdomain 2.6.18-92.el5 #1 SMP Tue Jun 10 18:49:47
> EDT 2008 i686 i686 i386 GNU/Linux
>
> And here is a output from strace for mount: http://www.heypasteit.com/clip/8WT
>
> Any further debugging hints?
>
> Thanks,
> CS.
>
> On 6/16/09, Cliff White <Cliff.White at sun.com> wrote:
>> Carlos Santana wrote:
>>> The '$ modprobe -l lustre*' did not show any module on a patchless
>>> client. modprobe -v returns 'FATAL: Module lustre not found'.
>>>
>>> How do I install a patchless client?
>>> I have tried lustre-client-modules and lustre-client-ver rpm packages in
>>> both sequences. Am I missing anything?
>>>
>> Make sure the lustre-client-modules package matches your running kernel.
>> Run depmod -a to be sure
>> cliffw
>>
>>> Thanks,
>>> CS.
>>>
>>>
>>>
>>> On Tue, Jun 16, 2009 at 2:28 PM, Cliff White <Cliff.White at sun.com
>>> <mailto:Cliff.White at sun.com>> wrote:
>>>
>>> Carlos Santana wrote:
>>>
>>> The lctlt ping and 'net up' failed with the following messages:
>>> --- ---
>>> [root at localhost ~]# lctl ping 10.0.0.42
>>> opening /dev/lnet failed: No such device
>>> hint: the kernel modules may not be loaded
>>> failed to ping 10.0.0.42 at tcp: No such device
>>>
>>> [root at localhost ~]# lctl network up
>>> opening /dev/lnet failed: No such device
>>> hint: the kernel modules may not be loaded
>>> LNET configure error 19: No such device
>>>
>>>
>>> Make sure modules are unloaded, then try modprobe -v.
>>> Looks like you have lnet mis-configured, if your module options are
>>> wrong, you will see an error during the modprobe.
>>> cliffw
>>>
>>> --- ---
>>>
>>>
>>> I tried lustre_rmmod and depmod commands and it did not return
>>> any error messages. Any further clues? Reinstall patchless
>>> client again?
>>>
>>> -
>>> CS.
>>>
>>>
>>> On Tue, Jun 16, 2009 at 1:32 PM, Cliff White
>>> <Cliff.White at sun.com <mailto:Cliff.White at sun.com>
>>> <mailto:Cliff.White at sun.com <mailto:Cliff.White at sun.com>>> wrote:
>>>
>>> Carlos Santana wrote:
>>>
>>> I was able to run lustre_rmmod and depmod successfully. The
>>> '$lctl list_nids' returned the server ip address and
>>> interface
>>> (tcp0).
>>>
>>> I tried to mount the file system on a remote client, but it
>>> failed with the following message.
>>> --- ---
>>> [root at localhost ~]# mount -t lustre 10.0.0.42 at tcp0:/lustre
>>> /mnt/lustre
>>> mount.lustre: mount 10.0.0.42 at tcp0:/lustre at /mnt/lustre
>>> failed: No such device
>>> Are the lustre modules loaded?
>>> Check /etc/modprobe.conf and /proc/filesystems
>>> Note 'alias lustre llite' should be removed from
>>> modprobe.conf
>>> --- ---
>>>
>>> However, the mounting is successful on a single node
>>> configuration - with client on the same machine as MDS
>>> and OST.
>>> Any clues? Where to look for logs and debug messages?
>>>
>>>
>>> Syslog || /var/log/messages is the normal place.
>>>
>>> You can use 'lctl ping' to verify that the client can reach
>>> the server.
>>> Usually in these cases, it's a network/name misconfiguration.
>>>
>>> Run 'tunefs.lustre --print' on your servers, and verify that
>>> mgsnode=
>>> is correct.
>>>
>>> cliffw
>>>
>>>
>>> Thanks,
>>> CS.
>>>
>>>
>>>
>>>
>>>
>>> On Tue, Jun 16, 2009 at 12:16 PM, Cliff White
>>> <Cliff.White at sun.com <mailto:Cliff.White at sun.com>
>>> <mailto:Cliff.White at sun.com <mailto:Cliff.White at sun.com>>
>>> <mailto:Cliff.White at sun.com <mailto:Cliff.White at sun.com>
>>> <mailto:Cliff.White at sun.com <mailto:Cliff.White at sun.com>>>> wrote:
>>>
>>> Carlos Santana wrote:
>>>
>>> Thanks Kevin..
>>>
>>> Please read:
>>>
>>>
>>> http://manual.lustre.org/manual/LustreManual16_HTML/ConfiguringLustre.html#50401328_pgfId-1289529
>>>
>>> Those instructions are identical for 1.6 and 1.8.
>>>
>>> For current lustre, only two commands are used for
>>> configuration.
>>> mkfs.lustre and mount.
>>>
>>>
>>> Usually when lustre_rmmod returns that error, you run
>>> it a second
>>> time, and it will clear things. Unless you have live
>>> mounts or
>>> network connections.
>>>
>>> cliffw
>>>
>>>
>>> I am referring to 1.8 manual, but I was also
>>> referring to
>>> HowTo
>>> page on wiki which seems to be for 1.6. The HowTo
>>> page
>>>
>>>
>>> http://wiki.lustre.org/index.php/Lustre_Howto#Using_Supplied_Configuration_Tools
>>> mentions abt lmc, lconf, and lctl.
>>>
>>> The modules are installed in the right place. The '$
>>> lustre_rmmod' resulted in following o/p:
>>> [root at localhost 2.6.18-92.1.17.el5_lustre.1.8.0smp]#
>>> lustre_rmmod
>>> ERROR: Module obdfilter is in use
>>> ERROR: Module ost is in use
>>> ERROR: Module mds is in use
>>> ERROR: Module fsfilt_ldiskfs is in use
>>> ERROR: Module mgs is in use
>>> ERROR: Module mgc is in use by mgs
>>> ERROR: Module ldiskfs is in use by fsfilt_ldiskfs
>>> ERROR: Module lov is in use
>>> ERROR: Module lquota is in use by obdfilter,mds
>>> ERROR: Module osc is in use
>>> ERROR: Module ksocklnd is in use
>>> ERROR: Module ptlrpc is in use by
>>> obdfilter,ost,mds,mgs,mgc,lov,lquota,osc
>>> ERROR: Module obdclass is in use by
>>>
>>> obdfilter,ost,mds,fsfilt_ldiskfs,mgs,mgc,lov,lquota,osc,ptlrpc
>>> ERROR: Module lnet is in use by
>>> ksocklnd,ptlrpc,obdclass
>>> ERROR: Module lvfs is in use by
>>>
>>>
>>> obdfilter,ost,mds,fsfilt_ldiskfs,mgs,mgc,lov,lquota,osc,ptlrpc,obdclass
>>> ERROR: Module libcfs is in use by
>>>
>>>
>>> obdfilter,ost,mds,fsfilt_ldiskfs,mgs,mgc,lov,lquota,osc,ksocklnd,ptlrpc,obdclass,lnet,lvfs
>>>
>>> Do I need to shutdown these services? How can I do
>>> that?
>>>
>>> Thanks,
>>> CS.
>>>
>>>
>>> On Tue, Jun 16, 2009 at 11:36 AM, Kevin Van Maren
>>> <Kevin.Vanmaren at sun.com
>>> <mailto:Kevin.Vanmaren at sun.com> <mailto:Kevin.Vanmaren at sun.com
>>> <mailto:Kevin.Vanmaren at sun.com>>
>>> <mailto:Kevin.Vanmaren at sun.com
>>> <mailto:Kevin.Vanmaren at sun.com> <mailto:Kevin.Vanmaren at sun.com
>>> <mailto:Kevin.Vanmaren at sun.com>>>
>>> <mailto:Kevin.Vanmaren at sun.com
>>> <mailto:Kevin.Vanmaren at sun.com>
>>> <mailto:Kevin.Vanmaren at sun.com
>>> <mailto:Kevin.Vanmaren at sun.com>> <mailto:Kevin.Vanmaren at sun.com
>>> <mailto:Kevin.Vanmaren at sun.com>
>>> <mailto:Kevin.Vanmaren at sun.com
>>> <mailto:Kevin.Vanmaren at sun.com>>>>>
>>>
>>> wrote:
>>>
>>> I think lconf and lmc went away with Lustre
>>> 1.6. Are you
>>> sure you
>>> are looking at the 1.8 manual, and not
>>> directions for 1.4?
>>>
>>> /usr/sbin/lctl should be in the
>>> lustre-<version> RPM.
>>> Do a:
>>> # rpm -q -l
>>> lustre-1.8.0-2.6.18_92.1.17.el5_lustre.1.8.0smp
>>>
>>>
>>> Do make sure the modules are installed in the
>>> right place:
>>> # cd /lib/modules/`uname -r`
>>> # find . | grep lustre.ko
>>>
>>> If it shows up, then do:
>>> # lustre_rmmod
>>> # depmod
>>> and try again.
>>>
>>> Otherwise, figure out where your modules are
>>> installed:
>>> # uname -r
>>> # cd /lib/modules
>>> # find . | grep lustre.ko
>>>
>>>
>>> You can also double-check the NID. On the MSD
>>> server, do
>>> # lctl list_nids
>>>
>>> Should show 10.0.0.42 at tcp0
>>>
>>> Kevin
>>>
>>>
>>>
>>>
>>>
>>> ------------------------------------------------------------------------
>>>
>>> _______________________________________________
>>> Lustre-discuss mailing list
>>> Lustre-discuss at lists.lustre.org
>>> <mailto:Lustre-discuss at lists.lustre.org>
>>> <mailto:Lustre-discuss at lists.lustre.org
>>> <mailto:Lustre-discuss at lists.lustre.org>>
>>>
>>> http://lists.lustre.org/mailman/listinfo/lustre-discuss
>>>
>>>
>>>
>>>
>>>
>>> ------------------------------------------------------------------------
>>>
>>> _______________________________________________
>>> Lustre-discuss mailing list
>>> Lustre-discuss at lists.lustre.org
>>> <mailto:Lustre-discuss at lists.lustre.org>
>>> http://lists.lustre.org/mailman/listinfo/lustre-discuss
>>>
>>>
>>>
>>>
>>> ------------------------------------------------------------------------
>>>
>>> _______________________________________________
>>> Lustre-discuss mailing list
>>> Lustre-discuss at lists.lustre.org
>>> http://lists.lustre.org/mailman/listinfo/lustre-discuss
>>
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-discuss
More information about the lustre-discuss
mailing list