[lustre-discuss] lustre-discuss Digest, Vol 123, Issue 35
Yu Chen
chenyu at umbc.edu
Tue Jun 28 14:03:34 PDT 2016
opps, sorry, Sohamm, not sure why I missed your reply.
Thanks for sharing, so lnet is not needed? then how then connect to each
other? will check doc more carefully.
Chen
On Sun, Jun 26, 2016 at 12:02 AM, sohamm <sohamm at gmail.com> wrote:
> Hi YuChen,
>
> My setup also uses zfs on lustre and below is the output log from my
> system.
> The only difference is i did not setup the lnet ( as the lustre doc says
> its optional ). Not sure if thats the reason you see the error and i do
> not. I am also new to lustre and trying the config on bunch of VM's.
>
> [root at lustre_mgs01_vm03 ~]# systemctl status lustre
> ● lustre.service - SYSV: Part of the lustre file system.
> Loaded: loaded (/etc/rc.d/init.d/lustre)
> Active: inactive (dead)
> Docs: man:systemd-sysv-generator(8)
>
> [root at lustre_mgs01_vm03 ~]# systemctl start lustre
> [root at lustre_mgs01_vm03 ~]# systemctl status lustre
> ● lustre.service - SYSV: Part of the lustre file system.
> Loaded: loaded (/etc/rc.d/init.d/lustre)
> Active: active (exited) since Sat 2016-06-25 23:49:45 EDT; 2s ago
> Docs: man:systemd-sysv-generator(8)
> Process: 17751 ExecStart=/etc/rc.d/init.d/lustre start (code=exited,
> status=0/SUCCESS)
>
> Jun 25 23:49:45 lustre_mgs01_vm03 systemd[1]: Starting SYSV: Part of the
> lustre file system....
> Jun 25 23:49:45 lustre_mgs01_vm03 systemd[1]: Started SYSV: Part of the
> lustre file system..
>
> [root at lustre_mgs01_vm03 ~]# ls /etc/modprobe.d/
> ko2iblnd.conf mlx4.conf
>
> [root at lustre_mgs01_vm03 ~]# yum list installed | grep zfs
> Repodata is over 2 weeks old. Install yum-cron? Or run: yum makecache fast
> libnvpair1.x86_64 0.6.5.7-1.el7.centos @zfs
> libuutil1.x86_64 0.6.5.7-1.el7.centos @zfs
> libzfs2.x86_64 0.6.5.7-1.el7.centos @zfs
> libzpool2.x86_64 0.6.5.7-1.el7.centos @zfs
> lustre-osd-zfs.x86_64
> 2.8.54_1_g5f03bf9-3.10.0_327.18.2.el7_lustre.x86_64
> lustre-osd-zfs-mount.x86_64
> 2.8.54_1_g5f03bf9-3.10.0_327.18.2.el7_lustre.x86_64
> spl.x86_64 0.6.5.7-1.el7.centos @zfs
> spl-dkms.noarch 0.6.5.7-1.el7.centos @zfs
> zfs.x86_64 0.6.5.7-1.el7.centos @zfs
> zfs-dkms.noarch 0.6.5.7-1.el7.centos @zfs
> zfs-release.noarch 1-2.el7.centos
> @/zfs-release.el7.noarch
>
> [root at lustre_mgs01_vm03 ~]# yum list installed | grep lustre
> Repodata is over 2 weeks old. Install yum-cron? Or run: yum makecache fast
> kernel.x86_64 3.10.0-327.13.1.el7_lustre
> @hpddLustreserver
> kernel.x86_64 3.10.0-327.18.2.el7_lustre
> @hpddLustreserver
> kernel-devel.x86_64 3.10.0-327.13.1.el7_lustre
> @hpddLustreserver
> kernel-headers.x86_64 3.10.0-327.13.1.el7_lustre
> @hpddLustreserver
> lustre.x86_64
> 2.8.54_1_g5f03bf9-3.10.0_327.18.2.el7_lustre.x86_64
> lustre-iokit.x86_64
> 2.8.53_11_gfd4ab6e-3.10.0_327.13.1.el7.x86_64
> lustre-modules.x86_64
> 2.8.54_1_g5f03bf9-3.10.0_327.18.2.el7_lustre.x86_64
> lustre-osd-ldiskfs.x86_64
> 2.8.54_1_g5f03bf9-3.10.0_327.18.2.el7_lustre.x86_64
> lustre-osd-ldiskfs-mount.x86_64
> 2.8.54_1_g5f03bf9-3.10.0_327.18.2.el7_lustre.x86_64
> lustre-osd-zfs.x86_64
> 2.8.54_1_g5f03bf9-3.10.0_327.18.2.el7_lustre.x86_64
> lustre-osd-zfs-mount.x86_64
> 2.8.54_1_g5f03bf9-3.10.0_327.18.2.el7_lustre.x86_64
> lustre-tests.x86_64
> 2.8.54_1_g5f03bf9-3.10.0_327.18.2.el7_lustre.x86_64
>
> Thanks
>
> On Fri, Jun 24, 2016 at 1:07 PM, <lustre-discuss-request at lists.lustre.org>
> wrote:
>
>> Send lustre-discuss mailing list submissions to
>> lustre-discuss at lists.lustre.org
>>
>> To subscribe or unsubscribe via the World Wide Web, visit
>> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>> or, via email, send a message with subject or body 'help' to
>> lustre-discuss-request at lists.lustre.org
>>
>> You can reach the person managing the list at
>> lustre-discuss-owner at lists.lustre.org
>>
>> When replying, please edit your Subject line so it is more specific
>> than "Re: Contents of lustre-discuss digest..."
>>
>>
>> Today's Topics:
>>
>> 1. Re: Lustre Client Package for Rocks 6.2 (Yasir Israr)
>> 2. Can't load module 'osd-zfs' (Yu Chen)
>>
>>
>> ----------------------------------------------------------------------
>>
>> Message: 1
>> Date: Fri, 24 Jun 2016 08:24:36 +0530
>> From: Yasir Israr <yasir at orionsolutions.co.in>
>> To: "Mannthey, Keith" <keith.mannthey at intel.com>
>> Cc: "lustre-discuss at lists.lustre.org"
>> <lustre-discuss at lists.lustre.org>
>> Subject: Re: [lustre-discuss] Lustre Client Package for Rocks 6.2
>> Message-ID:
>> <F43ED9E5-1716-4A3F-B0DF-1FE79B7FCA75 at orionsolutions.co.in>
>> Content-Type: text/plain; charset="utf-8"
>>
>> Hi Keith
>> Running below src it's asking again to downgrade existing kernel. & in
>> rocks compute I couldn't able to amend kernel because of rocks issue
>>
>> Thanks & Regards
>> Yasir Israr
>>
>> > On 24-Jun-2016, at 4:12 AM, Mannthey, Keith <keith.mannthey at intel.com>
>> wrote:
>> >
>> > Yasir,
>> > Did you try rebuilding from the src.rpm against your current kernel?
>> >
>> https://wiki.hpdd.intel.com/display/PUB/Rebuilding+the+Lustre-client+rpms+for+a+new+kernel
>> >
>> > Thanks,
>> > Keith
>> >
>> >
>> > From: lustre-discuss [mailto:lustre-discuss-bounces at lists.lustre.org]
>> On Behalf Of Yasir Israr
>> > Sent: Thursday, June 23, 2016 3:16 PM
>> > To: E.S. Rosenberg <esr+lustre at mail.hebrew.edu>
>> > Cc: lustre-discuss at lists.lustre.org
>> > Subject: Re: [lustre-discuss] Lustre Client Package for Rocks 6.2
>> >
>> > Hi Rosenberg
>> > I've tried using source code but problem not resolved every lustre
>> client package asking to downgrade below existing kernel but that I can't
>> do because of many application will get an issue.
>> >
>> > Thanks & Regards
>> > Yasir Israr
>> >
>> > On 23-Jun-2016, at 4:57 PM, E.S. Rosenberg <esr+lustre at mail.hebrew.edu>
>> wrote:
>> >
>> > Have you tried doing the lustre install from source?
>> >
>> > On Thu, Jun 23, 2016 at 10:45 AM, <yasir at orionsolutions.co.in> wrote:
>> > Dear Rocks User,
>> >
>> > Please help in getting correct Lustre Client Package for Rocks 6.2 with
>> Kernel 2.6.32-504.16.2.el6.x86_64
>> > Currently I?m trying to install Client Package of Lustre 2.7 & 2.8 but
>> it?s not working & asking to downgrade existing kernel.
>> >
>> > I couldn?t able to downgrade because we are running many packages which
>> is installed & working fine. So please help if we can do it without
>> downgrade kernel.
>> >
>> > Thanks & Regards,
>> > Yasir Israr
>> >
>> >
>> > _______________________________________________
>> > lustre-discuss mailing list
>> > lustre-discuss at lists.lustre.org
>> > http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>> >
>> >
>> -------------- next part --------------
>> An HTML attachment was scrubbed...
>> URL: <
>> http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20160624/29ba2ee6/attachment.html
>> >
>>
>> ------------------------------
>>
>> Message: 2
>> Date: Fri, 24 Jun 2016 10:48:20 -0400
>> From: Yu Chen <chenyu at umbc.edu>
>> To: lustre-discuss at lists.lustre.org
>> Subject: [lustre-discuss] Can't load module 'osd-zfs'
>> Message-ID:
>> <
>> CAO0EFYMN-0f54mP_oQXtXRNv-nK8rqrzrLqxQwfqKUA5axXgfA at mail.gmail.com>
>> Content-Type: text/plain; charset="utf-8"
>>
>> Hello,
>>
>> I am new to lustre system. Was trying to setup a lustre on Centos 7 with
>> zfs.
>>
>> When came to the step to start lustre.service from systemctl, got the
>> errors:
>> ....
>>
>> -------
>>
>> osd_zfs: disagrees about version of symbol dmu_objset_space
>>
>> osd_zfs: Unknown symbol dmu_objset_space (err -22)
>>
>> ------
>>
>> LustreError: 158-c: Can't load module 'osd-zfs'
>>
>> LustreError: 48041:0:(genops.c:318:class_newdev()) OBD: unknown type: osd
>> -zfs
>>
>> .....
>>
>>
>> Here is what I have done:
>>
>>
>> - I installed kernel-3.10.0-327.3.1.el7_lustre.x86_64,
>> - Then configured and installed zfs from archive.zfsonlinux.org
>> (0.6.5.7-1)
>> - Then downloaded and installed lustre rpms from el7.2.1511 from:
>> https://downloads.hpdd.intel.com/public/lustre/latest-feature-release/
>> - Then *zpool* create mdt01-pool /dev/sdb
>> - mkfs.lustre --reformat --fsname=lustre01 --mgs --mdt --backfstype=zfs
>> --index=0 --mgsnode=10.1.12.1 at o2ib --servicenode=10.1.12.1 at o2ib
>> mdt01-pool/mgsmdt01
>> - echo "mds1 - lustre01-MDT0000 zfs:mdt01-pool/mgsmdt01" >
>> /etc/ldev.conf
>> - echo "options lnet networks=o2ib(ib0)" > /etc/modprobe.d/lustre.conf
>> - systemctl start lustre.service
>>
>> This is where it failed.
>>
>> Then I think it might be because of the zfs version. So downloaded
>> lustre-2.8.0-3.10.0_327.3.1.el7_lustre.x86_64.src.rpm, and tried to
>> rebuild
>> from it. But after rebuilding, there were no lustre-osd-zfs and
>> lustre-osd-zfs-mount rpms?
>>
>>
>> Would greatly appreciate any suggestions!
>>
>>
>> Thanks in advance!
>>
>>
>> yc
>> -------------- next part --------------
>> An HTML attachment was scrubbed...
>> URL: <
>> http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20160624/ed3d8deb/attachment-0001.htm
>> >
>>
>> ------------------------------
>>
>> Subject: Digest Footer
>>
>> _______________________________________________
>> lustre-discuss mailing list
>> lustre-discuss at lists.lustre.org
>> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>>
>>
>> ------------------------------
>>
>> End of lustre-discuss Digest, Vol 123, Issue 35
>> ***********************************************
>>
>
>
> _______________________________________________
> lustre-discuss mailing list
> lustre-discuss at lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20160628/2b656ffa/attachment-0001.htm>
More information about the lustre-discuss
mailing list