[lustre-discuss] lustre-discuss Digest, Vol 123, Issue 35
Patrick Farrell
paf at cray.com
Tue Jun 28 14:20:23 PDT 2016
Chen,
Lnet is required, but for some configurations, the default settings - with no explicit configuration of lnet - will work. Notably, this is true for simple TCP/IP networks of the sort you often have in VM environments.
- Patrick
________________________________
From: lustre-discuss <lustre-discuss-bounces at lists.lustre.org> on behalf of Yu Chen <chenyu at umbc.edu>
Sent: Tuesday, June 28, 2016 4:03:34 PM
To: sohamm
Cc: lustre-discuss at lists.lustre.org
Subject: Re: [lustre-discuss] lustre-discuss Digest, Vol 123, Issue 35
opps, sorry, Sohamm, not sure why I missed your reply.
Thanks for sharing, so lnet is not needed? then how then connect to each other? will check doc more carefully.
Chen
On Sun, Jun 26, 2016 at 12:02 AM, sohamm <sohamm at gmail.com<mailto:sohamm at gmail.com>> wrote:
Hi YuChen,
My setup also uses zfs on lustre and below is the output log from my system.
The only difference is i did not setup the lnet ( as the lustre doc says its optional ). Not sure if thats the reason you see the error and i do not. I am also new to lustre and trying the config on bunch of VM's.
[root at lustre_mgs01_vm03 ~]# systemctl status lustre
● lustre.service - SYSV: Part of the lustre file system.
Loaded: loaded (/etc/rc.d/init.d/lustre)
Active: inactive (dead)
Docs: man:systemd-sysv-generator(8)
[root at lustre_mgs01_vm03 ~]# systemctl start lustre
[root at lustre_mgs01_vm03 ~]# systemctl status lustre
● lustre.service - SYSV: Part of the lustre file system.
Loaded: loaded (/etc/rc.d/init.d/lustre)
Active: active (exited) since Sat 2016-06-25 23:49:45 EDT; 2s ago
Docs: man:systemd-sysv-generator(8)
Process: 17751 ExecStart=/etc/rc.d/init.d/lustre start (code=exited, status=0/SUCCESS)
Jun 25 23:49:45 lustre_mgs01_vm03 systemd[1]: Starting SYSV: Part of the lustre file system....
Jun 25 23:49:45 lustre_mgs01_vm03 systemd[1]: Started SYSV: Part of the lustre file system..
[root at lustre_mgs01_vm03 ~]# ls /etc/modprobe.d/
ko2iblnd.conf mlx4.conf
[root at lustre_mgs01_vm03 ~]# yum list installed | grep zfs
Repodata is over 2 weeks old. Install yum-cron? Or run: yum makecache fast
libnvpair1.x86_64 0.6.5.7-1.el7.centos @zfs
libuutil1.x86_64 0.6.5.7-1.el7.centos @zfs
libzfs2.x86_64 0.6.5.7-1.el7.centos @zfs
libzpool2.x86_64 0.6.5.7-1.el7.centos @zfs
lustre-osd-zfs.x86_64 2.8.54_1_g5f03bf9-3.10.0_327.18.2.el7_lustre.x86_64
lustre-osd-zfs-mount.x86_64 2.8.54_1_g5f03bf9-3.10.0_327.18.2.el7_lustre.x86_64
spl.x86_64 0.6.5.7-1.el7.centos @zfs
spl-dkms.noarch 0.6.5.7-1.el7.centos @zfs
zfs.x86_64 0.6.5.7-1.el7.centos @zfs
zfs-dkms.noarch 0.6.5.7-1.el7.centos @zfs
zfs-release.noarch 1-2.el7.centos @/zfs-release.el7.noarch
[root at lustre_mgs01_vm03 ~]# yum list installed | grep lustre
Repodata is over 2 weeks old. Install yum-cron? Or run: yum makecache fast
kernel.x86_64 3.10.0-327.13.1.el7_lustre @hpddLustreserver
kernel.x86_64 3.10.0-327.18.2.el7_lustre @hpddLustreserver
kernel-devel.x86_64 3.10.0-327.13.1.el7_lustre @hpddLustreserver
kernel-headers.x86_64 3.10.0-327.13.1.el7_lustre @hpddLustreserver
lustre.x86_64 2.8.54_1_g5f03bf9-3.10.0_327.18.2.el7_lustre.x86_64
lustre-iokit.x86_64 2.8.53_11_gfd4ab6e-3.10.0_327.13.1.el7.x86_64
lustre-modules.x86_64 2.8.54_1_g5f03bf9-3.10.0_327.18.2.el7_lustre.x86_64
lustre-osd-ldiskfs.x86_64 2.8.54_1_g5f03bf9-3.10.0_327.18.2.el7_lustre.x86_64
lustre-osd-ldiskfs-mount.x86_64 2.8.54_1_g5f03bf9-3.10.0_327.18.2.el7_lustre.x86_64
lustre-osd-zfs.x86_64 2.8.54_1_g5f03bf9-3.10.0_327.18.2.el7_lustre.x86_64
lustre-osd-zfs-mount.x86_64 2.8.54_1_g5f03bf9-3.10.0_327.18.2.el7_lustre.x86_64
lustre-tests.x86_64 2.8.54_1_g5f03bf9-3.10.0_327.18.2.el7_lustre.x86_64
Thanks
On Fri, Jun 24, 2016 at 1:07 PM, <lustre-discuss-request at lists.lustre.org<mailto:lustre-discuss-request at lists.lustre.org>> wrote:
Send lustre-discuss mailing list submissions to
lustre-discuss at lists.lustre.org<mailto:lustre-discuss at lists.lustre.org>
To subscribe or unsubscribe via the World Wide Web, visit
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
or, via email, send a message with subject or body 'help' to
lustre-discuss-request at lists.lustre.org<mailto:lustre-discuss-request at lists.lustre.org>
You can reach the person managing the list at
lustre-discuss-owner at lists.lustre.org<mailto:lustre-discuss-owner at lists.lustre.org>
When replying, please edit your Subject line so it is more specific
than "Re: Contents of lustre-discuss digest..."
Today's Topics:
1. Re: Lustre Client Package for Rocks 6.2 (Yasir Israr)
2. Can't load module 'osd-zfs' (Yu Chen)
----------------------------------------------------------------------
Message: 1
Date: Fri, 24 Jun 2016 08:24:36 +0530
From: Yasir Israr <yasir at orionsolutions.co.in<mailto:yasir at orionsolutions.co.in>>
To: "Mannthey, Keith" <keith.mannthey at intel.com<mailto:keith.mannthey at intel.com>>
Cc: "lustre-discuss at lists.lustre.org<mailto:lustre-discuss at lists.lustre.org>"
<lustre-discuss at lists.lustre.org<mailto:lustre-discuss at lists.lustre.org>>
Subject: Re: [lustre-discuss] Lustre Client Package for Rocks 6.2
Message-ID:
<F43ED9E5-1716-4A3F-B0DF-1FE79B7FCA75 at orionsolutions.co.in<mailto:F43ED9E5-1716-4A3F-B0DF-1FE79B7FCA75 at orionsolutions.co.in>>
Content-Type: text/plain; charset="utf-8"
Hi Keith
Running below src it's asking again to downgrade existing kernel. & in rocks compute I couldn't able to amend kernel because of rocks issue
Thanks & Regards
Yasir Israr
> On 24-Jun-2016, at 4:12 AM, Mannthey, Keith <keith.mannthey at intel.com<mailto:keith.mannthey at intel.com>> wrote:
>
> Yasir,
> Did you try rebuilding from the src.rpm against your current kernel?
> https://wiki.hpdd.intel.com/display/PUB/Rebuilding+the+Lustre-client+rpms+for+a+new+kernel
>
> Thanks,
> Keith
>
>
> From: lustre-discuss [mailto:lustre-discuss-bounces at lists.lustre.org<mailto:lustre-discuss-bounces at lists.lustre.org>] On Behalf Of Yasir Israr
> Sent: Thursday, June 23, 2016 3:16 PM
> To: E.S. Rosenberg <esr+lustre at mail.hebrew.edu<mailto:esr%2Blustre at mail.hebrew.edu>>
> Cc: lustre-discuss at lists.lustre.org<mailto:lustre-discuss at lists.lustre.org>
> Subject: Re: [lustre-discuss] Lustre Client Package for Rocks 6.2
>
> Hi Rosenberg
> I've tried using source code but problem not resolved every lustre client package asking to downgrade below existing kernel but that I can't do because of many application will get an issue.
>
> Thanks & Regards
> Yasir Israr
>
> On 23-Jun-2016, at 4:57 PM, E.S. Rosenberg <esr+lustre at mail.hebrew.edu<mailto:esr%2Blustre at mail.hebrew.edu>> wrote:
>
> Have you tried doing the lustre install from source?
>
> On Thu, Jun 23, 2016 at 10:45 AM, <yasir at orionsolutions.co.in<mailto:yasir at orionsolutions.co.in>> wrote:
> Dear Rocks User,
>
> Please help in getting correct Lustre Client Package for Rocks 6.2 with Kernel 2.6.32-504.16.2.el6.x86_64
> Currently I?m trying to install Client Package of Lustre 2.7 & 2.8 but it?s not working & asking to downgrade existing kernel.
>
> I couldn?t able to downgrade because we are running many packages which is installed & working fine. So please help if we can do it without downgrade kernel.
>
> Thanks & Regards,
> Yasir Israr
>
>
> _______________________________________________
> lustre-discuss mailing list
> lustre-discuss at lists.lustre.org<mailto:lustre-discuss at lists.lustre.org>
> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20160624/29ba2ee6/attachment.html>
------------------------------
Message: 2
Date: Fri, 24 Jun 2016 10:48:20 -0400
From: Yu Chen <chenyu at umbc.edu<mailto:chenyu at umbc.edu>>
To: lustre-discuss at lists.lustre.org<mailto:lustre-discuss at lists.lustre.org>
Subject: [lustre-discuss] Can't load module 'osd-zfs'
Message-ID:
<CAO0EFYMN-0f54mP_oQXtXRNv-nK8rqrzrLqxQwfqKUA5axXgfA at mail.gmail.com<mailto:CAO0EFYMN-0f54mP_oQXtXRNv-nK8rqrzrLqxQwfqKUA5axXgfA at mail.gmail.com>>
Content-Type: text/plain; charset="utf-8"
Hello,
I am new to lustre system. Was trying to setup a lustre on Centos 7 with
zfs.
When came to the step to start lustre.service from systemctl, got the
errors:
....
-------
osd_zfs: disagrees about version of symbol dmu_objset_space
osd_zfs: Unknown symbol dmu_objset_space (err -22)
------
LustreError: 158-c: Can't load module 'osd-zfs'
LustreError: 48041:0:(genops.c:318:class_newdev()) OBD: unknown type: osd
-zfs
.....
Here is what I have done:
- I installed kernel-3.10.0-327.3.1.el7_lustre.x86_64,
- Then configured and installed zfs from archive.zfsonlinux.org<http://archive.zfsonlinux.org>
(0.6.5.7-1)
- Then downloaded and installed lustre rpms from el7.2.1511 from:
https://downloads.hpdd.intel.com/public/lustre/latest-feature-release/
- Then *zpool* create mdt01-pool /dev/sdb
- mkfs.lustre --reformat --fsname=lustre01 --mgs --mdt --backfstype=zfs
--index=0 --mgsnode=10.1.12.1 at o2ib --servicenode=10.1.12.1 at o2ib
mdt01-pool/mgsmdt01
- echo "mds1 - lustre01-MDT0000 zfs:mdt01-pool/mgsmdt01" > /etc/ldev.conf
- echo "options lnet networks=o2ib(ib0)" > /etc/modprobe.d/lustre.conf
- systemctl start lustre.service
This is where it failed.
Then I think it might be because of the zfs version. So downloaded
lustre-2.8.0-3.10.0_327.3.1.el7_lustre.x86_64.src.rpm, and tried to rebuild
from it. But after rebuilding, there were no lustre-osd-zfs and
lustre-osd-zfs-mount rpms?
Would greatly appreciate any suggestions!
Thanks in advance!
yc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20160624/ed3d8deb/attachment-0001.htm>
------------------------------
Subject: Digest Footer
_______________________________________________
lustre-discuss mailing list
lustre-discuss at lists.lustre.org<mailto:lustre-discuss at lists.lustre.org>
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
------------------------------
End of lustre-discuss Digest, Vol 123, Issue 35
***********************************************
_______________________________________________
lustre-discuss mailing list
lustre-discuss at lists.lustre.org<mailto:lustre-discuss at lists.lustre.org>
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20160628/82e7e8ac/attachment-0001.htm>
More information about the lustre-discuss
mailing list