<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">I'd like to report that we've had
success in setting up an<br>
ldiskfs MDT and zfs OSTs on a single node with the <br>
version lustre 2.4 g1cff80a. Something must have been<br>
fixed/changed in this tree since the install steps didn't <br>
change as far as I can tell. <br>
<br>
Also, I was making rpms from spl and zfs source, so didn't <br>
have to add anything to /etc/ld.so.conf.d, but thanks for the <br>
suggestions. On to the testing stage...<br>
<br>
-Anjana<br>
<br>
On 10/09/2013 05:30 AM, Thomas Stibor wrote:<br>
</div>
<blockquote cite="mid:52552248.9070608@gsi.de" type="cite">
<pre wrap="">Hello Anjana,
I can confirm that this setup works (ZFS-MGS/MDT or LDFISKFS-MGS/MDT and
ZFS-OSS/OST)
I used a Cent OS 6.4
build:
2.4.0-RC2-gd3f91c4-PRISTINE-2.6.32-358.6.2.el6_lustre.g230b174.x86_64
and the Lustre Packages from
<a class="moz-txt-link-freetext" href="http://downloads.whamcloud.com/public/lustre/latest-feature-release/el6/server/RPMS/x86_64/">http://downloads.whamcloud.com/public/lustre/latest-feature-release/el6/server/RPMS/x86_64/</a>
ZFS is downloaded from ZOL and compiled/installed.
SPL: Loaded module v0.6.2-1
SPL: using hostid 0x00000000
ZFS: Loaded module v0.6.2-1, ZFS pool version 5000, ZFS filesystem version 5
I first run in the same problem:
mkfs.lustre --fsname=lustrefs --reformat --ost --backfstype=zfs .....
mkfs.lustre FATAL: unable to prepare backend (22)
mkfs.lustre: exiting with 22 (Invalid argument)
and saw that ZFS libraries in /usr/local/lib where not known to Cent OS 6.4.
A quick:
echo "/usr/local/lib" >> /etc/ld.so.conf.d/zfs.conf
echo "/usr/local/lib64" >> /etc/ld.so.conf.d/zfs.conf
ldconfig
solved the problem.
(LDISKFS)
mkfs.lustre --reformat --mgs /dev/sda16
mkfs.lustre --reformat --fsname=zlust --mgsnode=10.16.0.104@o2ib0 --mdt
--index=0 /dev/sda5
(ZFS)
mkfs.lustre --reformat --mgs --backfstype=zfs mgs/mgs /dev/sda16
mkfs.lustre --reformat --fsname=zlust --mgsnode=10.16.0.104@o2ib0 --mdt
--index=0 --backfstype=zfs mdt0/mdt0 /dev/sda5
is working fine.
The OSS/OST is a debian wheezy box with 70 disks JBOD and kernel
3.6.11-lustre-tstibor-build with patch series 3.x-fc18.series
and SPL/ZFS v0.6.2-1
Best,
Thomas
On 10/08/2013 05:40 PM, Anjana Kar wrote:
</pre>
<blockquote type="cite">
<pre wrap="">The git checkout was on Sep. 20. Was the patch before or after?
The zpool create command successfully creates a raidz2 pool, and mkfs.lustre
does not complain, but
[root@cajal kar]# zpool list
NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT
lustre-ost0 36.2T 2.24M 36.2T 0% 1.00x ONLINE -
[root@cajal kar]# /usr/sbin/mkfs.lustre --fsname=cajalfs --ost
--backfstype=zfs --index=0 --mgsnode=10.10.101.171@o2ib lustre-ost0
[root@cajal kar]# /sbin/service lustre start lustre-ost0
lustre-ost0 is not a valid lustre label on this node
I think we'll be splitting up the MDS and OSTs on 2 nodes as some of you
said
there could be other issues down the road, but thanks for all the good
suggestions.
-Anjana
On 10/07/2013 07:24 PM, Ned Bass wrote:
</pre>
<blockquote type="cite">
<pre wrap="">I'm guessing your git checkout doesn't include this commit:
* 010a78e Revert "LU-3682 tunefs: prevent tunefs running on a mounted device"
It looks like the LU-3682 patch introduced a bug that could cause your issue,
so its reverted in the latest master.
Ned
On Mon, Oct 07, 2013 at 04:54:13PM -0400, Anjana Kar wrote:
</pre>
<blockquote type="cite">
<pre wrap="">On 10/07/2013 04:27 PM, Ned Bass wrote:
</pre>
<blockquote type="cite">
<pre wrap="">On Mon, Oct 07, 2013 at 02:23:32PM -0400, Anjana Kar wrote:
</pre>
<blockquote type="cite">
<pre wrap="">Here is the exact command used to create a raidz2 pool with 8+2 drives,
followed by the error messages:
mkfs.lustre --fsname=cajalfs --reformat --ost --backfstype=zfs
--index=0 --mgsnode=10.10.101.171@o2ib lustre-ost0/ost0 raidz2
/dev/sda /dev/sdc /dev/sde /dev/sdg /dev/sdi /dev/sdk /dev/sdm
/dev/sdo /dev/sdq /dev/sds
mkfs.lustre FATAL: Invalid filesystem name /dev/sds
</pre>
</blockquote>
<pre wrap="">It seems that either the version of mkfs.lustre you are using has a
parsing bug, or there was some sort of syntax error in the actual
command entered. If you are certain your command line is free from
errors, please post the version of lustre you are using, or report the
bug in the Lustre issue tracker.
Thanks,
Ned
</pre>
</blockquote>
<pre wrap="">For building this server, I followed steps from the walk-thru-build*
for Centos 6.4,
and added --with-spl and --with-zfs when configuring lustre..
*<a class="moz-txt-link-freetext" href="https://wiki.hpdd.intel.com/pages/viewpage.action?pageId=8126821">https://wiki.hpdd.intel.com/pages/viewpage.action?pageId=8126821</a>
spl and zfs modules were installed from source for the lustre 2.4 kernel
2.6.32.358.18.1.el6_lustre2.4
Device sds appears to be valid, but I will try issuing the command
using by-path
names..
-Anjana
</pre>
</blockquote>
</blockquote>
<pre wrap="">_______________________________________________
Lustre-discuss mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Lustre-discuss@lists.lustre.org">Lustre-discuss@lists.lustre.org</a>
<a class="moz-txt-link-freetext" href="http://lists.lustre.org/mailman/listinfo/lustre-discuss">http://lists.lustre.org/mailman/listinfo/lustre-discuss</a>
</pre>
</blockquote>
<pre wrap="">
</pre>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
Lustre-discuss mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Lustre-discuss@lists.lustre.org">Lustre-discuss@lists.lustre.org</a>
<a class="moz-txt-link-freetext" href="http://lists.lustre.org/mailman/listinfo/lustre-discuss">http://lists.lustre.org/mailman/listinfo/lustre-discuss</a>
</pre>
</blockquote>
<br>
</body>
</html>