<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
</head>
<body dir="auto">
<div>Using rsync or tar to backup/restore a ZFS MDT is not supported, because this changes the dnode numbering, but ZFS OI Scrub is not yet implemented (there is a Jira ticket for this, and some work is underway there).</div>
<div id="AppleMailSignature"><br>
</div>
<div id="AppleMailSignature">Options include using zfs send/recv, as you were using, or just incrementally replacing the disks in the pool one at a time and letting them resilver to the SSDs (assuming they are larger than the HDDs they are replacing). </div>
<div id="AppleMailSignature"><br>
</div>
<div id="AppleMailSignature">I'm not sure why send/recv is so slow and exploding the metadata size, but it might relate to the ashift=12 on the target and ashift=9 on the source? This can be particularly bad with RAIDz compared to mirrors, since small blocks
(as typically used on the MDT) will always need to write 16KB instead of 8 or 12KB (with 2 or 3 mirrors). </div>
<div id="AppleMailSignature"><br>
Cheers, Andreas</div>
<div><br>
On Jul 22, 2017, at 07:48, Raj <<a href="mailto:rajgautam@gmail.com">rajgautam@gmail.com</a>> wrote:<br>
<br>
</div>
<blockquote type="cite">
<div>
<div dir="ltr">Stu,
<div>Is there a reason why you picked Raidz 3 rather than 4 way mirror across 4 disks?</div>
<div>Raidz 3 parity calculation might take more cpu resources rather than mirroring across disks but also the latency may be higher in mirroring to sync across all the disks. Wondering if you did some testing before deciding it.</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr">On Fri, Jul 21, 2017 at 12:27 AM Stu Midgley <<a href="mailto:sdm900@gmail.com">sdm900@gmail.com</a>> wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">
<div class="gmail_default" style="font-family:monospace,monospace">we have been happily using 2.9.52+0.7.0-rc3 for a while now.</div>
<div class="gmail_default" style="font-family:monospace,monospace"><br>
</div>
<div class="gmail_default" style="font-family:monospace,monospace">The MDT is a raidz3 across 4 disks.</div>
</div>
<div class="gmail_extra"></div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Fri, Jul 21, 2017 at 1:19 PM, Isaac Huang <span dir="ltr">
<<a href="mailto:he.huang@intel.com" target="_blank">he.huang@intel.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<span>On Fri, Jul 21, 2017 at 12:54:15PM +0800, Stu Midgley wrote:<br>
> Afternoon<br>
><br>
> I have an MDS running on spinning media and wish to migrate it to SSD's.<br>
><br>
> Lustre 2.9.52<br>
> ZFS 0.7.0-rc3<br>
<br>
</span>This may not be a stable combination - I don't think Lustre officially<br>
supports 0.7.0-rc yet. Plus, there's a recent Lustre osd-zfs bug and<br>
its fix hasn't been back ported to 2.9 yet (to the best of my knowledge):<br>
<a href="https://jira.hpdd.intel.com/browse/LU-9305" rel="noreferrer" target="_blank">https://jira.hpdd.intel.com/browse/LU-9305</a><br>
<span><br>
> How do I do it?<br>
<br>
</span>Depends on how you've configured the MDT pool. If the disks are<br>
mirrored or just plan disks without any redundancy (i.e. not RAIDz),<br>
you can simply attach the SSDs to the hard drives to form or extend<br>
mirrors and then detach the hard drives - see zpool attach/detach.<br>
<span class="m_-7255051117978200532HOEnZb"><font color="#888888"><br>
-Isaac<br>
</font></span></blockquote>
</div>
<br>
<br clear="all">
<div><br>
</div>
</div>
<div class="gmail_extra">-- <br>
<div class="m_-7255051117978200532gmail_signature" data-smartmail="gmail_signature">
<div dir="ltr">
<div class="m_-7255051117978200532gmail_signature" data-smartmail="gmail_signature">
<font face="monospace, monospace">Dr Stuart Midgley<br>
<a href="mailto:sdm900@gmail.com" target="_blank">sdm900@gmail.com</a></font></div>
</div>
</div>
</div>
_______________________________________________<br>
lustre-discuss mailing list<br>
<a href="mailto:lustre-discuss@lists.lustre.org" target="_blank">lustre-discuss@lists.lustre.org</a><br>
<a href="http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org" rel="noreferrer" target="_blank">http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org</a><br>
</blockquote>
</div>
</div>
</blockquote>
<blockquote type="cite">
<div><span>_______________________________________________</span><br>
<span>lustre-discuss mailing list</span><br>
<span><a href="mailto:lustre-discuss@lists.lustre.org">lustre-discuss@lists.lustre.org</a></span><br>
<span><a href="http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org">http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org</a></span><br>
</div>
</blockquote>
</body>
</html>