<div dir="ltr">Okay. Got it.<div><br></div><div>Regards,</div><div><br></div><div><br></div><div>Indivar Nair</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sun, Feb 21, 2021 at 7:55 PM Spitz, Cory James <<a href="mailto:cory.spitz@hpe.com">cory.spitz@hpe.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div lang="EN-US" style="overflow-wrap: break-word;">
<div class="gmail-m_-4221470015216358567WordSection1">
<p class="MsoNormal">Right, except I should point out that a mirror sync operation doesn’t have to be from one pool to another. The FLR replica can use any specified layout. Using an OST Pool is just a convenience.<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">-Cory<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<p class="MsoNormal" style="margin-left:0.5in">On 2/20/21, 12:10 PM, "Indivar Nair" <<a href="mailto:indivar.nair@techterra.in" target="_blank">indivar.nair@techterra.in</a>> wrote:<u></u><u></u></p>
</div>
<p class="MsoNormal" style="margin-left:0.5in"><u></u> <u></u></p>
<div>
<p class="MsoNormal" style="margin-left:0.5in">Thanks, Cory.<u></u><u></u></p>
<div>
<p class="MsoNormal" style="margin-left:0.5in"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:0.5in">So -<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:0.5in"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:0.5in">1. When the "lfs mirror" command is run, the file(s) are copied by the client from one pool to another<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:0.5in"> Pool A -> Client Machine -> Pool B (mirror)<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:0.5in">2. If a mirrored file is modified, the modifications are saved only to the primary mirror<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:0.5in"> We will then have to run the "lfs mirror resync" command to synchronize all the mirror copies<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:0.5in"> Pool A (modified file) -> Client Machine -> Pool B (mirror)<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:0.5in"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:0.5in">So for this feature to be really effective, we will need a dedicated client that does this copying / re-copying continuously.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:0.5in">And if there are a lot of mirrored files, the dedicated client will have to be very fast too.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:0.5in"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:0.5in">3. In future it may be possible to move these files using HSM agents.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:0.5in"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:0.5in">Regards,<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:0.5in"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:0.5in"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:0.5in">Indivar Nair<u></u><u></u></p>
</div>
</div>
<p class="MsoNormal" style="margin-left:0.5in"><u></u> <u></u></p>
<div>
<div>
<p class="MsoNormal" style="margin-left:0.5in">On Fri, Feb 19, 2021 at 1:51 AM Spitz, Cory James <<a href="mailto:cory.spitz@hpe.com" target="_blank">cory.spitz@hpe.com</a>> wrote:<u></u><u></u></p>
</div>
<blockquote style="border-top:none;border-right:none;border-bottom:none;border-left:1pt solid rgb(204,204,204);padding:0in 0in 0in 6pt;margin-left:4.8pt;margin-right:0in">
<div>
<div>
<p class="MsoNormal" style="margin-left:0.5in">
Good questions, Indivar.<u></u><u></u></p>
<p class="MsoNormal" style="margin-left:0.5in">
<u></u><u></u></p>
<p class="MsoNormal" style="margin-left:0.5in">
1) The Lustre Operations Manual doesn’t make this clear at <a href="https://doc.lustre.org/lustre_manual.xhtml#flr.operations.resyncmirror" target="_blank">https://doc.lustre.org/lustre_manual.xhtml#flr.operations.resyncmirror</a> , but a mirror sync will be completed in the context of the client that it was executed on.
<u></u><u></u></p>
<p class="MsoNormal" style="margin-left:0.5in">
<u></u><u></u></p>
<p class="MsoNormal" style="margin-left:0.5in">
2) Mirror resync is entirely manual today. Again, it is done in the context of the client from which the lfs command was executed. Automatic or “Immediate” mirror is on the
<a href="https://www.lustre.org/roadmap/" target="_blank">https://www.lustre.org/roadmap/</a> for 2.16.0. However, as a side note, there are also plans to “externalize” and leverage the HSM coordinator to be able to hand-off the data movement to multiple data movers. See LU-6081, LU-12890, and LU-13384 for an outline
of that work. I only mention it here because you’ve hinted at the need for this improvement, but it is not committed to the roadmap yet.<u></u><u></u></p>
<p class="MsoNormal" style="margin-left:0.5in">
<u></u><u></u></p>
<p class="MsoNormal" style="margin-left:0.5in">
3) One way could use lfs find to find files needing a mirror update. There are multiple options to zero in on the mirror count and/or state. Consult the lfs-find man page for details.<u></u><u></u></p>
<p class="MsoNormal" style="margin-left:0.5in">
<u></u><u></u></p>
<p class="MsoNormal" style="margin-left:0.5in">
-Cory<u></u><u></u></p>
<p class="MsoNormal" style="margin-left:0.5in">
<u></u><u></u></p>
<div>
<p class="MsoNormal" style="margin-left:1in">
On 2/17/21, 6:43 AM, "lustre-discuss" <<a href="mailto:lustre-discuss-bounces@lists.lustre.org" target="_blank">lustre-discuss-bounces@lists.lustre.org</a>> wrote:<u></u><u></u></p>
</div>
<p class="MsoNormal" style="margin-left:1in">
<u></u><u></u></p>
<div>
<p class="MsoNormal" style="margin-left:1in">
Hi All,<u></u><u></u></p>
<div>
<p class="MsoNormal" style="margin-left:1in">
<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:1in">
I was going through the FLR feature in the Lustre Manual and had some questions.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:1in">
<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:1in">
1. If we run the "lfs mirror" command on an existing file or directory, how is the file ' directory copied from one OST to another?<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:1in">
Is the copying done by the Lustre client on which this command was run?<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:1in">
<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:1in">
2. And, in case of delayed mirroring of newly created files, who does the copying?<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:1in">
<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:1in">
3. How can we automate this process to automatically mirror certain file types?<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:1in">
<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:1in">
Regards,<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:1in">
<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:1in">
<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:1in">
Indivar Nair<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:1in">
<u></u><u></u></p>
</div>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</blockquote></div>