<div><div dir="auto">Thank you Emoly, This is a really useful feature! </div><div dir="auto"><br></div><div dir="auto">I must have been reading old setstripe doc last week - I appreciate the email!</div></div><div dir="auto"><br></div><div dir="auto">Mark</div><div><br><div class="gmail_quote"><div>On Mon, Jun 25, 2018 at 4:09 AM Emoly Liu <<a href="mailto:emoly@whamcloud.com">emoly@whamcloud.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div lang="EN-US" link="blue" vlink="purple">
<div class="m_4735082373127818205WordSection1">
<p class="MsoNormal">Hi Mark,<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">I just verified the following commands on my 2.11 system. It works.<u></u><u></u></p>
<p class="m_4735082373127818205p1"><span class="m_4735082373127818205s1">[root@centos7-3 tests]# ../utils/lfs setstripe -N2 /mnt/lustre/dir</span><u></u><u></u></p>
<p class="m_4735082373127818205p1"><span class="m_4735082373127818205s1">[root@centos7-3 tests]# touch /mnt/lustre/dir/file</span><u></u><u></u></p>
<p class="m_4735082373127818205p1"><span class="m_4735082373127818205s1">[root@centos7-3 tests]# ../utils/lfs getstripe /mnt/lustre/dir/file</span><span class="m_4735082373127818205apple-converted-space"> </span><u></u><u></u></p>
<p class="m_4735082373127818205p1"><span class="m_4735082373127818205s1">/mnt/lustre/dir/file</span><u></u><u></u></p>
<p class="m_4735082373127818205p1"><span class="m_4735082373127818205apple-converted-space"> </span><span class="m_4735082373127818205s1">lcm_layout_gen:</span><span class="m_4735082373127818205apple-converted-space">
</span><span class="m_4735082373127818205s1">2</span><u></u><u></u></p>
<p class="m_4735082373127818205p1"><span class="m_4735082373127818205apple-converted-space"> </span><span class="m_4735082373127818205s1">lcm_mirror_count:</span><span class="m_4735082373127818205apple-converted-space">
</span><span class="m_4735082373127818205s1">2</span><u></u><u></u></p>
<p class="m_4735082373127818205p1"><span class="m_4735082373127818205apple-converted-space"> </span><span class="m_4735082373127818205s1">lcm_entry_count:
</span><span class="m_4735082373127818205apple-converted-space"> </span><span class="m_4735082373127818205s1">2</span><u></u><u></u></p>
<p class="m_4735082373127818205p1"><span class="m_4735082373127818205apple-converted-space"> </span><span class="m_4735082373127818205s1">lcme_id:
</span><span class="m_4735082373127818205apple-converted-space"> </span><span class="m_4735082373127818205s1">65537</span><u></u><u></u></p>
<p class="m_4735082373127818205p1"><span class="m_4735082373127818205apple-converted-space"> </span><span class="m_4735082373127818205s1">lcme_mirror_id:</span><span class="m_4735082373127818205apple-converted-space">
</span><span class="m_4735082373127818205s1">1</span><u></u><u></u></p>
<p class="m_4735082373127818205p1"><span class="m_4735082373127818205apple-converted-space"> </span><span class="m_4735082373127818205s1">lcme_flags:</span><span class="m_4735082373127818205apple-converted-space">
</span><span class="m_4735082373127818205s1">init</span><u></u><u></u></p>
<p class="m_4735082373127818205p1"><span class="m_4735082373127818205apple-converted-space"> </span><span class="m_4735082373127818205s1">lcme_extent.e_start: 0</span><u></u><u></u></p>
<p class="m_4735082373127818205p1"><span class="m_4735082373127818205apple-converted-space"> </span><span class="m_4735082373127818205s1">lcme_extent.e_end:
</span><span class="m_4735082373127818205apple-converted-space"> </span><span class="m_4735082373127818205s1">EOF</span><u></u><u></u></p>
<p class="m_4735082373127818205p1"><span class="m_4735082373127818205apple-converted-space"> </span><span class="m_4735082373127818205s1">lmm_stripe_count:</span><span class="m_4735082373127818205apple-converted-space">
</span><span class="m_4735082373127818205s1">1</span><u></u><u></u></p>
<p class="m_4735082373127818205p1"><span class="m_4735082373127818205apple-converted-space"> </span><span class="m_4735082373127818205s1">lmm_stripe_size:
</span><span class="m_4735082373127818205apple-converted-space"> </span><span class="m_4735082373127818205s1">1048576</span><u></u><u></u></p>
<p class="m_4735082373127818205p1"><span class="m_4735082373127818205apple-converted-space"> </span><span class="m_4735082373127818205s1">lmm_pattern:
</span><span class="m_4735082373127818205apple-converted-space"> </span><span class="m_4735082373127818205s1">raid0</span><u></u><u></u></p>
<p class="m_4735082373127818205p1"><span class="m_4735082373127818205apple-converted-space"> </span><span class="m_4735082373127818205s1">lmm_layout_gen:</span><span class="m_4735082373127818205apple-converted-space">
</span><span class="m_4735082373127818205s1">0</span><u></u><u></u></p>
<p class="m_4735082373127818205p1"><span class="m_4735082373127818205apple-converted-space"> </span><span class="m_4735082373127818205s1">lmm_stripe_offset: 1</span><u></u><u></u></p>
<p class="m_4735082373127818205p1"><span class="m_4735082373127818205apple-converted-space"> </span><span class="m_4735082373127818205s1">lmm_objects:</span><u></u><u></u></p>
<p class="m_4735082373127818205p1"><span class="m_4735082373127818205apple-converted-space"> </span><span class="m_4735082373127818205s1">- 0: { l_ost_idx: 1, l_fid: [0x100010000:0x6:0x0] }</span><u></u><u></u></p>
<p class="m_4735082373127818205p2"><u></u> <u></u></p>
<p class="m_4735082373127818205p1"><span class="m_4735082373127818205apple-converted-space"> </span><span class="m_4735082373127818205s1">lcme_id:
</span><span class="m_4735082373127818205apple-converted-space"> </span><span class="m_4735082373127818205s1">131074</span><u></u><u></u></p>
<p class="m_4735082373127818205p1"><span class="m_4735082373127818205apple-converted-space"> </span><span class="m_4735082373127818205s1">lcme_mirror_id:</span><span class="m_4735082373127818205apple-converted-space">
</span><span class="m_4735082373127818205s1">2</span><u></u><u></u></p>
<p class="m_4735082373127818205p1"><span class="m_4735082373127818205apple-converted-space"> </span><span class="m_4735082373127818205s1">lcme_flags:</span><span class="m_4735082373127818205apple-converted-space">
</span><span class="m_4735082373127818205s1">init</span><u></u><u></u></p>
<p class="m_4735082373127818205p1"><span class="m_4735082373127818205apple-converted-space"> </span><span class="m_4735082373127818205s1">lcme_extent.e_start: 0</span><u></u><u></u></p>
<p class="m_4735082373127818205p1"><span class="m_4735082373127818205apple-converted-space"> </span><span class="m_4735082373127818205s1">lcme_extent.e_end:
</span><span class="m_4735082373127818205apple-converted-space"> </span><span class="m_4735082373127818205s1">EOF</span><u></u><u></u></p>
<p class="m_4735082373127818205p1"><span class="m_4735082373127818205apple-converted-space"> </span><span class="m_4735082373127818205s1">lmm_stripe_count:</span><span class="m_4735082373127818205apple-converted-space">
</span><span class="m_4735082373127818205s1">1</span><u></u><u></u></p>
<p class="m_4735082373127818205p1"><span class="m_4735082373127818205apple-converted-space"> </span><span class="m_4735082373127818205s1">lmm_stripe_size:
</span><span class="m_4735082373127818205apple-converted-space"> </span><span class="m_4735082373127818205s1">1048576</span><u></u><u></u></p>
<p class="m_4735082373127818205p1"><span class="m_4735082373127818205apple-converted-space"> </span><span class="m_4735082373127818205s1">lmm_pattern:
</span><span class="m_4735082373127818205apple-converted-space"> </span><span class="m_4735082373127818205s1">raid0</span><u></u><u></u></p>
<p class="m_4735082373127818205p1"><span class="m_4735082373127818205apple-converted-space"> </span><span class="m_4735082373127818205s1">lmm_layout_gen:</span><span class="m_4735082373127818205apple-converted-space">
</span><span class="m_4735082373127818205s1">0</span><u></u><u></u></p>
<p class="m_4735082373127818205p1"><span class="m_4735082373127818205apple-converted-space"> </span><span class="m_4735082373127818205s1">lmm_stripe_offset: 0</span><u></u><u></u></p>
<p class="m_4735082373127818205p1"><span class="m_4735082373127818205apple-converted-space"> </span><span class="m_4735082373127818205s1">lmm_objects:</span><u></u><u></u></p>
<p class="m_4735082373127818205p1"><span class="m_4735082373127818205apple-converted-space"> </span><span class="m_4735082373127818205s1">- 0: { l_ost_idx: 0, l_fid: [0x100000000:0x6:0x0] }</span><u></u><u></u></p>
<p class="m_4735082373127818205p2"><u></u> <u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">B.R.,<u></u><u></u></p>
<p class="MsoNormal">Emoly<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<div style="border:none;border-top:solid #b5c4df 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span style="font-size:12.0pt;color:black">From: </span></b><span style="font-size:12.0pt;color:black">Andreas Dilger <<a href="mailto:adilger@whamcloud.com" target="_blank">adilger@whamcloud.com</a>><br>
<b>Date: </b>Monday, June 25, 2018 at 3:59 PM<br>
<b>To: </b>Emoly Liu <<a href="mailto:emoly@whamcloud.com" target="_blank">emoly@whamcloud.com</a>><br>
<b>Subject: </b>Fwd: [lustre-discuss] Lustre 2.11 File Level Replication<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<p class="MsoNormal">Begin forwarded message:<u></u><u></u></p>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><b>From:</b> Patrick Farrell <<a href="mailto:paf@cray.com" target="_blank">paf@cray.com</a>><br>
<b>Date:</b> June 22, 2018 at 21:54:26 GMT+2<br>
<b>To:</b> Mark Roper <<a href="mailto:markroper@gmail.com" target="_blank">markroper@gmail.com</a>><br>
<b>Cc:</b> "<a href="mailto:lustre-discuss@lists.lustre.org" target="_blank">lustre-discuss@lists.lustre.org</a>" <<a href="mailto:lustre-discuss@lists.lustre.org" target="_blank">lustre-discuss@lists.lustre.org</a>></p></div></blockquote></div></div><div lang="EN-US" link="blue" vlink="purple"><div class="m_4735082373127818205WordSection1"><blockquote style="margin-top:5.0pt;margin-bottom:5.0pt"><div><p class="MsoNormal" style="margin-bottom:12.0pt"><br>
<b>Subject:</b> <b>Re: [lustre-discuss] Lustre 2.11 File Level Replication</b><u></u><u></u></p></div></blockquote></div></div><div lang="EN-US" link="blue" vlink="purple"><div class="m_4735082373127818205WordSection1">
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal">Mark,<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">Hmm.<u></u><u></u></p>
<p class="MsoNormal">I\u2019m adding the list back on here, because that *<b>seems</b>* like it\u2019s wrong. Don\u2019t have time to check right now, but I\u2019m curious if others can weigh in.<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<ul style="margin-top:0cm" type="disc">
<li class="m_4735082373127818205MsoListParagraph" style="margin-left:0cm">Patrick<u></u><u></u></li></ul>
<p class="MsoNormal"> <u></u><u></u></p>
<div style="border:none;border-top:solid #b5c4df 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span style="font-size:12.0pt;color:black">From: </span></b><span style="font-size:12.0pt;color:black">Mark Roper <<a href="mailto:markroper@gmail.com" target="_blank">markroper@gmail.com</a>><br>
<b>Date: </b>Friday, June 22, 2018 at 2:29 PM<br>
<b>To: </b>Patrick Farrell <<a href="mailto:paf@cray.com" target="_blank">paf@cray.com</a>><br>
<b>Subject: </b>Re: [lustre-discuss] Lustre 2.11 File Level Replication</span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Thanks Patrick! It looks like you can't set default mirroring with setstripe. I appreciate the lead and the response!<u></u><u></u></p>
</div>
<p class="MsoNormal"> <u></u><u></u></p>
<div>
<div>
<p class="MsoNormal">On Thu, Jun 21, 2018 at 10:09 AM Patrick Farrell <<a href="mailto:paf@cray.com" target="_blank">paf@cray.com</a>> wrote:<u></u><u></u></p>
</div>
<blockquote style="border:none;border-left:solid #cccccc 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt">
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><br>
Mark,<br>
<br>
I haven\u2019t played specifically with FLR and inheritance/templates, but if you want to set a default layout on a directory, you\u2019ll want to look at lfs setstripe. Mirror extend is specifically for modifying individual, existing files.<br>
<br>
- Patrick<u></u><u></u></p>
<div class="MsoNormal" align="center" style="text-align:center">
<hr size="0" width="68%" align="center">
</div>
<div id="m_4735082373127818205m_-185905444424043489divRplyFwdMsg">
<p class="MsoNormal"><b><span style="color:black">From:</span></b><span style="color:black"> lustre-discuss <<a href="mailto:lustre-discuss-bounces@lists.lustre.org" target="_blank">lustre-discuss-bounces@lists.lustre.org</a>> on behalf of Mark Roper <<a href="mailto:markroper@gmail.com" target="_blank">markroper@gmail.com</a>><br>
<b>Sent:</b> Thursday, June 21, 2018 8:36:43 AM<br>
<b>To:</b> <a href="mailto:lustre-discuss@lists.lustre.org" target="_blank">lustre-discuss@lists.lustre.org</a><br>
<b>Subject:</b> [lustre-discuss] Lustre 2.11 File Level Replication</span> <u></u><u></u></p>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;color:#212121">Hi Lustre Users,</span>
<u></u><u></u></p>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;color:#212121"> </span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;color:#212121">I have set up a Lustre 2.11 cluster with multiple OSS's in order to experiment with the File Level Replication feature in Lustre 2.11, which I'm excited about since it raises the bar on availability
for cloud deployment of Lustre. I cannot seem to get files created within a directory to inherit the mirror settings from the parent directory, which I thought might be possible given the documentation at <a href="http://wiki.lustre.org/File_Level_Replication_High_Level_Design" target="_blank">http://wiki.lustre.org/File_Level_Replication_High_Level_Design</a> :</span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;color:#212121"> </span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;color:#212121">'</span><span style="font-size:10.5pt;font-family:"Courier New";color:#212121;background:#f8f9fa">lfs mirror extend [--no-verify] <-N[mirror_count]> [other setstripe options|-f <victim_file>]
<file_name><br>
</span><span style="font-size:10.5pt;color:#222222">This command will append a replica indicated by setstripe options or just take the layout from existing file victim_file into the file file_name. The file_name must be an existing file but it can be a mirrored
or normal file. This command will create new volatile file with any optional setstripe options that are specified, or using the defaults inherited from the parent directory or filesystem.</span><span style="font-size:10.0pt;color:#212121">'</span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;color:#212121"> </span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;color:#212121">I was hoping to be able to make all files written to particular directories or within a filesystem inherit a mirror count transparently to the client writing or reading files. Does anyone know
if this is possible?</span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;color:#212121"> </span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;color:#212121">Cheers!</span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;color:#212121"> </span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;color:#212121">Mark Roper</span><u></u><u></u></p>
</div>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</blockquote>
</div></div><div lang="EN-US" link="blue" vlink="purple"><div class="m_4735082373127818205WordSection1"><blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal">_______________________________________________</p></div></blockquote></div></div><div lang="EN-US" link="blue" vlink="purple"><div class="m_4735082373127818205WordSection1"><blockquote style="margin-top:5.0pt;margin-bottom:5.0pt"><div><p class="MsoNormal"><br>
lustre-discuss mailing list<br>
<a href="mailto:lustre-discuss@lists.lustre.org" target="_blank">lustre-discuss@lists.lustre.org</a><br>
<a href="http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org" target="_blank">http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org</a><u></u><u></u></p></div></blockquote></div></div></blockquote></div></div>