<div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Jul 13, 2017 at 5:49 AM, Bob Ball <span dir="ltr"><<a href="mailto:ball@umich.edu" target="_blank">ball@umich.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF">
On the mgs/mgt do something like:<br>
lctl --device <fsname>-OST0019-osc-MDT0000 deactivate<br>
<br>
No further files will be assigned to that OST. Reverse with
"activate". Or reboot the mgs/mdt as this is not persistent. "lctl
dl" will tell you exactly what that device name should be for you.<span class="HOEnZb"><font color="#888888"><br></font></span></div></blockquote><div>Doesn't that also disable reads from the OST though? <br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div text="#000000" bgcolor="#FFFFFF"><span class="HOEnZb"><font color="#888888">
<br>
bob</font></span><div><div class="h5"><br>
<br>
<div class="m_4682020740092110866moz-cite-prefix">On 7/12/2017 6:04 PM, Alexander I
Kulyavtsev wrote:<br>
</div>
<blockquote type="cite">
<div>You may find advise from Andreas on this list (also
attached below). I did not try setting fail_loc myself.</div>
<div><br>
</div>
<div>In 2.9 there is setting <span style="color:rgb(112,112,112);font-family:monospace;font-size:14px;background-color:rgb(255,255,255)">osp.*.max_create_count=0</span><font face="Arial, sans-serif" color="#707070"><span style="font-size:14px;background-color:rgb(255,255,255)"> </span></font>desc<wbr>ribed at LUDOC-305.</div>
<div><br>
</div>
<div>We used to set OST degraded as described in lustre
manual. </div>
<div>It works most of the time but at some point I saw
lustre errors in logs for some ops. Sorry, I do not recall
details.</div>
<div><br>
</div>
<div>I still not sure either of these approaches will
work for you: setting OST degraded or fail_loc will makes some
osts selected instead of others.</div>
<div>You may want to verify if these settings will
trigger clean error on user side (instead of blocking) when all
OSTs are degraded.</div>
<div><br>
</div>
<div>The other and also simpler approach would be to
enable lustre quota and set quota below used space for all users
(or groups).</div>
<div><br>
</div>
<div>Alex.</div>
<div><br>
<blockquote type="cite">
<div style="margin:0px"><span style="font-family:-webkit-system-font,'Helvetica Neue',Helvetica,sans-serif"><b>From: </b></span><span style="font-family:-webkit-system-font,'Helvetica Neue',Helvetica,sans-serif">"Dilger, Andreas" <<a href="mailto:andreas.dilger@intel.com" target="_blank">andreas.dilger@intel.com</a>><br>
</span></div>
<div style="margin:0px"><span style="font-family:-webkit-system-font,'Helvetica Neue',Helvetica,sans-serif"><b>Subject: </b></span><span style="font-family:-webkit-system-font,'Helvetica Neue',Helvetica,sans-serif"><b>Re:
[lustre-discuss] lustre 2.5.3 ost not draining</b><br>
</span></div>
<div style="margin:0px"><span style="font-family:-webkit-system-font,'Helvetica Neue',Helvetica,sans-serif"><b>Date: </b></span><span style="font-family:-webkit-system-font,'Helvetica Neue',Helvetica,sans-serif">July 28, 2015 at 11:51:38
PM CDT</span></div>
<div style="margin:0px"><span style="font-family:-webkit-system-font,'Helvetica Neue',Helvetica,sans-serif"><b>Cc: </b></span><span style="font-family:-webkit-system-font,'Helvetica Neue',Helvetica,sans-serif">"<a href="mailto:lustre-discuss@lists.lustre.org" target="_blank">lustre-discuss@lists.<wbr>lustre.org</a>"
<<a href="mailto:lustre-discuss@lists.lustre.org" target="_blank">lustre-discuss@lists.lustre.<wbr>org</a>><br>
</span></div>
<br>
<div><span style="float:none;display:inline!important">Setting it degraded means the MDS will
avoid allocations on that OST</span><br>
<span style="float:none;display:inline!important">unless there aren't enough OSTs to meet the
request (e.g. stripe_count =</span><br>
<span style="float:none;display:inline!important">-1), so it should work.</span><br>
<br>
<span style="float:none;display:inline!important">That is actually a very interesting
workaround for this problem, and it</span><br>
<span style="float:none;display:inline!important">will work for older versions of Lustre as
well. It doesn't disable the</span><br>
<span style="float:none;display:inline!important">OST completely, which is fine if you are
doing space balancing (and may</span><br>
<span style="float:none;display:inline!important">even be desirable to allow apps that need
more bandwidth for a widely</span><br>
<span style="float:none;display:inline!important">striped file), but it isn't good if you are
trying to empty the OST</span><br>
<span style="float:none;display:inline!important">completely to remove it.</span><br>
<br>
<span style="float:none;display:inline!important">It looks like another approach would be to
mark the OST as having no free</span><br>
<span style="float:none;display:inline!important">space using OBD_FAIL_OST_ENOINO (0x229) fault
injection on that OST:</span><br>
<br>
<span style="float:none;display:inline!important"> lctl set_param fail_loc=0x229
fail_val=<ost_index></span><br>
<br>
<span style="float:none;display:inline!important">This would cause the OST to return 0 free
inodes from OST_STATFS for the</span><br>
<span style="float:none;display:inline!important">specified OST index, and the MDT would skip
this OST completely. To</span><br>
<span style="float:none;display:inline!important">disable all of the OSTs on an OSS use
<ost_index> = -1. It isn't possible</span><br>
<span style="float:none;display:inline!important">to selectively disable a subset of OSTs using
this method. The</span><br>
<span style="float:none;display:inline!important">OBD_FAIL_OST_ENOINO fail_loc has been
available since Lustre 2.2, which</span><br>
<span style="float:none;display:inline!important">covers all of the 2.4+ versions that are
affected by this issue.</span><br>
<br>
<span style="float:none;display:inline!important">If this mechanism works for you (it should,
as this fail_loc is used</span><br>
<span style="float:none;display:inline!important">during regular testing) I'd be obliged if
someone could file an LUDOC bug</span><br>
<span style="float:none;display:inline!important">so the manual can be updated.</span><br>
<br>
<span style="float:none;display:inline!important">Cheers, Andreas</span></div>
</blockquote>
<br>
</div>
<br>
<div>
<blockquote type="cite">
<div>On Jul 12, 2017, at 4:20 PM, Riccardo Veraldi
<<a href="mailto:Riccardo.Veraldi@cnaf.infn.it" target="_blank">Riccardo.Veraldi@cnaf.infn.it</a><wbr>>
wrote:</div>
<br class="m_4682020740092110866Apple-interchange-newline">
<div>
<div>Hello,<br>
<br>
on one of my lustre FS I need to find a solution so that
users can still<br>
access data on the FS but cannot write new files on it.<br>
I have hundreds of clients accessing the FS so remounting
it ro is not<br>
really easily feasible.<br>
Is there an option on the OSS side to allow OSTs to be
accessed just to<br>
read data and not to store new data ?<br>
tunefs.lustre could do that ?<br>
thank you<br>
<br>
Rick<br>
<br>
______________________________<wbr>_________________<br>
lustre-discuss mailing list<br>
<a href="mailto:lustre-discuss@lists.lustre.org" target="_blank">lustre-discuss@lists.lustre.<wbr>org</a><br>
<a class="m_4682020740092110866moz-txt-link-freetext" href="http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org" target="_blank">http://lists.lustre.org/<wbr>listinfo.cgi/lustre-discuss-<wbr>lustre.org</a><br>
</div>
</div>
</blockquote>
</div>
<br>
<br>
<fieldset class="m_4682020740092110866mimeAttachmentHeader"></fieldset>
<br>
<pre>______________________________<wbr>_________________
lustre-discuss mailing list
<a class="m_4682020740092110866moz-txt-link-abbreviated" href="mailto:lustre-discuss@lists.lustre.org" target="_blank">lustre-discuss@lists.lustre.<wbr>org</a>
<a class="m_4682020740092110866moz-txt-link-freetext" href="http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org" target="_blank">http://lists.lustre.org/<wbr>listinfo.cgi/lustre-discuss-<wbr>lustre.org</a>
</pre>
</blockquote>
<br>
</div></div></div>
<br>______________________________<wbr>_________________<br>
lustre-discuss mailing list<br>
<a href="mailto:lustre-discuss@lists.lustre.org">lustre-discuss@lists.lustre.<wbr>org</a><br>
<a href="http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org" rel="noreferrer" target="_blank">http://lists.lustre.org/<wbr>listinfo.cgi/lustre-discuss-<wbr>lustre.org</a><br>
<br></blockquote></div><br></div></div>