<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
<style type="text/css" style="display:none;"><!-- P {margin-top:0;margin-bottom:0;} --></style>
</head>
<body dir="ltr">
<div id="divtagdefaultwrapper" style="font-size:12pt;color:#000000;font-family:Calibri,Helvetica,sans-serif;" dir="ltr">
<p style="margin-top:0;margin-bottom:0">Sorry for the spam, but here are a few more interesting results:</p>
<p style="margin-top:0;margin-bottom:0"><br>
</p>
<p style="margin-top:0;margin-bottom:0">If I create a file that stripes only on the full OST, and then disable the OST, I get the following:</p>
<p style="margin-top:0;margin-bottom:0"><br>
</p>
<p style="margin-top:0;margin-bottom:0"></p>
<ol style="margin-bottom: 0px; margin-top: 0px;">
<li><span style="font-size: 12pt;"></span>I can over write the file within it's original size and it takes up space on the "disabled" OST.</li><li>I can zero the file.</li><li>I can write more data to the file than it originally had, ie. the original file before disabling the OST was 1G, I can overwrite the file with > 1G with the OST disabled.</li><li>If I create a new file asking for that OST with the OST disabled, I get a different OST </li></ol>
<br>
<p></p>
<p style="margin-top:0;margin-bottom:0"># 4 and #2 are the only expected behavior. I'm not sure what the behavior should be in the case of #1 and #3.</p>
<p style="margin-top:0;margin-bottom:0"><br>
</p>
<p style="margin-top:0;margin-bottom:0"><br>
</p>
<div id="Signature">
<div id="divtagdefaultwrapper" dir="ltr" style="font-size: 12pt; color: rgb(0, 0, 0); font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols;">
<p style="margin-top:0; margin-bottom:0"><span id="ms-rterangepaste-start"></span></p>
<div style="color:rgb(33,33,33); font-family:wf_segoe-ui_normal,"Segoe UI","Segoe WP",Tahoma,Arial,sans-serif,serif,EmojiFont; font-size:15px; margin:0px">
<font face="Calibri,sans-serif" size="2"><span style="font-size:11pt">--</span></font></div>
<div style="color:rgb(33,33,33); font-family:wf_segoe-ui_normal,"Segoe UI","Segoe WP",Tahoma,Arial,sans-serif,serif,EmojiFont; font-size:15px; margin:0px">
<font face="Calibri,sans-serif" size="2"><span style="font-size:11pt">Jason Williams</span></font></div>
<div style="color:rgb(33,33,33); font-family:wf_segoe-ui_normal,"Segoe UI","Segoe WP",Tahoma,Arial,sans-serif,serif,EmojiFont; font-size:15px; margin:0px">
<font face="Calibri,sans-serif" size="2"><span style="font-size:11pt">Assistant Director</span></font></div>
<div style="color:rgb(33,33,33); font-family:wf_segoe-ui_normal,"Segoe UI","Segoe WP",Tahoma,Arial,sans-serif,serif,EmojiFont; font-size:15px; margin:0px">
<font face="Calibri,sans-serif" size="2"><span style="font-size:11pt">Systems and Data Center Operations.<br>
Maryland Advanced Research Computing Center (MARCC)<br>
Johns Hopkins University</span></font></div>
<div style="color:rgb(33,33,33); font-family:wf_segoe-ui_normal,"Segoe UI","Segoe WP",Tahoma,Arial,sans-serif,serif,EmojiFont; font-size:15px; margin:0px">
<font face="Calibri,sans-serif" size="2"><span style="font-size:11pt"><a href="mailto:jasonw@jhu.edu" target="_blank" rel="noopener noreferrer" id="LPNoLP"><font color="#0563C1">jasonw@jhu.edu</font></a></span></font></div>
<span id="ms-rterangepaste-end"></span><br>
<p></p>
</div>
</div>
</div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> lustre-discuss <lustre-discuss-bounces@lists.lustre.org> on behalf of Jason Williams <jasonw@jhu.edu><br>
<b>Sent:</b> Monday, January 7, 2019 11:47:09 AM<br>
<b>To:</b> Mohr Jr, Richard Frank (Rick Mohr)<br>
<b>Cc:</b> lustre-discuss@lists.lustre.org<br>
<b>Subject:</b> Re: [lustre-discuss] Full OST, any way of avoiding it without hanging?</font>
<div> </div>
</div>
<style type="text/css" style="display:none">
<!--
p
{margin-top:0;
margin-bottom:0}
-->
</style>
<div dir="ltr">
<div id="x_divtagdefaultwrapper" dir="ltr" style="font-size:12pt; color:#000000; font-family:Calibri,Helvetica,sans-serif">
<p style="margin-top:0; margin-bottom:0">So I found this: <a href="http://wiki.lustre.org/Handling_Full_OSTs" class="x_OWAAutoLink" id="LPlnk558116">http://wiki.lustre.org/Handling_Full_OSTs</a> which is what I thought I had followed before but ran into hang
issues. I did some quick testing with this and found that:</p>
<p style="margin-top:0; margin-bottom:0"><br>
</p>
<p style="margin-top:0; margin-bottom:0">1. if I deactivate the OST in the MDS, no new files appear to be created on that OST (expected behavior) and no hangs.</p>
<p style="margin-top:0; margin-bottom:0">2. If I first create a file on the OST with it activated, then deactivate the OST, and OVERWRITE a file what was spanned on that OST, the indexes stay the same and the file successfully overwrites (the file spanned 4
OSTs, so perhaps a little more testing with a single OST in the index is necessary) </p>
<p style="margin-top:0; margin-bottom:0">3. Deactivating the OST shows it as inactive in the MDS but UP in the Client. (not expected.)</p>
<p style="margin-top:0; margin-bottom:0">4. I am able to delete a file that spans that OST with the OST deactivated, no hang.</p>
<div><br>
</div>
I think the only thing here that concerns me a bit is #2.<br>
<p style="margin-top:0; margin-bottom:0"><br>
</p>
<div id="x_Signature">
<div id="x_divtagdefaultwrapper" dir="ltr" style="font-size:12pt; color:rgb(0,0,0); font-family:Calibri,Helvetica,sans-serif,EmojiFont,"Apple Color Emoji","Segoe UI Emoji",NotoColorEmoji,"Segoe UI Symbol","Android Emoji",EmojiSymbols">
<p style="margin-top:0; margin-bottom:0"><span id="x_ms-rterangepaste-start"></span></p>
<div style="color:rgb(33,33,33); font-family:wf_segoe-ui_normal,"Segoe UI","Segoe WP",Tahoma,Arial,sans-serif,serif,EmojiFont; font-size:15px; margin:0px">
<font face="Calibri,sans-serif" size="2"><span style="font-size:11pt">--</span></font></div>
<div style="color:rgb(33,33,33); font-family:wf_segoe-ui_normal,"Segoe UI","Segoe WP",Tahoma,Arial,sans-serif,serif,EmojiFont; font-size:15px; margin:0px">
<font face="Calibri,sans-serif" size="2"><span style="font-size:11pt">Jason Williams</span></font></div>
<div style="color:rgb(33,33,33); font-family:wf_segoe-ui_normal,"Segoe UI","Segoe WP",Tahoma,Arial,sans-serif,serif,EmojiFont; font-size:15px; margin:0px">
<font face="Calibri,sans-serif" size="2"><span style="font-size:11pt">Assistant Director</span></font></div>
<div style="color:rgb(33,33,33); font-family:wf_segoe-ui_normal,"Segoe UI","Segoe WP",Tahoma,Arial,sans-serif,serif,EmojiFont; font-size:15px; margin:0px">
<font face="Calibri,sans-serif" size="2"><span style="font-size:11pt">Systems and Data Center Operations.<br>
Maryland Advanced Research Computing Center (MARCC)<br>
Johns Hopkins University</span></font></div>
<div style="color:rgb(33,33,33); font-family:wf_segoe-ui_normal,"Segoe UI","Segoe WP",Tahoma,Arial,sans-serif,serif,EmojiFont; font-size:15px; margin:0px">
<font face="Calibri,sans-serif" size="2"><span style="font-size:11pt"><a href="mailto:jasonw@jhu.edu" target="_blank" rel="noopener noreferrer" id="LPNoLP"><font color="#0563C1">jasonw@jhu.edu</font></a></span></font></div>
<span id="x_ms-rterangepaste-end"></span><br>
<p></p>
</div>
</div>
</div>
<hr tabindex="-1" style="display:inline-block; width:98%">
<div id="x_divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>From:</b> lustre-discuss <lustre-discuss-bounces@lists.lustre.org> on behalf of Jason Williams <jasonw@jhu.edu><br>
<b>Sent:</b> Sunday, January 6, 2019 5:22:16 PM<br>
<b>To:</b> Mohr Jr, Richard Frank (Rick Mohr)<br>
<b>Cc:</b> lustre-discuss@lists.lustre.org<br>
<b>Subject:</b> Re: [lustre-discuss] Full OST, any way of avoiding it without hanging?</font>
<div> </div>
</div>
<style id="x_x_ms-outlook-ios-style" type="text/css">
<!--
html
{background-color:transparent}
body
{color:#333;
line-height:150%;
font-family:"-apple-system","HelveticaNeue";
margin:0}
.x_x_ms-outlook-ios-reference-expand
{display:block;
color:#999;
padding:20px 0px;
text-decoration:none}
.x_x_ms-outlook-ios-availability-container
{max-width:500px;
margin:auto;
padding:12px 15px 15px 15px;
border:1px solid #C7E0F4;
border-radius:4px}
.x_x_ms-outlook-ios-availability-container > .x_x_ms-outlook-ios-availability-delete-button
{width:25px;
height:25px;
right:-12px;
top:-12px;
background-image:url("data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAEsAAABLCAYAAAA4TnrqAAAAAXNSR0IArs4c6QAACxpJREFUeAHlnFuMXlUVx/fcOzNtp0ynF0U7hWKrEmKLosZEjUZ9MgZIQBNC0uAtJr745oOJIT74xgskJkQbAlQNJmBMfNDEG0YjEC7GIBQZ6IAI005L79O5+/+dfut0f5dzzt7nu8w37UrWt893zt5rr/U/e6+zL+ucHrcGtLq62q9qd4gnxTeKb6kcc267eEI8Kz4mnhFPi58Rv1g5nunp6VnS8ZVHAqdHPCk+KP6zuBWEHORNinvWNWoYIN4q/o74mLidhHzqob71AxzKij8g/p14LYh6qb97QUM58T7x38TdQOiBPt0FmhQaEf9M3I2EXiNr7tOkBK3pVvGCuJsJ/dCzqVbWWxZxVTygso+InxBz3M2Efuj5SEXvUrqWQloV7lRtT4vfX6rWtS30pqr/uMZp78SqEQ2WgPqQKvmnuKWtaWnFuaWVVbciXl51rk+a9fb2uP6EY80qzL+oHB8RYC8V5vQyRIEloD6tsk965UsdAsyZ+WV35uKyOz+/4uZ0YgmEMqhfyA3397rRoV63eUOf2zzUJxAzMsed/owA+2tokWCwmgVKDcadvLDsZs8vutMXV5zkhepYl08GurENvW5idMCNj/Q5Nb5mKBiwoGoqXe/fZTQCkmNnl9x/Ty+4xZzW0yeLB9WC6Hp0QbLSJRd0sAzSGTSgzO8bG3TbN/W7IGMay/lwSJcslC+gcOZviKN9FC3pjVML7uKi+l0NDakf0Sq2DPe5kYFeh9FZBMgXJOPU3HLSOufpxzW0QTJ2bRlMZNZcCvmLD9slwHKdfraGKi2gAGhKHPXUm1tcdVMn5t05+SWfMGjbaL+7RiABUFkCuHd1I46fX6q7ERvlz/ZsHXLDA7mmNaqap+QeAQZwDSlTooDiGuOouxqWzDjJ3X91dj55slkWWs216io7musqJi5N6Zwz6uJv1XRxnqA3TAwlrTbNHHZwWNnuFmAN+30eWLeqIAO5YHr7zKK63WLqvPFDOzcNuPeODSR+KFhQZEb82/9OL7p3zi6m/k0Gq1sOuPdsjvYet6nsrxup0BAstSrmUqfEQTVxG147seCOn7vcguly+7ZtKNMdGukZdI7uf+T4xaquuW3jgLt+62CM88eILQLsQm2ldY6j0v3uV8YgoBBYC9SYxkI37RzuKFDogZ+iXuo34gaiXwRh9/0VHKqK1bUsZdqnHC9X5cr5Q9ebfveyMnS73eODOSU6c+noyYWkW1ptk9cMxnbJD6p1HbHypFUtq4LmIT9D3jHOHB9l1C1AoQ83DH2M0BN9I+hQbeuqAkuCbhB/KkQg/oGnngQm2Wn63dCifN3Rx7okeqIvegcSOIBHSilYFRQfSK8UHDCOYuIL4cz3ypl3I6EX+kHoi94R9IDfulKwJGBc/KUQQYzMbcDJ8ICnXp8vKURIh/Kg1yX9Lrln9Eb/QAIPcEnIN/FOO5mX0paYwhjhF0qMlq14R1L0q/ZfCy64MzqX4pKAVWlq94ZozqTY5nqMzBlwrgdCT5t/oj92BNK91hWtZe1SwW1FhXFRrB4YMYXJmf9atiRl7vvz52fd4/86GXNXq2TYH1oFch59blZ+yM7mp+iJvkbYkbOYYdlIwQV8HNvo0OcuJfm/9HVbZsFpMtcLpV++MOvuPvyfJPs9n9jufnrnnphRdVoNQH3jsSl36Cl29l0i466b2e0vJvRlSkTLwg7smRi9PIDNkQA+D1nL+nZOxvQSC3dGrB7oZgXTcOWJRAEMxeAIv5HUUwsUJ325SaacH/RFbyPfHjuXkR7kfK/6I03sk/zJI5o7K5xGLLPE0O03jTtalFEsYI2AQt5tkhtDvt7YE9iNPyuckpXsj4VUxnq5CiRZWbiLXY/irtL1ygCWBVSZroze6A9hD3YF0g5KMRcsJDYYjFjhLENlAGslUKazr79vl13PSCeDwWIXxoil4LIUA1g7gEJvX3/frgKbbgSsvQWZkstsVxnFdkErZ2kIYO0CCh18/X27TL+M9BbA2ppxMT0NTravx/TGBndphhIHeYCx8ukPDxDfzHCjVj30xw4Iu7x2UJvV/z/Jc3STf6bRsU2YucZ2VavIAEOejZtIn5w6qxWCubSaVgJlQrFjrjIqxT7W7QsocfCFYPn7dnZHCgQHXzbA/Kdku4FCOd8O374cxXfSDYdzMiSX/GlB8Q0oklZ/HcAevGOPdmSqVeE/5wvveb3IwjO+Hb59OQXHAatuYb62QAnBtSJy/+PMv/WrqaquRwFaGOe53mrCLxoFepZZwDpnhbLSEk02S1TdeXSudeZ+C4sd6ddVkHGC0AAjQgYC6BhgnS3K6Ds/Yg9aRY2Awne9/P39pUb6MXr5dvj25ciYAawTORmSS8wOCPuBcIa28pCcKPmTBRRTGKoqOzUKUQf9zaljV2X2U1R0GrBeKcrFdeKjjIg1aIbygLIOQdouwHz9fbsKbHoGBKr2xrIKEEhmFLmlZMWSNAQoK9AuwHz9fbus3oz0xWCwiLYziljwtyJJGgOUFWwHYL7+RBIGUtINnw3JjFCCLSDio/ymHFK+DFAmt5WAobfFd2GP3wisvox0plcFpnXxtYwM6WlcFqGJRsRHxdATWjO3KQ3lYqcwWYAhN4Z8vbHHc8V5Yv4inJbM+j/l5bRrxHAaEUhGawmlOe+hEAuU1dEIMF+u5ctK0Re9jXx77FxG+hDnqZ8Vw68p+QXHecQ47vm3LqRDh93jQ9qPu7ymnVeWmT2bFqyZs8ScVJxXIOcaRtOiAOqr+ydCW4c2K5bc0ZOXdqRZeThw7Uho8O5ueqCBtVH1E085mqNjcolIu9e9CverwsoQrKjoml5nLP2Cd6Ov040O3J06LsV3CKzVpBvqgClPUJQfUcEWO8Dgjoi79UDoaYNp9MeOQPohQJHXfBbHD/NTRDRFooKN2IeLiEyxYh1N0e9t6WmE/hFu4DEr54P1B50MGs2z4E9UMMS0gdDE5eYG9YmsdvygF/rZxBm9/Q2Lgjp/r+vp4zYFS00Nc39cUDi9TPi0TUDZ4X1FCnUjoZfFZqAvekfQd60LUiYFqyLgUaXTlePchMgUwqclLMl3WvtvhCZ2E6EPekHoib4RET9/V7FXk8KVnyqwJJBByI/8DHnHbCkRPm2E/+oWwGpjStHT3wIznXPSe/xWRb4qsCoFDyl9qnJcmBBnTvi0EYC9NLN2PgwfRf3oYYR+kfHwYFDnvxs+FDRIPaDMfHQiaJbJc7U2vJvH85UWB98QLNnOqP4+Jd/jOJTW+g0Lhgf21MNHdeQNC8ARWAymcHIf5X8osVZ01b27AzgC7Holz4nH+B9KDAKvqrfCDBgB9hUdPy4O8l9WjpRFtqvmfUMzXIB9U8cP2v+YFOcf8yYr227sTLHCwexgXb3JasAIsB/oOHgMZuUsxXha2hX/jrQZ3CxgJoe1LSLuCCSLfvteczuWuANXOK3KrDT4ZXIEZA4dsqRXuuRPdD3ah2XJ5DwAEs1C16MV0hXpksznWgSMXz0j1vZ+18FqE2A4/YfFUU9JK7/G6Zuqv9QXQxpNdwpt0YDvN8p0szhoZ6hQYOcyHFZVvDSe+5Z9W9RRCxsU3ydeEnczteQrRy0BUSgdEP+jS9Hqju9n+UgLKL6l9XXx0S4BrTu/zFYDWr/AOig+skagdf83/3zAOBZQvOryRTEf+Donbid15GuS0eOsWlBC/gsl9iW/LP6C+PPi68TN0usS8EcxH6z4be2qZrPCG5XvCFi1FQu8SZ1j6YdXYeC9YuLxiZyGicQltpuoRPiEmJVLwqPgZwXOtNKO0v8BzRAPSFNM7HEAAAAASUVORK5CYII=");
background-size:25px 25px;
background-position:center}
#x_x_ms-outlook-ios-main-container
{margin:0 0 0 0;
margin-top:120;
padding:8}
#x_x_ms-outlook-ios-content-container
{padding:0;
padding-top:12;
padding-bottom:20}
.x_x_ms-outlook-ios-mention
{color:#333;
background-color:#f1f1f1;
border-radius:4px;
padding:0 2px 0 2px;
text-decoration:none}
.x_x_ms-outlook-ios-mention-external
{color:#ba8f0d;
background-color:#fdf7e7}
.x_x_ms-outlook-ios-mention-external-clear-design
{color:#ba8f0d;
background-color:#f1f1f1}
-->
</style>
<div style="">
<div style="direction:ltr">
<div style="direction:ltr">Hi Rick, </div>
<div style="direction:ltr">I thought what I had done was disable it on the MDS, but perhaps I was following the wrong instructions. Do you know where the best instructions for what you are describing can be found? I would be willing to try again.</div>
<div><br>
</div>
<div class="x_x_ms-outlook-ios-signature">
<div style="direction:ltr">—</div>
<div style="direction:ltr">Sent you tersely from my phone</div>
<div style="direction:ltr">Jason Williams</div>
</div>
</div>
<div>
<hr tabindex="-1" style="display:inline-block; width:98%">
<div id="x_x_divRplyFwdMsg" dir="dir="ltr""><font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>From:</b> Mohr Jr, Richard Frank (Rick Mohr) <rmohr@utk.edu><br>
<b>Sent:</b> Sunday, January 6, 2019 4:56 PM<br>
<b>To:</b> Jason Williams<br>
<b>Cc:</b> lustre-discuss@lists.lustre.org<br>
<b>Subject:</b> Re: [lustre-discuss] Full OST, any way of avoiding it without hanging?
<div> </div>
</font></div>
<br>
<br>
> On Jan 5, 2019, at 9:49 PM, Jason Williams <jasonw@jhu.edu> wrote:<br>
> <br>
> I have looked around the internet and found you can disable an OST, but when I have tried that, any writes (including deletes) to the OST hang the clients indefinitely. Does anyone know a way to make an OST basically "read-only" with the exception of deletes
so we can work to clear out the OST? <br>
<br>
What command did you use to disable the OST?<br>
<br>
There is a way to disable the OST on all the clients, but there is also a way to deactivate it on the MDS. The latter method should prevent the MDS from allocating any new files to the OST, but still allow clients to read and delete files on that OST.<br>
<br>
--<br>
Rick Mohr<br>
Senior HPC System Administrator<br>
National Institute for Computational Sciences<br>
http://www.nics.tennessee.edu<br>
<br>
<br>
</div>
</div>
</div>
</body>
</html>