<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Title" content="">
<meta name="Keywords" content="">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:Calibri;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal;
font-family:Calibri;
color:windowtext;}
span.EmailStyle18
{mso-style-type:personal;
font-family:Calibri;
color:#1F497D;}
span.EmailStyle19
{mso-style-type:personal-reply;
font-family:Calibri;
color:windowtext;}
span.msoIns
{mso-style-type:export-only;
mso-style-name:"";
text-decoration:underline;
color:teal;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style>
</head>
<body bgcolor="white" lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal">There isn't Data-on-MDT (DoM) functionality available yet, so there is no data being stored on the MDT. With DNE1 there is no automatic distribution of files between MDTs, so unless you have explicitly created subdirectories on the second
MDT with "lfs mkdir -i 1" there will not be any files created there. DNE1 is intended to create some high-level directories (e.g. per user or per project) on the second MDT, and files created in those directories will be created on the second MDT.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;color:black">Cheers, Andreas<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;color:black">-- <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;color:black">Andreas Dilger<o:p></o:p></span></p>
</div>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;color:black">Lustre Principal Architect<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><span style="font-size:10.5pt;color:black">Intel High Performance Data Division</span><o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal">On 2016/07/27, 17:53, "lustre-discuss on behalf of Andrus, Brian Contractor" <<a href="mailto:lustre-discuss-bounces@lists.lustre.org">lustre-discuss-bounces@lists.lustre.org</a> on behalf of
<a href="mailto:bdandrus@nps.edu">bdandrus@nps.edu</a>> wrote:<span style="font-size:12.0pt"><o:p></o:p></span></p>
</div>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<blockquote style="border:none;border-left:solid #B5C4DF 4.5pt;padding:0in 0in 0in 4.0pt;margin-left:3.75pt;margin-right:0in" id="MAC_OUTLOOK_ATTRIBUTION_BLOCKQUOTE">
<div>
<div>
<p class="MsoNormal"><span style="color:#1F497D">Rick,</span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">Thanks for the info. I added more to the ZFS pool and it was immediately available and all was happy.</span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">Now I need to figure out how/why the MDT got so full.</span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">I have 2 MDTs and the other was barely used. Both were 100GB, which I would have thought was fine. I suspect I have a user that put many small files on the filesystem and they all landed on the MDT for storage.</span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal">Brian Andrus<o:p></o:p></p>
<p class="MsoNormal">ITACS/Research Computing<o:p></o:p></p>
<p class="MsoNormal">Naval Postgraduate School<o:p></o:p></p>
<p class="MsoNormal">Monterey, California<o:p></o:p></p>
<p class="MsoNormal">voice: 831-656-6238<o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span><o:p></o:p></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b>From:</b> Rick Wagner [mailto:rpwagner@sdsc.edu] <br>
<b>Sent:</b> Tuesday, July 26, 2016 5:58 PM<br>
<b>To:</b> Andrus, Brian Contractor <bdandrus@nps.edu><br>
<b>Cc:</b> lustre-discuss@lists.lustre.org; Cai, Haisong <cai@sdsc.edu><br>
<b>Subject:</b> Re: [lustre-discuss] MDT 100% full<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"> <o:p></o:p></p>
<div>
<p class="MsoNormal">Hi Brian,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><br>
On Jul 26, 2016, at 5:45 PM, Andrus, Brian Contractor <<a href="mailto:bdandrus@nps.edu">bdandrus@nps.edu</a>> wrote:<o:p></o:p></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal">All,<o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal">Ok, I thought 100GB would be sufficient for an MDT.<o:p></o:p></p>
<p class="MsoNormal">I have 2 MDTs as well, BUT…<o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal">MDT0 is 100% full and now I cannot write anything to my lustre filesystem.<o:p></o:p></p>
<p class="MsoNormal">The MDT is on a ZFS backing filesystem.<o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal">So, what is the proper way to grow my MDT using ZFS? Do I need to shut the filesystem down completely? Can I just add a disk or space to the pool and Lustre will see it?<o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal">Any advice or direction is appreciated.<o:p></o:p></p>
</div>
</blockquote>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Times New Roman""> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Times New Roman"">We just did this successfully on the two MDTs backing one of our Lustre file systems and everything happened at the ZFS layer. We added drives to the pool and Lustre immediately
saw the additional capacity. Whether you take down the file system or do it live is a question of your architecture, skills, and confidence. Having test file system is also worthwhile to go over the steps.</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Times New Roman""> </span><o:p></o:p></p>
</div>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Times New Roman"">--Rick<br>
<br>
<br>
<br>
</span><o:p></o:p></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal">Brian Andrus<o:p></o:p></p>
<p class="MsoNormal">ITACS/Research Computing<o:p></o:p></p>
<p class="MsoNormal">Naval Postgraduate School<o:p></o:p></p>
<p class="MsoNormal">Monterey, California<o:p></o:p></p>
<p class="MsoNormal">voice: 831-656-6238<o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Times New Roman"">_______________________________________________<br>
lustre-discuss mailing list<br>
<a href="mailto:lustre-discuss@lists.lustre.org">lustre-discuss@lists.lustre.org</a><br>
<a href="http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org">http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org</a></span><o:p></o:p></p>
</div>
</blockquote>
</div>
</div>
</blockquote>
</div>
</body>
</html>