<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Title" content="">
<meta name="Keywords" content="">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
p
{mso-style-priority:99;
margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman";}
p.MsoQuote, li.MsoQuote, div.MsoQuote
{mso-style-priority:29;
mso-style-link:"Quote Char";
margin-top:10.0pt;
margin-right:.6in;
margin-bottom:8.0pt;
margin-left:.6in;
text-align:center;
font-size:12.0pt;
font-family:"Times New Roman";
color:#404040;
font-style:italic;}
span.QuoteChar
{mso-style-name:"Quote Char";
mso-style-priority:29;
mso-style-link:Quote;
font-family:Calibri;
color:#404040;
font-style:italic;}
span.quote1
{mso-style-name:quote1;}
span.EmailStyle21
{mso-style-type:personal-reply;
font-family:Calibri;
color:windowtext;}
span.msoIns
{mso-style-type:export-only;
mso-style-name:"";
text-decoration:underline;
color:teal;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style>
</head>
<body bgcolor="white" lang="EN-US" link="#0563C1" vlink="#954F72">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri">Craig, FWIW, this sounds a lot like
<a href="https://jira.hpdd.intel.com/browse/LU-5038">https://jira.hpdd.intel.com/browse/LU-5038</a>, which was addressed in 2.7.0.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri">-Cory<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri"><o:p> </o:p></span></p>
<div>
<div>
<div>
<p class="MsoNormal">-- <o:p></o:p></p>
</div>
</div>
</div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri"><o:p> </o:p></span></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-family:Calibri;color:black">From: </span>
</b><span style="font-family:Calibri;color:black">lustre-discuss <lustre-discuss-bounces@lists.lustre.org> on behalf of "Prescott,Craig P" <prescott@rc.ufl.edu><br>
<b>Date: </b>Monday, December 5, 2016 at 3:02 PM<br>
<b>To: </b>"lustre-discuss@lists.lustre.org" <lustre-discuss@lists.lustre.org><br>
<b>Subject: </b>[lustre-discuss] Changelog record cleanup in /O/1/d*<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p><o:p> </o:p></p>
<p>We were running 2.5.3.90 with changelogs enabled earlier this summer. We ran into a catalog corruption issue (<span class="quote1">LU-6556) - we decided to deregister our changelog users, move the
</span>CONFIGS/changelog_{catalog,users} files out of the way, and carry on until we had an opportunity to upgrade. We did not remove anything from /O/1/d* at that time (though we probably should have).<o:p></o:p></p>
<p><o:p> </o:p></p>
<p>We've observed that mounting our MDT can take several-to-many minutes - I can see with iostat that the MDT is very busy with reads while it is being mounted. I suspect that those stale files in /O/1/d* are the reason (there are lots of them), as they are
processed by the OSP sync at MDT startup. I looked with debugfs at the /O/1/d* directories - there are 1000s of files and their timestamps are consistent with when we were using changelogs. I dumped a few randomly selected ones and checked with llog_reader
that the types of records they contain are CHANGELOG_REC (type=10660000). <o:p></o:p></p>
<p><o:p> </o:p></p>
<p>At the least, I think we should to remove the files in /O/1/d* that contain CHANGELOG_REC entries. Can I just delete every file in /O/1/d*, or do I need to be careful and only remove the CHANGELOG_REC entries?
<o:p></o:p></p>
<p><o:p> </o:p></p>
<p>The reason I ask is that I do see a handful of files that are not changelog-related in these directories - their timestamps are newer and their record type as reported by llog_reader is not CHANGELOG_REC or CHANGELOG_USER. There are only a small number
of such files, though.<o:p></o:p></p>
<p><o:p> </o:p></p>
<p>Thanks,<o:p></o:p></p>
<p>Craig Prescott<o:p></o:p></p>
<p class="MsoNormal">University of Florida Research Computing <o:p></o:p></p>
</div>
</body>
</html>