<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">
<div style="font-size:15px;font-family:"Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, BlinkMacSystemFont, Roboto, "Helvetica Neue", sans-serif;margin:0px;color:rgb(36, 36, 36);background-color:rgb(255, 255, 255)">
<p style="margin:0cm;font-size:11pt;font-family:Calibri, sans-serif"><span style="font-family: "inherit", serif; margin: 0px; color: black;" class="ContentPasted0">Marc,</span></p>
<p style="margin:0cm;font-size:11pt;font-family:Calibri, sans-serif"><span style="font-family: "inherit", serif; margin: 0px; color: black;" class="ContentPasted0"><br>
</span></p>
<p style="margin:0cm;font-size:11pt;font-family:Calibri, sans-serif"><span style="font-family: "inherit", serif; margin: 0px; color: black;" class="ContentPasted0">[Re-posting to the list...]</span></p>
</div>
<div style="font-size:15px;font-family:"Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, BlinkMacSystemFont, Roboto, "Helvetica Neue", sans-serif;margin:0px;color:rgb(36, 36, 36);background-color:rgb(255, 255, 255)">
<p style="margin:0cm;font-size:11pt;font-family:Calibri, sans-serif"><span style="font-family: "inherit", serif; margin: 0px; color: black;" class="ContentPasted0"> </span></p>
</div>
<div style="font-size:15px;font-family:"Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, BlinkMacSystemFont, Roboto, "Helvetica Neue", sans-serif;margin:0px;color:rgb(36, 36, 36);background-color:rgb(255, 255, 255)">
<p style="margin:0cm;font-size:11pt;font-family:Calibri, sans-serif"><span style="font-family: "inherit", serif; margin: 0px; color: black;" class="ContentPasted0">No, it’s fine to have interaction during those times. The system is designed to do that work
online. Depending what you’re trying to do and what you’re accessing, some client operations will experience delays, but that’s it. For example, during failover/recovery for a particular OST or MDT, no new IO to that target will complete. But the user programs
will just wait - it’s safe to leave them running.</span></p>
</div>
<div style="font-size:15px;font-family:"Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, BlinkMacSystemFont, Roboto, "Helvetica Neue", sans-serif;margin:0px;color:rgb(36, 36, 36);background-color:rgb(255, 255, 255)">
<p style="margin:0cm;font-size:11pt;font-family:Calibri, sans-serif"><span style="font-family: "inherit", serif; margin: 0px; color: black;" class="ContentPasted0"> </span></p>
</div>
<div style="font-size:15px;font-family:"Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, BlinkMacSystemFont, Roboto, "Helvetica Neue", sans-serif;margin:0px;color:rgb(36, 36, 36);background-color:rgb(255, 255, 255)">
<p style="margin:0cm;font-size:11pt;font-family:Calibri, sans-serif"><span style="font-family: "inherit", serif; margin: 0px; color: black;" class="ContentPasted0">So recovery, etc, will show up to users as delays in some requests, but it’s safe to do with
users accessing the system.</span></p>
</div>
<div style="font-size:15px;font-family:"Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, BlinkMacSystemFont, Roboto, "Helvetica Neue", sans-serif;margin:0px;color:rgb(36, 36, 36);background-color:rgb(255, 255, 255)">
<p style="margin:0cm;font-size:11pt;font-family:Calibri, sans-serif"><span style="font-family: "inherit", serif; margin: 0px; color: black;" class="ContentPasted0"> </span></p>
</div>
<div style="font-size:15px;font-family:"Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, BlinkMacSystemFont, Roboto, "Helvetica Neue", sans-serif;margin:0px;color:rgb(36, 36, 36);background-color:rgb(255, 255, 255)">
<p style="margin:0cm;font-size:11pt;font-family:Calibri, sans-serif"><span style="font-family: "inherit", serif; margin: 0px; color: black;" class="ContentPasted0">Regards,</span></p>
</div>
<div style="font-size:15px;font-family:"Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, BlinkMacSystemFont, Roboto, "Helvetica Neue", sans-serif;margin:0px;color:rgb(36, 36, 36);background-color:rgb(255, 255, 255)">
<p style="margin:0cm;font-size:11pt;font-family:Calibri, sans-serif"><span style="font-family: "inherit", serif; margin: 0px; color: black;" class="ContentPasted0">Patrick</span></p>
</div>
</div>
<div id="appendonsend"></div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> lustre-discuss <lustre-discuss-bounces@lists.lustre.org> on behalf of Marc O'Brien via lustre-discuss <lustre-discuss@lists.lustre.org><br>
<b>Sent:</b> Tuesday, March 14, 2023 7:24 AM<br>
<b>To:</b> lustre-discuss@lists.lustre.org <lustre-discuss@lists.lustre.org><br>
<b>Subject:</b> [lustre-discuss] Question regarding user access during recovery and journal replay</font>
<div> </div>
</div>
<style>
<!--
@font-face
{font-family:"Cambria Math"}
@font-face
{font-family:Calibri}
p.x_MsoNormal, li.x_MsoNormal, div.x_MsoNormal
{margin:0cm;
font-size:12.0pt;
font-family:"Calibri",sans-serif}
span.x_EmailStyle17
{font-family:"Calibri",sans-serif;
color:windowtext}
.x_MsoChpDefault
{font-size:12.0pt;
font-family:"Calibri",sans-serif}
.x_MsoPapDefault
{}
@page WordSection1
{margin:72.0pt 72.0pt 72.0pt 72.0pt}
div.x_WordSection1
{}
-->
</style>
<div lang="EN-GB" link="#0563C1" vlink="#954F72" style="word-wrap:break-word">
<div class="x_WordSection1">
<p class="x_MsoNormal"><span style="font-size:11.0pt; color:black">Hi,</span><span style="font-size:11.0pt"></span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt; color:black">When I was first taught some Lustre file system administration, it was stressed that when recovering a Lustre file system and while the journal replay was occurring on each host, there should
be no user interaction with the file system. Any recovery was done with cluster access denied to HPC users, or when the cluster was deemed to be quiescent. This seemed to make sense as during journal replay the file system is in R/W state, but the distributed
file system may not have reached a stable state. We now have multiple Lustre file systems (2 Ext4 based and 1 ZFS based) and evicting users or finding a quiescent time is problematic (luckily there are maintenance windows for the routine stuff).</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt; color:black">I have searched online and have yet to see in print that there should be no user interaction with Lustre during recovery or journal replay (I may have missed it).</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt; color:black">So, my question is, is the no cluster user interaction during recovery and journal replay restriction, actually a thing?</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt; color:black">Thanks in advance for any enlightenment :)</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt; color:black">Marc</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt"> </span></p>
</div>
</div>
</body>
</html>