<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body dir="auto">
Hi Jan,
<div>There is "lctl --device XXXX recover" that will trigger a reconnect to the named OST device (per "lctl dl" output), but not sure if that will help. </div>
<div><br>
</div>
<div><br id="lineBreakAtBeginningOfSignature">
<div dir="ltr">Cheers, Andreas</div>
<div dir="ltr"><br>
<blockquote type="cite">On Aug 22, 2024, at 06:36, Haarst, Jan van via lustre-discuss <lustre-discuss@lists.lustre.org> wrote:<br>
<br>
</blockquote>
</div>
<blockquote type="cite">
<div dir="ltr">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style>@font-face { font-family: "Cambria Math"; }
@font-face { font-family: Calibri; }
@font-face { font-family: Aptos; }
@font-face { font-family: Consolas; }
p.MsoNormal, li.MsoNormal, div.MsoNormal { margin: 0cm; font-size: 11pt; font-family: Aptos, sans-serif; }
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph { margin: 0cm 0cm 0cm 36pt; font-size: 11pt; font-family: Aptos, sans-serif; }
span.EmailStyle17 { font-family: Aptos, sans-serif; color: windowtext; }
.MsoChpDefault { font-size: 11pt; }
@page WordSection1 { size: 612pt 792pt; margin: 72pt; }
div.WordSection1 { page: WordSection1; }
ol { margin-bottom: 0cm; }
ul { margin-bottom: 0cm; }</style>
<div class="WordSection1">
<p class="MsoNormal"><span lang="NL">Hi, <o:p></o:p></span></p>
<p class="MsoNormal"><span lang="NL"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Probably the wording of the subject doesn’t actually cover the issue, what we see is this :<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">We have a client behind a router (linking tcp to Omnipath) that shows an inactive OST (all on 2.15.5).<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Other clients that go through the router do not have this issue.
<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">One client had the same issue, although it showed a different OST as inactive.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">After a reboot, all was well again on that machine.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">The clients can lctl ping the OSSs.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">So although we have a workaround (reboot the client), it would be nice to:<o:p></o:p></span></p>
<ol style="margin-top:0cm" start="1" type="1">
<li class="MsoListParagraph" style="margin-left:0cm;mso-list:l0 level1 lfo1"><span lang="EN-US">Fix the issue without a reboot<o:p></o:p></span></li><li class="MsoListParagraph" style="margin-left:0cm;mso-list:l0 level1 lfo1"><span lang="EN-US">Fix the underlying issue.<o:p></o:p></span></li></ol>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">It might be unrelated, but we also see another routing issue every now and then:<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">The router stops routing request toward a certain OSS, and this can be fixed by deleting the peer_nid of the OSS from the router.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">I am probably missing informative logs, but I’m more than happy to try to generate them, if somebody has a pointer to how.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">We are a bit stumped right now.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">With kind regards,<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<div>
<div>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:Consolas;color:black;border:none windowtext 1.0pt;padding:0cm;mso-ligatures:none;mso-fareast-language:EN-GB">-- </span><span style="font-family:"Calibri",sans-serif;color:black;mso-ligatures:none;mso-fareast-language:EN-GB"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:Consolas;color:black;border:none windowtext 1.0pt;padding:0cm;mso-ligatures:none;mso-fareast-language:EN-GB">Jan van Haarst</span><span style="font-family:"Calibri",sans-serif;color:black;mso-ligatures:none;mso-fareast-language:EN-GB"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="NL" style="font-size:9.0pt;font-family:Consolas;color:black;border:none windowtext 1.0pt;padding:0cm;mso-ligatures:none;mso-fareast-language:EN-GB">HPC
</span><span style="font-size:9.0pt;font-family:Consolas;color:black;border:none windowtext 1.0pt;padding:0cm;mso-ligatures:none;mso-fareast-language:EN-GB">Administrator</span><span style="font-family:"Calibri",sans-serif;color:black;mso-ligatures:none;mso-fareast-language:EN-GB"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:Consolas;color:black;border:none windowtext 1.0pt;padding:0cm;mso-ligatures:none;mso-fareast-language:EN-GB">For Anunna/HPC questions, please use <a href="https://support.wur.nl"><span style="color:#0563C1">https://support.wur.nl</span></a> (with
HPC as service)</span><span style="font-family:"Calibri",sans-serif;color:black;mso-ligatures:none;mso-fareast-language:EN-GB"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="NL" style="font-size:9.0pt;font-family:Consolas;color:black;border:none windowtext 1.0pt;padding:0cm;mso-ligatures:none;mso-fareast-language:EN-GB">Aanwezig: maandag, dinsdag, donderdag & vrijdag </span><span style="font-family:"Calibri",sans-serif;color:black;mso-ligatures:none;mso-fareast-language:EN-GB"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="NL" style="font-size:9.0pt;font-family:Consolas;color:black;border:none windowtext 1.0pt;padding:0cm;mso-ligatures:none;mso-fareast-language:EN-GB">Facilitair Bedrijf, onderdeel van Wageningen University & Research </span><span style="font-family:"Calibri",sans-serif;color:black;mso-ligatures:none;mso-fareast-language:EN-GB"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="NL" style="font-size:9.0pt;font-family:Consolas;color:black;border:none windowtext 1.0pt;padding:0cm;mso-ligatures:none;mso-fareast-language:EN-GB">Afdeling Informatie Technologie </span><span style="font-family:"Calibri",sans-serif;color:black;mso-ligatures:none;mso-fareast-language:EN-GB"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="NL" style="font-size:9.0pt;font-family:Consolas;color:black;border:none windowtext 1.0pt;padding:0cm;mso-ligatures:none;mso-fareast-language:EN-GB">Postbus 59, 6700 AB, Wageningen </span><span style="font-family:"Calibri",sans-serif;color:black;mso-ligatures:none;mso-fareast-language:EN-GB"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="NL" style="font-size:9.0pt;font-family:Consolas;color:black;border:none windowtext 1.0pt;padding:0cm;mso-ligatures:none;mso-fareast-language:EN-GB">Gebouw 116, Akkermaalsbos 12, 6700 WB, Wageningen </span><span style="font-family:"Calibri",sans-serif;color:black;mso-ligatures:none;mso-fareast-language:EN-GB"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:Consolas;color:black;border:none windowtext 1.0pt;padding:0cm;mso-ligatures:none;mso-fareast-language:EN-GB"><a href="http://www.wur.nl/nl/Disclaimer.htm" title="http://www.wur.nl/nl/Disclaimer.htm"><span style="color:#0563C1">http://www.wur.nl/nl/Disclaimer.htm</span></a></span><span style="font-family:"Calibri",sans-serif;color:black;mso-ligatures:none;mso-fareast-language:EN-GB"><o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<span>_______________________________________________</span><br>
<span>lustre-discuss mailing list</span><br>
<span>lustre-discuss@lists.lustre.org</span><br>
<span>http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org</span><br>
</div>
</blockquote>
</div>
</body>
</html>