<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Aptos;
panose-1:2 11 0 4 2 2 2 2 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
font-size:11.0pt;
font-family:"Aptos",sans-serif;
mso-ligatures:standardcontextual;
mso-fareast-language:EN-US;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#467886;
text-decoration:underline;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;
mso-ligatures:none;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:70.85pt 70.85pt 2.0cm 70.85pt;}
div.WordSection1
{page:WordSection1;}
--></style>
</head>
<body lang="DE" link="#467886" vlink="#96607D" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:12.0pt;mso-ligatures:none;mso-fareast-language:DE">Hi everyone,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;mso-ligatures:none;mso-fareast-language:DE"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;mso-ligatures:none;mso-fareast-language:DE">we have a problem and I would be very grateful for help or advice.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;mso-ligatures:none;mso-fareast-language:DE"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;mso-ligatures:none;mso-fareast-language:DE">We are using ZFS for metadata and object storage target in our Lustre. One of the pools is a draid3:12d:42c:2s-0 is became no longer importable after one of the disks
failed.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;mso-ligatures:none;mso-fareast-language:DE"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;mso-ligatures:none;mso-fareast-language:DE">on startup zpool import:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;mso-ligatures:none;mso-fareast-language:DE"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;mso-ligatures:none;mso-fareast-language:DE">pool: l1fs-OST010b<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;mso-ligatures:none;mso-fareast-language:DE"> id: 6548008278833985886<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;mso-ligatures:none;mso-fareast-language:DE"> state: FAULTED<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;mso-ligatures:none;mso-fareast-language:DE">status: One or more devices contains corrupted data.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;mso-ligatures:none;mso-fareast-language:DE">action: The pool cannot be imported due to damaged devices or data.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;mso-ligatures:none;mso-fareast-language:DE"> see:
<a href="https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-5E">https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-5E</a><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;mso-ligatures:none;mso-fareast-language:DE"> ...<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;mso-ligatures:none;mso-fareast-language:DE"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;mso-ligatures:none;mso-fareast-language:DE">the commands<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;mso-ligatures:none;mso-fareast-language:DE"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;mso-ligatures:none;mso-fareast-language:DE">zpool import -f l1fs-OST010b<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;mso-ligatures:none;mso-fareast-language:DE">zpool import -F l1fs-OST010b<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;mso-ligatures:none;mso-fareast-language:DE">zpool import -F -X l1fs-OST010b<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;mso-ligatures:none;mso-fareast-language:DE"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;mso-ligatures:none;mso-fareast-language:DE">yield:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;mso-ligatures:none;mso-fareast-language:DE">cannot import 'l1fs-OST010b': I/O error<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;mso-ligatures:none;mso-fareast-language:DE"> Destroy and re-create the pool from<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;mso-ligatures:none;mso-fareast-language:DE"> a backup source.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;mso-ligatures:none;mso-fareast-language:DE"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;mso-ligatures:none;mso-fareast-language:DE">zdb -e ends very quickly with the I/O error<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;mso-ligatures:none;mso-fareast-language:DE">Some tool said there is missing metadata in the zpool.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;mso-ligatures:none;mso-fareast-language:DE"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;mso-ligatures:none;mso-fareast-language:DE">Does anyone have any productive advice for us other than what is on the website<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;mso-ligatures:none;mso-fareast-language:DE"><a href="https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-5E">https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-5E</a><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;mso-ligatures:none;mso-fareast-language:DE">?<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;mso-ligatures:none;mso-fareast-language:DE"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;mso-ligatures:none;mso-fareast-language:DE">Now the question: how do we proceed if we have to reformat the OST?<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;mso-ligatures:none;mso-fareast-language:DE">At the moment we see the plan as:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;mso-ligatures:none;mso-fareast-language:DE">Disable the broken OST (or is it possible to use the same name for OST?)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;mso-ligatures:none;mso-fareast-language:DE">create a new pool and integrate it into the Lustre<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;mso-ligatures:none;mso-fareast-language:DE"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;mso-ligatures:none;mso-fareast-language:DE">What do we do with the files on the broken OST, can we just delete them now? Can the difference in metadata and object targets be repaired with Lustre-FSCK or somehow?<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;mso-ligatures:none;mso-fareast-language:DE"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;mso-ligatures:none;mso-fareast-language:DE">Thanks in advance</span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal"><span lang="EN-US" style="color:black;mso-ligatures:none;mso-fareast-language:DE">With best regards</span><span lang="EN-US" style="font-family:"Calibri",sans-serif;color:black;mso-ligatures:none;mso-fareast-language:DE"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="color:black;mso-ligatures:none;mso-fareast-language:DE"> <o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="color:black;mso-ligatures:none;mso-fareast-language:DE">Sergey Noskov<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="color:black;mso-ligatures:none;mso-fareast-language:DE"> </span><span style="color:black;mso-ligatures:none;mso-fareast-language:DE"><o:p></o:p></span></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</body>
</html>