<div dir="ltr">Hi, <div><br></div><div>Start a ZFS scrub on your pool, this will ensure that all the content is fine since the short resilver when re-adding dead disks to a pool does not check everything, only what changed on the pool while that disk was gone. </div><div><br></div><div>I sadly often see that kind of error on my personal NAS due to some bad hardware but ZFS is always able to fix everything even if it detects "permanent errors" and those permanent errors disappear after the scrub. </div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Apr 28, 2022 at 4:10 AM Alastair Basden via lustre-discuss <<a href="mailto:lustre-discuss@lists.lustre.org">lustre-discuss@lists.lustre.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hi,<br>
<br>
We have OSDs on ZFS (0.7.9) / Lustre 2.12.6.<br>
<br>
Recently, one of our JBODs had a wobble, and the disks (as presented to <br>
the OS) disappeared for a few seconds (and then returned).<br>
<br>
This upset a few zpools which SUSPENDED.<br>
<br>
A zpool clear on these then started the resilvering process, and zpool <br>
status gave e.g.:<br>
errors: Permanent errors have been detected in the following files:<br>
<br>
<metadata>:<0x0><br>
<metadata>:<0xb01><br>
<metadata>:<0x15><br>
<metadata>:<0x383><br>
cos6-ost7/ost7:/O/400000400/d11/10617643<br>
cos6-ost7/ost7:/O/400000400/d21/583029<br>
<br>
<br>
However, once the resilvering had completed, these permanent errors had <br>
gone.<br>
<br>
The question is then, are these errors really permanent, or was zfs able <br>
to correct them?<br>
<br>
Lustre continues to remain fine (though obviously froze while the pools <br>
were suspended).<br>
<br>
Should we be worried that there might be some under-the-hood corruption <br>
that will present itself when we need to remount (e.g. after a reboot) the <br>
OST? In particular the <metadata>:<0x0> file worries me a bit!<br>
<br>
Thanks,<br>
Alastair.<br>
_______________________________________________<br>
lustre-discuss mailing list<br>
<a href="mailto:lustre-discuss@lists.lustre.org" target="_blank">lustre-discuss@lists.lustre.org</a><br>
<a href="http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org" rel="noreferrer" target="_blank">http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org</a><br>
</blockquote></div>