[lustre-discuss] [EXTERNAL] [BULK] mds and mst are lost

[Vicker, Darby J. (JSC-EG111)[Jacobs Technology, Inc.]]

Sorry to hear this.  We are dealing with MDT data loss right now as well and its no fun.  Please look at my posts to the list from last week for some more information about what we are doing to recover.  Our situation is not as bad as yours, we only lost part of our MDT data (the last 3 months of a filesystem we’ve had in service for over 7 years).  If you get to the point where you can get the OST’s attached to a functioning lustre filesystem, you might be able to run an lfsck to move all of your files into lost+found.  We started our lfsck as follows:

[root at hpfs-fsl-mds1 hpfs3-eg3]# lctl lfsck_start -M scratch-MDT0000 -o


When finished, this is on a client:


[root at hpfs-fsl-lmon0 MDT0000]# \ls | head
[0x20000bdbe:0x1eae1:0x0]-R-0
[0x20000bdeb:0x12c3e:0x0]-R-0
[0x20001f801:0x296f:0x0]-R-0
[0x20001f801:0x57:0x0]-R-0
[0x20001f801:0x58:0x0]-R-0
[0x20001f805:0x1000:0x0]-R-0
[0x20001f805:0x100:0x0]-R-0
[0x20001f805:0x1001:0x0]-R-0
[0x20001f805:0x1002:0x0]-R-0
[0x20001f805:0x1003:0x0]-R-0
[root at hpfs-fsl-lmon0 MDT0000]# ls -l [0x20000bdbe:0x1eae1:0x0]-R-0
-r-------- 1 damocles_runner damocles 3162 Sep 24 14:45 [0x20000bdbe:0x1eae1:0x0]-R-0
[root at hpfs-fsl-lmon0 MDT0000]# pwd
/scratch-lustre/.lustre/lost+found/MDT0000
[root at hpfs-fsl-lmon0 MDT0000]#

From: lustre-discuss <lustre-discuss-bounces at lists.lustre.org> on behalf of Sergey Astafyev via lustre-discuss <lustre-discuss at lists.lustre.org>
Reply-To: Sergey Astafyev <astafyev.sergey at gmail.com>
Date: Monday, September 25, 2023 at 4:27 AM
To: "lustre-discuss at lists.lustre.org" <lustre-discuss at lists.lustre.org>
Subject: [EXTERNAL] [BULK] [lustre-discuss] mds and mst are lost

CAUTION: This email originated from outside of NASA.  Please take care when clicking links or opening attachments.  Use the "Report Message" button to report suspicious messages to the NASA SOC.



Hello.

Please help.

It doesn’t really matter how, but MDS and MDT servers are  lost.

I connected old OSTs to the new lustre, but I don’t see any information.

lfs find tells me cannot get lov name : inappropriate ioctl for device

Is there any mechanism for recovering information from orphaned OST partitions?

best regards

S.Astafev


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20230925/e9dec5cb/attachment-0001.htm>