[Lustre-discuss] recovering formatted OST

Wojciech Turek wjt27 at cam.ac.uk
Wed Oct 20 09:15:03 PDT 2010


Many thanks for prompt reply,

On 20 October 2010 16:32, Andreas Dilger <andreas.dilger at oracle.com> wrote:

> Right - you need to recreate the LV exactly as it was before. If you
> created it all at once on the whole LUN then it is likely to be allocated in
> a linear way. If there are multiple LVs on the same LUN and they were
> expanded after use the chance of recovering them is very low.
>
There was one LVM on that LUN I created it using  following commands:

pvcreate /dev/sdc
vgcreate ost16vg /dev/sdc
lvcreate --name ost16v -l 100%VG ost16vg

So in order to recreate that LVM on the formatted LUN i need to repeat above
steps, is that right?


> In the e2fsprogs there is a tool I wrote called findsuper to scan a device
> looking for ext3 superblock signatures. If needed, you could run findsuper
> to determine the starting offset of the filesystem, and then just create a
> simple partition with the right starting offset on order to run e2fsck.
>
Should I successfully  recreate the LVM volume I will run findsuper tool on
that volume. If the tool finds a superblock how I can tell if that
superblock belongs to the ldiskfs and not to the newly created filesystems?

>
> That said, if there were filesystems formatted in each partition, the
> amount of data loss may be large. You may have some saving grace if the
> first partitions are very small and fit inside the space previously used by
> the 400MB journal.
>
Unfortunately new partitions use much more space than 400mb
   8    32 7809904640 sdc
   8    33   10484719 sdc1
   8    34    4193280 sdc2
   8    35    4193280 sdc3
   8    36    8387584 sdc4
   8    37 7782640640 sdc5


> Cheers, Andreas
>
> On 2010-10-20, at 9:06, Wojciech Turek <wjt27 at cam.ac.uk> wrote:
>
> Thank you for quick reply.
> Unfortunately all partitions were formatted with ext3, also I didn't
> mention earlier but the OST was placed on the LVM volume which is now gone
> as the installation script formatted the physical device. I understand  that
> this complicates things even further. In that case i guess firstly I need to
> try to recover the LVM information otherwise fsck will not be able to find
> anything is that right?
>
> Best regards,
>
> Wojciech
>
> On 20 October 2010 08:46, Andreas Dilger < <andreas.dilger at oracle.com>
> andreas.dilger at oracle.com> wrote:
>
>> On 2010-10-19, at 17:01, Wojciech Turek wrote:
>> > Due to the locac disk failure in an OSS one of our /scratch OSTs was
>> formatted by automatic installation script. This script created 5 small
>> partitions and 6th partition consisting of the remaining space on that OST.
>> Nothing else was written to that device since then. Is there a way to
>> recover any data from that OST?
>>
>> Your best bet is to make a full "dd" backup of the OST to a new device
>> (for safety), first restore the original partition table.  If there was not
>> originally a partition table, then you can just erase the new partitions:
>>
>>  dd if=/dev/zero of=/dev/XXX bs=512 count=1
>>
>> Then run e2fsck -fy, followed by "ll_recover_lost_found_objs" (from a
>> newer lustre RPM, if you don't have it).  It is likely that you will get
>> some or most of the data back.  This depends heavily on exactly what was
>> written over the original filesystem.
>>
>> If it was just a new partition table, there should be relatively little
>> damage (ext3 is very robust this way, and can repair itself so long as the
>> starting alignment is correct).  If there were filesystems formatted in each
>> of these partitions, then the amount of data available will be reduced
>> significantly.
>>
>> Cheers, Andreas
>> --
>> Andreas Dilger
>> Lustre Technical Lead
>> Oracle Corporation Canada Inc.
>>
>>
>


-- 
Wojciech Turek

Senior System Architect

High Performance Computing Service
University of Cambridge
Email: wjt27 at cam.ac.uk
Tel: (+)44 1223 763517
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20101020/2c21beb5/attachment.htm>


More information about the lustre-discuss mailing list