[Lustre-discuss] Problem on OST data recovery

[Andreas Dilger]

On 2011-01-13, at 20:33, thhsieh wrote:
> Thanks very much for your response. In any case I am very appriciate.
> 
> I did not change the strip size of the Lustre defautl value. So I think
> our Lustre filesystem uses 1-strip. Now the problem is how to re-attach
> the recovered files to the Lustre name space.
> 
> In fact, we have a completed list of filenames which are migrated to
> the new OST. However, only filenames, but without file modified time,
> size, owner, permission .... etc. But we can still guess about them.
> 
> So, is there a way to re-attach the files in OST to the Lustre name
> space? This is the major difficulty we are facing now.

If you are able to recover the object files on the original OST, the easiest way to re-attach them into the Lustre namespace is to just copy them from the ldiskfs-mounted OST into the Lustre filesystem.  This can be also be done by running lfsck with the "-l" option.

> On Thu, Jan 13, 2011 at 11:09:55AM -0700, Andreas Dilger wrote:
>> On 2011-01-13, at 01:41, thhsieh wrote:
>>> I am wondering whether it is possible to recover the OST data ?
>>> 
>>> We have faced the following problem. We installed a new OST server
>>> which is intended to replace an old one. We do the following steps
>>> to migrate the data from old OST to new OST:
>>> 
>>> 1. Figure out the files which located in old OST server:
>>> 
>>>  lfs find --obd {OST_UUID} /mount/point > list
>>> 
>>> 2. For each file in the list, we do the following migration:
>>> 
>>> 	cp -a ..../file ..../file.tmp
>>> 	mv ..../file.tmp ..../file
>>> 
>>> 
>>> However, after doing this data transfer for a few days, we suddently
>>> found that the hardware of the new OST is failed, and all the data
>>> migrated on it may not be recovered.
>>> 
>>> Therefore, I am wondering that whether it is possible to recover
>>> data from the old OST server ? This may sounds quit impossible, but
>>> we would like to try if there is any possibility to do that.
>> 
>> This partly boils down to an ext3 data recovery problem, because the files have been deleted from the old OST where you want to recover them.  There are a few tools you could try, including "ext3grep".
>> 
>> If/when you are able to recover the objects on the old OST, then the secondary problem becomes re-attaching them into the Lustre namespace.  If you have 1-stiped files (the default) this is possible, but if you have 2-striped (or wider) files, then it is nearly impossible because the objects on the other OSTs have likely been overwritten by your migration process.
>> 
>> Cheers, Andreas
>> --
>> Andreas Dilger 
>> Principal Engineer
>> Whamcloud, Inc.
>> 
>> 
>> 


Cheers, Andreas
--
Andreas Dilger 
Principal Engineer
Whamcloud, Inc.