[Lustre-discuss] Problem on OST data recovery

[Andreas Dilger]

On 2011-01-13, at 01:41, thhsieh wrote:
> I am wondering whether it is possible to recover the OST data ?
> 
> We have faced the following problem. We installed a new OST server
> which is intended to replace an old one. We do the following steps
> to migrate the data from old OST to new OST:
> 
> 1. Figure out the files which located in old OST server:
> 
>   lfs find --obd {OST_UUID} /mount/point > list
> 
> 2. For each file in the list, we do the following migration:
> 
> 	cp -a ..../file ..../file.tmp
> 	mv ..../file.tmp ..../file
> 
> 
> However, after doing this data transfer for a few days, we suddently
> found that the hardware of the new OST is failed, and all the data
> migrated on it may not be recovered.
> 
> Therefore, I am wondering that whether it is possible to recover
> data from the old OST server ? This may sounds quit impossible, but
> we would like to try if there is any possibility to do that.

This partly boils down to an ext3 data recovery problem, because the files have been deleted from the old OST where you want to recover them.  There are a few tools you could try, including "ext3grep".

If/when you are able to recover the objects on the old OST, then the secondary problem becomes re-attaching them into the Lustre namespace.  If you have 1-stiped files (the default) this is possible, but if you have 2-striped (or wider) files, then it is nearly impossible because the objects on the other OSTs have likely been overwritten by your migration process.

Cheers, Andreas
--
Andreas Dilger 
Principal Engineer
Whamcloud, Inc.