[Lustre-discuss] how to replace a ost

[Andreas Dilger]

On Apr 29, 2008  18:14 +0800, Felix New wrote:
> i want to replace a ost, and i read the manual(820-3681.pdf from sun
> site), i get this section 4.2.3.1 about how to do:
> 
> 4.2.3.1 Removing an OST
> To remove an OST:
> 1. Deactivate the OST (make it read-only) on the MDS so no new objects are
> allocated to it.
> 2. Use lfs find to discover all files that have objects residing on
> the deactivated
> OST.
> 3. Copy these files to a new location and then move them back to their original
> location to force their object re-creation on the active OSTs and the object
> deletion on the OST to be removed.
> 
> 
> and i do as what it say, deactivate the ost(cfs-OST0003_UUID) at my
> MGS, and find the files reside in ost cfs-OST0003_UUID from my client,
> and copy the file...ouch, i can't copy the file, the error msg is:

Did you deactivate it on MDS only?

> cp /data//v/ff/b6/0b142400670.flv ./
> cp: cannot stat `/data//v/ff/b6/0b142400670.flv ': Input/output error
> 
> because the ost is deactive, so i can't stat and read the file? how to do that?

You shouldn't deactivate the OST on the client.  If you are mounting
a client on the MDS node and doing this testing there, then you need
to be aware that there are 2 OST devices on such a setup - one for
the client and one for the MDS:

  9 UP mds myth-MDT0000 mds-myth-0_UUID 7
 10 UP osc myth-OST0000-osc myth-mdtlov_UUID 5
 11 UP osc myth-OST0001-osc myth-mdtlov_UUID 5
 12 UP osc myth-OST0002-osc myth-mdtlov_UUID 5
 13 UP osc myth-OST0003-osc myth-mdtlov_UUID 5
:
:
 17 UP mdc myth-MDT0000-mdc-ffff810027cd7138 34d9e3f0-6b7a-5412-f602-3cce5b5247f2 5
 18 UP osc myth-OST0000-osc-ffff810027cd7138 34d9e3f0-6b7a-5412-f602-3cce5b5247f2 5
 19 UP osc myth-OST0001-osc-ffff810027cd7138 34d9e3f0-6b7a-5412-f602-3cce5b5247f2 5
 20 UP osc myth-OST0002-osc-ffff810027cd7138 34d9e3f0-6b7a-5412-f602-3cce5b5247f2 5
 21 UP osc myth-OST0003-osc-ffff810027cd7138 34d9e3f0-6b7a-5412-f602-3cce5b5247f2 5
 22 UP osc myth-OST0004-osc-ffff810027cd7138 34d9e3f0-6b7a-5412-f602-3cce5b5247f2 5

You need to deactivate ONLY the osc device for e.g. OST0003 for the MDS
(myth-OST0003-osc in my case) and not the client one.

Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.