[Lustre-discuss] Removing an OST

[Andreas Dilger]

On Jan 16, 2009  13:45 +0100, Heiko Schroeter wrote:
> the manual (May 2008) describes on page 4-14 how to remove an OST from LUSTRE.
> Its says "deactivate the OST (make it read-only).
> 
> Hm, i'am a bit puzzled here.
> When i deactivate the OST on the MDS using
> 'lctl --device 18 conf_param foo-OST000d.osc.active=0' the device is 
> deactivated but i cannot read any files from it any longer. Quite logical.

Ah, it looks like you are confusing two different commands here.
By using "lctl conf_param ..." you are storing this setting in the
filesystem-wide configuration file, permanently deactivating the device
on all nodes.

If you just want to deactivate the OST only on the MDS you can use:

	lctl set_param osc.foo-OST000d.active=0

This means the MDS will not allocate any new objects on this OST.  It will
still be active on all of the clients, so they can read or write to this
file, or delete it.

> On the other hand using lctl "readonly" option is specified as "dangerous".

Yes, this isn't something you should be using - it is for regression
testing only and doesn't do what you want.

> So what is the correct approach of replacing an OST and/or what is the
> correct syntax for lctl (or whatever command) putting a device in
> read-only mode ?
> 
> When the OST is in read-only mode and i "recopy" the file over the original 
> one what happens to the directory structure?

If the OST is deactivated on only the MDS, but not the clients, then the
deleted file will be cleaned up correctly on the OST.

> Is it necessary to recreate 
> it? Or is it enough to do something like this (mentioned under 3. on the 
> above lustre manual page):


> cp /lustrefs/dir1/foo/dat.file    /tmp/.
> cp /tmp/dat.file     /lustrefs/dir1/foo/.

Using "cp" likely isn't what you want to do, because "cp" doesn't change
the inode (object) allocation (even on local filesystems).  More likely
you want to do:

	cp .../dat.file .../dat.file.tmp
	mv .../dat.file.tmp .../dat.file

This will create a new file (not on the OST deactivated at the MDS),
and delete the old one when the new one is moved over top of it.

Once you have moved all of your files off of this OST then it is safe
to remove and the "conf_param" command can be used to deactivate it
permanently.

Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.