[Lustre-discuss] What says an OST is deactivated?

[Andreas Dilger]

On Mar 25, 2008  08:53 -0600, Chris Worley wrote:
> On Tue, Mar 25, 2008 at 2:13 AM, Andreas Dilger <adilger at sun.com> wrote:
> > On Mar 25, 2008  01:28 -0600, Chris Worley wrote:
> >  > I do an "lctl dl" and it shows "UP" in the first column for all
> >  > OST's... even though I've deactivated many disks.  "iostat" shows the
> >  > disks are still in use too.
> >
> >  What does it mean when you say "deactivated many disks"?
>
> To deactivate the disk, I use an incantation like:
> 
> lctl --device ddnlfs-OST001f-osc deactivate

Note that "deactivate" only affects the node on which it is run.
The normal place to do this is on the MDS.  Note that you also
mount the client filesystem on the MDS node you need to deactivate
the MDS OSC connection, and not the client filesystem one:

> ...but new files are still going there, and, if I'm reading it right,
> the disk is still "up" in Lustre:
> 
> # lctl dl | grep 1f
>  36 UP osc ddnlfs-OST001f-osc ddnlfs-mdtlov_UUID 5

This does look like you have the right device.  Using "device_list"
only shows which devices are configured.  A deactivated device is
still configured...   The "UP" status is related to the configuration
status and not the current connection state.  Have a look at the file
/proc/fs/lustre/ddnlfs-mdtlov/target_obd to see the device status.

# lfs df
UUID                 1K-blocks      Used Available  Use% Mounted on
mds-myth-0_UUID        9174328    678000   8496328    7% /myth[MDT:0]
ost-myth-0_UUID      292223856 286837752   5386104   98% /myth[OST:0]
ost-myth-1_UUID       94442984  92833972   1609012   98% /myth[OST:1]
ost-myth-2_UUID      487388376 474792788  12595588   97% /myth[OST:2]
ost-myth-3_UUID      487865304 472221312  15643992   96% /myth[OST:3]

filesystem summary:  1361920520 1326685824  35234696   97% /myth

# lctl --device %myth-OST0001-osc deactivate
# cat /proc/fs/lustre/lov/myth-mdtlov/target_obd 
0: ost-myth-0_UUID ACTIVE
1: ost-myth-1_UUID INACTIVE
2: ost-myth-2_UUID ACTIVE
3: ost-myth-3_UUID ACTIVE

# lctl --device %myth-OST0001-osc recover

> >  > I'm trying to get rid of slow disks... what's the right way to tell
> >  > Lustre to quit using a disk?
> >
> >  If you deactivate an OST on the MDS node it will stop allocating new
> >  files there
> 
> For now, that's all I want to do... but new files are still going there.
> 
> ... both a way to deactivate the disk and a way to know which are
> deactivated would be nice.

It was confusing when you say "deactivate the disk" because that could
mean a great many things like deactivating a disk from a RAID set, or
similar.  An OST may reside on many disks (via hardware/software RAID,
LVM, etc).

What you are trying to do is the right process.

Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.