[Lustre-discuss] lctl --device XX deactivate doesn't make OST read only

Dilger, Andreas andreas.dilger at intel.com
Mon Jun 17 16:26:53 PDT 2013

On 2013/17/06 1:12 AM, "Alastair Ferguson" <aferguson at cmcrc.com> wrote:
>OK, bit of a weird one, so 3 OSTs are 100%, but there is 30TB of free
>space around the other OSTs, so I do:
>lfs df -h
>Get this part as one of the OSTs I need to deactivate:
>AC3-OST000c_UUID           14.3T       13.6T       87.4M 100%
>lctl dl
> 19 UP osc AC3-OST000c-osc AC3-mdtlov_UUID 5
>lctl --device 19 deactivate
>lctl dl:
> 19 IN osc AC3-OST000c-osc AC3-mdtlov_UUID 5
>Should be read only right>>?

Right, this is the MDS OSC device, so no new files should be allocated on
that OST.

>lfs getstripe -O AC3-OST000c_UUID -rv -d /data | grep /data >>
>To find the files in the filesystem (/data) and strip out all the stuff
>you don't need.  Then:
>while read line; do cp -p "$line" "$line.___bak"; rm -f "$line"; mv
>"$line.___bak" "$line";  done < ost000c_raw.txt
>This should move the data off the OST but it doesn't. I have used this
>procedure before to remove data from a whole server (which worked) and I
>can see when I  lfs df -h
>the ost emptying but in this case it goes up and down suggesting it is
>copying BACK to the same OST despite the fact it is IN not UP when lctl
>dl is run.

You should look at "lfs_migrate" and its man page, for a more robust
mechanism for
doing the above migration.  Your script is unsafe if interrupted after "rm
-f" but
before "mv" moves the old file into place.  You can also use "lfs_migrate"
in a
pipeline, so that it only moves new files, while your script would re-move
the same
files repeatedly if interrupted and restarted.

>How can I get files off this as I get errors saying no space on device??

Your process _should_ be working, but if you are moving small files the
effects may
be slow.  As mentioned in the "lfs_migrate" man page, you should select
large files
to migrate, since you will get better IO performance, and will free space
more quickly.

Cheers, Andreas
Andreas Dilger

Lustre Software Architect
Intel High Performance Data Division

More information about the lustre-discuss mailing list