[Lustre-discuss] lctl --device XX deactivate doesn't make OST read only

Dilger, Andreas andreas.dilger at intel.com
Mon Jun 17 16:26:53 PDT 2013


On 2013/17/06 1:12 AM, "Alastair Ferguson" <aferguson at cmcrc.com> wrote:
>OK, bit of a weird one, so 3 OSTs are 100%, but there is 30TB of free
>space around the other OSTs, so I do:
>
>lfs df -h
>
>Get this part as one of the OSTs I need to deactivate:
>
>AC3-OST000c_UUID           14.3T       13.6T       87.4M 100%
>/data[OST:12]
>
>then
>
>lctl dl
>
> 19 UP osc AC3-OST000c-osc AC3-mdtlov_UUID 5
>
>Then
>
>lctl --device 19 deactivate
>
>then
>
>lctl dl:
>
> 19 IN osc AC3-OST000c-osc AC3-mdtlov_UUID 5
>
>Should be read only right>>?

Right, this is the MDS OSC device, so no new files should be allocated on
that OST.

>Then
>
>lfs getstripe -O AC3-OST000c_UUID -rv -d /data | grep /data >>
>ost000c_raw.txt
>
>To find the files in the filesystem (/data) and strip out all the stuff
>you don't need.  Then:
>
>while read line; do cp -p "$line" "$line.___bak"; rm -f "$line"; mv
>"$line.___bak" "$line";  done < ost000c_raw.txt
>
>This should move the data off the OST but it doesn't. I have used this
>procedure before to remove data from a whole server (which worked) and I
>can see when I  lfs df -h
>the ost emptying but in this case it goes up and down suggesting it is
>copying BACK to the same OST despite the fact it is IN not UP when lctl
>dl is run.

You should look at "lfs_migrate" and its man page, for a more robust
mechanism for
doing the above migration.  Your script is unsafe if interrupted after "rm
-f" but
before "mv" moves the old file into place.  You can also use "lfs_migrate"
in a
pipeline, so that it only moves new files, while your script would re-move
the same
files repeatedly if interrupted and restarted.

>How can I get files off this as I get errors saying no space on device??

Your process _should_ be working, but if you are moving small files the
effects may
be slow.  As mentioned in the "lfs_migrate" man page, you should select
large files
to migrate, since you will get better IO performance, and will free space
more quickly.

Cheers, Andreas
-- 
Andreas Dilger

Lustre Software Architect
Intel High Performance Data Division





More information about the lustre-discuss mailing list