[Lustre-discuss] lctl --device XX deactivate doesn't make OST read only

Alastair Ferguson aferguson at cmcrc.com
Mon Jun 17 16:56:03 PDT 2013

Awesome thanks!

Doing a:

lfs find /data -obd AC3-OST000a_UUID -size +4G | lfs_migrate -y

now (as per a man page I found)

Alastair Ferguson
IT Manager
Capital Markets CRC Limited (CMCRC)
Telephone: +61 2 8088 4222
Mobile: +61 424 235 159
Fax: +61 2 8088 4201

Capital Markets CRC Ltd - Confidential Communication
The information contained in this e-mail is confidential.  It is intended for the addressee only.  If you receive this e-mail by mistake please promptly inform us by reply e-mail and then delete the e-mail and destroy any printed copy. You must not disclose or use in any way the information in the e-mail. There is no warranty that this e-mail is error or virus free. It may be a private communication, and if so, does not represent the views of the CMCRC and its associates.  If it is a private communication, care should be taken in opening it to ensure that undue offence is not given.

On 18/06/2013, at 9:26 AM, "Dilger, Andreas" <andreas.dilger at intel.com> wrote:

> On 2013/17/06 1:12 AM, "Alastair Ferguson" <aferguson at cmcrc.com> wrote:
>> OK, bit of a weird one, so 3 OSTs are 100%, but there is 30TB of free
>> space around the other OSTs, so I do:
>> lfs df -h
>> Get this part as one of the OSTs I need to deactivate:
>> AC3-OST000c_UUID           14.3T       13.6T       87.4M 100%
>> /data[OST:12]
>> then
>> lctl dl
>> 19 UP osc AC3-OST000c-osc AC3-mdtlov_UUID 5
>> Then
>> lctl --device 19 deactivate
>> then
>> lctl dl:
>> 19 IN osc AC3-OST000c-osc AC3-mdtlov_UUID 5
>> Should be read only right>>?
> Right, this is the MDS OSC device, so no new files should be allocated on
> that OST.
>> Then
>> lfs getstripe -O AC3-OST000c_UUID -rv -d /data | grep /data >>
>> ost000c_raw.txt
>> To find the files in the filesystem (/data) and strip out all the stuff
>> you don't need.  Then:
>> while read line; do cp -p "$line" "$line.___bak"; rm -f "$line"; mv
>> "$line.___bak" "$line";  done < ost000c_raw.txt
>> This should move the data off the OST but it doesn't. I have used this
>> procedure before to remove data from a whole server (which worked) and I
>> can see when I  lfs df -h
>> the ost emptying but in this case it goes up and down suggesting it is
>> copying BACK to the same OST despite the fact it is IN not UP when lctl
>> dl is run.
> You should look at "lfs_migrate" and its man page, for a more robust
> mechanism for
> doing the above migration.  Your script is unsafe if interrupted after "rm
> -f" but
> before "mv" moves the old file into place.  You can also use "lfs_migrate"
> in a
> pipeline, so that it only moves new files, while your script would re-move
> the same
> files repeatedly if interrupted and restarted.
>> How can I get files off this as I get errors saying no space on device??
> Your process _should_ be working, but if you are moving small files the
> effects may
> be slow.  As mentioned in the "lfs_migrate" man page, you should select
> large files
> to migrate, since you will get better IO performance, and will free space
> more quickly.
> Cheers, Andreas
> -- 
> Andreas Dilger
> Lustre Software Architect
> Intel High Performance Data Division

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20130618/178ed9bf/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 13489 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20130618/178ed9bf/attachment.png>

More information about the lustre-discuss mailing list