[Lustre-discuss] lctl --device XX deactivate doesn't make OST read only
Alastair Ferguson
aferguson at cmcrc.com
Mon Jun 17 16:56:03 PDT 2013
Awesome thanks!
Doing a:
lfs find /data -obd AC3-OST000a_UUID -size +4G | lfs_migrate -y
now (as per a man page I found)
Alastair Ferguson
IT Manager
Capital Markets CRC Limited (CMCRC)
Telephone: +61 2 8088 4222
Mobile: +61 424 235 159
Fax: +61 2 8088 4201
www.cmcrc.com
Capital Markets CRC Ltd - Confidential Communication
The information contained in this e-mail is confidential. It is intended for the addressee only. If you receive this e-mail by mistake please promptly inform us by reply e-mail and then delete the e-mail and destroy any printed copy. You must not disclose or use in any way the information in the e-mail. There is no warranty that this e-mail is error or virus free. It may be a private communication, and if so, does not represent the views of the CMCRC and its associates. If it is a private communication, care should be taken in opening it to ensure that undue offence is not given.
On 18/06/2013, at 9:26 AM, "Dilger, Andreas" <andreas.dilger at intel.com> wrote:
> On 2013/17/06 1:12 AM, "Alastair Ferguson" <aferguson at cmcrc.com> wrote:
>> OK, bit of a weird one, so 3 OSTs are 100%, but there is 30TB of free
>> space around the other OSTs, so I do:
>>
>> lfs df -h
>>
>> Get this part as one of the OSTs I need to deactivate:
>>
>> AC3-OST000c_UUID 14.3T 13.6T 87.4M 100%
>> /data[OST:12]
>>
>> then
>>
>> lctl dl
>>
>> 19 UP osc AC3-OST000c-osc AC3-mdtlov_UUID 5
>>
>> Then
>>
>> lctl --device 19 deactivate
>>
>> then
>>
>> lctl dl:
>>
>> 19 IN osc AC3-OST000c-osc AC3-mdtlov_UUID 5
>>
>> Should be read only right>>?
>
> Right, this is the MDS OSC device, so no new files should be allocated on
> that OST.
>
>> Then
>>
>> lfs getstripe -O AC3-OST000c_UUID -rv -d /data | grep /data >>
>> ost000c_raw.txt
>>
>> To find the files in the filesystem (/data) and strip out all the stuff
>> you don't need. Then:
>>
>> while read line; do cp -p "$line" "$line.___bak"; rm -f "$line"; mv
>> "$line.___bak" "$line"; done < ost000c_raw.txt
>>
>> This should move the data off the OST but it doesn't. I have used this
>> procedure before to remove data from a whole server (which worked) and I
>> can see when I lfs df -h
>> the ost emptying but in this case it goes up and down suggesting it is
>> copying BACK to the same OST despite the fact it is IN not UP when lctl
>> dl is run.
>
> You should look at "lfs_migrate" and its man page, for a more robust
> mechanism for
> doing the above migration. Your script is unsafe if interrupted after "rm
> -f" but
> before "mv" moves the old file into place. You can also use "lfs_migrate"
> in a
> pipeline, so that it only moves new files, while your script would re-move
> the same
> files repeatedly if interrupted and restarted.
>
>> How can I get files off this as I get errors saying no space on device??
>
> Your process _should_ be working, but if you are moving small files the
> effects may
> be slow. As mentioned in the "lfs_migrate" man page, you should select
> large files
> to migrate, since you will get better IO performance, and will free space
> more quickly.
>
> Cheers, Andreas
> --
> Andreas Dilger
>
> Lustre Software Architect
> Intel High Performance Data Division
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20130618/178ed9bf/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 13489 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20130618/178ed9bf/attachment.png>
More information about the lustre-discuss
mailing list