[Lustre-discuss] redistribute used space to other OSTs (free space)

[Wojciech Turek]

Alex wrote:
> On Tuesday 04 November 2008 18:52, Brian J. Murrell wrote:
>   
>> On Tue, 2008-11-04 at 16:23 +0000, Wojciech Turek wrote:
>>     
>>> I don't know how to move a particular object but you could move a
>>> whole file to another OST and that would release some space from the
>>> full OST.
>>> mkdir /mnt/lustre/tmp
>>> lfs setstripe -i 2 -c 1 /mnt/lustre/tmp
>>>       
>
> supposing that i have enough space on all OSTs, but i want to create file1 on 
> 2 OSTs, the next command will be enough: lfs setstripe -c 2 /mnt/lustre/file1 
> but... i can predict which OSTs will be used!
>
> Is possible to specify somehow OSTs index, for more then one OST? someting 
> like: lfs setstripe -i 2 -i 4 -c 2 /mnt/lustre/file1 in order to create file1 
> on OST:2 and OST4?
>   
AFAIK no it isn't possible.
Currently when using lfs setstripe command with --index switch you can 
only specify index of the first OST of the striping pattern.
>   
>>> and then copy or move file test3.img into tmp dir
>>>       
>
> using cp/rm command after i deactivated my full OST:0 caused that the new file 
> to be restriped (that's good and what i want).
>
> but this technique (cp followed by rm) it has a major disavantage: you need 
> more free space on /mnt/lustre. what is happen if we are trying to move a 
> test3.img and we have /mnt/lustre 99% full -> we need free space equivalent 
> with minimum another test3.img size -> cp && rm command can not be used -> we 
> go to MOVE command but here comes my problem again. See below:
>
> ON CLIENT:
> [root at rs1 ~]# lfs df -h
> UUID                     bytes      Used Available  Use% Mounted on
> testfs-MDT0000_UUID     130.4G    460.1M    122.5G    0% /mnt/lustre[MDT:0]
> testfs-OST0000_UUID      18.3G     17.4G      2.0M   94% /mnt/lustre[OST:0]
> testfs-OST0001_UUID      18.3G     15.5G      2.0G   84% /mnt/lustre[OST:1]
> testfs-OST0002_UUID      36.7G     15.5G     19.4G   42% /mnt/lustre[OST:2]
> testfs-OST0003_UUID      36.7G     15.5G     19.4G   42% /mnt/lustre[OST:3]
> filesystem summary:     110.0G     63.8G     40.7G   57% /mnt/lustre
> [root at rs1 ~]#
>
> ON MGS:
> [root at rhclm ~]# lctl --device testfs-OST0000-osc deactivate
> [root at rhclm ~]# lctl --device testfs-OST0001-osc deactivate
>
> [root at rhclm ~]# lctl dl|grep -i in
>   5 IN osc testfs-OST0000-osc testfs-mdtlov_UUID 5
>   6 IN osc testfs-OST0001-osc testfs-mdtlov_UUID 5
> [root at rhclm ~]#
>
> ON CLIENT AGAIN:
> [root at rs1 ~]# mkdir /mnt/lustre/tmp
> [root at rs1 ~]# lfs getstripe /mnt/lustre/tmp/
> OBDS:
> 0: testfs-OST0000_UUID ACTIVE
> 1: testfs-OST0001_UUID ACTIVE
> 2: testfs-OST0002_UUID ACTIVE
> 3: testfs-OST0003_UUID ACTIVE
> /mnt/lustre/tmp/
> stripe_count: -1 stripe_size: 0 stripe_offset: -1
> [root at rs1 ~]#
> [root at rs1 ~]# mv /mnt/lustre/test3.img /mnt/lustre/tmp/
> [root at rs1 ~]# lfs getstripe /mnt/lustre/tmp/test3.img
> OBDS:
> 0: testfs-OST0000_UUID ACTIVE
> 1: testfs-OST0001_UUID ACTIVE
> 2: testfs-OST0002_UUID ACTIVE
> 3: testfs-OST0003_UUID ACTIVE
> /mnt/lustre/tmp/test3.img
>         obdidx           objid          objid            group
>              0               7            0x7                0
>              2               6            0x6                0
>              3              70           0x46                0
>              1              69           0x45                0
> [root at rs1 ~]#
>
> So, using move command, test3.img is not restriped when moved 
> in /mnt/lustre/tmp directory, even OST:0 and OST:1 are deactivated! In this 
> case why round-robin allocator is not working? Is any way to use mv command 
> and have test3.img file restriped on the fly when moved to new tmp directory? 
> If yes, how can i do that? I would like to avoid cp usage (which i tested and 
> is working)?
>
> Regards,
> Alx
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-discuss
>   

-- 
Wojciech Turek

Assistant System Manager
High Performance Computing Service
University of Cambridge
Email: wjt27 at cam.ac.uk
Tel: (+)44 1223 763517