[Lustre-discuss] redistribute used space to other OSTs (free space)

Brian J. Murrell Brian.Murrell at Sun.COM
Wed Nov 5 07:49:55 PST 2008


On Wed, 2008-11-05 at 17:18 +0200, Alex wrote:
> 
> supposing that i have enough space on all OSTs, but i want to create file1 on 
> 2 OSTs, the next command will be enough: lfs setstripe -c 2 /mnt/lustre/file1 
> but... i can predict which OSTs will be used!

No, you can only predict the first OST.

> Is possible to specify somehow OSTs index, for more then one OST?

No.

> using cp/rm command after i deactivated my full OST:0 caused that the new file 
> to be restriped (that's good and what i want).

Good.

> but this technique (cp followed by rm) it has a major disavantage: you need 
> more free space on /mnt/lustre.

Of course.

> what is happen if we are trying to move a 
> test3.img and we have /mnt/lustre 99% full

Well, if all of your OSTs are full then you don't have a "rebalancing"
problem but a free space problem.  That is solved by adding more OSTs.
Once you add space you can proceed with your rebalance if you wish, or
simply let lustre fill up your new OST(s) in due course.  You of course
won't benefit from the bandwidth of having as many OSTs as you do since
the existing ones are full.

> with minimum another test3.img size -> cp && rm command can not be used

If it cannot be used then all of your OSTs are full and you need to add
more space.

> ON CLIENT:
> [root at rs1 ~]# lfs df -h
> UUID                     bytes      Used Available  Use% Mounted on
> testfs-MDT0000_UUID     130.4G    460.1M    122.5G    0% /mnt/lustre[MDT:0]
> testfs-OST0000_UUID      18.3G     17.4G      2.0M   94% /mnt/lustre[OST:0]
> testfs-OST0001_UUID      18.3G     15.5G      2.0G   84% /mnt/lustre[OST:1]
> testfs-OST0002_UUID      36.7G     15.5G     19.4G   42% /mnt/lustre[OST:2]
> testfs-OST0003_UUID      36.7G     15.5G     19.4G   42% /mnt/lustre[OST:3]
> filesystem summary:     110.0G     63.8G     40.7G   57% /mnt/lustre
> [root at rs1 ~]#
> 
> ON MGS:
> [root at rhclm ~]# lctl --device testfs-OST0000-osc deactivate
> [root at rhclm ~]# lctl --device testfs-OST0001-osc deactivate
> 
> [root at rhclm ~]# lctl dl|grep -i in
>   5 IN osc testfs-OST0000-osc testfs-mdtlov_UUID 5
>   6 IN osc testfs-OST0001-osc testfs-mdtlov_UUID 5
> [root at rhclm ~]#
> 
> ON CLIENT AGAIN:
> [root at rs1 ~]# mkdir /mnt/lustre/tmp
> [root at rs1 ~]# lfs getstripe /mnt/lustre/tmp/
> OBDS:
> 0: testfs-OST0000_UUID ACTIVE
> 1: testfs-OST0001_UUID ACTIVE
> 2: testfs-OST0002_UUID ACTIVE
> 3: testfs-OST0003_UUID ACTIVE
> /mnt/lustre/tmp/
> stripe_count: -1 stripe_size: 0 stripe_offset: -1
> [root at rs1 ~]#
> [root at rs1 ~]# mv /mnt/lustre/test3.img /mnt/lustre/tmp/
> [root at rs1 ~]# lfs getstripe /mnt/lustre/tmp/test3.img
> OBDS:
> 0: testfs-OST0000_UUID ACTIVE
> 1: testfs-OST0001_UUID ACTIVE
> 2: testfs-OST0002_UUID ACTIVE
> 3: testfs-OST0003_UUID ACTIVE
> /mnt/lustre/tmp/test3.img
>         obdidx           objid          objid            group
>              0               7            0x7                0
>              2               6            0x6                0
>              3              70           0x46                0
>              1              69           0x45                0
> [root at rs1 ~]#
> 
> So, using move command, test3.img is not restriped when moved 
> in /mnt/lustre/tmp directory, even OST:0 and OST:1 are deactivated! In this 
> case why round-robin allocator is not working? Is any way to use mv command 
> and have test3.img file restriped on the fly when moved to new tmp directory? 
> If yes, how can i do that? I would like to avoid cp usage (which i tested and 
> is working)?

As I said before, I suspect that mv won't work.  You must use cp/rm.  I
don't understand why in your above scenario you can't use cp/rm.  There
is plenty of space for it to work.

b.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: This is a digitally signed message part
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20081105/430a6773/attachment.pgp>


More information about the lustre-discuss mailing list