[Lustre-discuss] redistribute used space to other OSTs (free space)

Alex linux at vfemail.net
Wed Nov 5 07:18:09 PST 2008


On Tuesday 04 November 2008 18:52, Brian J. Murrell wrote:
> On Tue, 2008-11-04 at 16:23 +0000, Wojciech Turek wrote:
> > I don't know how to move a particular object but you could move a
> > whole file to another OST and that would release some space from the
> > full OST.
> > mkdir /mnt/lustre/tmp
> > lfs setstripe -i 2 -c 1 /mnt/lustre/tmp

supposing that i have enough space on all OSTs, but i want to create file1 on 
2 OSTs, the next command will be enough: lfs setstripe -c 2 /mnt/lustre/file1 
but... i can predict which OSTs will be used!

Is possible to specify somehow OSTs index, for more then one OST? someting 
like: lfs setstripe -i 2 -i 4 -c 2 /mnt/lustre/file1 in order to create file1 
on OST:2 and OST4?

> > and then copy or move file test3.img into tmp dir

using cp/rm command after i deactivated my full OST:0 caused that the new file 
to be restriped (that's good and what i want).

but this technique (cp followed by rm) it has a major disavantage: you need 
more free space on /mnt/lustre. what is happen if we are trying to move a 
test3.img and we have /mnt/lustre 99% full -> we need free space equivalent 
with minimum another test3.img size -> cp && rm command can not be used -> we 
go to MOVE command but here comes my problem again. See below:

ON CLIENT:
[root at rs1 ~]# lfs df -h
UUID                     bytes      Used Available  Use% Mounted on
testfs-MDT0000_UUID     130.4G    460.1M    122.5G    0% /mnt/lustre[MDT:0]
testfs-OST0000_UUID      18.3G     17.4G      2.0M   94% /mnt/lustre[OST:0]
testfs-OST0001_UUID      18.3G     15.5G      2.0G   84% /mnt/lustre[OST:1]
testfs-OST0002_UUID      36.7G     15.5G     19.4G   42% /mnt/lustre[OST:2]
testfs-OST0003_UUID      36.7G     15.5G     19.4G   42% /mnt/lustre[OST:3]
filesystem summary:     110.0G     63.8G     40.7G   57% /mnt/lustre
[root at rs1 ~]#

ON MGS:
[root at rhclm ~]# lctl --device testfs-OST0000-osc deactivate
[root at rhclm ~]# lctl --device testfs-OST0001-osc deactivate

[root at rhclm ~]# lctl dl|grep -i in
  5 IN osc testfs-OST0000-osc testfs-mdtlov_UUID 5
  6 IN osc testfs-OST0001-osc testfs-mdtlov_UUID 5
[root at rhclm ~]#

ON CLIENT AGAIN:
[root at rs1 ~]# mkdir /mnt/lustre/tmp
[root at rs1 ~]# lfs getstripe /mnt/lustre/tmp/
OBDS:
0: testfs-OST0000_UUID ACTIVE
1: testfs-OST0001_UUID ACTIVE
2: testfs-OST0002_UUID ACTIVE
3: testfs-OST0003_UUID ACTIVE
/mnt/lustre/tmp/
stripe_count: -1 stripe_size: 0 stripe_offset: -1
[root at rs1 ~]#
[root at rs1 ~]# mv /mnt/lustre/test3.img /mnt/lustre/tmp/
[root at rs1 ~]# lfs getstripe /mnt/lustre/tmp/test3.img
OBDS:
0: testfs-OST0000_UUID ACTIVE
1: testfs-OST0001_UUID ACTIVE
2: testfs-OST0002_UUID ACTIVE
3: testfs-OST0003_UUID ACTIVE
/mnt/lustre/tmp/test3.img
        obdidx           objid          objid            group
             0               7            0x7                0
             2               6            0x6                0
             3              70           0x46                0
             1              69           0x45                0
[root at rs1 ~]#

So, using move command, test3.img is not restriped when moved 
in /mnt/lustre/tmp directory, even OST:0 and OST:1 are deactivated! In this 
case why round-robin allocator is not working? Is any way to use mv command 
and have test3.img file restriped on the fly when moved to new tmp directory? 
If yes, how can i do that? I would like to avoid cp usage (which i tested and 
is working)?

Regards,
Alx



More information about the lustre-discuss mailing list