[Lustre-discuss] redistribute used space to other OSTs (free space)

Andreas Dilger adilger at sun.com
Wed Nov 5 11:31:46 PST 2008


On Nov 05, 2008  17:18 +0200, Alex wrote:
> supposing that i have enough space on all OSTs, but i want to create file1 on 
> 2 OSTs, the next command will be enough: lfs setstripe -c 2 /mnt/lustre/file1 
> but... i can predict which OSTs will be used!

I question why you need to know which OST will be used?  The files should not
be allocated on the full OST, which is a problem for the MDS to handle, but
otherwise it shouldn't really matter where the files are stored.

> Is possible to specify somehow OSTs index, for more then one OST? someting 
> like: lfs setstripe -i 2 -i 4 -c 2 /mnt/lustre/file1 in order to create file1 
> on OST:2 and OST4?

In the 1.8.0 release it will be possible to specify more arbitrary groupings
of OSTs using the "OST Pools" feature.  Until then you can only specify the
starting OST index.

> but this technique (cp followed by rm) it has a major disavantage: you need 
> more free space on /mnt/lustre. what is happen if we are trying to move a 
> test3.img and we have /mnt/lustre 99% full -> we need free space equivalent 
> with minimum another test3.img size -> cp && rm command can not be used -> we 
> go to MOVE command but here comes my problem again.

I agree with Brian - if the target OSTs do not have enough free space to
hold the file data, how can you move the file there?

> [root at rs1 ~]# lfs getstripe /mnt/lustre/tmp/
> /mnt/lustre/tmp/
> stripe_count: -1 stripe_size: 0 stripe_offset: -1

Note that you are requesting that all files be striped over all OSTs,
which will always result  in the full OST being used.  If you specify
the files be striped over only 2 or 3 OSTs then the full OST can be
skipped.

>  I found that /mnt/lustre/test3.img is 12Gb in size and is striped between
> all OSTs (arround 3GB/OST)

I would also suggest that instead of having such small OSTs (18GB and 36GB)
it is much better to have fewer large OSTs.  Even a single disk today is
hundreds of GB or more, so it seems you are only using a small partition
for each OST, and you will run into this sort of "free space fragmentation"
problem.  This is especially true if the size of a single file is a very
large fraction of the total OST size.

Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.




More information about the lustre-discuss mailing list