[Lustre-discuss] Reclaiming Reserved Disk Space

[Andreas Dilger]

On 2010-11-26, at 05:51, Wojciech Turek wrote:
> Thanks for your reply Andreas. I have one more question regarding the size of the reserved space. Our average file size is 1MB, OST size is 7.2TB and I am planning to set reserved space to 0.15% which  will give us 12GB per OST. Would you agree that this is enough to keep fragmentation low?

I can't really say for sure, since this is highly dependent upon your usage pattern.  In any case, keeping some space in reserve is much better than no reserved space at all.

We have observed significant performance degradation as filesystems approach 99% full, so I'm not confident that 0.15% reserved will avoid this problem.  12GB over 7.2TB is only about 218kB (54 blocks) per block group of unused space, so it means you have to be packing files very densely to get this kind of utilization.

Even with expensive storage, 5% reserved is only about $200 per OST, so you have to balance, say, 25% performance loss from 95% - 99% full vs the $200 saved.

> On 25 November 2010 23:57, Andreas Dilger <andreas.dilger at oracle.com> wrote:
> On 2010-11-25, at 13:12, Wojciech Turek wrote:
> > I forgot to ask, is setting reserved % to 0 is safe on Lustre filesysytem?
> 
> It is safe, but can lead to serious (and permanent) fragmentation of the filesystem, which can lead to permanent performance loss.  Consider reducing it to some smaller fraction than 5%, but leaving some reserve.
> 
> > On 25 November 2010 19:15, Wojciech Turek <wjt27 at cam.ac.uk> wrote:
> >> I am trying to reclaim the default 5% of reserved disk space on Lustre filesystem following procedure describe in the Lustre manual (chapter 23.3.10). File system is in use and I would like to avoid any downtime.
> >>
> >> [root at oss17 ~]# tune2fs -m 1 /dev/sdg
> >> tune2fs 1.41.12.2.ora1 (14-Aug-2010)
> >> tune2fs: MMP: device currently active while trying to open /dev/sdg
> >>
> >> I understand it is the MMP flag which prevent tune2fs to access the device, is there a way to work around this or is my only option to umount OSTs (eek, downtime)
> 
> It should be possible to do this with a modification to tune2fs.
> 
> The issue is that this operation is only safe on a mounted filesystem if it is a local mount, and not a remote mount.  tune2fs probably needs to skip the MMP checking in this case, but it needs to check that the filesystem is mounted locally.
> 
> Cheers, Andreas
> --
> Andreas Dilger
> Lustre Technical Lead
> Oracle Corporation Canada Inc.
> 
> 
> 


Cheers, Andreas
--
Andreas Dilger
Lustre Technical Lead
Oracle Corporation Canada Inc.