[Lustre-discuss] Maximum OST Size

Bernd Schubert bernd.schubert at fastmail.fm
Wed Oct 20 10:00:16 PDT 2010


On Wednesday, October 20, 2010, Andreas Dilger wrote:
> On 2010-10-19, at 08:27, Roger Spellman wrote:
> > I don't understand this comment:
> >> For the MDT, yes, you could potentially use "-i 1500" as about the
> >> minimum space per inode, but then you risk running out of space in the
> >> filesystem before running out of inodes.
> > 
> > If we set -I to 512, then on an MDT, what else is there that would cause
> > require 1500 bytes per inode?
> 
> With "-I 512" that means the actual inode will consume 512 bytes, so with
> "-i 1536" there would be 1024 bytes per inode of block space still
> available.  That extra space is needed for everything else in the
> filesystem, including the journal, directory blocks, Lustre metadata
> (last_rcvd, distributed transaction logs, etc), and any external xattr
> blocks for widely-striped files (beyond 12 stripes or so).
> 

I have to admit, I entirely fail to understand why we should need 2/3 of the 
filesystem reserved for real file data.

- journal - 400MB -> negligible with recent decent MDT sizes (1TiB+)
- directory blocks, maybe, but I have noticed any system where that takes more 
than 5%
- Lustre metadata > (last_rcvd, distributed transaction logs, etc) -> 
negligible with recent decent MDT sizes

- external xattr for Lustre lov and additional ACLs: Maybe, depends on the 
customer

With the default -i 4096, it looks like that for most customers I know of:

df -h:
973G   57G  861G   7% /lustre/lustre/mdt

df -ih:
278M    248M     31M   89% /lustre/lustre/mdt


So doubling inode ratio to -i2048 or even quadrupling it to -i1024 seems to be 
recommendable. 


Cheers,
Bernd



More information about the lustre-discuss mailing list