[Lustre-discuss] Maximum OST Size

[Andreas Dilger]

On 2010-10-19, at 08:27, Roger Spellman wrote:
> I don't understand this comment:
>> For the MDT, yes, you could potentially use "-i 1500" as about the 
>> minimum space per inode, but then you risk running out of space in the
>> filesystem before running out of inodes.
> 
> If we set -I to 512, then on an MDT, what else is there that would cause
> require 1500 bytes per inode?

With "-I 512" that means the actual inode will consume 512 bytes, so with "-i 1536" there would be 1024 bytes per inode of block space still available.  That extra space is needed for everything else in the filesystem, including the journal, directory blocks, Lustre metadata (last_rcvd, distributed transaction logs, etc), and any external xattr blocks for widely-striped files (beyond 12 stripes or so).

> Just ACLs and striping?  If there are no ACLs, and all files are single-striped, then could both -i and -I be set to the same value, say 512?

No, this will cause mke2fs to fail.  There needs to be some free space in the filesystem for the above filesysem/Lustre metadata.  In any case, since the maximum number of inodes is 2^32 the total filesystem size is not the limiting factor.

> Andreas Dilger wrote:
>> On 2010-10-18, at 14:47, Roger Spellman wrote:
>>> In Lustre 2.x, what is the largest number of files that we could
>>> possibly have?
>>> 
>>> I noticed that mkfs.lustre on the MDT passes the following
>>> parameters to mkfs.ext2:  -i 4096 -I 512
>> 
>> Do you mean the maximum OST size (as mentioned in the subject) or the
>> maximum MDT size (above)?  For the ext4-based ldiskfs the maximum size
>> is 16TB and 4B inodes (this listed in the manual).
>> 
>>> Can these params be smaller?
>> 
>> For the MDT, yes, you could potentially use "-i 1500" as about the
>> minimum space per inode, but then you risk running out of space in the
>> filesystem before running out of inodes.  The "-I 512" parameter controls
>> the size of the inode itself, which holds the xattrs.  If there are
>> single-striped files and no use of ACLs, user_xattrs, etc. then you might
>> get by with "-I 256", but if this xattr space is exceeded then each such
>> inode will consume 4096 bytes of space and also be slower to access.
>> 
>>> Can we get more inodes if we use zfs?
>> 
>> Definitely yes.
>> 
>> Cheers, Andreas
>> --
>> Andreas Dilger
>> Lustre Technical Lead
>> Oracle Corporation Canada Inc.
> 


Cheers, Andreas
--
Andreas Dilger
Lustre Technical Lead
Oracle Corporation Canada Inc.