[lustre-discuss] ldiskfs / mdt size limits

[Andreas Dilger]

Thomas,
You are exactly correct that large MDTs can be useful for DoM if you have HDD OSTs. The benefit is relatively small if you have NVMe OSTs. 

If the MDT is larger than 16TB it must be formatted with the extents feature to address block numbers over 2^32. Unfortunately, this is _slightly_ less efficient than the (in)direct block addressing for very fragmented allocations, like those of directories, so this feature is not used for MDTs below 16TiB. 

Cheers, Andreas

> On Feb 2, 2024, at 06:35, Thomas Roth via lustre-discuss <lustre-discuss at lists.lustre.org> wrote:
> 
> Hi all,
> 
> confused  about size limits:
> 
> I distinctly remember trying to format a ~19 TB disk / LV for use as an MDT, with ldiskfs, and failing to do so: the max size for the underlying ext4 is 16 TB.
> Knew that, had ignoed that, but not a problem back then - just adapted the logical volume's size.
> 
> Now I have a 24T disk, and neither mkfs.lustre nor Lustre itself have show any issues with it.
> 'df -h' does show the 24T, 'df -ih' shows the expected 4G of inodes.
> I suppose this MDS has a lot of space for directories and stuff, or for DOM.
> But why does it work in the first place? ldiskfs extends beyond all limits these days?
> 
> Regards,
> Thomas
> 
> --
> --------------------------------------------------------------------
> Thomas Roth
> Department: Informationstechnologie
> Location: SB3 2.291
> 
> 
> GSI Helmholtzzentrum für Schwerionenforschung GmbH
> Planckstraße 1, 64291 Darmstadt, Germany, www.gsi.de
> 
> Commercial Register / Handelsregister: Amtsgericht Darmstadt, HRB 1528
> Managing Directors / Geschäftsführung:
> Professor Dr. Paolo Giubellino, Jörg Blaurock
> Chairman of the Supervisory Board / Vorsitzender des GSI-Aufsichtsrats:
> Ministerialdirigent Dr. Volkmar Dietz
> 
> _______________________________________________
> lustre-discuss mailing list
> lustre-discuss at lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org