[Lustre-discuss] explanation about inodes in MDT/OST

Dilger, Andreas andreas.dilger at intel.com
Wed Jun 19 08:33:14 PDT 2013

On 2013-06-19, at 8:52, "Lee, Brett" <brett.lee at intel.com<mailto:brett.lee at intel.com>> wrote:
We probably should clarify the context of inodes:

1) Each OST has "inodes" – these inodes refer to objects on that OST that contain block data for a Lustre file.

I prefer to call these as OST objects, to keep them distinct from the MDT inodes.

2) Each (as of 2.4) MDT has "inodes" – these inodes contain metadata for a single Lustre file.

When adding an OST you gain the inodes on that OST for objects (block data) on that OST, but the total amount of Lustre files (MDT inodes) hasn't changed.

Right, though Mr. Rosenberg is correct that since the MDT uses an ext4-based backing storage it is possible to use offline resize2fs to increase the size of the MDT afterward, if it is using LVM or some other form of storage that can be resized.  Online resizing is blocked behind LU-137 being finished.

Note that resize2fs is not something that we test on any regular basis, because most of our customers have large fixed-size devices for the MDT (over-provisioning on spindles). If someone needs to do this, is encourage them to make a "dd" copy of the MDT to a new device first (could just be a USB SATA disk) and test out the resize2fs first, and verify with e2fsck before and after that the filesystem is still valid.

I'll see whether i can get resize2fs testing added into our normal test suite.

Cheers, Andreas

From: lustre-discuss-bounces at lists.lustre.org<mailto:lustre-discuss-bounces at lists.lustre.org> [mailto:lustre-discuss-bounces at lists.lustre.org] On Behalf Of Parinay Kondekar
Sent: Wednesday, June 19, 2013 8:23 AM
To: E.S. Rosenberg
Cc: lustre-discuss at lists.lustre.org<mailto:lustre-discuss at lists.lustre.org>
Subject: Re: [Lustre-discuss] explanation about inodes in MDT/OST

IMO there are two limiting factors

1. total number of inodes supported by MDT and
2.  OSTs  i.e. num_ost_inodes = 4 * number_of_mds_inodes * default_stripe_count / number_of_osts

so your Q
I am not sure I understand this 100%, when adding an OST I do or don't
gain inodes?or you basically only get more inodes in the data part of
the filesystem but if you don't have enough inodes on the MDT you
still have a problem?
Yes, if there are no enough inodes on MDT adding OSTs wont help much.

I don't think you can change no. of inodes w/o formatting. You might want to take a look at
lfs_migrate, on how to increase the no_of_inodes on OST's.


On 19 June 2013 19:19, E.S. Rosenberg <esr+lustre at mail.hebrew.edu<mailto:esr+lustre at mail.hebrew.edu>> wrote:
In a Lustre ldiskfs file system, all the inodes are allocated on the
MDT and OSTs when the file system is first formatted. The total number
of inodes on a formatted MDT or OST cannot be easily changed, although
it is possible to add OSTs with additional space and corresponding
inodes. Thus, the number of inodes created at format time should be
generous enough to anticipate future expansion.

Also if lustrefs is really based on ext4 doesn't ext4 support growing
the amount of inodes with resize2fs?

Lustre-discuss mailing list
Lustre-discuss at lists.lustre.org<mailto:Lustre-discuss at lists.lustre.org>

Lustre-discuss mailing list
Lustre-discuss at lists.lustre.org<mailto:Lustre-discuss at lists.lustre.org>

More information about the lustre-discuss mailing list