[Lustre-discuss] Understanding lfs output

[Mag Gam]

Thanks Andreas.

Is this documented anywhere? We are facing inode shortage problems on
our filesystem and I would like to show this to my professors. The
reason why this is  occurring is..

I checked the operations manual and Lustre "inter galactic" file
system but no luck of this issue.

TIA


On Wed, Aug 20, 2008 at 1:55 AM, Andreas Dilger <adilger at sun.com> wrote:
> On Aug 19, 2008  20:27 -0400, Mag Gam wrote:
>> Yes, I have looked thru the lists but I could not really get this
>> question answered. tune2fs is a ext2/3 setting, and I though each file
>> I create on a Lustre filesystem an inode gets created on the MDS and
>> the OST as an object.
>>
>> But, I don't understand how there is that much of a space discrepancy.
>
> The reason for this discrepancy is because of "safety".  It is possible
> that the striping is too large to fit into the inode, so it needs an
> external block.  If that is needed for every remaining inode on the MDS
> then the limit on the number of files created is equal to the number of
> free blocks on the MDS.  In order to avoid over-stating the number of
> possible files that can be created (sometimes important to e.g. RPM
> or regression tests), "df" and "lfs df" will return min(free inodes,
> free blocks) for the inode count.
>
> If (as usually happens) inodes are created that do not need external blocks
> to store the striping information then more inodes "appear" (i.e. as
> "IUsed" increases so does "Inodes" and "IFree" stays constant).
>
>> Also, how does striping account for in inodes? Lets say I stipe a file
>> on 4 OSTs, I suppose 5 (1 for MDS, 4 for OSTs) inodes are occupied,
>> correct?
>
> Yes, though each of the inodes are in different filesystems.
>
>> On Tue, Aug 19, 2008 at 4:30 PM, Brian J. Murrell <Brian.Murrell at sun.com> wrote:
>> > On Sun, 2008-08-17 at 15:50 -0700, Mag Gam wrote:
>> >> My departments Lustre's lfs df -i looks like this
>> >>
>> >> UUID                    Inodes     IUsed     IFree IUse% Mounted on
>> >> xfs001-MDT0000_UUID   82565322  19073586  63491736   23%
>> >> /xfs/engine1/xfs001[MDT:0]
>> >> xfs001-OST0000_UUID   41943040   5076445  36866595   12%
>> >> /xfs/engine1/xfs001[OST:0]
>> >> xfs001-OST0001_UUID   41943040   3735715  38207325    8%
>> >> /xfs/engine1/xfs001[OST:1]
>> >> xfs001-OST0002_UUID   41943040   4784447  37158593   11%
>> >> /xfs/engine1/xfs001[OST:2]
>> >> xfs001-OST0003_UUID   41943040   4556627  37386413   10%
>> >> /xfs/engine1/xfs001[OST:3]
>> >>
>> >> filesystem summary:   82565322  19073586  63491736   23% /xfs/engine1/xfs001
>> >>
>> >>
>> >> Is it safe to say, "
>> >> My MDT has "19073586" 19 million inodes free? or should I consider.
>> >>
>> >> However, when I check my inode count on my mds using tune2fs it reports this.
>> >> Inode count:              524288000
>> >> Free inodes:              505214414
>> >>
>> >> Am I missing something?
>> >
>> > Yes.  lfs df does not simply report the free inodes on the MDT.  It's
>> > more complicated than that.  This has been discussed either here or in a
>> > bugzilla bug (don't recall which).  I don't recall the bug or when the
>> > lustre-discussion was, offhand, but a search should not be terribly
>> > difficult to find the answer.
>> >
>> > b.
>> >
>> >
>> > _______________________________________________
>> > Lustre-discuss mailing list
>> > Lustre-discuss at lists.lustre.org
>> > http://lists.lustre.org/mailman/listinfo/lustre-discuss
>> >
>> >
>> _______________________________________________
>> Lustre-discuss mailing list
>> Lustre-discuss at lists.lustre.org
>> http://lists.lustre.org/mailman/listinfo/lustre-discuss
>
> Cheers, Andreas
> --
> Andreas Dilger
> Sr. Staff Engineer, Lustre Group
> Sun Microsystems of Canada, Inc.
>
>