[Lustre-discuss] Unbalanced OST--for discussion purposes

[Ms. Megan Larko]

Thanks to both Brian and Andreas for the timely responses.
Brian posed the question as to whether or not the OSTs were more or
less balanced a week ago.  The answer is that I believe that they
were.   Usually all OSTs report a similar percentage of usage (between
1%  and 3% of one another).   I believe that is why this new report
piqued my curiosity.

Regarding Andreas remark about individual OST size, yes I understand
that having larger individual OSTs can preempt any one OST from
becoming so full that the others degrade in performance (per A.
Dilger, not B. Murrel).   For that reason I personally like the option
available in newer Lustre releases (I think 1.8.x and higher) to allow
up to 16Tb in a single OST slice.  I know the previous limit was 8Tb
per OST slice for precaution against data corruption.   (I was able to
build a larger OST slice with 1.6.7 but I was cautioned that some data
may become unreachable and/or corrupted as the Lustre system had not
at that time been modified to accept the larger partition sizes which
the underlying files systems--ext4, xfs---would accept.)    The OST
formatted size of 6.3Tb fit nicely into the JBOD scheme of
evenly-sized partitions.

Thanks,
megan

On Tue, 2010-03-02 at 15:45 -0500, Ms. Megan Larko wrote:
> Hi,

Hi,

> I logged directly into the OSS (OSS4) and just ran a df (along with a
> periodic check of the log files).  I last looked about two weeks ago
> (I know it was after 17 Feb).

Is the implication that at this point the OSTs were more or less well
balanced?

> Anyway, the OST0007 is more full than
> any of the other OSTs.  The default lustre stripe (I believe that is
> set to 1) is used.    Can just one file shift the size used of one OST
> that significantly?

Sure.  As an example, if one had a 1KiB file on that OST, called, let's
say, "1K_file.dat" and one did:

$ dd if=/dev/zero of=1K_file.dat bs=1G count=1024

that would overwrite the 1KiB file on that OST with a 1TiB file.
Recognizing of course that that would be 1TiB in a single object on an
OST.

> What other reasonable explanation for a
> difference on one OST in comparison with the others?

Any kind of variation on the above.

> Could this cause
> a lustre performance hit at this point?

Not really.

b.