[lustre-discuss] Mixed size OST's

Martin Hecht hecht at hlrs.de
Fri Mar 16 05:25:36 PDT 2018


On 03/15/2018 04:48 PM, Steve Thompson wrote:
> If I go with one OST per system (one zpool comprising 8 x 6 RAIDZ2
> vdevs), I will have a lustre f/s comprised of two 60 TB OST's and two
> 192 TB OST's (minus RAIDZ2 overhead). This is obviously a big mismatch
> between OST sizes.
Depending on how full your file system is going to be, it may be better
to create more OSTs on the new OSSes to have all OSTs roughly of the
same size and avoid trouble balancing the fill level of the OSTs.

We had a lustre system (back in lustre 1.8 times) with different disk
sizes. We did put them into pools such that each pool contains only OSTs
of the same size. We balanced the users between the pools such that the
larger OSTs were filled more quickly than the smaller ones, or put in
other words such that the percentage how much an OST was filled remained
homogeneous across the whole file system. It worked, but this manual
interaction was needed to prevent the small OSTs from reaching a
critical filling level more quickly than the large ones.

Maybe the internal algorithm has been improved in the meantime, but as
far as I know it is just round robin until a critical difference of
levels is reached and the weighted stripe allocation impacts
performance. A rough description can be found on
wiki.lustre.org/Managing_Free_Space or in the Lustre Manual in the
corresponding chapter. However, I'm not sure if these sections are all
up to date.


-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 5254 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20180316/05a2ccfa/attachment.bin>


More information about the lustre-discuss mailing list