[Lustre-discuss] Bad distribution of files among OSTs

Thomas Roth t.roth at gsi.de
Fri Oct 30 11:07:51 PDT 2009


Hi all,

in our 196 OST - Cluster, the previously perfect distribution of files
among the OSTs is not working anymore, since ~ 2 weeks.
The filling for most OSTs is between 57% and 62%, but some (~10)  have
risen up to 94%. I'm trying to fix that by having these OSTs deactivated
on the MDT and finding and migrating away data from them, but it seems
I'm not fast enough and it's a ongoing problem - I've just deactivated
another OST with threatening 67%.

Our qos_prio_free is at the default 90%.

Our OST's sizes are between 2.3TB and 4.5TB. We use striping level 1, so
it would be possible to fill up an OST by just creating a 2TB file.
However, I'm not aware of any such gigafiles (using robinhood to get a
picture of our file system).

In addition, our user's behavior should not have changed recently. In
August, the entire cluster had filled up to almost 80% in a neatly even
distribution among the OSTs, so we extended the cluster by more OSTs,
migrating data to even the filling between old and new ones. This also
succeeded, and up to October there was no indication of something not
working.

There are no error message in the logs that would point to some OSTs
being favored ;-)

So, what could be the cause of this misdistribution?

Regards,
Thomas





More information about the lustre-discuss mailing list