[Lustre-discuss] Slow du

[Daniel Kobras]

Hi!

On Wed, Feb 03, 2010 at 04:36:43PM -0500, Larry Brown wrote:
> We have a cluster set up with 60 nodes.  We have a folder with the
> striping count set to 20.  We loaded up two folders with a large number
> of small files.  I know the bigger the file the better the more Lustre
> shines but we have multiple things to test and this is the first.
> According to "lfs df -h" there is a total disk use of 76.3G.  I ran "du
> -sh" at the top level of the folders and am now at 45 minutes run time
> without a result yet.  Top only shows du taking between 3 and 5% cpu
> time and 1% of memory.
> 
> Does anyone know what causes this?  Shouldn't the server be able to
> examine the MDS to sum the space used?  At worst wouldn't the first
> object on the stripe return the total file size if it isn't kept on the
> MDS?

In Lustre 1.8, the MDS doesn't know the accurate size of a file. This feature
is planned for Lustre 2.x. Size information therefore currently needs to be
obtained from the OSTs, and as files can contain holes, there's no way
knowing the end of the file until all objects have been examined--even if
for a small file most of them turn out to be empty.

Furthermore, unless you set a stripe count hint, or otherwise overrode the
default inode size when creating the MDT, the stripe information won't fit
the inode and therefore require an extra seek for each file.

Regards,

Daniel.