[Lustre-discuss] abnormally long ftruncates on Cray XT4

[Rob Latham]

On Fri, Dec 11, 2009 at 10:58:19AM -0800, Mark Howison wrote:
> The purple indicates ftruncate, which is only called from task 0 (that
> is how MPI_File_set_size is implemented). The salmon at the far right
> actually indicates fsyncs, not reads. And the brown (hard to see) is
> fclose.

Just to add a bit of ROMIO MPI-IO background while you explore this:

In the old days every MPI process called ftruncate, and of course that
gave brutally poor performance.  This "ftruncate from one" approach
has been in place since 2004.  I would be both shocked and appalled if
you encountered an ancient ROMIO-based MPI-IO implementation that
still does ftruncate from all.

if you are nfs-exporting this lustre volume for some reason, ROMIO
will behave quite differently (and slowly).  But you aren't doing that
or you'd see a very different IPM graph.
 
==rob

-- 
Rob Latham
Mathematics and Computer Science Division
Argonne National Lab, IL USA