[Lustre-discuss] Optimize parallel compilation on lustre

[Andreas Dilger]

On 2010-07-16, at 0:27, Maxence Dunnewind <maxence at dunnewind.net> wrote:
> I just tried on qt4, and it compiles correctly, the results are :
> -j 16  : 30min35 against 32 min
> -j 8 : same time (34min25 vs 34min36)

Thanks for testing this. What it means is that there is very little contention on the client's single metadata write RPC until there are a lot (16) of concurrent threads.  

I'm assuming the two numbers are for compiles on Lustre with and without the patch?

How does this performance compare to local filesystem performance?

If you are reading all of the input files into cache before the start of the run (e.g. find . | xargs cat > /dev/null) then the slowdown isn't from reading the small input files from the OSTs.

 If you are writing the output to a separate directory, then the namespace cache of the input files is not being invalidated. Even so, the fact that the patch didn't improve performance means the metadata changes are not blocking the threads much. 

You also increased the write cache size, so it _shouldn't_ be that writing the output is taking too long. 

I'm a bit at a loss for further suggestions for you to test, unless you want to start into things like profile on the client. 

Cheers, Andreas