<div><br> Thanks Andreas! </div>
<div> </div>
<div> Disabling checksumming certainly leads to a big performance impact on the client side.</div>
<div> However it looks like we still have some performance gap between <a href="http://1.6.4.3">1.6.4.3</a> and <a href="http://1.6.5.">1.6.5.</a></div>
<div> I have repeated the tests making sure that the file sizes are much larger than available</div>
<div> RAM on the client, to avoid any caching effects. Here is what came out:</div>
<div> </div>
<div> Single stream writing: (lmdd of=/lustre/tstfileXX bs=1M time=200 fsync=1) </div>
<div> </div>
<div> Client: AMD 2354 @ 2.21 GHz 2xQuad core, 16GB RAM, Infiniband, Servers: official <a href="http://1.6.4.1">1.6.4.1</a></div>
<div> --------------------------------------------------------------------------------------------------------------------------------------</div>
<div> Kernel: official 2.6.9-67.0.4.EL_lustre.1.6.4.3smp, Lustre: <a href="http://1.6.4.3">1.6.4.3</a> official : 681 MB/sec</div>
<div> Kernel: patchless 2.6.7-67.0.20.ELsmp x86_64, Lustre: 1.6.5 (no checksum) : 590 MB/sec</div>
<div> Kernel: patchless 2.6.7-67.0.20.ELsmp x86_64, Lustre: 1.6.5 (with checksum) : 265 MB/sec</div>
<div> </div>
<div> </div>
<div> Client: Intel X5450 @ 3.00GHz 2xQuad core, 16GB RAM, Infiniband, Servers: official <a href="http://1.6.4.1">1.6.4.1</a></div>
<div> --------------------------------------------------------------------------------------------------------------------------------------</div>
<div>
<div> Kernel: official 2.6.9-67.0.4.EL_lustre.1.6.4.3smp, Lustre: <a href="http://1.6.4.3">1.6.4.3</a> : 832 MB/sec</div>
<div> Kernel: official 2.6.9-67.0.7.EL_lustre.1.6.5smp, Lustre: 1.6.5 (no checksum) : 681 MB/sec</div>
<div> Kernel: patchless 2.6.7-67.0.20.ELsmp x86_64, Lustre: 1.6.5 (no checksum) : 675 MB/sec</div>
<div> Kernel: official 2.6.9-67.0.7.EL_lustre.1.6.5smp, Lustre: 1.6.5 (with checksum) : 326 MB/sec </div>
<div> Kernel: patchless 2.6.7-67.0.20.ELsmp x86_64, Lustre: 1.6.5 (with checksum) : 322 MB/sec</div></div>
<div><br> Here we see that <a href="http://1.6.5.0">1.6.5.0</a> with fully patched client and no checksumming still performs worse</div>
<div> than <a href="http://1.6.4.3">1.6.4.3</a> with fully patched client (only 681 MB/sec against 832 MB/sec, almost 18% less).</div>
<div> Is there some other parameter to play with?</div>
<div> </div>
<div> Regards - Andrei.</div>
<div> </div>
<div> </div>
<div> </div>
<div class="gmail_quote">On Fri, Jul 4, 2008 at 6:52 AM, Andreas Dilger <<a href="mailto:adilger@sun.com">adilger@sun.com</a>> wrote:<br>
<blockquote class="gmail_quote" style="PADDING-LEFT: 1ex; MARGIN: 0px 0px 0px 0.8ex; BORDER-LEFT: #ccc 1px solid">
<div>
<div></div>
<div class="Wj3C7c"> </div></div>Can you try disabling checksumming on the client:<br><br> lctl set_param osc.*.checksums=0<br><br>Cheers, Andreas<br><font color="#888888">--<br>Andreas Dilger<br>Sr. Staff Engineer, Lustre Group<br>
Sun Microsystems of Canada, Inc.<br><br></font></blockquote></div><br>