[Lustre-discuss] Performance drop (1.6.5 vs 1.6.4.3, OFED 1.2)?

[Andrei Maslennikov]

 Thanks Andreas!

 Disabling checksumming certainly leads to a big performance impact on the
client side.
 However it looks like we still have some performance gap between 1.6.4.3and
1.6.5.
 I have repeated the tests making sure that the file sizes are much larger
than available
 RAM on the client, to avoid any caching effects. Here is what came out:

 Single stream writing: (lmdd of=/lustre/tstfileXX bs=1M time=200 fsync=1)

 Client: AMD 2354 @ 2.21 GHz 2xQuad core, 16GB RAM, Infiniband, Servers:
official 1.6.4.1
 --------------------------------------------------------------------------------------------------------------------------------------
 Kernel: official      2.6.9-67.0.4.EL_lustre.1.6.4.3smp, Lustre:
1.6.4.3 official
 :  681 MB/sec
 Kernel: patchless 2.6.7-67.0.20.ELsmp x86_64, Lustre: 1.6.5 (no
checksum)        :  590 MB/sec
 Kernel: patchless 2.6.7-67.0.20.ELsmp x86_64, Lustre: 1.6.5 (with
checksum)      :  265 MB/sec


 Client: Intel  X5450  @ 3.00GHz 2xQuad core, 16GB RAM, Infiniband, Servers:
official 1.6.4.1
 --------------------------------------------------------------------------------------------------------------------------------------
  Kernel: official      2.6.9-67.0.4.EL_lustre.1.6.4.3smp, Lustre:
1.6.4.3
          :  832 MB/sec
 Kernel: official      2.6.9-67.0.7.EL_lustre.1.6.5smp, Lustre: 1.6.5 (no
checksum)   :  681 MB/sec
 Kernel: patchless 2.6.7-67.0.20.ELsmp x86_64, Lustre: 1.6.5 (no
checksum)        :  675 MB/sec
 Kernel: official      2.6.9-67.0.7.EL_lustre.1.6.5smp, Lustre: 1.6.5 (with
checksum) :  326 MB/sec
 Kernel: patchless 2.6.7-67.0.20.ELsmp x86_64, Lustre: 1.6.5 (with
checksum)      :  322 MB/sec

 Here we see that 1.6.5.0 with fully patched client and no checksumming
still performs worse
 than 1.6.4.3 with fully patched client (only 681 MB/sec against 832 MB/sec,
almost 18% less).
 Is there some other parameter to play with?

 Regards - Andrei.



On Fri, Jul 4, 2008 at 6:52 AM, Andreas Dilger <adilger at sun.com> wrote:

>
> Can you try disabling checksumming on the client:
>
>        lctl set_param osc.*.checksums=0
>
> Cheers, Andreas
> --
> Andreas Dilger
> Sr. Staff Engineer, Lustre Group
> Sun Microsystems of Canada, Inc.
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20080706/302b23a5/attachment.htm>