[Lustre-discuss] Performance drop (1.6.5 vs 1.6.4.3, OFED 1.2)?

[Andrei Maslennikov]

  *New performance numbers (1.6.5.1 vs 1.6.4.3):*


---------------------------------------------------------------------------------------
   Client   : Intel  X5450 at 3.00GHz 2xQuad core, 16GB RAM,
               Infiniband, RHEL4 x86_64
  Servers: Official 1.6.4.1
  Single stream writing: (lmdd of=/lustre/tstfileXX bs=1M time=200 fsync=1)

---------------------------------------------------------------------------------------



  *2.6.9-67.0.20.ELsmp unmodified, OFED 1.2,                      319 MB/sec
*  *Lustre 1.6.5.1 (with checksumming): *
                                        **
  Client loads: lmdd - 100% (1 CPU), ptlrpcd - 5% , pdflush- 15%
  On 2 OSS servers in use: circa 50% total sys (2 CPUs), circa 10% I/O wait.

  *2.6.9-67.0.7.EL_lustre.1.6.5.1smp, OFED 1.3,                      340
MB/sec
*  *Lustre 1.6.5.1 (with checksumming):*   Client loads: lmdd - 100% (1
CPU), ptlrpcd - 5%, pdflush- 15%
  On 2 OSS servers in use: circa 50% total sys (2 CPUs), circa 12% I/O wait.

  *2.6.9-67.0.20.ELsmp unmodified, OFED 1.2,                       671
MB/sec
*  *Lustre 1.6.5.1 (no checksumming)  :*  *
*
  Client loads: lmdd - 100% (1 CPU), ptlrpcd - 15%, pdflush- 2-3%
   On 2 OSS servers in use: circa 35% total sys (2 CPUs), circa 35% I/O
wait.

  *2.6.9-67.0.7.EL_lustre.1.6.5.1smp, OFED 1.3,                       670
MB/sec
*  *Lustre 1.6.5.1 (no checksumming)  :*   Client loads: lmdd - 100% (1
CPU), ptlrpcd - 12%, pdflush- 2-3%
   On 2 OSS servers in use: circa 32% total sys (2 CPUs), circa 32% I/O
wait.

  *2.6.9-67.0.4.EL_lustre.1.6.4.3smp, OFED 1.2,                       843
MB/sec*
  *Lustre 1.6.4.3 *
  Client loads: lmdd - 100% (1 CPU), ptlrpcd - 20%, pdflush - 1%
   On 2 OSS servers in use: circa 33 % total sys (2 CPUs), circa 30% I/O
wait.



--------------------------------------------------------------------------------------
  Running several (2,4) simultaneous jobs on the same 1.6.4.3 client
  does not improve the aggregate performance. I have seen 750 MB/sec
  aggregate with 4 streams, and 806 MB/sec aggregate with 2 streams.

  With 1.6.5.1 client with no checksumming I can get up to 800 MB/sec
  aggregate with 4 streams, and some 730 MB/sec with 2 streams.

  But Lustre 1.6.5.1 is visibly (20%) less performant on a single stream
when
  compared with 1.6.4.3.

  Andrei.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20080716/4f02ff48/attachment.htm>