[Lustre-discuss] OST - low MB/s

[Rafael David Tinoco]

Hello,
 
I'm having problems now with my "OSTs" throughput.


I have 4 OSS each one with 2 OSTs. These OSTs are RAID6 with 10 disks, chunk size of 128k.
These disks are from J4400 (JBOD) connected in multipath using multipathd.
 
Each disk speed is giving me 50MB/s with dd.
With lustre, using IOR or DD I can get only arround 80MB/s. I was expecting for 8 active disks in raid 8*50 = something between 300
and 400MB/s.
 
avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           0.00    0.00    6.00    9.06    0.00   84.94
 
Device:         rrqm/s   wrqm/s   r/s   w/s    rMB/s    wMB/s avgrq-sz avgqu-sz   await  svctm  %util
md10              0.00     0.00  0.00 398.00     0.00     1.55     8.00     0.00    0.00   0.00   0.00
md11              0.00     0.00  0.00 380.00     0.00     1.48     8.00     0.00    0.00   0.00   0.00
md20              0.00     0.00  0.00 158.00     0.00    79.00  1024.00     0.00    0.00   0.00   0.00
md21              0.00     0.00  0.00 159.00     0.00    79.50  1024.00     0.00    0.00   0.00   0.00
 
avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           0.00    0.00    5.94    9.32    0.00   84.74
 
Device:         rrqm/s   wrqm/s   r/s   w/s    rMB/s    wMB/s avgrq-sz avgqu-sz   await  svctm  %util
md10              0.00     0.00  0.00 407.50     0.00     1.59     8.00     0.00    0.00   0.00   0.00
md11              0.00     0.00  0.00 394.00     0.00     1.54     8.00     0.00    0.00   0.00   0.00
md20              0.00     0.00  0.00 159.00     0.00    79.50  1024.00     0.00    0.00   0.00   0.00
md21              0.00     0.00  0.00 158.00     0.00    79.00  1024.00     0.00    0.00   0.00   0.00
 
avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           0.00    0.00    6.37    9.43    0.00   84.21
 
Device:         rrqm/s   wrqm/s   r/s   w/s    rMB/s    wMB/s avgrq-sz avgqu-sz   await  svctm  %util
md10              0.00     0.00  0.00 410.50     0.00     1.60     8.00     0.00    0.00   0.00   0.00
md11              0.00     0.00  0.00 376.00     0.00     1.47     8.00     0.00    0.00   0.00   0.00
md20              0.00     0.00  0.00 165.00     0.00    82.50  1024.00     0.00    0.00   0.00   0.00
md21              0.00     0.00  0.00 165.00     0.00    82.50  1024.00     0.00    0.00   0.00   0.00
 
Any clues ? 
 
Rafael David Tinoco - Sun Microsystems
Systems Engineer - High Performance Computing
Rafael.Tinoco at Sun.COM - 55.11.5187.2194
 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20090910/0f901adb/attachment.htm>