[Lustre-discuss] OST - low MB/s

[Rafael David Tinoco]

With RAID 10 using 10 disks:
 
root at a02n00:~# dd if=/dev/zero of=/dev/md20 bs=128k count=10000
10000+0 records in
10000+0 records out
1310720000 bytes (1.3 GB) copied, 3.18607 seconds, 411 MB/s
 
Maybe, problems in performance RAID5 from CentOS 5.3 ?
 
Anybody ?
 
From: lustre-discuss-bounces at lists.lustre.org [mailto:lustre-discuss-bounces at lists.lustre.org] On Behalf Of Rafael David Tinoco
Sent: Thursday, September 10, 2009 5:10 PM
To: lustre-discuss at lists.lustre.org
Subject: Re: [Lustre-discuss] OST - low MB/s
 
With this RAID5 configuration Im getting:
 
root at a02n00:~# dd if=/dev/zero of=/dev/md20 bs=128k count=10000
10000+0 records in
10000+0 records out
1310720000 bytes (1.3 GB) copied, 5.20774 seconds, 252 MB/s
 
root at a02n00:~# dd if=/dev/zero of=/dev/md20 bs=128k count=10000
10000+0 records in
10000+0 records out
1310720000 bytes (1.3 GB) copied, 5.12 seconds, 256 MB/s
 
So, 80MB/s using these md20 as OSTs isnt quite right . 
 
 
From: lustre-discuss-bounces at lists.lustre.org [mailto:lustre-discuss-bounces at lists.lustre.org] On Behalf Of Rafael David Tinoco
Sent: Thursday, September 10, 2009 4:26 PM
To: lustre-discuss at lists.lustre.org
Subject: [Lustre-discuss] OST - low MB/s
 
Hello,
 
I'm having problems now with my "OSTs" throughput.


I have 4 OSS each one with 2 OSTs. These OSTs are RAID6 with 10 disks, chunk size of 128k.
These disks are from J4400 (JBOD) connected in multipath using multipathd.
 
Each disk speed is giving me 50MB/s with dd.
With lustre, using IOR or DD I can get only arround 80MB/s. I was expecting for 8 active disks in raid 8*50 = something between 300
and 400MB/s.
 
avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           0.00    0.00    6.00    9.06    0.00   84.94
 
Device:         rrqm/s   wrqm/s   r/s   w/s    rMB/s    wMB/s avgrq-sz avgqu-sz   await  svctm  %util
md10              0.00     0.00  0.00 398.00     0.00     1.55     8.00     0.00    0.00   0.00   0.00
md11              0.00     0.00  0.00 380.00     0.00     1.48     8.00     0.00    0.00   0.00   0.00
md20              0.00     0.00  0.00 158.00     0.00    79.00  1024.00     0.00    0.00   0.00   0.00
md21              0.00     0.00  0.00 159.00     0.00    79.50  1024.00     0.00    0.00   0.00   0.00
 
avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           0.00    0.00    5.94    9.32    0.00   84.74
 
Device:         rrqm/s   wrqm/s   r/s   w/s    rMB/s    wMB/s avgrq-sz avgqu-sz   await  svctm  %util
md10              0.00     0.00  0.00 407.50     0.00     1.59     8.00     0.00    0.00   0.00   0.00
md11              0.00     0.00  0.00 394.00     0.00     1.54     8.00     0.00    0.00   0.00   0.00
md20              0.00     0.00  0.00 159.00     0.00    79.50  1024.00     0.00    0.00   0.00   0.00
md21              0.00     0.00  0.00 158.00     0.00    79.00  1024.00     0.00    0.00   0.00   0.00
 
avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           0.00    0.00    6.37    9.43    0.00   84.21
 
Device:         rrqm/s   wrqm/s   r/s   w/s    rMB/s    wMB/s avgrq-sz avgqu-sz   await  svctm  %util
md10              0.00     0.00  0.00 410.50     0.00     1.60     8.00     0.00    0.00   0.00   0.00
md11              0.00     0.00  0.00 376.00     0.00     1.47     8.00     0.00    0.00   0.00   0.00
md20              0.00     0.00  0.00 165.00     0.00    82.50  1024.00     0.00    0.00   0.00   0.00
md21              0.00     0.00  0.00 165.00     0.00    82.50  1024.00     0.00    0.00   0.00   0.00
 
Any clues ? 
 
Rafael David Tinoco - Sun Microsystems
Systems Engineer - High Performance Computing
Rafael.Tinoco at Sun.COM - 55.11.5187.2194
 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20090910/6732febb/attachment.htm>