[Lustre-discuss] OST - low MB/s

[Dr. Hung-Sheng Tsao (LaoTsao)]

raid 6 chunk size=128k the full strip size will be 128k*8 (for 10 disks 
8+2 raid 6)
in Ur dd test one should use bs=128k*8, then each 8 HDD will be busy
regards


Rafael David Tinoco wrote:
>
> With this RAID5 configuration Im getting:
>
> root at a02n00:~# dd if=/dev/zero of=/dev/md20 bs=128k count=10000
>
> 10000+0 records in
>
> 10000+0 records out
>
> 1310720000 bytes (1.3 GB) copied, 5.20774 seconds, 252 MB/s
>
> root at a02n00:~# dd if=/dev/zero of=/dev/md20 bs=128k count=10000
>
> 10000+0 records in
>
> 10000+0 records out
>
> 1310720000 bytes (1.3 GB) copied, 5.12 seconds, 256 MB/s
>
> So, 80MB/s using these md20 as OSTs isnt quite right …
>
> *From:* lustre-discuss-bounces at lists.lustre.org 
> [mailto:lustre-discuss-bounces at lists.lustre.org] *On Behalf Of *Rafael 
> David Tinoco
> *Sent:* Thursday, September 10, 2009 4:26 PM
> *To:* lustre-discuss at lists.lustre.org
> *Subject:* [Lustre-discuss] OST - low MB/s
>
> Hello,
>
> I’m having problems now with my “OSTs” throughput.
>
> I have 4 OSS each one with 2 OSTs. These OSTs are RAID6 with 10 disks, 
> chunk size of 128k.
>
> These disks are from J4400 (JBOD) connected in multipath using multipathd.
>
> Each disk speed is giving me 50MB/s with dd.
>
> With lustre, using IOR or DD I can get only arround 80MB/s. I was 
> expecting for 8 active disks in raid 8*50 = something between 300 and 
> 400MB/s.
>
> avg-cpu: %user %nice %system %iowait %steal %idle
>
> 0.00 0.00 6.00 9.06 0.00 84.94
>
> Device: rrqm/s wrqm/s r/s w/s rMB/s wMB/s avgrq-sz avgqu-sz await 
> svctm %util
>
> md10 0.00 0.00 0.00 398.00 0.00 1.55 8.00 0.00 0.00 0.00 0.00
>
> md11 0.00 0.00 0.00 380.00 0.00 1.48 8.00 0.00 0.00 0.00 0.00
>
> md20 0.00 0.00 0.00 158.00 0.00 79.00 1024.00 0.00 0.00 0.00 0.00
>
> md21 0.00 0.00 0.00 159.00 0.00 79.50 1024.00 0.00 0.00 0.00 0.00
>
> avg-cpu: %user %nice %system %iowait %steal %idle
>
> 0.00 0.00 5.94 9.32 0.00 84.74
>
> Device: rrqm/s wrqm/s r/s w/s rMB/s wMB/s avgrq-sz avgqu-sz await 
> svctm %util
>
> md10 0.00 0.00 0.00 407.50 0.00 1.59 8.00 0.00 0.00 0.00 0.00
>
> md11 0.00 0.00 0.00 394.00 0.00 1.54 8.00 0.00 0.00 0.00 0.00
>
> md20 0.00 0.00 0.00 159.00 0.00 79.50 1024.00 0.00 0.00 0.00 0.00
>
> md21 0.00 0.00 0.00 158.00 0.00 79.00 1024.00 0.00 0.00 0.00 0.00
>
> avg-cpu: %user %nice %system %iowait %steal %idle
>
> 0.00 0.00 6.37 9.43 0.00 84.21
>
> Device: rrqm/s wrqm/s r/s w/s rMB/s wMB/s avgrq-sz avgqu-sz await 
> svctm %util
>
> md10 0.00 0.00 0.00 410.50 0.00 1.60 8.00 0.00 0.00 0.00 0.00
>
> md11 0.00 0.00 0.00 376.00 0.00 1.47 8.00 0.00 0.00 0.00 0.00
>
> md20 0.00 0.00 0.00 165.00 0.00 82.50 1024.00 0.00 0.00 0.00 0.00
>
> md21 0.00 0.00 0.00 165.00 0.00 82.50 1024.00 0.00 0.00 0.00 0.00
>
> Any clues ?
>
> Rafael David Tinoco - Sun Microsystems
>
> Systems Engineer - High Performance Computing
>
> Rafael.Tinoco at Sun.COM - 55.11.5187.2194
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-discuss
>