Hi Guys,<br><br>I have setup a new cluster with a infiniband interconnect and I am a bit confused with the performance I am getting as far as transfer rates. Is there something that I am missing here? Shouldn't the transfer rate over the ib interface be much faster and not the same compared with the 1G (bonded mode=6) interface?<br>
<br>(I am new to infiniband but this does not seem right. My routing is also setup correctly. Not sure what I might be missing here):<br><br>--<br>hpc101:/var/tmp # time scp SLES-11-DVD-x86_64-GM-DVD1.iso root@hpc103:/var/tmp/<br>
SLES-11-DVD-x86_64-GM-DVD1.iso 100% 2749MB 62.5MB/s 00:44 <br>
<br>real 0m43.882s<br>user 0m29.482s<br>sys 0m6.736s<br><br>hpc101:/var/tmp # time scp SLES-11-DVD-x86_64-GM-DVD1.iso root@hpc103-ib:/var/tmp/<br>SLES-11-DVD-x86_64-GM-DVD1.iso 100% 2749MB 80.9MB/s 00:34 <br>
<br>real 0m35.757s<br>user 0m24.498s<br>sys 0m6.292s<br><br>hpc101:/var/tmp # netstat -rn<br>Kernel IP routing table<br>Destination Gateway Genmask Flags MSS Window irtt Iface<br>10.0.250.0 0.0.0.0 255.255.255.0 U 0 0 0 ib0<br>
128.137.126.0 0.0.0.0 255.255.255.0 U 0 0 0 bond0<br>127.0.0.0 0.0.0.0 255.0.0.0 U 0 0 0 lo<br>0.0.0.0 128.137.126.253 0.0.0.0 UG 0 0 0 bond0<br>
<br>hpc101:/var/tmp # ifconfig ib0<br>ib0 Link encap:UNSPEC HWaddr 80-00-00-48-FE-80-00-00-00-00-00-00-00-00-00-00 <br> inet addr:10.0.250.5 Bcast:10.0.250.255 Mask:255.255.255.0<br> inet6 addr: fe80::223:7dff:ff93:bf4d/64 Scope:Link<br>
UP BROADCAST RUNNING MULTICAST MTU:2044 Metric:1<br> RX packets:493685 errors:0 dropped:0 overruns:0 frame:0<br> TX packets:890402 errors:0 dropped:0 overruns:0 carrier:0<br> collisions:0 txqueuelen:256 <br>
RX bytes:22848079 (21.7 Mb) TX bytes:7349365631 (7008.9 Mb)<br>--<br><br>Thanks in advance for your assistance.<br><br>Regards,<br>-J<br>