<html><head></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><div>OK, IB back to back is at 1,2 GB/s, 10GE back to back at 950 MB/s, with additional lnet router I see 550 MB/s. Time for lnet tuning?</div><div><br></div><div>Michael</div><div><br><blockquote type="cite"><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">Hi Andreas,<div><br><div><div><div>Am 10.09.2010 um 16:35 schrieb Andreas Dilger:</div><br class="Apple-interchange-newline"><blockquote type="cite"><div>On 2010-09-10, at 08:23, Michael Kluge wrote:<br><blockquote type="cite">I have a Lustre 1.8.3 setup where I'd like to some lnet router performance tests with routing between DDR IB<->10GE networks. Currently I have three nodes, one with DDR IB, one with 10GE and one with both that does the routing. A first short lnet test shows 520-550 MB/s performance. <br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">Has anyone an idea which of the variables of the lnet module are worth playing with to get this number a bit closer to 1GB/s? <br></blockquote><br>I would start by testing the performance on just the 10GigE side, and then separately on the IB side, to verify you are getting the expected performance from the components before trying them both together. Often it is necessary to tune the ethernet send/receive buffers.</div></blockquote><br></div><div>Ethernet back to back is at 950 MB/s. I have not looked at IB back to back yet.</div><div><br></div><div><br></div><div>Michael</div><div><span class="Apple-style-span" style="border-collapse: separate; font-family: 'Andale Mono'; font-size: medium; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; "><span class="Apple-style-span" style="border-collapse: separate; font-family: 'Andale Mono'; font-size: medium; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><br>-- <br><br>Michael Kluge, M.Sc.<br><br>Technische Universität Dresden<br>Center for Information Services and<br>High Performance Computing (ZIH)<br>D-01062 Dresden<br>Germany<br><br>Contact:<br>Willersbau, Room WIL A 208<br>Phone: (+49) 351 463-34217<br>Fax: (+49) 351 463-37773<br>e-mail: <a href="mailto:michael.kluge@tu-dresden.de">michael.kluge@tu-dresden.de</a><br>WWW: <a href="http://www.tu-dresden.de/zih">http://www.tu-dresden.de/zih</a></div></span></span>
</div>
<br></div></div></div>_______________________________________________<br>Lustre-discuss mailing list<br><a href="mailto:Lustre-discuss@lists.lustre.org">Lustre-discuss@lists.lustre.org</a><br>http://lists.lustre.org/mailman/listinfo/lustre-discuss<br></blockquote></div><br><div>
<span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: 'Andale Mono'; font-size: medium; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; "><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: 'Andale Mono'; font-size: medium; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><br>-- <br><br>Michael Kluge, M.Sc.<br><br>Technische Universität Dresden<br>Center for Information Services and<br>High Performance Computing (ZIH)<br>D-01062 Dresden<br>Germany<br><br>Contact:<br>Willersbau, Room WIL A 208<br>Phone: (+49) 351 463-34217<br>Fax: (+49) 351 463-37773<br>e-mail: <a href="mailto:michael.kluge@tu-dresden.de">michael.kluge@tu-dresden.de</a><br>WWW: <a href="http://www.tu-dresden.de/zih">http://www.tu-dresden.de/zih</a></div></span></span>
</div>
<br></body></html>