<html><head><base href="x-msg://97/"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">Hi Eric,<div><br></div><div>basically right now I have one IB node, one 10GE node and one router node that has both types of network interfaces.</div><div><div><br></div><div>I've got a small lnet test script on the router node, that does the work:</div><div>export LST_SESSION=$$</div><div>lst new_session rw</div><div>lst add_group readers 192.168.10.8@tcp</div><div>lst add_group writers 10.148.0.94@o2ib</div><div>lst add_batch bulk_rw</div><div>lst add_test --batch bulk_rw --from writers --to readers brw read check=simple size=1M</div><div>lst run bulk_rw</div><div>lst stat writers & sleep 30; kill $!</div><div>lst end_session</div></div><div><br></div><div>Is there a way to figure out the messages in flight? I remember to have a "rpc's in flight" tunable but this is connected to the OSC layer which does not do anything in my case (I think).</div><div><br></div><div><br></div><div>Michael</div><div><br></div><div><br></div><div><br><div><div>Am 13.09.2010 um 03:08 schrieb Eric Barton:</div><br class="Apple-interchange-newline"><blockquote type="cite"><div lang="EN-GB" link="blue" vlink="purple" style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><div class="WordSection1" style="page: WordSection1; "><div style="margin-top: 0cm; margin-right: 0cm; margin-bottom: 0.0001pt; margin-left: 0cm; font-size: 12pt; font-family: 'Times New Roman', serif; "><span style="color: rgb(31, 73, 125); "><o:p> </o:p></span></div><div style="margin-top: 0cm; margin-right: 0cm; margin-bottom: 0.0001pt; margin-left: 0cm; font-size: 12pt; font-family: 'Times New Roman', serif; "><span style="color: rgb(31, 73, 125); ">Michael,<o:p></o:p></span></div><div style="margin-top: 0cm; margin-right: 0cm; margin-bottom: 0.0001pt; margin-left: 0cm; font-size: 12pt; font-family: 'Times New Roman', serif; "><span style="color: rgb(31, 73, 125); "><o:p> </o:p></span></div><div style="margin-top: 0cm; margin-right: 0cm; margin-bottom: 0.0001pt; margin-left: 0cm; font-size: 12pt; font-family: 'Times New Roman', serif; "><span style="color: rgb(31, 73, 125); "><o:p> </o:p></span></div><div style="margin-top: 0cm; margin-right: 0cm; margin-bottom: 0.0001pt; margin-left: 0cm; font-size: 12pt; font-family: 'Times New Roman', serif; "><span style="color: rgb(31, 73, 125); ">How are you generating load and measuring the throughput? I’m particularly interested in the number<o:p></o:p></span></div><div style="margin-top: 0cm; margin-right: 0cm; margin-bottom: 0.0001pt; margin-left: 0cm; font-size: 12pt; font-family: 'Times New Roman', serif; "><span style="color: rgb(31, 73, 125); ">of nodes on each side of the router and how many messages you have in flight between each one.<o:p></o:p></span></div><div style="margin-top: 0cm; margin-right: 0cm; margin-bottom: 0.0001pt; margin-left: 0cm; font-size: 12pt; font-family: 'Times New Roman', serif; "><span style="color: rgb(31, 73, 125); "><o:p> </o:p></span></div><div style="margin-top: 0cm; margin-right: 0cm; margin-bottom: 0.0001pt; margin-left: 0cm; font-size: 12pt; font-family: 'Times New Roman', serif; "><span style="color: rgb(31, 73, 125); "><o:p> </o:p></span></div><blockquote style="margin-top: 5pt; margin-bottom: 5pt; "><div style="margin-top: 0cm; margin-right: 0cm; margin-bottom: 0.0001pt; margin-left: 0cm; font-size: 12pt; font-family: 'Times New Roman', serif; "><span style="color: rgb(31, 73, 125); ">Cheers,<br> Eric<o:p></o:p></span></div></blockquote><div style="margin-top: 0cm; margin-right: 0cm; margin-bottom: 0.0001pt; margin-left: 0cm; font-size: 12pt; font-family: 'Times New Roman', serif; "><span style="color: rgb(31, 73, 125); "><o:p> </o:p></span></div><div style="margin-top: 0cm; margin-right: 0cm; margin-bottom: 0.0001pt; margin-left: 0cm; font-size: 12pt; font-family: 'Times New Roman', serif; "><span style="color: rgb(31, 73, 125); "><o:p> </o:p></span></div><div style="margin-top: 0cm; margin-right: 0cm; margin-bottom: 0.0001pt; margin-left: 0cm; font-size: 12pt; font-family: 'Times New Roman', serif; "><span style="color: rgb(31, 73, 125); "><o:p> </o:p></span></div><div style="margin-top: 0cm; margin-right: 0cm; margin-bottom: 0.0001pt; margin-left: 0cm; font-size: 12pt; font-family: 'Times New Roman', serif; "><span style="color: rgb(31, 73, 125); "><o:p> </o:p></span></div><div style="border-top-style: none; border-right-style: none; border-bottom-style: none; border-width: initial; border-color: initial; border-left-style: solid; border-left-color: blue; border-left-width: 1.5pt; padding-top: 0cm; padding-right: 0cm; padding-bottom: 0cm; padding-left: 4pt; "><div><div style="border-right-style: none; border-bottom-style: none; border-left-style: none; border-width: initial; border-color: initial; border-top-style: solid; border-top-color: rgb(181, 196, 223); border-top-width: 1pt; padding-top: 3pt; padding-right: 0cm; padding-bottom: 0cm; padding-left: 0cm; "><div style="margin-top: 0cm; margin-right: 0cm; margin-bottom: 0.0001pt; margin-left: 0cm; font-size: 12pt; font-family: 'Times New Roman', serif; "><b><span lang="EN-US" style="font-size: 10pt; font-family: Tahoma, sans-serif; ">From:</span></b><span lang="EN-US" style="font-size: 10pt; font-family: Tahoma, sans-serif; "><span class="Apple-converted-space"> </span><a href="mailto:lustre-discuss-bounces@lists.lustre.org" style="color: blue; text-decoration: underline; ">lustre-discuss-bounces@lists.lustre.org</a><span class="Apple-converted-space"> </span>[mailto:lustre-discuss-bounces@lists.lustre.org]<span class="Apple-converted-space"> </span><b>On Behalf Of<span class="Apple-converted-space"> </span></b>Michael Kluge<br><b>Sent:</b><span class="Apple-converted-space"> </span>11 September 2010 12:56 AM<br><b>To:</b><span class="Apple-converted-space"> </span>Michael Kluge<br><b>Cc:</b><span class="Apple-converted-space"> </span>Lustre Diskussionsliste<br><b>Subject:</b><span class="Apple-converted-space"> </span>Re: [Lustre-discuss] lnet router tuning<o:p></o:p></span></div></div></div><div style="margin-top: 0cm; margin-right: 0cm; margin-bottom: 0.0001pt; margin-left: 0cm; font-size: 12pt; font-family: 'Times New Roman', serif; "><o:p> </o:p></div><div><div style="margin-top: 0cm; margin-right: 0cm; margin-bottom: 0.0001pt; margin-left: 0cm; font-size: 12pt; font-family: 'Times New Roman', serif; ">And here are my params:<o:p></o:p></div></div><div><div style="margin-top: 0cm; margin-right: 0cm; margin-bottom: 0.0001pt; margin-left: 0cm; font-size: 12pt; font-family: 'Times New Roman', serif; "><o:p> </o:p></div></div><div><div style="margin-top: 0cm; margin-right: 0cm; margin-bottom: 0.0001pt; margin-left: 0cm; font-size: 12pt; font-family: 'Times New Roman', serif; ">root@doss05:/home/tests/lnet# for F in /sys/module/lnet/parameters/* ; do echo -n "$F: "; cat $F ; done<o:p></o:p></div></div><div><div style="margin-top: 0cm; margin-right: 0cm; margin-bottom: 0.0001pt; margin-left: 0cm; font-size: 12pt; font-family: 'Times New Roman', serif; ">/sys/module/lnet/parameters/accept: secure<o:p></o:p></div></div><div><div style="margin-top: 0cm; margin-right: 0cm; margin-bottom: 0.0001pt; margin-left: 0cm; font-size: 12pt; font-family: 'Times New Roman', serif; ">/sys/module/lnet/parameters/accept_backlog: 127<o:p></o:p></div></div><div><div style="margin-top: 0cm; margin-right: 0cm; margin-bottom: 0.0001pt; margin-left: 0cm; font-size: 12pt; font-family: 'Times New Roman', serif; ">/sys/module/lnet/parameters/accept_port: 988<o:p></o:p></div></div><div><div style="margin-top: 0cm; margin-right: 0cm; margin-bottom: 0.0001pt; margin-left: 0cm; font-size: 12pt; font-family: 'Times New Roman', serif; ">/sys/module/lnet/parameters/accept_timeout: 5<o:p></o:p></div></div><div><div style="margin-top: 0cm; margin-right: 0cm; margin-bottom: 0.0001pt; margin-left: 0cm; font-size: 12pt; font-family: 'Times New Roman', serif; ">/sys/module/lnet/parameters/auto_down: 1<o:p></o:p></div></div><div><div style="margin-top: 0cm; margin-right: 0cm; margin-bottom: 0.0001pt; margin-left: 0cm; font-size: 12pt; font-family: 'Times New Roman', serif; ">/sys/module/lnet/parameters/avoid_asym_router_failure: 0<o:p></o:p></div></div><div><div style="margin-top: 0cm; margin-right: 0cm; margin-bottom: 0.0001pt; margin-left: 0cm; font-size: 12pt; font-family: 'Times New Roman', serif; ">/sys/module/lnet/parameters/check_routers_before_use: 0<o:p></o:p></div></div><div><div style="margin-top: 0cm; margin-right: 0cm; margin-bottom: 0.0001pt; margin-left: 0cm; font-size: 12pt; font-family: 'Times New Roman', serif; ">/sys/module/lnet/parameters/config_on_load: 0<o:p></o:p></div></div><div><div style="margin-top: 0cm; margin-right: 0cm; margin-bottom: 0.0001pt; margin-left: 0cm; font-size: 12pt; font-family: 'Times New Roman', serif; ">/sys/module/lnet/parameters/dead_router_check_interval: 0<o:p></o:p></div></div><div><div style="margin-top: 0cm; margin-right: 0cm; margin-bottom: 0.0001pt; margin-left: 0cm; font-size: 12pt; font-family: 'Times New Roman', serif; ">/sys/module/lnet/parameters/forwarding: enabled<o:p></o:p></div></div><div><div style="margin-top: 0cm; margin-right: 0cm; margin-bottom: 0.0001pt; margin-left: 0cm; font-size: 12pt; font-family: 'Times New Roman', serif; ">/sys/module/lnet/parameters/ip2nets: <o:p></o:p></div></div><div><div style="margin-top: 0cm; margin-right: 0cm; margin-bottom: 0.0001pt; margin-left: 0cm; font-size: 12pt; font-family: 'Times New Roman', serif; ">/sys/module/lnet/parameters/large_router_buffers: 512<o:p></o:p></div></div><div><div style="margin-top: 0cm; margin-right: 0cm; margin-bottom: 0.0001pt; margin-left: 0cm; font-size: 12pt; font-family: 'Times New Roman', serif; ">/sys/module/lnet/parameters/live_router_check_interval: 0<o:p></o:p></div></div><div><div style="margin-top: 0cm; margin-right: 0cm; margin-bottom: 0.0001pt; margin-left: 0cm; font-size: 12pt; font-family: 'Times New Roman', serif; ">/sys/module/lnet/parameters/local_nid_dist_zero: 1<o:p></o:p></div></div><div><div style="margin-top: 0cm; margin-right: 0cm; margin-bottom: 0.0001pt; margin-left: 0cm; font-size: 12pt; font-family: 'Times New Roman', serif; ">/sys/module/lnet/parameters/networks: tcp0(eth2),o2ib(ib1)<o:p></o:p></div></div><div><div style="margin-top: 0cm; margin-right: 0cm; margin-bottom: 0.0001pt; margin-left: 0cm; font-size: 12pt; font-family: 'Times New Roman', serif; ">/sys/module/lnet/parameters/peer_buffer_credits: 0<o:p></o:p></div></div><div><div style="margin-top: 0cm; margin-right: 0cm; margin-bottom: 0.0001pt; margin-left: 0cm; font-size: 12pt; font-family: 'Times New Roman', serif; ">/sys/module/lnet/parameters/portals_compatibility: none<o:p></o:p></div></div><div><div style="margin-top: 0cm; margin-right: 0cm; margin-bottom: 0.0001pt; margin-left: 0cm; font-size: 12pt; font-family: 'Times New Roman', serif; ">/sys/module/lnet/parameters/router_ping_timeout: 50<o:p></o:p></div></div><div><div style="margin-top: 0cm; margin-right: 0cm; margin-bottom: 0.0001pt; margin-left: 0cm; font-size: 12pt; font-family: 'Times New Roman', serif; ">/sys/module/lnet/parameters/routes: <o:p></o:p></div></div><div><div style="margin-top: 0cm; margin-right: 0cm; margin-bottom: 0.0001pt; margin-left: 0cm; font-size: 12pt; font-family: 'Times New Roman', serif; ">/sys/module/lnet/parameters/small_router_buffers: 8192<o:p></o:p></div></div><div><div style="margin-top: 0cm; margin-right: 0cm; margin-bottom: 0.0001pt; margin-left: 0cm; font-size: 12pt; font-family: 'Times New Roman', serif; ">/sys/module/lnet/parameters/tiny_router_buffers: 1024<o:p></o:p></div></div><div><div style="margin-top: 0cm; margin-right: 0cm; margin-bottom: 0.0001pt; margin-left: 0cm; font-size: 12pt; font-family: 'Times New Roman', serif; "><o:p> </o:p></div></div><div><div style="margin-top: 0cm; margin-right: 0cm; margin-bottom: 0.0001pt; margin-left: 0cm; font-size: 12pt; font-family: 'Times New Roman', serif; ">I have not used ip2nets but configure routing but put explict routing statements into the modprobe.d/ files. Is that OK? <o:p></o:p></div></div><div><div style="margin-top: 0cm; margin-right: 0cm; margin-bottom: 0.0001pt; margin-left: 0cm; font-size: 12pt; font-family: 'Times New Roman', serif; "><o:p> </o:p></div></div><div><div style="margin-top: 0cm; margin-right: 0cm; margin-bottom: 0.0001pt; margin-left: 0cm; font-size: 12pt; font-family: 'Times New Roman', serif; "><o:p> </o:p></div></div><div><div style="margin-top: 0cm; margin-right: 0cm; margin-bottom: 0.0001pt; margin-left: 0cm; font-size: 12pt; font-family: 'Times New Roman', serif; ">Michael<o:p></o:p></div></div><div><div style="margin-top: 0cm; margin-right: 0cm; margin-bottom: 0.0001pt; margin-left: 0cm; font-size: 12pt; font-family: 'Times New Roman', serif; "><o:p> </o:p></div></div><div><div style="margin-top: 0cm; margin-right: 0cm; margin-bottom: 0.0001pt; margin-left: 0cm; font-size: 12pt; font-family: 'Times New Roman', serif; "><o:p> </o:p></div></div><div><div><div style="margin-top: 0cm; margin-right: 0cm; margin-bottom: 0.0001pt; margin-left: 0cm; font-size: 12pt; font-family: 'Times New Roman', serif; ">Am 10.09.2010 um 17:48 schrieb Michael Kluge:<o:p></o:p></div></div><div style="margin-top: 0cm; margin-right: 0cm; margin-bottom: 0.0001pt; margin-left: 0cm; font-size: 12pt; font-family: 'Times New Roman', serif; "><br><br><o:p></o:p></div><div><div><div style="margin-top: 0cm; margin-right: 0cm; margin-bottom: 0.0001pt; margin-left: 0cm; font-size: 12pt; font-family: 'Times New Roman', serif; ">OK, IB back to back is at 1,2 GB/s, 10GE back to back at 950 MB/s, with additional lnet router I see 550 MB/s. Time for lnet tuning?<o:p></o:p></div></div><div><div style="margin-top: 0cm; margin-right: 0cm; margin-bottom: 0.0001pt; margin-left: 0cm; font-size: 12pt; font-family: 'Times New Roman', serif; "><o:p> </o:p></div></div><div><div style="margin-top: 0cm; margin-right: 0cm; margin-bottom: 0.0001pt; margin-left: 0cm; font-size: 12pt; font-family: 'Times New Roman', serif; ">Michael<o:p></o:p></div></div><div><div style="margin-top: 0cm; margin-right: 0cm; margin-bottom: 0.0001pt; margin-left: 0cm; font-size: 12pt; font-family: 'Times New Roman', serif; "><br><br><o:p></o:p></div><div><div style="margin-top: 0cm; margin-right: 0cm; margin-bottom: 0.0001pt; margin-left: 0cm; font-size: 12pt; font-family: 'Times New Roman', serif; ">Hi Andreas,<o:p></o:p></div><div><div style="margin-top: 0cm; margin-right: 0cm; margin-bottom: 0.0001pt; margin-left: 0cm; font-size: 12pt; font-family: 'Times New Roman', serif; "><o:p> </o:p></div><div><div><div><div style="margin-top: 0cm; margin-right: 0cm; margin-bottom: 0.0001pt; margin-left: 0cm; font-size: 12pt; font-family: 'Times New Roman', serif; ">Am 10.09.2010 um 16:35 schrieb Andreas Dilger:<o:p></o:p></div></div><div style="margin-top: 0cm; margin-right: 0cm; margin-bottom: 0.0001pt; margin-left: 0cm; font-size: 12pt; font-family: 'Times New Roman', serif; "><br><br><o:p></o:p></div><div><div style="margin-top: 0cm; margin-right: 0cm; margin-bottom: 0.0001pt; margin-left: 0cm; font-size: 12pt; font-family: 'Times New Roman', serif; ">On 2010-09-10, at 08:23, Michael Kluge wrote:<br><br><o:p></o:p></div><div style="margin-top: 0cm; margin-right: 0cm; margin-bottom: 0.0001pt; margin-left: 0cm; font-size: 12pt; font-family: 'Times New Roman', serif; ">I have a Lustre 1.8.3 setup where I'd like to some lnet router performance tests with routing between DDR IB<->10GE networks. Currently I have three nodes, one with DDR IB, one with 10GE and one with both that does the routing. A first short lnet test shows 520-550 MB/s performance.<o:p></o:p></div><blockquote style="margin-top: 5pt; margin-bottom: 5pt; "><div style="margin-top: 0cm; margin-right: 0cm; margin-bottom: 0.0001pt; margin-left: 0cm; font-size: 12pt; font-family: 'Times New Roman', serif; "><o:p> </o:p></div></blockquote><blockquote style="margin-top: 5pt; margin-bottom: 5pt; "><div style="margin-top: 0cm; margin-right: 0cm; margin-bottom: 0.0001pt; margin-left: 0cm; font-size: 12pt; font-family: 'Times New Roman', serif; ">Has anyone an idea which of the variables of the lnet module are worth playing with to get this number a bit closer to 1GB/s?<o:p></o:p></div></blockquote><div style="margin-top: 0cm; margin-right: 0cm; margin-bottom: 0.0001pt; margin-left: 0cm; font-size: 12pt; font-family: 'Times New Roman', serif; "><br>I would start by testing the performance on just the 10GigE side, and then separately on the IB side, to verify you are getting the expected performance from the components before trying them both together. Often it is necessary to tune the ethernet send/receive buffers.<o:p></o:p></div></div><div style="margin-top: 0cm; margin-right: 0cm; margin-bottom: 0.0001pt; margin-left: 0cm; font-size: 12pt; font-family: 'Times New Roman', serif; "><o:p> </o:p></div></div><div><div style="margin-top: 0cm; margin-right: 0cm; margin-bottom: 0.0001pt; margin-left: 0cm; font-size: 12pt; font-family: 'Times New Roman', serif; ">Ethernet back to back is at 950 MB/s. I have not looked at IB back to back yet.<o:p></o:p></div></div><div><div style="margin-top: 0cm; margin-right: 0cm; margin-bottom: 0.0001pt; margin-left: 0cm; font-size: 12pt; font-family: 'Times New Roman', serif; "><o:p> </o:p></div></div><div><div style="margin-top: 0cm; margin-right: 0cm; margin-bottom: 0.0001pt; margin-left: 0cm; font-size: 12pt; font-family: 'Times New Roman', serif; "><o:p> </o:p></div></div><div><div style="margin-top: 0cm; margin-right: 0cm; margin-bottom: 0.0001pt; margin-left: 0cm; font-size: 12pt; font-family: 'Times New Roman', serif; ">Michael<o:p></o:p></div></div><div><div><div style="margin-top: 0cm; margin-right: 0cm; margin-bottom: 0.0001pt; margin-left: 0cm; font-size: 12pt; font-family: 'Times New Roman', serif; "><br>-- <br><br>Michael Kluge, M.Sc.<br><br>Technische Universität Dresden<br>Center for Information Services and<br>High Performance Computing (ZIH)<br>D-01062 Dresden<br>Germany<br><br>Contact:<br>Willersbau, Room WIL A 208<br>Phone: (+49) 351 463-34217<br>Fax: (+49) 351 463-37773<br>e-mail: <a href="mailto:michael.kluge@tu-dresden.de" style="color: blue; text-decoration: underline; ">michael.kluge@tu-dresden.de</a><br>WWW: <a href="http://www.tu-dresden.de/zih" style="color: blue; text-decoration: underline; ">http://www.tu-dresden.de/zih</a><o:p></o:p></div></div></div><div style="margin-top: 0cm; margin-right: 0cm; margin-bottom: 0.0001pt; margin-left: 0cm; font-size: 12pt; font-family: 'Times New Roman', serif; "><o:p> </o:p></div></div></div></div><div style="margin-top: 0cm; margin-right: 0cm; margin-bottom: 0.0001pt; margin-left: 0cm; font-size: 12pt; font-family: 'Times New Roman', serif; ">_______________________________________________<br>Lustre-discuss mailing list<br><a href="mailto:Lustre-discuss@lists.lustre.org" style="color: blue; text-decoration: underline; ">Lustre-discuss@lists.lustre.org</a><br><a href="http://lists.lustre.org/mailman/listinfo/lustre-discuss" style="color: blue; text-decoration: underline; ">http://lists.lustre.org/mailman/listinfo/lustre-discuss</a><o:p></o:p></div></div><div style="margin-top: 0cm; margin-right: 0cm; margin-bottom: 0.0001pt; margin-left: 0cm; font-size: 12pt; font-family: 'Times New Roman', serif; "><o:p> </o:p></div><div><div><div style="margin-top: 0cm; margin-right: 0cm; margin-bottom: 0.0001pt; margin-left: 0cm; font-size: 12pt; font-family: 'Times New Roman', serif; "><span style="font-size: 13.5pt; font-family: 'Andale Mono', serif; "><br>-- <br><br>Michael Kluge, M.Sc.<br><br>Technische Universität Dresden<br>Center for Information Services and<br>High Performance Computing (ZIH)<br>D-01062 Dresden<br>Germany<br><br>Contact:<br>Willersbau, Room WIL A 208<br>Phone: (+49) 351 463-34217<br>Fax: (+49) 351 463-37773<br>e-mail: <a href="mailto:michael.kluge@tu-dresden.de" style="color: blue; text-decoration: underline; ">michael.kluge@tu-dresden.de</a><br>WWW: <a href="http://www.tu-dresden.de/zih" style="color: blue; text-decoration: underline; ">http://www.tu-dresden.de/zih</a><o:p></o:p></span></div></div></div><div style="margin-top: 0cm; margin-right: 0cm; margin-bottom: 0.0001pt; margin-left: 0cm; font-size: 12pt; font-family: 'Times New Roman', serif; "><o:p> </o:p></div></div><div style="margin-top: 0cm; margin-right: 0cm; margin-bottom: 0.0001pt; margin-left: 0cm; font-size: 12pt; font-family: 'Times New Roman', serif; ">_______________________________________________<br>Lustre-discuss mailing list<br><a href="mailto:Lustre-discuss@lists.lustre.org" style="color: blue; text-decoration: underline; ">Lustre-discuss@lists.lustre.org</a><br><a href="http://lists.lustre.org/mailman/listinfo/lustre-discuss" style="color: blue; text-decoration: underline; ">http://lists.lustre.org/mailman/listinfo/lustre-discuss</a><o:p></o:p></div></div><div style="margin-top: 0cm; margin-right: 0cm; margin-bottom: 0.0001pt; margin-left: 0cm; font-size: 12pt; font-family: 'Times New Roman', serif; "><o:p> </o:p></div><div><div><div style="margin-top: 0cm; margin-right: 0cm; margin-bottom: 0.0001pt; margin-left: 0cm; font-size: 12pt; font-family: 'Times New Roman', serif; "><span style="font-size: 13.5pt; font-family: 'Andale Mono', serif; color: black; "><br>-- <br><br>Michael Kluge, M.Sc.<br><br>Technische Universität Dresden<br>Center for Information Services and<br>High Performance Computing (ZIH)<br>D-01062 Dresden<br>Germany<br><br>Contact:<br>Willersbau, Room WIL A 208<br>Phone: (+49) 351 463-34217<br>Fax: (+49) 351 463-37773<br>e-mail: <a href="mailto:michael.kluge@tu-dresden.de" style="color: blue; text-decoration: underline; ">michael.kluge@tu-dresden.de</a><br>WWW: <a href="http://www.tu-dresden.de/zih" style="color: blue; text-decoration: underline; ">http://www.tu-dresden.de/zih</a><o:p></o:p></span></div></div></div><div style="margin-top: 0cm; margin-right: 0cm; margin-bottom: 0.0001pt; margin-left: 0cm; font-size: 12pt; font-family: 'Times New Roman', serif; "><o:p> </o:p></div></div></div>_______________________________________________<br>Lustre-discuss mailing list<br><a href="mailto:Lustre-discuss@lists.lustre.org" style="color: blue; text-decoration: underline; ">Lustre-discuss@lists.lustre.org</a><br><a href="http://lists.lustre.org/mailman/listinfo/lustre-discuss" style="color: blue; text-decoration: underline; ">http://lists.lustre.org/mailman/listinfo/lustre-discuss</a><br></div></blockquote></div><br><div>
<span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: 'Andale Mono'; font-size: medium; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; "><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: 'Andale Mono'; font-size: medium; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><br>-- <br><br>Michael Kluge, M.Sc.<br><br>Technische Universität Dresden<br>Center for Information Services and<br>High Performance Computing (ZIH)<br>D-01062 Dresden<br>Germany<br><br>Contact:<br>Willersbau, Room WIL A 208<br>Phone: (+49) 351 463-34217<br>Fax: (+49) 351 463-37773<br>e-mail: <a href="mailto:michael.kluge@tu-dresden.de">michael.kluge@tu-dresden.de</a><br>WWW: <a href="http://www.tu-dresden.de/zih">http://www.tu-dresden.de/zih</a></div></span></span>
</div>
<br></div></body></html>