[Lustre-discuss] lnet router tuning

[Michael Kluge]

Hi all,

I have a Lustre 1.8.3 setup where I'd like to some lnet router
performance tests with routing between DDR IB<->10GE networks. Currently
I have three nodes, one with DDR IB, one with 10GE and one with both
that does the routing. A first short lnet test shows 520-550 MB/s
performance. 

Has anyone an idea which of the variables of the lnet module are worth
playing with to get this number a bit closer to 1GB/s? 

parm:           tiny_router_buffers:# of 0 payload messages to buffer in
the router (int)
parm:           small_router_buffers:# of small (1 page) messages to
buffer in the router (int)
parm:           large_router_buffers:# of large messages to buffer in
the router (int)
parm:           peer_buffer_credits:# router buffer credits per peer
(int)

The CPU on the router node is less utilized than it was when I did back
to back 10GE tests. I have 6 cores in the machine, 5 have been idle and
one showing a load of about 60%.


Michael

-- 

Michael Kluge, M.Sc.

Technische Universität Dresden
Center for Information Services and
High Performance Computing (ZIH)
D-01062 Dresden
Germany

Contact:
Willersbau, Room A 208
Phone:  (+49) 351 463-34217
Fax:    (+49) 351 463-37773
e-mail: michael.kluge at tu-dresden.de
WWW:    http://www.tu-dresden.de/zih
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20100910/aa42fc56/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 5997 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20100910/aa42fc56/attachment.bin>