[lustre-discuss] weird issue w. lnet routers

[Raj]

John, increasing MTU size on Ethernet side should increase the b/w. I also
have a feeling that some lnet routers and/or intermediate switches/routers
does not have jumbo frame turned on (some switches needs to be set at 9212
bytes ).
How many LNet  routers are you using? I believe you are routing between EDR
IB and 100GbE.


On Tue, Nov 28, 2017 at 7:21 PM John Casu <john at chiraldynamics.com> wrote:

> just built a system w. lnet routers that bridge Infiniband & 100GbE, using
> Centos built in Infiniband support
> servers are Infiniband, clients are 100GbE (connectx-4 cards)
>
> my direct write performance from clients over Infiniband is around 15GB/s
>
> When I introduced the lnet routers, performance dropped to 10GB/s
>
> Thought the problem was an MTU of 1500, but when I changed the MTUs to 9000
> performance dropped to 3GB/s.
>
> When I tuned according to John Fragella's LUG slides, things went even
> slower (1.5GB/s write)
>
> does anyone have any ideas on what I'm doing wrong??
>
> thanks,
> -john c.
>
> _______________________________________________
> lustre-discuss mailing list
> lustre-discuss at lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20171129/1009d2d2/attachment.html>