[lustre-discuss] weird issue w. lnet routers

John Casu john at chiraldynamics.com
Tue Nov 28 17:21:10 PST 2017

just built a system w. lnet routers that bridge Infiniband & 100GbE, using Centos built in Infiniband support
servers are Infiniband, clients are 100GbE (connectx-4 cards)

my direct write performance from clients over Infiniband is around 15GB/s

When I introduced the lnet routers, performance dropped to 10GB/s

Thought the problem was an MTU of 1500, but when I changed the MTUs to 9000
performance dropped to 3GB/s.

When I tuned according to John Fragella's LUG slides, things went even slower (1.5GB/s write)

does anyone have any ideas on what I'm doing wrong??

-john c.

More information about the lustre-discuss mailing list