[lustre-discuss] Bad performance lustre
Benjamin GALINA
benjamin.galina at eolen.com
Mon Nov 30 06:54:14 PST 2020
Hi Lustre Community ,
Could you help me solve my luster perfomance problem. ?
I have just set up several Lustre gateways between two infiniband networks A and B.
Node A <--> Getway <--> Node B
lustre 2.12.2 Lustre 2.13.3 Lustre 2.12.3
I am using ConnectX 5 Mellanox cards on servers with AMD Epyc processors.
I have tested and validated all speeds at infiniband (ib_read_bw and ib_write_bw) and tcp / ip (iperf3) level.
>From a lustre point of view (lnet selfttest the bit rates are correct between the nodes of A and the gateways.
On the other hand, the performance of the B nodes and the gateways are very very low:
[LNet Rates of servers]
[R] Avg: 26 RPC/s Min: 26 RPC/s Max: 26 RPC/s
[W] Avg: 36 RPC/s Min: 36 RPC/s Max: 36 RPC/s
[LNet Bandwidth of servers]
[R] Avg: 8.61 MiB/s Min: 8.61 MiB/s Max: 8.61 MiB/s
[W] Avg: 9.59 MiB/s Min: 9.59 MiB/s Max: 9.59 MiB/s
[LNet Rates of servers]
[R] Avg: 33 RPC/s Min: 33 RPC/s Max: 33 RPC/s
[W] Avg: 42 RPC/s Min: 42 RPC/s Max: 42 RPC/s
[LNet Bandwidth of servers]
[R] Avg: 8.62 MiB/s Min: 8.62 MiB/s Max: 8.62 MiB/s
[W] Avg: 8.81 MiB/s Min: 8.81 MiB/s Max: 8.81 MiB/s
Do you have any idea on the causes of these ridiculous flow rates?
Regards,
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20201130/a9de4ac3/attachment.html>
More information about the lustre-discuss
mailing list