[lustre-discuss] Lustre RoCE v2 traffic issue

[Serguei Smirnov]

Hi Victor,

You may want to try setting the o2iblnd module parameter "tos".

For example (in the conf file):
  options ko2iblnd tos=0x68

This controls the TOS value used by o2iblnd when establishing
RDMA CM connections, and may influence which SL/priority
queue is selected for RoCE v2 traffic.

Note that:

- This is applied at connection setup time, so existing
  connections would need to be re-established to pick up the change.

- The actual mapping to hardware queues depends on the NIC

Given that “ib_send_bw -x <gid>” works in your setup, it’s also possible
that the effective behavior is driven more by GID selection than by
TOS alone, and o2iblnd does not currently expose a way to control GID
index explicitly.

Still, setting "tos" is probably the first thing worth trying.

Cheers,
Serguei

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20260505/ed750d21/attachment.htm>