<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
font-size:12.0pt;
font-family:"Calibri",sans-serif;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:12.0pt;
font-family:"Calibri",sans-serif;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style>
</head>
<body lang="EN-CA" link="#0563C1" vlink="#954F72" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">Hi Victor,<br>
<br>
You may want to try setting the o2iblnd module parameter "tos".<br>
<br>
For example (in the conf file):<br>
options ko2iblnd tos=0x68<br>
<br>
This controls the TOS value used by o2iblnd when establishing<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">RDMA CM connections, and may influence which SL/priority<br>
queue is selected for RoCE v2 traffic.<br>
<br>
Note that:<br>
<br>
- This is applied at connection setup time, so existing<br>
connections would need to be re-established to pick up the change.<br>
<br>
- The actual mapping to hardware queues depends on the NIC<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New""><br>
Given that “ib_send_bw -x <gid>” works in your setup, it’s also possible<br>
that the effective behavior is driven more by GID selection than by<br>
TOS alone, and o2iblnd does not currently expose a way to control GID<br>
index explicitly.<br>
<br>
Still, setting "tos" is probably the first thing worth trying.<br>
<br>
Cheers,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">Serguei<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
</div>
</body>
</html>