<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body>
<div dir="ltr" style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Thanks, Chris, for the suggestion!</div>
<div dir="ltr" style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
I should have mentioned my use-case in the initial e-mail.</div>
<div dir="ltr" style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div dir="ltr" style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
My scenario is that this client is co-located with the MDS server and other nice on that host machine are tied to different MDTs. With that, the UDSP priority wouldn’t be the right choice I believe.</div>
<div dir="ltr" style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div dir="ltr" style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
I was looking (thinking if feasible) for a solution (something like NFSv4 has, I believe?) where during client mount, we can specify a NIC ( optional) that is used by the client to send its requests and Lustre servers also map that NIC to the client and use
that to route the responses to the specific client.</div>
<div dir="ltr" style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div dir="ltr" style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Best</div>
<div dir="ltr" style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Sonia</div>
<div dir="ltr" id="ms-outlook-mobile-signature"></div>
<div id="mail-editor-reference-message-container">
<div dir="ltr" class="ms-outlook-mobile-reference-message skipProofing"></div>
<div class="ms-outlook-mobile-reference-message skipProofing" style="text-align: left; padding: 3pt 0in 0in; border-width: 1pt medium medium; border-style: solid none none; border-color: rgb(181, 196, 223) currentcolor currentcolor; font-family: Aptos; font-size: 12pt; color: black;">
<b><br>
<p style="font-family:Calibri;font-size:10pt;color:#000000;margin:5pt;font-style:normal;font-weight:normal;text-decoration:none;" align="Left">
Confidential – Oracle Internal<br>
</p>
From: </b>Horn, Chris <chris.horn@hpe.com><br>
<b>Date: </b>Friday, October 3, 2025 at 12:08 PM<br>
<b>To: </b>Sonia Sharma <sonia.sh.sharma@oracle.com>, lustre-discuss <lustre-discuss@lists.lustre.org><br>
<b>Subject: </b>[External] : Re: [lustre-discuss] Lustre Client pinned to use specific NIC on host<br>
<br>
</div>
<div dir="ltr" class="ms-outlook-mobile-reference-message skipProofing" style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
You can use a local network user-defined selection policy (UDSP) to assign a priority to a local interface. This does not guarantee that LNet will
<i>only</i> use that interface, but it will prefer it over other, lower-priority interfaces.</div>
<div dir="ltr" class="ms-outlook-mobile-reference-message skipProofing" style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div dir="ltr" class="ms-outlook-mobile-reference-message skipProofing" style="font-family: Consolas, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
# lnetctl net show -v --net tcp | grep -e send_count -e recv_count -e nid</div>
<div class="ms-outlook-mobile-reference-message skipProofing" style="font-family: Consolas, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
- nid: 172.18.2.5@tcp</div>
<div class="ms-outlook-mobile-reference-message skipProofing" style="font-family: Consolas, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
send_count: 0</div>
<div class="ms-outlook-mobile-reference-message skipProofing" style="font-family: Consolas, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
recv_count: 0</div>
<div class="ms-outlook-mobile-reference-message skipProofing" style="font-family: Consolas, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
- nid: 172.18.2.1@tcp</div>
<div class="ms-outlook-mobile-reference-message skipProofing" style="font-family: Consolas, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
send_count: 0</div>
<div class="ms-outlook-mobile-reference-message skipProofing" style="font-family: Consolas, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
recv_count: 0</div>
<div dir="ltr" class="ms-outlook-mobile-reference-message skipProofing" style="font-family: Consolas, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
# lnetctl udsp add --src 172.18.2.1@tcp --priority 0</div>
<div dir="ltr" class="ms-outlook-mobile-reference-message skipProofing" style="font-family: Consolas, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
# lst.sh -t 172.18.2.2@tcp -f 172.18.2.5@tcp -m rw</div>
<div dir="ltr" class="ms-outlook-mobile-reference-message skipProofing" style="font-family: Consolas, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
…</div>
<div dir="ltr" class="ms-outlook-mobile-reference-message skipProofing" style="font-family: Consolas, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
brw_rw is running now</div>
<div dir="ltr" class="ms-outlook-mobile-reference-message skipProofing" style="font-family: Consolas, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
…</div>
<div dir="ltr" class="ms-outlook-mobile-reference-message skipProofing" style="font-family: Consolas, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
# lnetctl net show -v --net tcp | grep -e send_count -e recv_count -e nid</div>
<div class="ms-outlook-mobile-reference-message skipProofing" style="font-family: Consolas, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
- nid: 172.18.2.5@tcp</div>
<div class="ms-outlook-mobile-reference-message skipProofing" style="font-family: Consolas, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
send_count: 14</div>
<div class="ms-outlook-mobile-reference-message skipProofing" style="font-family: Consolas, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
recv_count: 14</div>
<div class="ms-outlook-mobile-reference-message skipProofing" style="font-family: Consolas, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
- nid: 172.18.2.1@tcp</div>
<div class="ms-outlook-mobile-reference-message skipProofing" style="font-family: Consolas, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
send_count: 162088</div>
<div class="ms-outlook-mobile-reference-message skipProofing" style="font-family: Consolas, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
recv_count: 224920</div>
<div dir="ltr" class="ms-outlook-mobile-reference-message skipProofing" style="font-family: Consolas, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
#</div>
<div dir="ltr" class="ms-outlook-mobile-reference-message skipProofing" style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div dir="ltr" class="ms-outlook-mobile-reference-message skipProofing" style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div dir="ltr" class="ms-outlook-mobile-reference-message skipProofing" style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div dir="ltr" class="ms-outlook-mobile-reference-message skipProofing" style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Chris Horn</div>
<div dir="ltr" class="ms-outlook-mobile-reference-message skipProofing" style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div id="mail-editor-reference-message-container" style="color:inherit; background-color:inherit">
<div class="ms-outlook-mobile-reference-message skipProofing" style="text-align: left; padding: 3pt 0in 0in; border-width: 1pt medium medium; border-style: solid none none; border-color: rgb(181, 196, 223) currentcolor currentcolor; font-family: Aptos; font-size: 12pt; color: black;">
<b>From: </b>lustre-discuss <lustre-discuss-bounces@lists.lustre.org> on behalf of Sonia Sharma via lustre-discuss <lustre-discuss@lists.lustre.org><br>
<b>Date: </b>Friday, October 3, 2025 at 1:47 PM<br>
<b>To: </b>lustre-discuss <lustre-discuss@lists.lustre.org><br>
<b>Subject: </b>[lustre-discuss] Lustre Client pinned to use specific NIC on host<br>
<br>
</div>
<div class="ms-outlook-mobile-reference-message skipProofing">Hello Dear Lustre Community</div>
<div dir="ltr" class="ms-outlook-mobile-reference-message skipProofing"><br>
</div>
<div dir="ltr" class="ms-outlook-mobile-reference-message skipProofing" style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt;">
I was wondering if there is some setting/way to make a Lustre Client use only a specific NIC to route its LNet messages from even when multiple NICs on that host are configured with the same LNet network.</div>
<div dir="ltr" class="ms-outlook-mobile-reference-message skipProofing" style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt;">
<br>
</div>
<div dir="ltr" class="ms-outlook-mobile-reference-message skipProofing"><span style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt;">For example, let’s say I have a host with this below LNet configuration, then is there a way that I can
have the Lustre client to use only “</span><span style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 16px; color: rgb(0, 0, 0);"><i>10.30.201.26@tcp</i></span>”<span style="font-size: 16px;"> for all its communication</span></div>
<div dir="ltr" class="ms-outlook-mobile-reference-message skipProofing" style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt;">
<br>
</div>
<div dir="ltr" class="ms-outlook-mobile-reference-message skipProofing" style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<i>[node1]# lnetctl net show</i></div>
<div class="ms-outlook-mobile-reference-message skipProofing" style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<i>net:</i></div>
<div class="ms-outlook-mobile-reference-message skipProofing" style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<i> - net type: lo</i></div>
<div class="ms-outlook-mobile-reference-message skipProofing" style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<i> local NI(s):</i></div>
<div class="ms-outlook-mobile-reference-message skipProofing" style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<i> - nid: 0@lo</i></div>
<div class="ms-outlook-mobile-reference-message skipProofing" style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<i> status: up</i></div>
<div class="ms-outlook-mobile-reference-message skipProofing" style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<i> - net type: tcp</i></div>
<div class="ms-outlook-mobile-reference-message skipProofing" style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<i> local NI(s):</i></div>
<div class="ms-outlook-mobile-reference-message skipProofing" style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<i> - nid: 10.30.201.26@tcp</i></div>
<div class="ms-outlook-mobile-reference-message skipProofing" style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<i> status: up</i></div>
<div class="ms-outlook-mobile-reference-message skipProofing" style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<i> interfaces:</i></div>
<div class="ms-outlook-mobile-reference-message skipProofing" style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<i> 0: enp1s0</i></div>
<div class="ms-outlook-mobile-reference-message skipProofing" style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<i> - nid: 10.30.201.10@tcp</i></div>
<div class="ms-outlook-mobile-reference-message skipProofing" style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<i> status: up</i></div>
<div class="ms-outlook-mobile-reference-message skipProofing" style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<i> interfaces:</i></div>
<div class="ms-outlook-mobile-reference-message skipProofing" style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<i> 0: enp1s0:10269</i></div>
<div class="ms-outlook-mobile-reference-message skipProofing" style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<i> - nid: 10.30.202.241@tcp</i></div>
<div class="ms-outlook-mobile-reference-message skipProofing" style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<i> status: up</i></div>
<div class="ms-outlook-mobile-reference-message skipProofing" style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<i> interfaces:</i></div>
<div class="ms-outlook-mobile-reference-message skipProofing" style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<i> 0: enp1s0:10257</i></div>
<div dir="ltr" class="ms-outlook-mobile-reference-message skipProofing" style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<i><br>
</i></div>
<div dir="ltr" class="ms-outlook-mobile-reference-message skipProofing" style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Best regards</div>
<div dir="ltr" class="ms-outlook-mobile-reference-message skipProofing" style="font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Sonia</div>
<div id="ms-outlook-mobile-signature" dir="ltr" style="color:inherit; background-color:inherit">
</div>
<div dir="ltr" class="ms-outlook-mobile-reference-message skipProofing"><br>
</div>
<p class="ms-outlook-mobile-reference-message skipProofing" style="margin: 5pt; font-family: Calibri; font-size: 10pt; color: rgb(0, 0, 0);">
Confidential – Oracle Internal</p>
</div>
</div>
</body>
</html>