[lustre-discuss] [External] : Re: Lustre Client pinned to use specific NIC on host

[Sonia Sharma]

Thanks, Chris, for the suggestion!
I should have mentioned my use-case in the initial e-mail.

My scenario is that this client is co-located with the MDS server and other nice on that host machine are tied to different MDTs. With that, the UDSP priority wouldn’t be the right choice I believe.

I was looking (thinking if feasible)  for a solution (something like NFSv4 has, I believe?) where during client mount, we can specify a NIC ( optional) that is used by the client to send its requests and Lustre servers also map that NIC to the client and use that to route the responses to the specific client.

Best
Sonia


Confidential – Oracle Internal

From: Horn, Chris <chris.horn at hpe.com>
Date: Friday, October 3, 2025 at 12:08 PM
To: Sonia Sharma <sonia.sh.sharma at oracle.com>, lustre-discuss <lustre-discuss at lists.lustre.org>
Subject: [External] : Re: [lustre-discuss] Lustre Client pinned to use specific NIC on host

You can use a local network user-defined selection policy (UDSP) to assign a priority to a local interface. This does not guarantee that LNet will only​ use that interface, but it will prefer it over other, lower-priority interfaces.

# lnetctl net show -v --net tcp | grep -e send_count -e recv_count -e nid
      -     nid: 172.18.2.5 at tcp
                  send_count: 0
                  recv_count: 0
      -     nid: 172.18.2.1 at tcp
                  send_count: 0
                  recv_count: 0
# lnetctl udsp add --src 172.18.2.1 at tcp --priority 0
# lst.sh -t 172.18.2.2 at tcp -f 172.18.2.5 at tcp -m rw
…
brw_rw is running now
…
# lnetctl net show -v --net tcp | grep -e send_count -e recv_count -e nid
      -     nid: 172.18.2.5 at tcp
                  send_count: 14
                  recv_count: 14
      -     nid: 172.18.2.1 at tcp
                  send_count: 162088
                  recv_count: 224920
#



Chris Horn

From: lustre-discuss <lustre-discuss-bounces at lists.lustre.org> on behalf of Sonia Sharma via lustre-discuss <lustre-discuss at lists.lustre.org>
Date: Friday, October 3, 2025 at 1:47 PM
To: lustre-discuss <lustre-discuss at lists.lustre.org>
Subject: [lustre-discuss] Lustre Client pinned to use specific NIC on host

Hello Dear Lustre Community

I was wondering if there is some setting/way to make a Lustre Client use only a specific NIC to route its LNet messages from even when multiple NICs on that host are configured with the same LNet network.

For example, let’s say I have a host with this below LNet configuration, then is there a way that I can have the Lustre client to use only “10.30.201.26 at tcp” for all its communication

[node1]# lnetctl net show
net:
    - net type: lo
      local NI(s):
        - nid: 0 at lo
          status: up
    - net type: tcp
      local NI(s):
        - nid: 10.30.201.26 at tcp
          status: up
          interfaces:
              0: enp1s0
        - nid: 10.30.201.10 at tcp
          status: up
          interfaces:
              0: enp1s0:10269
        - nid: 10.30.202.241 at tcp
          status: up
          interfaces:
              0: enp1s0:10257

Best regards
Sonia


Confidential – Oracle Internal
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20251003/d0a6f52c/attachment-0001.htm>