[lustre-discuss] : Re: Lustre Client pinned to use specific NIC on host

[Horn, Chris]

I didn’t see this context when I wrote my other reply. UDSP doesn’t work here because LNet cannot distinguish between client/server traffic, so the UDSP rule would apply to both the client and co-located MDS traffic.

The only way I can think of that might work would be to disable LNet peer discovery, manually configure your multi-rail peer tables, create a separate tcp1 network for enp1s0, add tcp1 NIDs to all the servers, and mount the client using network=tcp1. This is very complicated, and not something I’ve tried so cannot guarantee that it will work.

Chris Horn

From: lustre-discuss <lustre-discuss-bounces at lists.lustre.org> on behalf of Sonia Sharma via lustre-discuss <lustre-discuss at lists.lustre.org>
Date: Friday, October 3, 2025 at 2:04 PM
To: Tim Schnedler <timothyschnedler at gmail.com>
Cc: lustre-discuss <lustre-discuss at lists.lustre.org>
Subject: Re: [lustre-discuss] [External] : Re: Lustre Client pinned to use specific NIC on host

Thanks, Tim for a quick response.
I should have mentioned the specific scenario - In this case, lustre client is co-located with an MDS lustre server and the rest of the nice on the host are specifically for MDTs.

I believe the lustre.conf approach wouldn’t work in that case, or there is still a way ?

Best
Sonia


Confidential – Oracle Internal

From: Tim Schnedler <timothyschnedler at gmail.com>
Date: Friday, October 3, 2025 at 11:54 AM
To: Sonia Sharma <sonia.sh.sharma at oracle.com>
Cc: lustre-discuss <lustre-discuss at lists.lustre.org>
Subject: [External] : Re: [lustre-discuss] Lustre Client pinned to use specific NIC on host

One way to do this is to make a lustre conf file in etc/ modprobe.d

On Fri, Oct 3, 2025 at 12:48 PM Sonia Sharma via lustre-discuss <lustre-discuss at lists.lustre.org<mailto:lustre-discuss at lists.lustre.org>> wrote:
Hello Dear Lustre Community

I was wondering if there is some setting/way to make a Lustre Client use only a specific NIC to route its LNet messages from even when multiple NICs on that host are configured with the same LNet network.

For example, let’s say I have a host with this below LNet configuration, then is there a way that I can have the Lustre client to use only “10.30.201.26 at tcp” for all its communication

[node1]# lnetctl net show
net:
    - net type: lo
      local NI(s):
        - nid: 0 at lo
          status: up
    - net type: tcp
      local NI(s):
        - nid: 10.30.201.26 at tcp
          status: up
          interfaces:
              0: enp1s0
        - nid: 10.30.201.10 at tcp
          status: up
          interfaces:
              0: enp1s0:10269
        - nid: 10.30.202.241 at tcp
          status: up
          interfaces:
              0: enp1s0:10257

Best regards
Sonia


Confidential – Oracle Internal

_______________________________________________
lustre-discuss mailing list
lustre-discuss at lists.lustre.org<mailto:lustre-discuss at lists.lustre.org>
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org<https://urldefense.com/v3/__http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org__;!!ACWV5N9M2RV99hQ!JK-5U3bPxJEQg0wm-92Mtqb9akrLlnZ1vY2azcQ5KmoD9Azb5QAtP0Vxm1xhag_lMu1sQU3Zs6lmaLEGcltmBmWysWgLsw$>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20251003/5d23029d/attachment-0001.htm>