[lustre-discuss] Lustre Multi-Rail Troubles

[Chris Horn]

What Lustre version are you using on clients and servers?

Chris Horn

From: lustre-discuss <lustre-discuss-bounces at lists.lustre.org> on behalf of Stepan Beskrovnyy via lustre-discuss <lustre-discuss at lists.lustre.org>
Date: Thursday, July 23, 2026 at 12:56 PM
To: lustre-discuss at lists.lustre.org <lustre-discuss at lists.lustre.org>
Subject: [lustre-discuss] Lustre Multi-Rail Troubles

Hey everyone ! I got a simple multi-rail configure without any routers, only peers with direct ROCEv2 connection.


I got mdt/mds with 2 networks: ens30, ens60

I got OSS with same networks

I got client with same networks


2 networks works in Multi-Rail and everything fine.


Then I starting I/O benchmark on mounted fs on client. Then I disable ens30 on OSS with ip link set ens30 down.


lnetctl net show marks ens30 as down and all of network bandwith goes through ens60(Thats good)


Then I set ens30 up, by ip link set ens30

After a small timeout lnetctl marks ens30 as up, but all bandwidth still goes through ens60, and nothing goes through upped ens30.


Is there any way to restore traffic on ens30 without stopping I/O and without reloading lnet kernel module ? I did any tricks with adding and deeting peers, but they do nothing.


Any ideas?


Thanks,
Stepan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss_lists.lustre.org/attachments/20260723/35949bee/attachment-0001.html>