[Lustre-discuss] Multi-Rail Configurations on a Multi-Port IB HCA

[Dardo D Kleiner - CONTRACTOR]

Mellanox ConnectX MT25418, two ports, each connected to a separate
IB fabric - ib0 and ib1 have distinct IP subnets, each connected
to a separate Lustre router.

ibstat:
CA 'mlx4_0'
         CA type: MT25418
         Number of ports: 2
         Firmware version: 2.7.0
         Hardware version: a0
         Port 1:
                 State: Active
                 Physical state: LinkUp
                 Rate: 20
                 Base lid: 302
                 LMC: 0
                 SM lid: 2
                 Capability mask: 0x02510868
         Port 2:
                 State: Active
                 Physical state: LinkUp
                 Rate: 20
                 Base lid: 5
                 LMC: 0
                 SM lid: 1
                 Capability mask: 0x02510868

ip ad ls:
4: ib0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 65520 qdisc pfifo_fast state UP qlen 4096
     inet xxx.xxx.182.130/26 brd xxx.xxx.182.191 scope global ib0
5: ib1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 65520 qdisc pfifo_fast state UP qlen 4096
     inet xxx.xxx.182.194/26 brd xxx.xxx.182.255 scope global ib1

/etc/modprobe.d/lustre:
options lnet \
         ip2nets=" \
                 o2ib1 xxx.xxx.[176-177].[0-255];
                 o2ib3(ib0) xxx.xxx.182.[128-191];
                 o2ib4(ib1) xxx.xxx.182.[192-255]"
         routes=" \
                 o2ib1 xxx.xxx.182.129 at o2ib3,xxx.xxx.182.193 at o2ib4"

dmesg:
.
.
Lustre: Listener bound to ib0:xxx.xxx.182.130:987:mlx4_0
.
.


Why don't I also get "Listener bound to ib1:xxx.xxx.182.194:987:mlx4_0"?

- Dardo