[Lustre-discuss] Multi-Rail Configurations on a Multi-Port IB HCA

[Isaac Huang]

On Fri, Nov 13, 2009 at 03:34:14PM -0500, Dardo D Kleiner - CONTRACTOR wrote:
> Mellanox ConnectX MT25418, two ports, each connected to a separate
> IB fabric - ib0 and ib1 have distinct IP subnets, each connected
> to a separate Lustre router.
> ......
> ip ad ls:
> 4: ib0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 65520 qdisc pfifo_fast state UP qlen 4096
>      inet xxx.xxx.182.130/26 brd xxx.xxx.182.191 scope global ib0
> 5: ib1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 65520 qdisc pfifo_fast state UP qlen 4096
>      inet xxx.xxx.182.194/26 brd xxx.xxx.182.255 scope global ib1
> 
> /etc/modprobe.d/lustre:
> options lnet \
>          ip2nets=" \
>                  o2ib1 xxx.xxx.[176-177].[0-255];
>                  o2ib3(ib0) xxx.xxx.182.[128-191];
>                  o2ib4(ib1) xxx.xxx.182.[192-255]"
>          routes=" \
>                  o2ib1 xxx.xxx.182.129 at o2ib3,xxx.xxx.182.193 at o2ib4"
> 
> dmesg:
> .
> .
> Lustre: Listener bound to ib0:xxx.xxx.182.130:987:mlx4_0
> .
> .
> 
> 
> Why don't I also get "Listener bound to ib1:xxx.xxx.182.194:987:mlx4_0"?

What did 'lctl list_nids' show? It looked like only one NI was
initialized.

Isaac