[Lustre-discuss] multihomed clients ignoring lnet options

[Joe Little]

I have all of my servers and clients using eth1 for the tcp lustre lnet.

All have modprobe.conf entries of:

options lnet networks="tcp0(eth1)"

and all report with "lctl list_nids" that they are using the IP
address associated with that interface (a net 192.168.200.x address)

However, when my client connects, it ignores the above and goes with
eth0 for routing, even though the mds/mgs is on that network range:

client dmesg:

Lustre: 4756:0:(module.c:382:init_libcfs_module()) maximum lustre stack 8192
Lustre: Added LNI 192.168.200.100 at tcp [8/256]
Lustre: Accept secure, port 988
Lustre: OBD class driver, info at clusterfs.com
        Lustre Version: 1.6.4.2
        Build Version:
1.6.4.2-19691231190000-PRISTINE-.cache.build.BUILD.lustre-kernel-2.6.9.lustre.linux-2.6.9-55.0.9.EL_lustre.1.6.4.2smp
Lustre: Lustre Client File System; info at clusterfs.com
LustreError: 4799:0:(socklnd_cb.c:2167:ksocknal_recv_hello()) Error
-104 reading HELLO from 192.168.2.201
LustreError: 11b-b: Connection to 192.168.2.201 at tcp at host
192.168.2.201 on port 988 was reset: is it running a compatible
version of Lustre and is 192.168.2.201 at tcp one of its NIDs?

server dmesg:
LustreError: 120-3: Refusing connection from 192.168.2.192 for
192.168.2.201 at tcp: No matching NI