[Lustre-discuss] How to force client-oss communication over IB when the MDS has only ethernet?

[Tero Hakala]

Hi,
our MDS is temporarily missing IB connection and has only eth available. 
However, OSS and clients have both IB and eth.

At the moment, it seems that all the traffic between clients-OSS goes 
also through the slow eth connection.  Is it possible to force them to 
use faster IB interfaces when communication with each other, and only 
use eth to communicate with the MDS?

Both clients and OSS have interfaces configured as
options lnet networks=o2ib0(ib0),tcp0(eth0)  (in modprobe.conf) also 
lctl ping works fine over IB.   The documents seem to suggests that the 
first interface is preferred, but apparently it is not when MDS is only 
available through the other.

  -t