[Lustre-discuss] How to force client-oss communication over IB when the MDS has only ethernet?

[Cliff White]

Tero Hakala wrote:
> Hi,
> our MDS is temporarily missing IB connection and has only eth available. 
> However, OSS and clients have both IB and eth.
> 
> At the moment, it seems that all the traffic between clients-OSS goes 
> also through the slow eth connection.  Is it possible to force them to 
> use faster IB interfaces when communication with each other, and only 
> use eth to communicate with the MDS?
> 
> Both clients and OSS have interfaces configured as
> options lnet networks=o2ib0(ib0),tcp0(eth0)  (in modprobe.conf) also 
> lctl ping works fine over IB.   The documents seem to suggests that the 
> first interface is preferred, but apparently it is not when MDS is only 
> available through the other.
> 

I believe in this case since you are using the tcp NID for the client 
mount, lnet assumes that you are using tcp0, since tcp0 can reach the 
OSS. I think you might be able to use options lnet ip2nets on the 
clients to force the client-OSS connection to use ib0.

cliffw

>   -t
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-discuss