[Lustre-discuss] Adding IB to tcp only cluster

[Brian J. Murrell]

On Fri, 2008-10-10 at 11:08 -0400, Brock Palen wrote:
> We have added a few IB nodes to our cluster (about 70 our of 600 nodes).
> What would it take to have lustre go over IB as well as tcp for the  
> rest of the hosts?

So I'm assuming that at least some of these IB nodes are servers (i.e.
OSS) then.

> would only the oss need HCA's?  or does the MDS need to have hca's  
> also?

No.  There is no requirement that the MDS use IB just because (some)
OSSes use it.

> It would be nice to have MDS traffic over TCP (fast enough for  
> this user) and IO over IB.

Fair enough.

> How does lustre figure out the preferred path?

An LNET node with multiple paths to another LNET node chooses the "best"
path.  How that decision is made, I'm not so sure, but I tend to think
that o2iblnd will be preferred over socklnd.

> How can we have the  
> nodes figure out IF I have IB talk to oss's over IB else use TCP?

Assuming you get the configuration right on the nodes, LNET will just do
that using it's "best path" algorithm.

b.