[Lustre-discuss] Adding IB to tcp only cluster

[Brock Palen]

We have added a few IB nodes to our cluster (about 70 our of 600 nodes).
What would it take to have lustre go over IB as well as tcp for the  
rest of the hosts?
I know we could use a router machine, but we could only provide maybe  
2 gige ports which would be poor.

So my questions:

would only the oss need HCA's?  or does the MDS need to have hca's  
also?  It would be nice to have MDS traffic over TCP (fast enough for  
this user) and IO over IB.

How does lustre figure out the preferred path? How can we have the  
nodes figure out IF I have IB talk to oss's over IB else use TCP?

Anything else to look out for?


Brock Palen
www.umich.edu/~brockp
Center for Advanced Computing
brockp at umich.edu
(734)936-1985