[Lustre-discuss] Adding IB to tcp only cluster

[Brock Palen]

On Oct 10, 2008, at 2:45 PM, Brian J. Murrell wrote:

> On Fri, 2008-10-10 at 11:08 -0400, Brock Palen wrote:
>> We have added a few IB nodes to our cluster (about 70 our of 600  
>> nodes).
>> What would it take to have lustre go over IB as well as tcp for the
>> rest of the hosts?
>
> So I'm assuming that at least some of these IB nodes are servers (i.e.
> OSS) then.

Not right now, the question was because we were thinking abou tit

>
>> would only the oss need HCA's?  or does the MDS need to have hca's
>> also?
>
> No.  There is no requirement that the MDS use IB just because (some)
> OSSes use it.

Really?  So given that lnet does the best path and it is not part of  
lustre its self.
So if we only hook some of the OSS by IB,  is there a way to have a  
user (who is a user of IB) IO prefer the IB connected OSS's.

If that is not possible now, I think some of the patches announced  
that are for 1.8 or 2.0 had the ability to select a OSS for only  
given users.  Am I correct?

>
>> It would be nice to have MDS traffic over TCP (fast enough for
>> this user) and IO over IB.
>
> Fair enough.
>
>> How does lustre figure out the preferred path?
>
> An LNET node with multiple paths to another LNET node chooses the  
> "best"
> path.  How that decision is made, I'm not so sure, but I tend to think
> that o2iblnd will be preferred over socklnd.
>
>> How can we have the
>> nodes figure out IF I have IB talk to oss's over IB else use TCP?
>
> Assuming you get the configuration right on the nodes, LNET will  
> just do
> that using it's "best path" algorithm.
>
> b.
>
>
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-discuss
>
>