[Lustre-discuss] Adding IB to tcp only cluster

Brian J. Murrell Brian.Murrell at Sun.COM
Fri Oct 10 12:09:41 PDT 2008

On Fri, 2008-10-10 at 14:56 -0400, Brock Palen wrote:
> Not right now, the question was because we were thinking abou tit

OK.  In any case, I guess the point I was making is that some servers
would need IB as well as the clients, or it would be pointless.  Just to
be absolutely clear.

> Really?

LNET configuration/routing is not (yet) one of my strong points, but I'm
fairly sure, yes.

> So given that lnet does the best path and it is not part of  
> lustre its self.
> So if we only hook some of the OSS by IB,  is there a way to have a  
> user (who is a user of IB) IO prefer the IB connected OSS's.

If you have a client which is connected to multiple networks (i.e. IB
and TCP) LNET will use them both.  You might have to poke LNET to do so
using module parameters, but I think it will use both automatically.

Regardless, once LNET has it's list of interfaces and networks it routes
requests accordingly depending on their destination.  If a target
(either an MDS or OSS) only has a TCP path that path will be used.  If a
target has more than one path the "best" path will be chosen.  I tend to
think o2iblnd trumps socklnd.

Maybe if there is an LNET engineer reading he can give you more details
on how this best path is chosen.

> If that is not possible now, I think some of the patches announced  
> that are for 1.8 or 2.0 had the ability to select a OSS for only  
> given users.  Am I correct?

I think you are talking about OST pools.  I'm not sure which release
that is targeted for.


More information about the lustre-discuss mailing list