[Lustre-discuss] Adding IB to tcp only cluster

[Andreas Dilger]

On Oct 10, 2008  15:09 -0400, Brian J. Murrell wrote:
> Regardless, once LNET has it's list of interfaces and networks it routes
> requests accordingly depending on their destination.  If a target
> (either an MDS or OSS) only has a TCP path that path will be used.  If a
> target has more than one path the "best" path will be chosen.  I tend to
> think o2iblnd trumps socklnd.
> 
> Maybe if there is an LNET engineer reading he can give you more details
> on how this best path is chosen.

I'm no LNET expert, but my understanding is:

LNET will pick the network with the fewest LNET network hops (TCP network
hops are not a consideration), and if they are equal then it will (AFAIK)
pick the first network listed in the "networks" module option as the "best"
network and use that exclusively until it fails.

That means "for clients with IB, put the o2iblnd interface first and the
TCP interface second" in the "lnet networks" module option.  I think this
is discussed in the Lustre manual already.

> > If that is not possible now, I think some of the patches announced  
> > that are for 1.8 or 2.0 had the ability to select a OSS for only  
> > given users.  Am I correct?
> 
> I think you are talking about OST pools.  I'm not sure which release
> that is targeted for.

While OST pools is going to be included into 1.8.0, it doesn't have
the policy engine yet, so the "pick a pool for UID X" doesn't exist yet.

Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.