[Lustre-discuss] Multi-Rail Configurations on a Multi-Port IB HCA

[Isaac Huang]

On Mon, Nov 16, 2009 at 04:38:03PM -0500, Dardo D Kleiner - CONTRACTOR wrote:
> Stand down.  Don't know what was wrong with my configuration at first,
> but it does instantiate the two NIDs on the host with multiple ports
> on a single HCA.  Unfortunately,
> 
> LustreError: 17771:0:(router.c:464:lnet_check_routes()) Routes to o2ib1 via xxx.xxx.182.193 at o2ib4 and xxx.xxx.182.129 at o2ib3 not supported

In fact, this limitation could be lifted. The reason it was there was
that upper layers would rely on source NID in lnet messages to
identify clients - i.e., it was assumed that messages from a same
client would carry a same source NID in lnet message headers.

It seems that it's becoming an annoyance as multi-rail configurations
grow more popular.

> So I couldn't have done what I wanted to anyway, the answer to my
> question below "Should I be able to route over multiple lnets?" is
> clearly no...
> 
> - Dardo