[Lustre-discuss] ksocklnd multiple connections

[Andreas Dilger]

On Aug 28, 2008  22:11 +0800, Tim Burgess wrote:
> - all dual connected hosts are connected to both LeftSwitch and RightSwitch
> - clients network interfaces are 172.16.4.x/16 (eth0,leftswitch) and
> 172.16.5.x/16 (eth1,rightswitch)
> - OSS/MDS network interfaces are 172.16.0.x/16 (eth0,leftswitch) and
> 172.16.1.x/16 (eth1,rightswitch)
> 
> So lustre has done the right thing in connecting eth0<->eth0 and eth1<->eth1
> in this case.  But how does it know?  Does the client connect to both server
> addresses and throw away any connections originating from the same address?
> Is there some check of the return path?

It does this matching based on the subnet addresses.  I think it refuses
to have multiple connections between nodes on the same interfaces, as
this sometimes happens when there is a A->B vs B->A connection race.

> My motive here is that I also have a set of singly-connected machines, and
> want to have their traffic balanced across both server networks (single
> connect machines come in via topswitch).  Right now, these clients all
> connect to the eth0 address (172.16.0.x) on all OSSes and the MDS.

Normally what is done is using ip2nets module option, so that clients
will load-balance their connections between the 2 interfaces.  That
doesn't help the single client, but is normally fine in a cluster.

I believe that is documented in the manual.

Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.