[Lustre-discuss] Mount 2 clusters, different networks - LNET tcp1-tcp2-o2ib - solved?

Thomas Roth t.roth at gsi.de
Tue Jun 14 11:44:02 PDT 2011

Hi all,

this seems to work with the correct IPs and correct network names ;-[]
I now have the following modprobe on my ethernet client:
 > options lnet networks=tcp1(eth0),tcp0(eth0:0) routes="o2ib LNET-Router at tcp1; tcp Default-Route at tcp1"

With these options, loading the modules gives me
 > Jun 14 20:12:55 kernel: Lustre: Added LNI at tcp1 [8/256/0/180]
 > Jun 14 20:12:55 kernel: Lustre: Added LNI at tcp [8/256/0/180]
which are the IPs of eth0 and eth0:0.

Now I still wonder why the alias interface eth0:0 is necessary (if left out, the whole endeavor 
fails). The routes=statement seems to say: "If you have data for tcp, use the Default-Router-IP and go 
via the interace that is on network tcp1".

Oh well, I should probably take some networking lectures...


On 06/14/2011 06:23 PM, Thomas Roth wrote:
> Hi all,
> I'd like to mount two Lustre filesystems on one client. Issues with more than one MGS set aside,
> the point here is that one of them is an Infiniband-cluster, the other is ethernet-based.
> And my client is on the ethernet.
> I have managed to mount the o2ib-fs by setting up an LNET router, but now this client's LNET doesn;t
> known how to reach the ethernet-fs.
> So the basic modprobe.conf reads
>   >  options lnet networks=tcp1(eth0) routes="o2ib LNET-Router-IP at tcp1"
> This mounts the MGS on the o2ib network.
> What do I have to add to get to the MGS on the tpc network?
> Meanwhile I have studied more posts here and came up with
>   >  options lnet networks=tcp1(eth0),tcp2(eth0:0) routes="o2ib LNET-Router-IP at tcp1; tcp
> Default-Gateway-IP at tcp2"
> Doesn't work either, but I see in the log of the (tcp-)MGS:
>   >  LustreError: 120-3: Refusing connection from Client-IP for MGS-IP at tcp2: No matching NI
> Somethings getting through ...
> Any ideas?
> Regards,
> Thomas
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-discuss

More information about the lustre-discuss mailing list