[Lustre-discuss] accessing different Lustre filesystems using different network interconnects

[Thomas Zeiser]

Hello Mr. Buch,

On Thu, Feb 16, 2012 at 12:50:56PM +0100, Jonathan Buch wrote:
> Hello Mr. Zeiser,
> 
> without knowledge about your current lnet setup:
> 
> 3 networks - 2 infiniband, 1 eth
> 
> cluster-a: i2ib0
> cluster-b: i2ib1
> conn-ab: tcp0
> 
> # /etc/modprobe.d/lustre
> options lnet ip2nets="tcp0 192.168.0.[1,2]; ib2ib0 10.0.1.*; ib2ib1
> 10.0.2.*"
> options lnet routes="tcp0 1 10.0.1.1 at i2ib0; tcp0 1 10.0.2.1 at i2ib1;
> i2ib0 1 192.168.0.1; i2ib1 1 192.168.0.2; "
> 
> 
> A `lctl list_nids` on login-a:
>    192.168.0.1 at tcp0
>    10.0.1.1 at i2ib0
> 
> And the other gateway of course gets the i2ib1 and second tcp0.
> The gateways  get to be lustre routers then and you can even mount the

I hoped to go without lnet routers ...

> other lustre on the client nodes - the single modprobe file can be
> distributed on all lustre servers/clients.
> 
> mount -t lustre     192.168.0.1 at tcp0:10.0.1.1 at i2ib0:/lustre-a /lustre-a
> 
> I _think_ this should work (of course it depends on how you set up
> you current networks, if both cluster a and b have i2ib0 as network
> I think you need to rename one>

that's probably the important point; currently, both (independent)
installations of course default to i2ib0 :-(

> (which I guess would involve an
> ugly unmount of the full lustre system and --writeconf, as suggested
> by the manual)).

so, let's wait for the next downtime ...

> I hope this covers what you were asking.

exactly; thanks!


Thomas Zeiser
> 
> Greetings,
> 
> Jonathan Buch