[Lustre-discuss] How to configure routing

[Herb Wartens]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

No problem with disagreeing...=)
I guess I was just trying to impress upon the group that before enabling
routing it would be bet to make sure that is what you really want to do
since it can cause its own set of problems (like the one I described with
failover)...=)

- -Herb

D. Marc Stearman wrote:
> I hate to disagree with you Herb, but there are a great number of  
> reasons you would not want to set up routing between those networks.   
> First of all, making 'private' subnets visible to the public network  
> is a no-no.  Second, private networks are often secluded for security  
> reasons.  Bridging the gap could have many unintended consequences.
> 
> Lustre should be able to do all the routing with no problem.
> 
> Servers:  LNET tcp0 network
> Routers: LNET tcp0,tcp1 networks
> Clients: LNET tcp1 network
> 
> server (192.168.x)  <---->   (192.168.x) router (132.246.x) <---->  
> (132.246.x) client
> 
> Make sure you have ip_forwarding enabled on the router node.
> Make sure 'options lnet forwarding="enabled"' is set on the router node.
> 
> The router will find routes automatically.  You just need to specify  
> both networks:
> options lnet networks=tcp0(eth0),tcp1(eth1)
> 
> On the client you need:
> options lnet networks=tcp0(eth0) routes "tcp1  
> 132.246.x at tcp0" (routers 132.246.x IP addr)
> 
> On the server you need:
> options lnet networks=tcp1(eth0) routes "tcp0  
> 192.168.x at tcp1" (routers 192.168.x IP addr)
> 
> Does that make sense?
> 
> -Marc
> 
> ----
> D. Marc Stearman
> LC Lustre Systems Administrator
> marc at llnl.gov
> 925.423.9670
> Pager: 1.888.203.0641
> 
> 
> On Jan 16, 2008, at 2:07 PM, Herb Wartens wrote:
> 
> IMHO it would be better for you to set up host routing (or whatever it
> is that you wanted) to be ablr to route between these networks.  It  
> would
> be best to avoid the routing code if it is possible since you would  
> be introducing
> some new problems by having the routing turned on.
> With routing enabled you would not have immediate notification that  
> a server went down
> for example.  This causes longer delays on the client when doing  
> things like failover
> since you would have to wait for a timeout to occur before the  
> client would actually
> try the failover server.  I think with the adaptive timeouts this  
> could be resolved,
> but I think that things like this would be better to avoid if you  
> don't especially
> need to use it IMHO (we mainly use it here to route between  
> different transports
> ethernet to infiniband for example).
> 
> -Herb
> 
> Jerome, Ron wrote:
>>>> The issue is the fact that the luster filesystem is on a private
>>>> (192.168.xxx.xxx) network and the client is on a public network, so
>>>> bidirectional ip traffic routing gets messy (NATing, port forwarding
>>>> etc).  I was hoping to avoid this by using a gateway lustre node  
>>>> that is
>>>> connected to both networks.
>>>>
>>>> Ron.
>>>>
>>>>> -----Original Message-----
>>>>> From: lustre-discuss-bounces at clusterfs.com [mailto:lustre-discuss-
>>>>> bounces at clusterfs.com] On Behalf Of Herb Wartens
>>>>> Sent: January 16, 2008 2:32 PM
>>>>> To: Lustre Discuss
>>>>> Subject: Re: [Lustre-discuss] How to configure routing
>>>>>
>>>> Ron,
>>>> I am not exactly familiar with your particular setup, but when using
>>>> tcp
>>>> is there any reason why you can't use a single lnet network?  I
>>>>> believe
>>>> that
>>>> it would be easier for you to set up one socklnd network named tcp0
>>>> that
>>>> contains both interfaces.  Then all you have to do is make sure that
>>>>> it
>>>> is possible
>>>> through host routes (or other means) for the two networks to route to
>>>> each other.
>>>>
>>>> -Herb
>>>>
>>>> Jerome, Ron wrote:
>>>>>>> I would like to use a client as a router between two tcp networks
>>>> (eth0
>>>>>>> and eth1) but it is unclear to me how to configure this in
>>>> modprobe.conf.
>>>>>>>
>>>>>>> Any suggestions would be greatly appreciated.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> _________________________________________
>>>>>>> Ron Jerome
>>>>>>> Programmer/Analyst
>>>>>>> National Research Council Canada
>>>>>>> M-2, 1200 Montreal Road, Ottawa, Ontario K1A 0R6
>>>>>>> Government of Canada
>>>>>>> _________________________________________
>>>>>>>
>>>>>>>
>>>>>>>
>>>>> -------------------------------------------------------------------- 
>>>>> -
>>>> ---
>>>>>>> _______________________________________________
>>>>>>> Lustre-discuss mailing list
>>>>>>> Lustre-discuss at clusterfs.com
>>>>>>> https://mail.clusterfs.com/mailman/listinfo/lustre-discuss
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at clusterfs.com
> https://mail.clusterfs.com/mailman/listinfo/lustre-discuss
> 
>>
_______________________________________________
Lustre-discuss mailing list
Lustre-discuss at clusterfs.com
https://mail.clusterfs.com/mailman/listinfo/lustre-discuss

> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at clusterfs.com
> https://mail.clusterfs.com/mailman/listinfo/lustre-discuss
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (GNU/Linux)
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org

iD8DBQFHj57UP/62XqEEbMYRClQAAKCRkRZRU+wEGsLpU51I4mAZDCGEuACg2ZAy
gFcrhk+SiXob4xsWMdlBeQs=
=yLUj
-----END PGP SIGNATURE-----