[Lustre-discuss] How to configure routing

D. Marc Stearman marc at llnl.gov
Thu Jan 17 08:30:47 PST 2008


I hate to disagree with you Herb, but there are a great number of  
reasons you would not want to set up routing between those networks.   
First of all, making 'private' subnets visible to the public network  
is a no-no.  Second, private networks are often secluded for security  
reasons.  Bridging the gap could have many unintended consequences.

Lustre should be able to do all the routing with no problem.

Servers:  LNET tcp0 network
Routers: LNET tcp0,tcp1 networks
Clients: LNET tcp1 network

server (192.168.x)  <---->   (192.168.x) router (132.246.x) <---->  
(132.246.x) client

Make sure you have ip_forwarding enabled on the router node.
Make sure 'options lnet forwarding="enabled"' is set on the router node.

The router will find routes automatically.  You just need to specify  
both networks:
options lnet networks=tcp0(eth0),tcp1(eth1)

On the client you need:
options lnet networks=tcp0(eth0) routes "tcp1  
132.246.x at tcp0" (routers 132.246.x IP addr)

On the server you need:
options lnet networks=tcp1(eth0) routes "tcp0  
192.168.x at tcp1" (routers 192.168.x IP addr)

Does that make sense?

-Marc

----
D. Marc Stearman
LC Lustre Systems Administrator
marc at llnl.gov
925.423.9670
Pager: 1.888.203.0641


On Jan 16, 2008, at 2:07 PM, Herb Wartens wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA512
>
> IMHO it would be better for you to set up host routing (or whatever it
> is that you wanted) to be ablr to route between these networks.  It  
> would
> be best to avoid the routing code if it is possible since you would  
> be introducing
> some new problems by having the routing turned on.
> With routing enabled you would not have immediate notification that  
> a server went down
> for example.  This causes longer delays on the client when doing  
> things like failover
> since you would have to wait for a timeout to occur before the  
> client would actually
> try the failover server.  I think with the adaptive timeouts this  
> could be resolved,
> but I think that things like this would be better to avoid if you  
> don't especially
> need to use it IMHO (we mainly use it here to route between  
> different transports
> ethernet to infiniband for example).
>
> - -Herb
>
> Jerome, Ron wrote:
>> The issue is the fact that the luster filesystem is on a private
>> (192.168.xxx.xxx) network and the client is on a public network, so
>> bidirectional ip traffic routing gets messy (NATing, port forwarding
>> etc).  I was hoping to avoid this by using a gateway lustre node  
>> that is
>> connected to both networks.
>>
>> Ron.
>>
>>> -----Original Message-----
>>> From: lustre-discuss-bounces at clusterfs.com [mailto:lustre-discuss-
>>> bounces at clusterfs.com] On Behalf Of Herb Wartens
>>> Sent: January 16, 2008 2:32 PM
>>> To: Lustre Discuss
>>> Subject: Re: [Lustre-discuss] How to configure routing
>>>
>>
>> Ron,
>> I am not exactly familiar with your particular setup, but when using
>> tcp
>> is there any reason why you can't use a single lnet network?  I
>>> believe
>> that
>> it would be easier for you to set up one socklnd network named tcp0
>> that
>> contains both interfaces.  Then all you have to do is make sure that
>>> it
>> is possible
>> through host routes (or other means) for the two networks to route to
>> each other.
>>
>> -Herb
>>
>> Jerome, Ron wrote:
>>>>> I would like to use a client as a router between two tcp networks
>> (eth0
>>>>> and eth1) but it is unclear to me how to configure this in
>> modprobe.conf.
>>>>>
>>>>>
>>>>> Any suggestions would be greatly appreciated.
>>>>>
>>>>>
>>>>>
>>>>> _________________________________________
>>>>> Ron Jerome
>>>>> Programmer/Analyst
>>>>> National Research Council Canada
>>>>> M-2, 1200 Montreal Road, Ottawa, Ontario K1A 0R6
>>>>> Government of Canada
>>>>> _________________________________________
>>>>>
>>>>>
>>>>>
>>> -------------------------------------------------------------------- 
>>> -
>> ---
>>>>> _______________________________________________
>>>>> Lustre-discuss mailing list
>>>>> Lustre-discuss at clusterfs.com
>>>>> https://mail.clusterfs.com/mailman/listinfo/lustre-discuss
>>>
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at clusterfs.com
> https://mail.clusterfs.com/mailman/listinfo/lustre-discuss
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.7 (GNU/Linux)
> Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org
>
> iD8DBQFHjoApP/62XqEEbMYRCjNaAJ9f17fLOlcJzMb62LjBD/rt4u4MFQCgkWRD
> H69WcXuJaVymWWJEEc2ufv4=
> =IUFY
> -----END PGP SIGNATURE-----
>
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at clusterfs.com
> https://mail.clusterfs.com/mailman/listinfo/lustre-discuss




More information about the lustre-discuss mailing list