[Lustre-discuss] How to configure routing

[Jerome, Ron]

That's fantastic Marc,  

Just the information I was looking for.  I'll give it a try.

Ron. 

> -----Original Message-----
> From: lustre-discuss-bounces at clusterfs.com [mailto:lustre-discuss-
> bounces at clusterfs.com] On Behalf Of D. Marc Stearman
> Sent: January 17, 2008 11:31 AM
> To: Lustre Discuss
> Subject: Re: [Lustre-discuss] How to configure routing
> 
> I hate to disagree with you Herb, but there are a great number of
> reasons you would not want to set up routing between those networks.
> First of all, making 'private' subnets visible to the public network
> is a no-no.  Second, private networks are often secluded for security
> reasons.  Bridging the gap could have many unintended consequences.
> 
> Lustre should be able to do all the routing with no problem.
> 
> Servers:  LNET tcp0 network
> Routers: LNET tcp0,tcp1 networks
> Clients: LNET tcp1 network
> 
> server (192.168.x)  <---->   (192.168.x) router (132.246.x) <---->
> (132.246.x) client
> 
> Make sure you have ip_forwarding enabled on the router node.
> Make sure 'options lnet forwarding="enabled"' is set on the router
> node.
> 
> The router will find routes automatically.  You just need to specify
> both networks:
> options lnet networks=tcp0(eth0),tcp1(eth1)
> 
> On the client you need:
> options lnet networks=tcp0(eth0) routes "tcp1
> 132.246.x at tcp0" (routers 132.246.x IP addr)
> 
> On the server you need:
> options lnet networks=tcp1(eth0) routes "tcp0
> 192.168.x at tcp1" (routers 192.168.x IP addr)
> 
> Does that make sense?
> 
> -Marc
> 
> ----
> D. Marc Stearman
> LC Lustre Systems Administrator
> marc at llnl.gov
> 925.423.9670
> Pager: 1.888.203.0641
> 
> 
> On Jan 16, 2008, at 2:07 PM, Herb Wartens wrote:
> 
> > -----BEGIN PGP SIGNED MESSAGE-----
> > Hash: SHA512
> >
> > IMHO it would be better for you to set up host routing (or whatever
> it
> > is that you wanted) to be ablr to route between these networks.  It
> > would
> > be best to avoid the routing code if it is possible since you would
> > be introducing
> > some new problems by having the routing turned on.
> > With routing enabled you would not have immediate notification that
> > a server went down
> > for example.  This causes longer delays on the client when doing
> > things like failover
> > since you would have to wait for a timeout to occur before the
> > client would actually
> > try the failover server.  I think with the adaptive timeouts this
> > could be resolved,
> > but I think that things like this would be better to avoid if you
> > don't especially
> > need to use it IMHO (we mainly use it here to route between
> > different transports
> > ethernet to infiniband for example).
> >
> > - -Herb
> >
> > Jerome, Ron wrote:
> >> The issue is the fact that the luster filesystem is on a private
> >> (192.168.xxx.xxx) network and the client is on a public network, so
> >> bidirectional ip traffic routing gets messy (NATing, port
forwarding
> >> etc).  I was hoping to avoid this by using a gateway lustre node
> >> that is
> >> connected to both networks.
> >>
> >> Ron.
> >>
> >>> -----Original Message-----
> >>> From: lustre-discuss-bounces at clusterfs.com [mailto:lustre-discuss-
> >>> bounces at clusterfs.com] On Behalf Of Herb Wartens
> >>> Sent: January 16, 2008 2:32 PM
> >>> To: Lustre Discuss
> >>> Subject: Re: [Lustre-discuss] How to configure routing
> >>>
> >>
> >> Ron,
> >> I am not exactly familiar with your particular setup, but when
using
> >> tcp
> >> is there any reason why you can't use a single lnet network?  I
> >>> believe
> >> that
> >> it would be easier for you to set up one socklnd network named tcp0
> >> that
> >> contains both interfaces.  Then all you have to do is make sure
that
> >>> it
> >> is possible
> >> through host routes (or other means) for the two networks to route
> to
> >> each other.
> >>
> >> -Herb
> >>
> >> Jerome, Ron wrote:
> >>>>> I would like to use a client as a router between two tcp
networks
> >> (eth0
> >>>>> and eth1) but it is unclear to me how to configure this in
> >> modprobe.conf.
> >>>>>
> >>>>>
> >>>>> Any suggestions would be greatly appreciated.
> >>>>>
> >>>>>
> >>>>>
> >>>>> _________________________________________
> >>>>> Ron Jerome
> >>>>> Programmer/Analyst
> >>>>> National Research Council Canada
> >>>>> M-2, 1200 Montreal Road, Ottawa, Ontario K1A 0R6
> >>>>> Government of Canada
> >>>>> _________________________________________
> >>>>>
> >>>>>
> >>>>>
> >>>
-------------------------------------------------------------------
> -
> >>> -
> >> ---
> >>>>> _______________________________________________
> >>>>> Lustre-discuss mailing list
> >>>>> Lustre-discuss at clusterfs.com
> >>>>> https://mail.clusterfs.com/mailman/listinfo/lustre-discuss
> >>>
> > _______________________________________________
> > Lustre-discuss mailing list
> > Lustre-discuss at clusterfs.com
> > https://mail.clusterfs.com/mailman/listinfo/lustre-discuss
> >
> > -----BEGIN PGP SIGNATURE-----
> > Version: GnuPG v1.4.7 (GNU/Linux)
> > Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org
> >
> > iD8DBQFHjoApP/62XqEEbMYRCjNaAJ9f17fLOlcJzMb62LjBD/rt4u4MFQCgkWRD
> > H69WcXuJaVymWWJEEc2ufv4=
> > =IUFY
> > -----END PGP SIGNATURE-----
> >
> > _______________________________________________
> > Lustre-discuss mailing list
> > Lustre-discuss at clusterfs.com
> > https://mail.clusterfs.com/mailman/listinfo/lustre-discuss
> 
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at clusterfs.com
> https://mail.clusterfs.com/mailman/listinfo/lustre-discuss