[Lustre-discuss] How to configure routing

[Isaac Huang]

Lnet routing is generic - it's not only able to route between
different network types (e.g. tcp0 and o2ib0) but also capable 
of routing between networks of a same type (e.g. tcp0 and tcp1).

Before trying to mount lustre, I'd suggest you ensure lnet is working.
A simple way is: lctl ping 192.168.0.200 at tcp.

If it works, then go on and try to mount lustre. Otherwise, 'dmesg'
shall shed some light on what went wrong.

Isaac

On Wed, Jan 16, 2008 at 10:54:31AM -0500, Jerome, Ron wrote:
> That's what I was hoping, but I'm still having trouble getting the
> routing to work.
> 
> My MGS is at 192.168.0.200 at tcp
> 
> On my gateway node, I have...
> 
>       options lnet networks="tcp0(eth0),tcp1(eth1)" routes="tcp1
> 192.168.0.2 at tcp ;\
>                                                             tcp0
> 132.246.xxx.xxx at tcp1"
> 
> Then from the node solely connected to the public (132.246) network, I
> try to mount 
> 
>       mount  -t lustre 192.168.0.200 at tcp:/data /mnt/data 
> 
> but regrettably this does not work :-(
> 
> 
> Ron. 
> 
> > -----Original Message-----
> > From: Canon, Richard Shane [mailto:canonrs at ornl.gov]
> > Sent: January 16, 2008 10:03 AM
> > To: Jerome, Ron
> > Cc: He.Huang at Sun.COM; Lustre-discuss at clusterfs.com
> > Subject: RE: [Lustre-discuss] How to configure routing
> > 
> > 
> > Ron,
> > 
> > You don't have to rely on IP routing.  You can use Lustre routing
> > instead.  You have the basic config line below.  The external clients
> > need to know about both networks and the routers.  You should be able
> > to
> > use the same LNET module options everywhere.  You would do a mount
> > like...
> > 
> > mount -t lustre <mds ip>@tcp1:/lustre/client /mnt
> > 
> > Assuming the private network is tcp1 and tcp0 is the public network.
> > 
> > 
> > --Shane
> > 
> > -----Original Message-----
> > From: lustre-discuss-bounces at clusterfs.com
> > [mailto:lustre-discuss-bounces at clusterfs.com] On Behalf Of Balagopal
> > Pillai
> > Sent: Wednesday, January 16, 2008 9:27 AM
> > To: Jerome, Ron
> > Cc: He.Huang at Sun.COM; Lustre-discuss at clusterfs.com
> > Subject: Re: [Lustre-discuss] How to configure routing
> > 
> > Hi,
> > 
> >           Interesting scenario if routing between public and private
> > subnets for mounting lustre filesystem is possible
> > without disturbing the network admins for public subnet :-) One way
> > around would be
> > to build tunnels like ipip for example between the gateway machine on
> > the cluster
> > that has access to both nets and with other hosts on the public net.
> > Then Lustre mount
> > could work over the tunnel with static routes to access the
> MGS/MDS/OST
> > over the tunnel.
> > 
> > Regards
> > Balagopal
> > 
> > Jerome, Ron wrote:
> > > Hi Isaac,
> > >
> > > Yes I had read the documentation prior to posting my question but a
> > > couple of things were still unclear to me, but first a bit more
> > > background... My lustre filesystem & my cluster reside on a private
> > > (192.169.xxx.xxx) network I have a couple of machines which are
> > > connected to both the private network and our public network and I
> > would
> > > like a machine which only has access to the public network to mount
> > the
> > > lustre filesystem on the private network via one of the nodes which
> > > spans both.
> > >
> > > So my questions are...
> > >
> > > a) is it possible to route to and from a private (192.168.xxx.xxx)
> > > Ethernet network to a public one (132.246.xxx.xxx)? The documents
> > only
> > > show routing between Ethernet and other networks such as elan,
> > Myrinet
> > > etc.
> > >
> > > b) if this is possible, what is the correct syntax for the
> > modprobe.conf
> > > file?  I currently have...
> > >
> > >   options lnet networks="tcp0(eth0),tcp1(eth1)" routes="tcp1
> > > 192.168.0.2 at tcp"
> > >
> > > c) from a client on the public network what would the syntax be for
> > > accessing (mounting) the mgs through the routing node?
> > >
> > >
> > > Thanks very much,
> > >
> > > Ron.
> > >
> > >
> > >> -----Original Message-----
> > >> From: He.Huang at Sun.COM [mailto:He.Huang at Sun.COM]
> > >> Sent: January 16, 2008 8:25 AM
> > >> To: Jerome, Ron
> > >> Cc: Lustre-discuss at clusterfs.com
> > >> Subject: Re: [Lustre-discuss] How to configure routing
> > >>
> > >> On Tue, Jan 15, 2008 at 02:27:39PM -0500, Jerome, Ron wrote:
> > >>
> > >>>    I would like to use a client as a router between two tcp
> > networks
> > >>>
> > >> (eth0
> > >>
> > >>>    and eth1) but it is unclear to me how to configure this in
> > >>>    modprobe.conf.
> > >>>
> > >>>
> > >>>    Any suggestions would be greatly appreciated.
> > >>>
> > >> http://manual.lustre.org/manual/LustreManual16_HTML/DynamicHTML-06-
> > >> 1.html
> > >>
> > >> Please read section 3.2.2.
> > >>
> > >> Isaac
> > >>
> > >
> > > _______________________________________________
> > > Lustre-discuss mailing list
> > > Lustre-discuss at clusterfs.com
> > > https://mail.clusterfs.com/mailman/listinfo/lustre-discuss
> > >
> > 
> > _______________________________________________
> > Lustre-discuss mailing list
> > Lustre-discuss at clusterfs.com
> > https://mail.clusterfs.com/mailman/listinfo/lustre-discuss
> 
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at clusterfs.com
> https://mail.clusterfs.com/mailman/listinfo/lustre-discuss