[Lustre-discuss] Configuring Lustre routring between two tcp networks

[Isaac Huang]

On Thu, Jun 11, 2009 at 10:51:01PM -0400, Erik Froese wrote:
>    OK here's where I am now.
> 
>    The public client can ping the routers public address but not the
>    private address.
> 
>    [root at routed-client lnet]$ cat /etc/modprobe.conf
>    ......
>    options lnet accept=all

This would allow connections from unprivileged ports, which is
probably not what you want unless you have liblustre clients. The
default "accept" setting should work fine.

>    ......
>    [root at routed-client lnet]$ lctl ping 10.1.255.252 at tcp
>    failed to ping 10.1.255.252 at tcp: Input/output error
>    I can see traffic between the routed-client and the router as well as
>    between the router and the MGS/MDS (10.1.255.252 at tcp)
> 
>    The mgs has the following config.
> 
>    [root at mgs-0-0 lnet]# cat /etc/modprobe.conf
>    ......
>    options lnet forwarding="enabled"

Only needed for routers.

>    options lnet networks=tcp(eth0) routes="tcp1 10.1.255.247 at tcp"

This looks good.

>    [root at mgs-0-0 lnet]# lctl network up
>    LNET configured
>    But it doesn't see any routes or routers.

Which was why the client couldn't ping the MGS - ping request arrived
at the MGS, but the MGS didn't have a route to send back its reply to
the client network.

>    [root at mgs-0-0 lnet]# cat /proc/sys/lnet/routes
>    Routing disabled
>    net      hops   state router

Could you try to unload and load the lnet module again? Or add quotes
over in modprobe.conf around all options, or add the options directly
at modprobe command line. Somehow the routes option seemed to be
ignored.

Thanks,
Isaac