[lustre-discuss] lnetctl fails to recreate exact config when importing exported lnet.conf

[Mohr Jr, Richard Frank]

> On Sep 4, 2020, at 12:11 AM, Angelos Ching <angelosching at clustertech.com> wrote:
> 
> All steps below carried out on Lustre client:
> 
> 1. Restart lnet service with empty /etc/lnet.conf
> 2. lnetctl net add: TCP network using Ethernet
> 3. lnetctl peer add: 2 peers with "Lnet router + server"@o2ib,tcp NIDs

The commands you ran were:

[root at access2 ~]# lnetctl peer add --nid 10.1.4.24 at o2ib,10.4.7.24 at tcp
[root at access2 ~]# lnetctl peer add --nid 10.1.4.25 at o2ib,10.4.7.25 at tcp

Commands like this can be used when a node has a multirail setup, like when a node has multiple interfaces on the same network.  But for your routers, it looks like the tcp network is available to the client and the o2ib network is available to the server.  Since those interfaces are not on the same network so you don’t need to add both of them as a peer.

> 4. lnetctl route add: 2 gateways to o2ib network using "Lnet router +
> server"@TCP NID

[root at access2 ~]# lnetctl route add --net o2ib --gateway 10.4.7.24 at tcp
[root at access2 ~]# lnetctl route add --net o2ib --gateway 10.4.7.25 at tcp

These should be the only commands you need to run to configure your routing.

-Rick