[lustre-discuss] using an lnet router for certain connections... but not for others

[Dilger, Andreas]

On Mar 14, 2017, at 15:46, Jessica Otey <jotey at nrao.edu> wrote:
> 
> We have three lustre systems, two "production" and one "recovery" system, which is in essence a (partial) backup of ONE of the production systems.
> 
> The main difference between the two production systems is that one uses IB and one uses 10G ethernet among the OSSes.
> 
> I am in the process of moving my robinhood instance from one box to another. The previous box was a normal "production" client, meaning it accessed both clients via our lnet routers (which are identical, and are used by both production lustre file systems).
> 
> The new box is a "data mover" for the backup lustre file system, which was configured WITHOUT an lnet router. Because this box has IB, I am able to connect directly to production lustre file system that also has IB. The thing is, I would also like to keep using robinhood for the other file system, the one using 10gig ethernet.
> 
> Is there a way to specify in the lustre.conf configuration a set up whereby the lnet routers could be used to access only the production file systems but not the backup file system?

I'm not a router expert, but routers should only be used if the client isn't on the LNet network where the target is connected (assuming the configuration lists all of the LNet interfaces of the server.  For example, if the OST is on ib0 and the client is also on ib0 then it shouldn't use e.g. a tcp0->ib0 router even if it also has a tcp0 interface.  It will preferentially take the route with the fewest hops.

Cheers, Andreas
--
Andreas Dilger
Lustre Principal Architect
Intel Corporation