[lustre-discuss] lustre clients cannot access different OSS groups on TCP and infiniband at same time

[Dilger, Andreas]

It sounds like you have two different filesystems, each using the same LNet networks "tcp0" and "o2ib0".  While "tcp" is a shorthand for network "tcp0", the network names need to be unique if the same clients are connecting to both filesystems.  One of the filesystems will need to regenerate the configuration to use "tcp1" and "o2ib1" (or whatever) to allow the clients to distinguish between the different networks.

Cheers, Andreas

On Jul 13, 2016, at 18:27, Riccardo Veraldi <Riccardo.Veraldi at cnaf.infn.it> wrote:
> 
> Hello,
> 
> I am in a situation in which I have a bunch of lustre clients which may access two different lustre
> infrastructures. One is working on TCP  and the other on infiniband.
> So I have 2 OSS groups. One group is reachable thought TCP/Ethernet and the other group through infiniband.
> 
> The problem is that I am not able to have both of them working client side.
> 
> Clients can mount both OSS groups via TCPnd
> 
> options lnet networks=tcp(ens1d1),o2ib(ib0)
> 
> but I'd want to be able to use infiniband for the OSS which has infiniband interface.
> 
> So if I set infiniband as first option
> 
> options lnet networks=o2ib(ib0),tcp(ens1d1)
> 
> only the OSTs related to infiniband OSSs are mounted while the OSTs belonging to OSSs
> reachable through Infiniband are not available.
> 
> I am not able to deploy a configuration which would allow me to do both things at the same time.
> 
> I Tried also to do this:
> 
> options lnet 'ip2nets="o2ib0(ib0) 172.21.49.[224-229]"' 'routes="o2ib0 172.21.49.[224-229]@o2ib0"'
> 
> where 172.21.49.[224-229] are my o21b network IDs (IPoIB) regarding the OSSs in the infiniband group.
> 
> but this still cannot fix.
> 
> What do I need to do to have my clients to be able to mount one group of OSS through TCP and
> the other group of OSS through infiniband ?
> 
> thank you
> 
> Riccardo
> 
> _______________________________________________
> lustre-discuss mailing list
> lustre-discuss at lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org