[Lustre-discuss] problem with clients and multiple transports

[John White]

Unfortunately, no lnet routers are involved.  If I just have o2ib NIDs available on the client, the IB connected FS mounts fine.  If I have TCP NIDs listed first in the modprobe.conf along with o2ib, IB does not mount.
----------------
John White
HPC Systems Engineer
(510) 486-7307
One Cyclotron Rd, MS: 50C-3396
Lawrence Berkeley National Lab
Berkeley, CA 94720

On Nov 9, 2011, at 9:13 AM, Thomas Roth wrote:

> Your clients have both ib and tcp nids? Because I encountered a strange behavior trying to mount an ib 
> based FS and a tcp based FS on the same (ethernet-only) client. To connect to the ib MDS it had to go 
> through a lnet router, of course.
> Experimentally, I found
>> options lnet networks=tcp0(eth0:0),tcp1(eth0) routes="o2ib 10.81.2.108 at tcp1; tcp 10.12.0.1 at tcp1"
> to be working.
> This client sits with its eth0 interface in the 10.12.x.y subnet, 10.81.2.108 is the IP of the lnet 
> router. I have no idea why a second IP on that client should be necessary, but without it, no 
> connection could be established.
> Also, the order 'networks=tcp0(eth0:0),tcp1(eth0)' proved to be important, 'tcp1(eth0),tcp0(eth0:0)' 
> didn't work either.
> But I'm afraid this is a unrelated problem (the two Lustres are separate FS, different MGSes..)
> 
> Thomas
> 
> On 02.11.2011 16:55, John White wrote:
>> Hello Folks,
>> 	I'm having a problem pinning a given FS to a given transport type.  I have 2 file systems hosted on the OSS/MDS side by an ib and tcp nid (each) being mounted by a client specifying the appropriate transport for each of the file systems (one over tcp and one over o2ib).  I can not mount o2ib.  Period.  If my client's modprobe.conf lists o2ib before the tcp nid, any o2ib mount request fails.  If the modprobe.conf has tcp first, both mount requests will mount via tcp0, no matter how the mount request is crafted.
>> 
>> 	So, am I just missing documentation somewhere that states you can only use a single transport on a client at a time or is this a bug?
>> ----------------
>> John White
>> HPC Systems Engineer
>> (510) 486-7307
>> One Cyclotron Rd, MS: 50C-3396
>> University of California Berkeley
>> Berkeley, CA 94720
>> 
>> _______________________________________________
>> Lustre-discuss mailing list
>> Lustre-discuss at lists.lustre.org
>> http://lists.lustre.org/mailman/listinfo/lustre-discuss
> 
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-discuss