[Lustre-discuss] multi-homed issues

[Andreas Dilger]

On Sep 30, 2009  16:13 +1000, Philip Manuel wrote:
> My /etc/modprobe.d/lustre now contains
>
> options lnet networks=tcp0(eth0),tcp1(eth1)
>
> I updated the partitions with the following:-
>
> mgsnode=172.16.0.57 at tcp,172.17.11.250 at tcp1
>
> On both of the MDT's lctl list_nids reports (note they have the above  
> lnet entry in their /etc/modprobe.d/lustre file):-
>
> 172.16.0.51 at tcp
> 172.17.11.252 at tcp1
>
> On clients, for either network, they do not have the lnet entry and lctl  
> list_nids reports on the local machine ip address.  However, clients on  
> the tcp0 network can mount the lustre partition.  Clients on the tcp1  
> network cannot mount via the tcp1 IP address but can via the tcp network  
> (with a another machine routing the traffic).
>
> Do I need the same lnet module entry everywhere to make this work ?

Yes, all of the clients need to be told whether they are in the tcp0
network or the tcp1 network.  Otherwise they will default to tcp0
(which will cause the ones on the 172.17.11.* subnet to be rejected).

The manual describes different ways you can make this line the same
for all nodes so they can automatically pick the right LNET network
based on their subnet address.

> Andreas Dilger wrote:
>> On Sep 29, 2009  16:05 +1000, Philip Manuel wrote:
>>   
>>> Hi we would like the lustre servers available to two networks, one on 
>>> eth0 (192.168.1.0/24) the other on eth1(192.168.8.0/22), they are on  
>>> different networks.
>>>
>>> I've tried the following:-
>>>
>>> 1. Create /etc/modprobe.d/lustre that contains
>>> options lnet networks=tcp0,tcp1
>>>     
>>
>> This probably needs to be like:
>>
>> options lnet networks=tcp0(eth0),tcp1(eth1)
>>
>>   
>>> 2. Used tunefs.lustre to specify the ip address of both interfaces of 
>>> the MGS/MDT is on, i.e. tunefs.lustre --eraseparams  
>>> --mgsnode=<net1>@tcp0,<net2>@tcp1 /dev/md5
>>> 3. Started up lustre
>>>
>>> The clients could connect on network tcp0, but the clients that would 
>>> connect via the IP address of network tcp1 could not.
>>>
>>> What am I missing ?
>>>
>>> Thanks
>>>
>>> Phil.
>>> _______________________________________________
>>> Lustre-discuss mailing list
>>> Lustre-discuss at lists.lustre.org
>>> http://lists.lustre.org/mailman/listinfo/lustre-discuss
>>>     
>>
>> Cheers, Andreas
>> --
>> Andreas Dilger
>> Sr. Staff Engineer, Lustre Group
>> Sun Microsystems of Canada, Inc.
>>
>>   

Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.