[Lustre-discuss] controlling which eth interface lustre uses

Bob Ball ball at umich.edu
Thu Oct 21 06:59:37 PDT 2010


Why do you need both active?  If one is a backup to the other, then bond 
them as a primary/backup pair, meaning only one will be active at at a 
time, ie, your designated primary (unless it goes down).

bob

On 10/21/2010 9:51 AM, Brock Palen wrote:
> On Oct 21, 2010, at 9:48 AM, Joe Landman wrote:
>
>> On 10/21/2010 09:37 AM, Brock Palen wrote:
>>> We recently added a new oss, it has 1 1Gb interface and 1 10Gb
>>> interface,
>>>
>>> The 10Gb interface is eth4 10.164.0.166 The 1Gb   interface is eth0
>>> 10.164.0.10
>> They look like they are on the same subnet if you are using /24 ...
> You are correct
>
> Both interfaces are on the same subnet:
>
> [root at oss4-gb ~]# route
> Kernel IP routing table
> Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
> 10.164.0.0      *               255.255.248.0   U     0      0        0 eth0
> 10.164.0.0      *               255.255.248.0   U     0      0        0 eth4
> 169.254.0.0     *               255.255.0.0     U     0      0        0 eth4
> default         10.164.0.1      0.0.0.0         UG    0      0        0 eth0
>
> There is no way to mask the lustre service away from the 1Gb interface?
>
>>> In modprobe.conf I have:
>>>
>>> options lnet networks=tcp0(eth4)
>>>
>>> lctl list_nids 10.164.0.166 at tcp
>>>
>>>>  From a host I run:
>>> lctl which_nid oss4 10.164.0.166 at tcp
>>>
>>> But yet I still see traffic over eth0 the 1Gb management network,
>>> might higher than I would expect (upto 100MB/s) The management
>>> interface is oss4-gb  So If I do from a client:
>>>
>>> lctl which_nid oss4-gb 10.164.0.10 at tcp
>>>
>>> Why If I have netwroks=tcp0(eth4)  and that list_nids showa only the
>>> 10Gb interface, do I have so much traffic over the 1Gb interface?
>>> There is some traffic on the 10Gb interface, but I would like to tell
>>> lustre 'don't use the 1Gb interface'.
>> If they are on the same subnet, its possible that the 1GbE sees the arp
>> response first.  And then its pretty much guaranteed to have the traffic
>> go out that port.
>>
>> If your subnets are different, this shouldn't be the issue.
>>
>>> Thanks!
>>>
>>> Brock Palen www.umich.edu/~brockp Center for Advanced Computing
>>> brockp at umich.edu (734)936-1985
>>>
>>>
>>>
>>> _______________________________________________ Lustre-discuss
>>> mailing list Lustre-discuss at lists.lustre.org
>>> http://lists.lustre.org/mailman/listinfo/lustre-discuss
>>
>> -- 
>> Joseph Landman, Ph.D
>> Founder and CEO
>> Scalable Informatics Inc.
>> email: landman at scalableinformatics.com
>> web  : http://scalableinformatics.com
>>         http://scalableinformatics.com/jackrabbit
>> phone: +1 734 786 8423 x121
>> fax  : +1 866 888 3112
>> cell : +1 734 612 4615
>> _______________________________________________
>> Lustre-discuss mailing list
>> Lustre-discuss at lists.lustre.org
>> http://lists.lustre.org/mailman/listinfo/lustre-discuss
>>
>>
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-discuss
>
>



More information about the lustre-discuss mailing list