Maybe I am missing a point here but can you explain me why would you need to have two NICs in one host on the same subnet?<br>If you need additional access route to your host why not to configure eth0 on different subnet?<br>
<br><div class="gmail_quote">On 21 October 2010 15:29, Brock Palen <span dir="ltr"><<a href="mailto:brockp@umich.edu">brockp@umich.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
<div class="im"><br>
<br>
> Why do you need both active? If one is a backup to the other, then bond<br>
> them as a primary/backup pair, meaning only one will be active at at a<br>
> time, ie, your designated primary (unless it goes down).<br>
<br>
</div>We could do this, the 10Gb drivers have been such a pain for us we wanted to have a 'back door' management network to get to the box should we have issues with the 10Gb driver.<br>
<br>
Oddly I ran:<br>
<br>
ifconfig eth0 down<br>
<br>
and I could nolonger ping the box over the eth4 interface, I had to power cycle it form management. Very odd.<br>
<div><div></div><div class="h5"><br>
><br>
> bob<br>
><br>
> On 10/21/2010 9:51 AM, Brock Palen wrote:<br>
>> On Oct 21, 2010, at 9:48 AM, Joe Landman wrote:<br>
>><br>
>>> On 10/21/2010 09:37 AM, Brock Palen wrote:<br>
>>>> We recently added a new oss, it has 1 1Gb interface and 1 10Gb<br>
>>>> interface,<br>
>>>><br>
>>>> The 10Gb interface is eth4 10.164.0.166 The 1Gb interface is eth0<br>
>>>> 10.164.0.10<br>
>>> They look like they are on the same subnet if you are using /24 ...<br>
>> You are correct<br>
>><br>
>> Both interfaces are on the same subnet:<br>
>><br>
>> [root@oss4-gb ~]# route<br>
>> Kernel IP routing table<br>
>> Destination Gateway Genmask Flags Metric Ref Use Iface<br>
>> 10.164.0.0 * 255.255.248.0 U 0 0 0 eth0<br>
>> 10.164.0.0 * 255.255.248.0 U 0 0 0 eth4<br>
>> 169.254.0.0 * 255.255.0.0 U 0 0 0 eth4<br>
>> default 10.164.0.1 0.0.0.0 UG 0 0 0 eth0<br>
>><br>
>> There is no way to mask the lustre service away from the 1Gb interface?<br>
>><br>
>>>> In modprobe.conf I have:<br>
>>>><br>
>>>> options lnet networks=tcp0(eth4)<br>
>>>><br>
>>>> lctl list_nids 10.164.0.166@tcp<br>
>>>><br>
>>>>> From a host I run:<br>
>>>> lctl which_nid oss4 10.164.0.166@tcp<br>
>>>><br>
>>>> But yet I still see traffic over eth0 the 1Gb management network,<br>
>>>> might higher than I would expect (upto 100MB/s) The management<br>
>>>> interface is oss4-gb So If I do from a client:<br>
>>>><br>
>>>> lctl which_nid oss4-gb 10.164.0.10@tcp<br>
>>>><br>
>>>> Why If I have netwroks=tcp0(eth4) and that list_nids showa only the<br>
>>>> 10Gb interface, do I have so much traffic over the 1Gb interface?<br>
>>>> There is some traffic on the 10Gb interface, but I would like to tell<br>
>>>> lustre 'don't use the 1Gb interface'.<br>
>>> If they are on the same subnet, its possible that the 1GbE sees the arp<br>
>>> response first. And then its pretty much guaranteed to have the traffic<br>
>>> go out that port.<br>
>>><br>
>>> If your subnets are different, this shouldn't be the issue.<br>
>>><br>
>>>> Thanks!<br>
>>>><br>
>>>> Brock Palen <a href="http://www.umich.edu/%7Ebrockp" target="_blank">www.umich.edu/~brockp</a> Center for Advanced Computing<br>
>>>> <a href="mailto:brockp@umich.edu">brockp@umich.edu</a> (734)936-1985<br>
>>>><br>
>>>><br>
>>>><br>
>>>> _______________________________________________ Lustre-discuss<br>
>>>> mailing list <a href="mailto:Lustre-discuss@lists.lustre.org">Lustre-discuss@lists.lustre.org</a><br>
>>>> <a href="http://lists.lustre.org/mailman/listinfo/lustre-discuss" target="_blank">http://lists.lustre.org/mailman/listinfo/lustre-discuss</a><br>
>>><br>
>>> --<br>
>>> Joseph Landman, Ph.D<br>
>>> Founder and CEO<br>
>>> Scalable Informatics Inc.<br>
>>> email: <a href="mailto:landman@scalableinformatics.com">landman@scalableinformatics.com</a><br>
>>> web : <a href="http://scalableinformatics.com" target="_blank">http://scalableinformatics.com</a><br>
>>> <a href="http://scalableinformatics.com/jackrabbit" target="_blank">http://scalableinformatics.com/jackrabbit</a><br>
>>> phone: +1 734 786 8423 x121<br>
>>> fax : +1 866 888 3112<br>
>>> cell : +1 734 612 4615<br>
>>> _______________________________________________<br>
>>> Lustre-discuss mailing list<br>
>>> <a href="mailto:Lustre-discuss@lists.lustre.org">Lustre-discuss@lists.lustre.org</a><br>
>>> <a href="http://lists.lustre.org/mailman/listinfo/lustre-discuss" target="_blank">http://lists.lustre.org/mailman/listinfo/lustre-discuss</a><br>
>>><br>
>>><br>
>> _______________________________________________<br>
>> Lustre-discuss mailing list<br>
>> <a href="mailto:Lustre-discuss@lists.lustre.org">Lustre-discuss@lists.lustre.org</a><br>
>> <a href="http://lists.lustre.org/mailman/listinfo/lustre-discuss" target="_blank">http://lists.lustre.org/mailman/listinfo/lustre-discuss</a><br>
>><br>
>><br>
> _______________________________________________<br>
> Lustre-discuss mailing list<br>
> <a href="mailto:Lustre-discuss@lists.lustre.org">Lustre-discuss@lists.lustre.org</a><br>
> <a href="http://lists.lustre.org/mailman/listinfo/lustre-discuss" target="_blank">http://lists.lustre.org/mailman/listinfo/lustre-discuss</a><br>
><br>
><br>
<br>
</div></div>_______________________________________________<br>
<div><div></div><div class="h5">Lustre-discuss mailing list<br>
<a href="mailto:Lustre-discuss@lists.lustre.org">Lustre-discuss@lists.lustre.org</a><br>
<a href="http://lists.lustre.org/mailman/listinfo/lustre-discuss" target="_blank">http://lists.lustre.org/mailman/listinfo/lustre-discuss</a><br>
</div></div></blockquote></div><br><br clear="all"><br><br>