[Lustre-discuss] controlling which eth interface lustre uses

Joe Landman landman at scalableinformatics.com
Thu Oct 21 06:48:42 PDT 2010


On 10/21/2010 09:37 AM, Brock Palen wrote:
> We recently added a new oss, it has 1 1Gb interface and 1 10Gb
> interface,
>
> The 10Gb interface is eth4 10.164.0.166 The 1Gb   interface is eth0
> 10.164.0.10

They look like they are on the same subnet if you are using /24 ...

>
> In modprobe.conf I have:
>
> options lnet networks=tcp0(eth4)
>
> lctl list_nids 10.164.0.166 at tcp
>
>> From a host I run:
>
> lctl which_nid oss4 10.164.0.166 at tcp
>
> But yet I still see traffic over eth0 the 1Gb management network,
> might higher than I would expect (upto 100MB/s) The management
> interface is oss4-gb  So If I do from a client:
>
> lctl which_nid oss4-gb 10.164.0.10 at tcp
>
> Why If I have netwroks=tcp0(eth4)  and that list_nids showa only the
> 10Gb interface, do I have so much traffic over the 1Gb interface?
> There is some traffic on the 10Gb interface, but I would like to tell
> lustre 'don't use the 1Gb interface'.

If they are on the same subnet, its possible that the 1GbE sees the arp 
response first.  And then its pretty much guaranteed to have the traffic 
go out that port.

If your subnets are different, this shouldn't be the issue.

>
> Thanks!
>
> Brock Palen www.umich.edu/~brockp Center for Advanced Computing
> brockp at umich.edu (734)936-1985
>
>
>
> _______________________________________________ Lustre-discuss
> mailing list Lustre-discuss at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-discuss


-- 
Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics Inc.
email: landman at scalableinformatics.com
web  : http://scalableinformatics.com
        http://scalableinformatics.com/jackrabbit
phone: +1 734 786 8423 x121
fax  : +1 866 888 3112
cell : +1 734 612 4615



More information about the lustre-discuss mailing list