[Lustre-discuss] Lustre over two TCP interfaces

Michael Shuey shuey at purdue.edu
Wed Jun 26 05:17:03 PDT 2013


That will probably be slow - the machine you use to proxy the IPVS address
would be a bottleneck.  Out of curiosity, what problem are you trying to
solve here?  Do you anticipate whole-subnet outages to be an issue (and if
so, why)?

--
Mike Shuey


On Wed, Jun 26, 2013 at 4:53 AM, Alfonso Pardo <alfonso.pardo at ciemat.es>wrote:

> oooh!
>
>
> Thanks for you reply! May be another way is a floating IP between two
> interfaces with IPVS (corosync).
>
> -----Mensaje original----- From: Brian O'Connor
> Sent: Wednesday, June 26, 2013 10:15 AM
> To: Alfonso Pardo
> Cc: 'Michael Shuey' ; 'WC-Discuss' ; lustre-discuss at lists.lustre.**org<lustre-discuss at lists.lustre.org>
> Subject: Re: [Lustre-discuss] Lustre over two TCP interfaces
>
>
>
>
>
> On 06/26/2013 04:16 PM, Alfonso Pardo wrote:
>
>> But.... if I configure the OST assigning to the first interface of the
>> OSS (bond0) and as failover OSS the second inteface of the OSS. If the
>> bond0 network down, the client will try to connect to the failover, that
>> is the second interface of the OSS.
>> is it possible?
>>
>
>
> I stand to be corrected, but no, I don't think so. As I understand it
> the failover code looks for a different server instance, rather than a
> different nid.
>
> See
>
> http://lists.opensfs.org/**pipermail/lustre-devel-**
> opensfs.org/2012-August/**000028.html<http://lists.opensfs.org/pipermail/lustre-devel-opensfs.org/2012-August/000028.html>
>
>
>  *From:* Brian O'Connor <mailto:briano at sgi.com>
>> *Sent:* Wednesday, June 26, 2013 1:09 AM
>> *To:* 'Alfonso Pardo' <mailto:alfonso.pardo at ciemat.**es<alfonso.pardo at ciemat.es>>
>> ; 'Michael Shuey'
>> <mailto:shuey at purdue.edu>
>> *Cc:* 'WC-Discuss' <mailto:WC-Discuss.Migration@**intel.com<WC-Discuss.Migration at intel.com>>
>> ;
>> mailto:lustre-discuss at lists.**lustre.org<lustre-discuss at lists.lustre.org>
>> *Subject:* RE: [Lustre-discuss] Lustre over two TCP interfaces
>> Unless something has changed in the new versions of lustre, I don't
>> think lustre can do failover between nids on the same machine.
>>
>> It can choose the available nid at mount time, but if an active nid goes
>> away after you are mounted then the client chooses the failover nid, and
>> this must be on a different server.
>>
>> Check the archives for more discussion in this topic :)
>>
>>
>>
>> -----Original Message-----
>> *From: *Alfonso Pardo [alfonso.pardo at ciemat.es
>> <mailto:alfonso.pardo at ciemat.**es <alfonso.pardo at ciemat.es>>]
>> *Sent: *Tuesday, June 25, 2013 07:23 AM Central Standard Time
>> *To: *Michael Shuey
>> *Cc: *WC-Discuss; lustre-discuss at lists.lustre.**org<lustre-discuss at lists.lustre.org>
>> *Subject: *Re: [Lustre-discuss] Lustre over two TCP interfaces
>>
>> thank Michael,
>> This is my second step, I will change the lnet with “options lnet
>> networks=tcp0(bond0,bond1)” because my machines has 4 nics. I have a
>> bond0 and bond1 with LACP. I need to comunicate the clients with two
>> network for HA network.
>> If the bond0 network is down, the clients can reach the OSS by the
>> second network bond1.
>> If I change the modprobe with “options lnet
>> networks=tcp0(bond0),tcp1(**bond1)”, how the clients mount the filesystem
>> to reach the OSS by two network?
>> *From:* Michael Shuey <mailto:shuey at purdue.edu>
>> *Sent:* Tuesday, June 25, 2013 2:14 PM
>> *To:* Alfonso Pardo <mailto:alfonso.pardo at ciemat.**es<alfonso.pardo at ciemat.es>
>> >
>> *Cc:* lustre-discuss at lists.lustre.**org <lustre-discuss at lists.lustre.org>
>> <mailto:lustre-discuss at lists.**lustre.org<lustre-discuss at lists.lustre.org>>
>> ; WC-Discuss
>> <mailto:WC-Discuss.Migration@**intel.com <WC-Discuss.Migration at intel.com>
>> >
>> *Subject:* Re: [Lustre-discuss] Lustre over two TCP interfaces
>> Different interfaces need to be declared with different LNET networks -
>> something like "networks=tcp0(eth0),tcp1(**eth1)".  Of course, that
>> assumes your clients are configured to use a mix of tcp0 and tcp1 for
>> connections (with each client only using one of the two).  This is
>> really only useful in corner cases, when you're doing something strange;
>> if eth0 and eth1 are in the same subnet (as in your example), this is
>> almost certainly not productive.
>> A better bet might be to use a single LNET, and bond the two interfaces
>> together - either as an active/passive pair, or active/active (e.g.,
>> LACP).  Then you'd declare networks=tcp0(bond0), give the bond a single
>> IP address, and client traffic would be split across the two members in
>> the bond more like you probably expect (given the limits of the bond
>> protocol you're using).
>> --
>> Mike Shuey
>>
>>
>> On Tue, Jun 25, 2013 at 8:06 AM, Alfonso Pardo <alfonso.pardo at ciemat.es
>> <mailto:alfonso.pardo at ciemat.**es <alfonso.pardo at ciemat.es>>> wrote:
>>
>>     hello friends,
>>     I need to comunicate my OSS by two ethernet TCP interfaces: eth0 and
>>     eth1.
>>     I have configured this feature in my modprobe.d with:
>>     “options lnet networks=tcp0(eth0,eth1)”
>>     And I can see two interfaces with:
>>     lctl --net tcp interface_list
>>     sa-d4-01.ceta-ciemat.es <http://sa-d4-01.ceta-ciemat.**es<http://sa-d4-01.ceta-ciemat.es>
>> >:
>>     (192.168.11.15/255.255.255.0 <http://192.168.11.15/255.255.**255.0<http://192.168.11.15/255.255.255.0>
>> >)
>>     npeer 0 nroute 2
>>     sa-d4-01.ceta-ciemat.es <http://sa-d4-01.ceta-ciemat.**es<http://sa-d4-01.ceta-ciemat.es>
>> >:
>>     (192.168.11.35/255.255.255.0 <http://192.168.11.35/255.255.**255.0<http://192.168.11.35/255.255.255.0>
>> >)
>>     npeer 0 nroute 0
>>     But, the clients only can communicate with the first interface:
>>     lctl ping 192.168.11.15
>>     12345-0 at lo
>>     12345-192.168.11.15 at tcp
>>     lctl ping 192.168.11.35
>>     failed to ping 192.168.11.35 at tcp: Input/output error
>>     Any suggestions how to “enable” the second interface?
>>     thank in advance
>>
>>     /Alfonso Pardo Diaz/
>>     /*System Administrator / Researcher*/
>>     /c/ Sola nº 1; 10200 TRUJILLO, SPAIN/
>>     /Tel: +34 927 65 93 17 <tel:%2B34%20927%2065%2093%**2017> Fax: +34
>> 927
>>     32 32 37/
>>
>>     CETA-Ciemat logo <http://www.ceta-ciemat.es/>
>>
>>     ---------------------------- Confidencialidad: Este mensaje y sus
>>     ficheros adjuntos se dirige exclusivamente a su destinatario y puede
>>     contener información privilegiada o confidencial. Si no es vd. el
>>     destinatario indicado, queda notificado de que la utilización,
>>     divulgación y/o copia sin autorización está prohibida en virtud de
>>     la legislación vigente. Si ha recibido este mensaje por error, le
>>     rogamos que nos lo comunique inmediatamente respondiendo al mensaje
>>     y proceda a su destrucción. Disclaimer: This message and its
>>     attached files is intended exclusively for its recipients and may
>>     contain confidential information. If you received this e-mail in
>>     error you are hereby notified that any dissemination, copy or
>>     disclosure of this communication is strictly prohibited and may be
>>     unlawful. In this case, please notify us by a reply and delete this
>>     email and its contents immediately. ----------------------------
>>
>>     ______________________________**_________________
>>     Lustre-discuss mailing list
>>     Lustre-discuss at lists.lustre.**org <Lustre-discuss at lists.lustre.org><mailto:
>> Lustre-discuss at lists.**lustre.org <Lustre-discuss at lists.lustre.org>>
>>     http://lists.lustre.org/**mailman/listinfo/lustre-**discuss<http://lists.lustre.org/mailman/listinfo/lustre-discuss>
>>
>>
>
> --
> Brian O'Connor
> ------------------------------**------------------------------**-
> SGI Consulting
> Email: briano at sgi.com, Mobile +61 417 746 452
> Phone: +61 3 9963 1900, Fax:  +61 3 9963 1902
> 691 Burke Road, Camberwell, Victoria, 3124
> AUSTRALIA
> http://www.sgi.com/support/**services<http://www.sgi.com/support/services>
> ------------------------------**------------------------------**-
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20130626/471df338/attachment.htm>


More information about the lustre-discuss mailing list