[Lustre-discuss] Lustre over two TCP interfaces
Indivar Nair
indivar.nair at techterra.in
Wed Jun 26 23:30:40 PDT 2013
Then follow the instructions in my earlier mail.
No need to have bond0 and bond1.
You will achieve high-availability even with one bonded interface.
Cheers,
Indivar Nair
On Thu, Jun 27, 2013 at 11:55 AM, Alfonso Pardo <alfonso.pardo at ciemat.es>wrote:
> Yes I have two swtiches, one to the bond0 interface and other switch to
> the second bond1 interface.
>
>
>
> *From:* Indivar Nair <indivar.nair at techterra.in>
> *Sent:* Wednesday, June 26, 2013 8:05 PM
> *To:* Alfonso Pardo <alfonso.pardo at ciemat.es>
> *Cc:* Michael Shuey <shuey at purdue.edu> ; WC-Discuss<WC-Discuss.Migration at intel.com>;
> lustre-discuss <lustre-discuss at lists.lustre.org>
> *Subject:* Re: [Lustre-discuss] Lustre over two TCP interfaces
>
> Hi Alfonso,
>
> I guess, you have two switches, with 2 interfaces (bond0) connected to one
> switch and the other 2 interfaces (bond1) to the second switch.
>
> ---
>
> What you need to do is merge the switches using a 'stacking' cable (if the
> switches are stackable) and create a single trunk using 2 ports from each
> switch.
> Then create a single bond on the Linux side using all the 4 Interfaces
> (and have just 1 IP).
>
> Use bonding mode balance-rr or 0 without LACP to get load balancing across
> all the 4 NICs.
>
> If the switches aren't stackable and a single trunk cannot be created on
> the switch side, then use bonding mode balance-alb or 6 on the Linux side.
>
> No changes need to be done to the cabling in either case.
>
> ---
>
> This way you get Load Balancing and H/A across NICs.
>
>
>
> Indivar Nair
>
>
>
>
> On Wed, Jun 26, 2013 at 5:47 PM, Michael Shuey <shuey at purdue.edu> wrote:
>
>> That will probably be slow - the machine you use to proxy the IPVS
>> address would be a bottleneck. Out of curiosity, what problem are you
>> trying to solve here? Do you anticipate whole-subnet outages to be an
>> issue (and if so, why)?
>>
>> --
>> Mike Shuey
>>
>>
>> On Wed, Jun 26, 2013 at 4:53 AM, Alfonso Pardo <alfonso.pardo at ciemat.es>wrote:
>>
>>> oooh!
>>>
>>>
>>> Thanks for you reply! May be another way is a floating IP between two
>>> interfaces with IPVS (corosync).
>>>
>>> -----Mensaje original----- From: Brian O'Connor
>>> Sent: Wednesday, June 26, 2013 10:15 AM
>>> To: Alfonso Pardo
>>> Cc: 'Michael Shuey' ; 'WC-Discuss' ; lustre-discuss at lists.lustre.**org<lustre-discuss at lists.lustre.org>
>>> Subject: Re: [Lustre-discuss] Lustre over two TCP interfaces
>>>
>>>
>>>
>>>
>>>
>>> On 06/26/2013 04:16 PM, Alfonso Pardo wrote:
>>>
>>>> But.... if I configure the OST assigning to the first interface of the
>>>> OSS (bond0) and as failover OSS the second inteface of the OSS. If the
>>>> bond0 network down, the client will try to connect to the failover, that
>>>> is the second interface of the OSS.
>>>> is it possible?
>>>>
>>>
>>>
>>> I stand to be corrected, but no, I don't think so. As I understand it
>>> the failover code looks for a different server instance, rather than a
>>> different nid.
>>>
>>> See
>>>
>>> http://lists.opensfs.org/**pipermail/lustre-devel-**
>>> opensfs.org/2012-August/**000028.html<http://lists.opensfs.org/pipermail/lustre-devel-opensfs.org/2012-August/000028.html>
>>>
>>>
>>> *From:* Brian O'Connor <mailto:briano at sgi.com>
>>>> *Sent:* Wednesday, June 26, 2013 1:09 AM
>>>> *To:* 'Alfonso Pardo' <mailto:alfonso.pardo at ciemat.**es<alfonso.pardo at ciemat.es>>
>>>> ; 'Michael Shuey'
>>>> <mailto:shuey at purdue.edu>
>>>> *Cc:* 'WC-Discuss' <mailto:WC-Discuss.Migration@**intel.com<WC-Discuss.Migration at intel.com>>
>>>> ;
>>>> mailto:lustre-discuss at lists.**lustre.org<lustre-discuss at lists.lustre.org>
>>>> *Subject:* RE: [Lustre-discuss] Lustre over two TCP interfaces
>>>> Unless something has changed in the new versions of lustre, I don't
>>>> think lustre can do failover between nids on the same machine.
>>>>
>>>> It can choose the available nid at mount time, but if an active nid goes
>>>> away after you are mounted then the client chooses the failover nid, and
>>>> this must be on a different server.
>>>>
>>>> Check the archives for more discussion in this topic :)
>>>>
>>>>
>>>>
>>>> -----Original Message-----
>>>> *From: *Alfonso Pardo [alfonso.pardo at ciemat.es
>>>> <mailto:alfonso.pardo at ciemat.**es <alfonso.pardo at ciemat.es>>]
>>>> *Sent: *Tuesday, June 25, 2013 07:23 AM Central Standard Time
>>>> *To: *Michael Shuey
>>>> *Cc: *WC-Discuss; lustre-discuss at lists.lustre.**org<lustre-discuss at lists.lustre.org>
>>>> *Subject: *Re: [Lustre-discuss] Lustre over two TCP interfaces
>>>>
>>>> thank Michael,
>>>> This is my second step, I will change the lnet with “options lnet
>>>> networks=tcp0(bond0,bond1)” because my machines has 4 nics. I have a
>>>> bond0 and bond1 with LACP. I need to comunicate the clients with two
>>>> network for HA network.
>>>> If the bond0 network is down, the clients can reach the OSS by the
>>>> second network bond1.
>>>> If I change the modprobe with “options lnet
>>>> networks=tcp0(bond0),tcp1(**bond1)”, how the clients mount the
>>>> filesystem
>>>> to reach the OSS by two network?
>>>> *From:* Michael Shuey <mailto:shuey at purdue.edu>
>>>> *Sent:* Tuesday, June 25, 2013 2:14 PM
>>>> *To:* Alfonso Pardo <mailto:alfonso.pardo at ciemat.**es<alfonso.pardo at ciemat.es>
>>>> >
>>>> *Cc:* lustre-discuss at lists.lustre.**org<lustre-discuss at lists.lustre.org>
>>>> <mailto:lustre-discuss at lists.**lustre.org<lustre-discuss at lists.lustre.org>>
>>>> ; WC-Discuss
>>>> <mailto:WC-Discuss.Migration@**intel.com<WC-Discuss.Migration at intel.com>
>>>> >
>>>> *Subject:* Re: [Lustre-discuss] Lustre over two TCP interfaces
>>>> Different interfaces need to be declared with different LNET networks -
>>>> something like "networks=tcp0(eth0),tcp1(**eth1)". Of course, that
>>>> assumes your clients are configured to use a mix of tcp0 and tcp1 for
>>>> connections (with each client only using one of the two). This is
>>>> really only useful in corner cases, when you're doing something strange;
>>>> if eth0 and eth1 are in the same subnet (as in your example), this is
>>>> almost certainly not productive.
>>>> A better bet might be to use a single LNET, and bond the two interfaces
>>>> together - either as an active/passive pair, or active/active (e.g.,
>>>> LACP). Then you'd declare networks=tcp0(bond0), give the bond a single
>>>> IP address, and client traffic would be split across the two members in
>>>> the bond more like you probably expect (given the limits of the bond
>>>> protocol you're using).
>>>> --
>>>> Mike Shuey
>>>>
>>>>
>>>> On Tue, Jun 25, 2013 at 8:06 AM, Alfonso Pardo <alfonso.pardo at ciemat.es
>>>> <mailto:alfonso.pardo at ciemat.**es <alfonso.pardo at ciemat.es>>> wrote:
>>>>
>>>> hello friends,
>>>> I need to comunicate my OSS by two ethernet TCP interfaces: eth0 and
>>>> eth1.
>>>> I have configured this feature in my modprobe.d with:
>>>> “options lnet networks=tcp0(eth0,eth1)”
>>>> And I can see two interfaces with:
>>>> lctl --net tcp interface_list
>>>> sa-d4-01.ceta-ciemat.es <http://sa-d4-01.ceta-ciemat.**es<http://sa-d4-01.ceta-ciemat.es>
>>>> >:
>>>> (192.168.11.15/255.255.255.0 <http://192.168.11.15/255.255.**255.0<http://192.168.11.15/255.255.255.0>
>>>> >)
>>>> npeer 0 nroute 2
>>>> sa-d4-01.ceta-ciemat.es <http://sa-d4-01.ceta-ciemat.**es<http://sa-d4-01.ceta-ciemat.es>
>>>> >:
>>>> (192.168.11.35/255.255.255.0 <http://192.168.11.35/255.255.**255.0<http://192.168.11.35/255.255.255.0>
>>>> >)
>>>> npeer 0 nroute 0
>>>> But, the clients only can communicate with the first interface:
>>>> lctl ping 192.168.11.15
>>>> 12345-0 at lo
>>>> 12345-192.168.11.15 at tcp
>>>> lctl ping 192.168.11.35
>>>> failed to ping 192.168.11.35 at tcp: Input/output error
>>>> Any suggestions how to “enable” the second interface?
>>>> thank in advance
>>>>
>>>> /Alfonso Pardo Diaz/
>>>> /*System Administrator / Researcher*/
>>>> /c/ Sola nº 1; 10200 TRUJILLO, SPAIN/
>>>> /Tel: +34 927 65 93 17 <tel:%2B34%20927%2065%2093%**2017> Fax: +34
>>>> 927
>>>> 32 32 37/
>>>>
>>>> CETA-Ciemat logo <http://www.ceta-ciemat.es/>
>>>>
>>>> ---------------------------- Confidencialidad: Este mensaje y sus
>>>> ficheros adjuntos se dirige exclusivamente a su destinatario y puede
>>>> contener información privilegiada o confidencial. Si no es vd. el
>>>> destinatario indicado, queda notificado de que la utilización,
>>>> divulgación y/o copia sin autorización está prohibida en virtud de
>>>> la legislación vigente. Si ha recibido este mensaje por error, le
>>>> rogamos que nos lo comunique inmediatamente respondiendo al mensaje
>>>> y proceda a su destrucción. Disclaimer: This message and its
>>>> attached files is intended exclusively for its recipients and may
>>>> contain confidential information. If you received this e-mail in
>>>> error you are hereby notified that any dissemination, copy or
>>>> disclosure of this communication is strictly prohibited and may be
>>>> unlawful. In this case, please notify us by a reply and delete this
>>>> email and its contents immediately. ----------------------------
>>>>
>>>> ______________________________**_________________
>>>> Lustre-discuss mailing list
>>>> Lustre-discuss at lists.lustre.**org <Lustre-discuss at lists.lustre.org><mailto:
>>>> Lustre-discuss at lists.**lustre.org <Lustre-discuss at lists.lustre.org>>
>>>> http://lists.lustre.org/**mailman/listinfo/lustre-**discuss<http://lists.lustre.org/mailman/listinfo/lustre-discuss>
>>>>
>>>>
>>>
>>> --
>>> Brian O'Connor
>>> ------------------------------**------------------------------**-
>>> SGI Consulting
>>> Email: briano at sgi.com, Mobile +61 417 746 452 <%2B61%20417%20746%20452>
>>> Phone: +61 3 9963 1900, Fax: +61 3 9963 1902 <%2B61%203%209963%201902>
>>> 691 Burke Road, Camberwell, Victoria, 3124
>>> AUSTRALIA
>>> http://www.sgi.com/support/**services<http://www.sgi.com/support/services>
>>> ------------------------------**------------------------------**-
>>>
>>
>>
>> _______________________________________________
>> Lustre-discuss mailing list
>> Lustre-discuss at lists.lustre.org
>> http://lists.lustre.org/mailman/listinfo/lustre-discuss
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20130627/1e268fc2/attachment.htm>
More information about the lustre-discuss
mailing list