[Lustre-discuss] Lustre over two TCP interfaces

Wed Jun 26 01:53:41 PDT 2013

oooh!

Thanks for you reply! May be another way is a floating IP between two 
interfaces with IPVS (corosync).

-----Mensaje original----- 
From: Brian O'Connor
Sent: Wednesday, June 26, 2013 10:15 AM
To: Alfonso Pardo
Cc: 'Michael Shuey' ; 'WC-Discuss' ; lustre-discuss at lists.lustre.org
Subject: Re: [Lustre-discuss] Lustre over two TCP interfaces

On 06/26/2013 04:16 PM, Alfonso Pardo wrote:
> But.... if I configure the OST assigning to the first interface of the
> OSS (bond0) and as failover OSS the second inteface of the OSS. If the
> bond0 network down, the client will try to connect to the failover, that
> is the second interface of the OSS.
> is it possible?

I stand to be corrected, but no, I don't think so. As I understand it
the failover code looks for a different server instance, rather than a
different nid.

See

http://lists.opensfs.org/pipermail/lustre-devel-opensfs.org/2012-August/000028.html

> *From:* Brian O'Connor <mailto:briano at sgi.com>
> *Sent:* Wednesday, June 26, 2013 1:09 AM
> *To:* 'Alfonso Pardo' <mailto:alfonso.pardo at ciemat.es> ; 'Michael Shuey'
> <mailto:shuey at purdue.edu>
> *Cc:* 'WC-Discuss' <mailto:WC-Discuss.Migration at intel.com> ;
> mailto:lustre-discuss at lists.lustre.org
> *Subject:* RE: [Lustre-discuss] Lustre over two TCP interfaces
> Unless something has changed in the new versions of lustre, I don't
> think lustre can do failover between nids on the same machine.
>
> It can choose the available nid at mount time, but if an active nid goes
> away after you are mounted then the client chooses the failover nid, and
> this must be on a different server.
>
> Check the archives for more discussion in this topic :)
>
>
>
> -----Original Message-----
> *From: *Alfonso Pardo [alfonso.pardo at ciemat.es
> <mailto:alfonso.pardo at ciemat.es>]
> *Sent: *Tuesday, June 25, 2013 07:23 AM Central Standard Time
> *To: *Michael Shuey
> *Cc: *WC-Discuss; lustre-discuss at lists.lustre.org
> *Subject: *Re: [Lustre-discuss] Lustre over two TCP interfaces
>
> thank Michael,
> This is my second step, I will change the lnet with “options lnet
> networks=tcp0(bond0,bond1)” because my machines has 4 nics. I have a
> bond0 and bond1 with LACP. I need to comunicate the clients with two
> network for HA network.
> If the bond0 network is down, the clients can reach the OSS by the
> second network bond1.
> If I change the modprobe with “options lnet
> networks=tcp0(bond0),tcp1(bond1)”, how the clients mount the filesystem
> to reach the OSS by two network?
> *From:* Michael Shuey <mailto:shuey at purdue.edu>
> *Sent:* Tuesday, June 25, 2013 2:14 PM
> *To:* Alfonso Pardo <mailto:alfonso.pardo at ciemat.es>
> *Cc:* lustre-discuss at lists.lustre.org
> <mailto:lustre-discuss at lists.lustre.org> ; WC-Discuss
> <mailto:WC-Discuss.Migration at intel.com>
> *Subject:* Re: [Lustre-discuss] Lustre over two TCP interfaces
> Different interfaces need to be declared with different LNET networks -
> something like "networks=tcp0(eth0),tcp1(eth1)".  Of course, that
> assumes your clients are configured to use a mix of tcp0 and tcp1 for
> connections (with each client only using one of the two).  This is
> really only useful in corner cases, when you're doing something strange;
> if eth0 and eth1 are in the same subnet (as in your example), this is
> almost certainly not productive.
> A better bet might be to use a single LNET, and bond the two interfaces
> together - either as an active/passive pair, or active/active (e.g.,
> LACP).  Then you'd declare networks=tcp0(bond0), give the bond a single
> IP address, and client traffic would be split across the two members in
> the bond more like you probably expect (given the limits of the bond
> protocol you're using).
> --
> Mike Shuey
>
>
> On Tue, Jun 25, 2013 at 8:06 AM, Alfonso Pardo <alfonso.pardo at ciemat.es
> <mailto:alfonso.pardo at ciemat.es>> wrote:
>
>     hello friends,
>     I need to comunicate my OSS by two ethernet TCP interfaces: eth0 and
>     eth1.
>     I have configured this feature in my modprobe.d with:
>     “options lnet networks=tcp0(eth0,eth1)”
>     And I can see two interfaces with:
>     lctl --net tcp interface_list
>     sa-d4-01.ceta-ciemat.es <http://sa-d4-01.ceta-ciemat.es>:
>     (192.168.11.15/255.255.255.0 <http://192.168.11.15/255.255.255.0>)
>     npeer 0 nroute 2
>     sa-d4-01.ceta-ciemat.es <http://sa-d4-01.ceta-ciemat.es>:
>     (192.168.11.35/255.255.255.0 <http://192.168.11.35/255.255.255.0>)
>     npeer 0 nroute 0
>     But, the clients only can communicate with the first interface:
>     lctl ping 192.168.11.15
>     12345-0 at lo
>     12345-192.168.11.15 at tcp
>     lctl ping 192.168.11.35
>     failed to ping 192.168.11.35 at tcp: Input/output error
>     Any suggestions how to “enable” the second interface?
>     thank in advance
>
>     /Alfonso Pardo Diaz/
>     /*System Administrator / Researcher*/
>     /c/ Sola nº 1; 10200 TRUJILLO, SPAIN/
>     /Tel: +34 927 65 93 17 <tel:%2B34%20927%2065%2093%2017> Fax: +34 927
>     32 32 37/
>
>     CETA-Ciemat logo <http://www.ceta-ciemat.es/>
>
>     ---------------------------- Confidencialidad: Este mensaje y sus
>     ficheros adjuntos se dirige exclusivamente a su destinatario y puede
>     contener información privilegiada o confidencial. Si no es vd. el
>     destinatario indicado, queda notificado de que la utilización,
>     divulgación y/o copia sin autorización está prohibida en virtud de
>     la legislación vigente. Si ha recibido este mensaje por error, le
>     rogamos que nos lo comunique inmediatamente respondiendo al mensaje
>     y proceda a su destrucción. Disclaimer: This message and its
>     attached files is intended exclusively for its recipients and may
>     contain confidential information. If you received this e-mail in
>     error you are hereby notified that any dissemination, copy or
>     disclosure of this communication is strictly prohibited and may be
>     unlawful. In this case, please notify us by a reply and delete this
>     email and its contents immediately. ----------------------------
>
>     _______________________________________________
>     Lustre-discuss mailing list
>     Lustre-discuss at lists.lustre.org 
> <mailto:Lustre-discuss at lists.lustre.org>
>     http://lists.lustre.org/mailman/listinfo/lustre-discuss
>

-- 
Brian O'Connor
-------------------------------------------------------------
SGI Consulting
Email: briano at sgi.com, Mobile +61 417 746 452
Phone: +61 3 9963 1900, Fax:  +61 3 9963 1902
691 Burke Road, Camberwell, Victoria, 3124
AUSTRALIA
http://www.sgi.com/support/services
------------------------------------------------------------- 