<HTML><HEAD></HEAD>
<BODY dir=ltr>
<DIV dir=ltr>
<DIV style="FONT-SIZE: 12pt; FONT-FAMILY: 'Calibri'; COLOR: #000000">
<DIV>Yes I have two swtiches, one to the bond0 interface and other switch to the
second bond1 interface.</DIV>
<DIV> </DIV>
<DIV> </DIV>
<DIV
style="FONT-SIZE: small; FONT-FAMILY: 'Calibri'; FONT-WEIGHT: normal; COLOR: #000000; FONT-STYLE: normal; TEXT-DECORATION: none; DISPLAY: inline">
<DIV style="FONT: 10pt tahoma">
<DIV> </DIV>
<DIV style="BACKGROUND: #f5f5f5">
<DIV style="font-color: black"><B>From:</B> <A title=indivar.nair@techterra.in
href="mailto:indivar.nair@techterra.in">Indivar Nair</A> </DIV>
<DIV><B>Sent:</B> Wednesday, June 26, 2013 8:05 PM</DIV>
<DIV><B>To:</B> <A title=alfonso.pardo@ciemat.es
href="mailto:alfonso.pardo@ciemat.es">Alfonso Pardo</A> </DIV>
<DIV><B>Cc:</B> <A title=shuey@purdue.edu href="mailto:shuey@purdue.edu">Michael
Shuey</A> ; <A title=WC-Discuss.Migration@intel.com
href="mailto:WC-Discuss.Migration@intel.com">WC-Discuss</A> ; <A
title=lustre-discuss@lists.lustre.org
href="mailto:lustre-discuss@lists.lustre.org">lustre-discuss</A> </DIV>
<DIV><B>Subject:</B> Re: [Lustre-discuss] Lustre over two TCP
interfaces</DIV></DIV></DIV>
<DIV> </DIV></DIV>
<DIV
style="FONT-SIZE: small; FONT-FAMILY: 'Calibri'; FONT-WEIGHT: normal; COLOR: #000000; FONT-STYLE: normal; TEXT-DECORATION: none; DISPLAY: inline">
<DIV dir=ltr>
<DIV>
<DIV>
<DIV>Hi Alfonso,<BR><BR></DIV>I guess, you have two switches, with 2 interfaces
(bond0) connected to one switch and the other 2 interfaces (bond1) to the second
switch.<BR><BR>---<BR><BR></DIV>What you need to do is merge the switches using
a 'stacking' cable (if the switches are stackable) and create a single trunk
using 2 ports from each switch.<BR>Then create a single bond on the Linux side
using all the 4 Interfaces (and have just 1 IP).<BR><BR></DIV>
<DIV>Use bonding mode balance-rr or 0 without LACP to get load balancing across
all the 4 NICs.<BR></DIV>
<DIV> </DIV>If the switches aren't stackable and a single trunk cannot be
created on the switch side, then use bonding mode balance-alb or 6 on the Linux
side.<BR>
<DIV><BR>No changes need to be done to the cabling in either
case.<BR><BR>---<BR><BR></DIV>
<DIV>This way you get Load Balancing and H/A across NICs.<BR></DIV>
<DIV><BR><BR><BR>Indivar Nair<BR><BR><BR></DIV></DIV>
<DIV class=gmail_extra><BR><BR>
<DIV class=gmail_quote>On Wed, Jun 26, 2013 at 5:47 PM, Michael Shuey <SPAN
dir=ltr><<A href="mailto:shuey@purdue.edu"
target=_blank>shuey@purdue.edu</A>></SPAN> wrote:<BR>
<BLOCKQUOTE class=gmail_quote
style="PADDING-LEFT: 1ex; MARGIN: 0px 0px 0px 0.8ex; BORDER-LEFT: #ccc 1px solid">
<DIV>That will probably be slow - the machine you use to proxy the IPVS
address would be a bottleneck. Out of curiosity, what problem are you
trying to solve here? Do you anticipate whole-subnet outages to be an
issue (and if so, why)?</DIV>
<DIV> </DIV>
<DIV>--<BR>Mike Shuey</DIV>
<DIV class=HOEnZb>
<DIV class=h5><BR><BR>
<DIV class=gmail_quote>On Wed, Jun 26, 2013 at 4:53 AM, Alfonso Pardo <SPAN
dir=ltr><<A href="mailto:alfonso.pardo@ciemat.es"
target=_blank>alfonso.pardo@ciemat.es</A>></SPAN> wrote:<BR>
<BLOCKQUOTE class=gmail_quote
style="PADDING-LEFT: 1ex; MARGIN: 0px 0px 0px 0.8ex; BORDER-LEFT: #ccc 1px solid">oooh!<BR><BR><BR>Thanks
for you reply! May be another way is a floating IP between two interfaces
with IPVS (corosync).<BR><BR>-----Mensaje original----- From: Brian
O'Connor<BR>Sent: Wednesday, June 26, 2013 10:15 AM<BR>To: Alfonso
Pardo<BR>Cc: 'Michael Shuey' ; 'WC-Discuss' ; <A
href="mailto:lustre-discuss@lists.lustre.org"
target=_blank>lustre-discuss@lists.lustre.<U></U>org</A><BR>Subject: Re:
[Lustre-discuss] Lustre over two TCP interfaces
<DIV>
<DIV><BR><BR><BR><BR><BR>On 06/26/2013 04:16 PM, Alfonso Pardo wrote:<BR>
<BLOCKQUOTE class=gmail_quote
style="PADDING-LEFT: 1ex; MARGIN: 0px 0px 0px 0.8ex; BORDER-LEFT: #ccc 1px solid">But....
if I configure the OST assigning to the first interface of the<BR>OSS
(bond0) and as failover OSS the second inteface of the OSS. If
the<BR>bond0 network down, the client will try to connect to the failover,
that<BR>is the second interface of the OSS.<BR>is it
possible?<BR></BLOCKQUOTE><BR><BR>I stand to be corrected, but no, I don't
think so. As I understand it<BR>the failover code looks for a different
server instance, rather than a<BR>different nid.<BR><BR>See<BR><BR><A
href="http://lists.opensfs.org/pipermail/lustre-devel-opensfs.org/2012-August/000028.html"
target=_blank>http://lists.opensfs.org/<U></U>pipermail/lustre-devel-<U></U>opensfs.org/2012-August/<U></U>000028.html</A><BR><BR><BR>
<BLOCKQUOTE class=gmail_quote
style="PADDING-LEFT: 1ex; MARGIN: 0px 0px 0px 0.8ex; BORDER-LEFT: #ccc 1px solid">*From:*
Brian O'Connor <mailto:<A href="mailto:briano@sgi.com"
target=_blank>briano@sgi.com</A>><BR>*Sent:* Wednesday, June 26, 2013
1:09 AM<BR>*To:* 'Alfonso Pardo' <mailto:<A
href="mailto:alfonso.pardo@ciemat.es"
target=_blank>alfonso.pardo@ciemat.<U></U>es</A>> ; 'Michael
Shuey'<BR><mailto:<A href="mailto:shuey@purdue.edu"
target=_blank>shuey@purdue.edu</A>><BR>*Cc:* 'WC-Discuss' <mailto:<A
href="mailto:WC-Discuss.Migration@intel.com"
target=_blank>WC-Discuss.Migration@<U></U>intel.com</A>> ;<BR>mailto:<A
href="mailto:lustre-discuss@lists.lustre.org"
target=_blank>lustre-discuss@lists.<U></U>lustre.org</A><BR>*Subject:* RE:
[Lustre-discuss] Lustre over two TCP interfaces<BR>Unless something has
changed in the new versions of lustre, I don't<BR>think lustre can do
failover between nids on the same machine.<BR><BR>It can choose the
available nid at mount time, but if an active nid goes<BR>away after you
are mounted then the client chooses the failover nid, and<BR>this must be
on a different server.<BR><BR>Check the archives for more discussion in
this topic :)<BR><BR><BR><BR>-----Original Message-----<BR>*From: *Alfonso
Pardo [<A href="mailto:alfonso.pardo@ciemat.es"
target=_blank>alfonso.pardo@ciemat.es</A><BR><mailto:<A
href="mailto:alfonso.pardo@ciemat.es"
target=_blank>alfonso.pardo@ciemat.<U></U>es</A>>]<BR>*Sent: *Tuesday,
June 25, 2013 07:23 AM Central Standard Time<BR>*To: *Michael
Shuey<BR>*Cc: *WC-Discuss; <A
href="mailto:lustre-discuss@lists.lustre.org"
target=_blank>lustre-discuss@lists.lustre.<U></U>org</A><BR>*Subject: *Re:
[Lustre-discuss] Lustre over two TCP interfaces<BR><BR>thank
Michael,<BR>This is my second step, I will change the lnet with “options
lnet<BR>networks=tcp0(bond0,bond1)” because my machines has 4 nics. I have
a<BR>bond0 and bond1 with LACP. I need to comunicate the clients with
two<BR>network for HA network.<BR>If the bond0 network is down, the
clients can reach the OSS by the<BR>second network bond1.<BR>If I change
the modprobe with “options
lnet<BR>networks=tcp0(bond0),tcp1(<U></U>bond1)”, how the clients mount
the filesystem<BR>to reach the OSS by two network?<BR>*From:* Michael
Shuey <mailto:<A href="mailto:shuey@purdue.edu"
target=_blank>shuey@purdue.edu</A>><BR>*Sent:* Tuesday, June 25, 2013
2:14 PM<BR>*To:* Alfonso Pardo <mailto:<A
href="mailto:alfonso.pardo@ciemat.es"
target=_blank>alfonso.pardo@ciemat.<U></U>es</A>><BR>*Cc:* <A
href="mailto:lustre-discuss@lists.lustre.org"
target=_blank>lustre-discuss@lists.lustre.<U></U>org</A><BR><mailto:<A
href="mailto:lustre-discuss@lists.lustre.org"
target=_blank>lustre-discuss@lists.<U></U>lustre.org</A>> ;
WC-Discuss<BR><mailto:<A href="mailto:WC-Discuss.Migration@intel.com"
target=_blank>WC-Discuss.Migration@<U></U>intel.com</A>><BR>*Subject:*
Re: [Lustre-discuss] Lustre over two TCP interfaces<BR>Different
interfaces need to be declared with different LNET networks -<BR>something
like "networks=tcp0(eth0),tcp1(<U></U>eth1)". Of course,
that<BR>assumes your clients are configured to use a mix of tcp0 and tcp1
for<BR>connections (with each client only using one of the two).
This is<BR>really only useful in corner cases, when you're doing something
strange;<BR>if eth0 and eth1 are in the same subnet (as in your example),
this is<BR>almost certainly not productive.<BR>A better bet might be to
use a single LNET, and bond the two interfaces<BR>together - either as an
active/passive pair, or active/active (e.g.,<BR>LACP). Then you'd
declare networks=tcp0(bond0), give the bond a single<BR>IP address, and
client traffic would be split across the two members in<BR>the bond more
like you probably expect (given the limits of the bond<BR>protocol you're
using).<BR>--<BR>Mike Shuey<BR><BR><BR>On Tue, Jun 25, 2013 at 8:06 AM,
Alfonso Pardo <<A href="mailto:alfonso.pardo@ciemat.es"
target=_blank>alfonso.pardo@ciemat.es</A><BR><mailto:<A
href="mailto:alfonso.pardo@ciemat.es"
target=_blank>alfonso.pardo@ciemat.<U></U>es</A>>>
wrote:<BR><BR> hello friends,<BR> I
need to comunicate my OSS by two ethernet TCP interfaces: eth0
and<BR> eth1.<BR> I have configured
this feature in my modprobe.d with:<BR> “options lnet
networks=tcp0(eth0,eth1)”<BR> And I can see two
interfaces with:<BR> lctl --net tcp
interface_list<BR> <A
href="http://sa-d4-01.ceta-ciemat.es"
target=_blank>sa-d4-01.ceta-ciemat.es</A> <<A
href="http://sa-d4-01.ceta-ciemat.es"
target=_blank>http://sa-d4-01.ceta-ciemat.<U></U>es</A>>:<BR>
(<A href="http://192.168.11.15/255.255.255.0"
target=_blank>192.168.11.15/255.255.255.0</A> <<A
href="http://192.168.11.15/255.255.255.0"
target=_blank>http://192.168.11.15/255.255.<U></U>255.0</A>>)<BR>
npeer 0 nroute 2<BR> <A
href="http://sa-d4-01.ceta-ciemat.es"
target=_blank>sa-d4-01.ceta-ciemat.es</A> <<A
href="http://sa-d4-01.ceta-ciemat.es"
target=_blank>http://sa-d4-01.ceta-ciemat.<U></U>es</A>>:<BR>
(<A href="http://192.168.11.35/255.255.255.0"
target=_blank>192.168.11.35/255.255.255.0</A> <<A
href="http://192.168.11.35/255.255.255.0"
target=_blank>http://192.168.11.35/255.255.<U></U>255.0</A>>)<BR>
npeer 0 nroute 0<BR> But, the clients only can
communicate with the first interface:<BR> lctl ping
192.168.11.15<BR> 12345-0@lo<BR>
12345-192.168.11.15@tcp<BR> lctl ping
192.168.11.35<BR> failed to ping 192.168.11.35@tcp:
Input/output error<BR> Any suggestions how to “enable”
the second interface?<BR> thank in
advance<BR><BR> /Alfonso Pardo
Diaz/<BR> /*System Administrator /
Researcher*/<BR> /c/ Sola nº 1; 10200 TRUJILLO,
SPAIN/<BR> /Tel: <A
href="tel:%2B34%20927%2065%2093%2017" target=_blank
value="+34927659317">+34 927 65 93 17</A>
<tel:%2B34%20927%2065%2093%<U></U>2017> Fax: +34
927<BR> 32 32 37/<BR><BR> CETA-Ciemat
logo <<A href="http://www.ceta-ciemat.es/"
target=_blank>http://www.ceta-ciemat.es/</A>><BR><BR>
---------------------------- Confidencialidad: Este mensaje y
sus<BR> ficheros adjuntos se dirige exclusivamente a su
destinatario y puede<BR> contener información
privilegiada o confidencial. Si no es vd. el<BR>
destinatario indicado, queda notificado de que la
utilización,<BR> divulgación y/o copia sin autorización
está prohibida en virtud de<BR> la legislación vigente.
Si ha recibido este mensaje por error, le<BR> rogamos
que nos lo comunique inmediatamente respondiendo al
mensaje<BR> y proceda a su destrucción. Disclaimer: This
message and its<BR> attached files is intended
exclusively for its recipients and may<BR> contain
confidential information. If you received this e-mail
in<BR> error you are hereby notified that any
dissemination, copy or<BR> disclosure of this
communication is strictly prohibited and may be<BR>
unlawful. In this case, please notify us by a reply and delete
this<BR> email and its contents immediately.
----------------------------<BR><BR>
______________________________<U></U>_________________<BR>
Lustre-discuss mailing list<BR> <A
href="mailto:Lustre-discuss@lists.lustre.org"
target=_blank>Lustre-discuss@lists.lustre.<U></U>org</A> <mailto:<A
href="mailto:Lustre-discuss@lists.lustre.org"
target=_blank>Lustre-discuss@lists.<U></U>lustre.org</A>><BR>
<A href="http://lists.lustre.org/mailman/listinfo/lustre-discuss"
target=_blank>http://lists.lustre.org/<U></U>mailman/listinfo/lustre-<U></U>discuss</A><BR><BR></BLOCKQUOTE><BR><BR>--
<BR>Brian
O'Connor<BR>------------------------------<U></U>------------------------------<U></U>-<BR>SGI
Consulting<BR>Email: <A href="mailto:briano@sgi.com"
target=_blank>briano@sgi.com</A>, Mobile <A
href="tel:%2B61%20417%20746%20452" target=_blank value="+61417746452">+61
417 746 452</A><BR>Phone: <A href="tel:%2B61%203%209963%201900"
target=_blank value="+61399631900">+61 3 9963 1900</A>, Fax: <A
href="tel:%2B61%203%209963%201902" target=_blank value="+61399631902">+61 3
9963 1902</A><BR>691 Burke Road, Camberwell, Victoria,
3124<BR>AUSTRALIA<BR><A href="http://www.sgi.com/support/services"
target=_blank>http://www.sgi.com/support/<U></U>services</A><BR>------------------------------<U></U>------------------------------<U></U>-
<BR></DIV></DIV></BLOCKQUOTE></DIV>
<DIV> </DIV></DIV></DIV><BR>_______________________________________________<BR>Lustre-discuss
mailing list<BR><A
href="mailto:Lustre-discuss@lists.lustre.org">Lustre-discuss@lists.lustre.org</A><BR><A
href="http://lists.lustre.org/mailman/listinfo/lustre-discuss"
target=_blank>http://lists.lustre.org/mailman/listinfo/lustre-discuss</A><BR><BR></BLOCKQUOTE></DIV>
<DIV> </DIV></DIV></DIV></DIV></DIV></BODY></HTML>