<div dir="ltr"><div><div><div>Then follow the instructions in my earlier mail.<br></div><div>No need to have bond0 and bond1.<br></div>You will achieve high-availability even with one bonded interface.<br><br></div>Cheers,<br>
<br><br></div>Indivar Nair<br></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Thu, Jun 27, 2013 at 11:55 AM, Alfonso Pardo <span dir="ltr"><<a href="mailto:alfonso.pardo@ciemat.es" target="_blank">alfonso.pardo@ciemat.es</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">
<div dir="ltr">
<div style="font-size:12pt;font-family:'Calibri'">
<div>Yes I have two swtiches, one to the bond0 interface and other switch to the
second bond1 interface.</div>
<div> </div>
<div> </div>
<div style="font-style:normal;font-size:small;display:inline;text-decoration:none;font-family:'Calibri';font-weight:normal">
<div style="FONT:10pt tahoma">
<div> </div>
<div style="BACKGROUND:#f5f5f5">
<div><b>From:</b> <a title="indivar.nair@techterra.in" href="mailto:indivar.nair@techterra.in" target="_blank">Indivar Nair</a> </div>
<div><b>Sent:</b> Wednesday, June 26, 2013 8:05 PM</div>
<div><b>To:</b> <a title="alfonso.pardo@ciemat.es" href="mailto:alfonso.pardo@ciemat.es" target="_blank">Alfonso Pardo</a> </div>
<div><b>Cc:</b> <a title="shuey@purdue.edu" href="mailto:shuey@purdue.edu" target="_blank">Michael
Shuey</a> ; <a title="WC-Discuss.Migration@intel.com" href="mailto:WC-Discuss.Migration@intel.com" target="_blank">WC-Discuss</a> ; <a title="lustre-discuss@lists.lustre.org" href="mailto:lustre-discuss@lists.lustre.org" target="_blank">lustre-discuss</a> </div>
<div><div class="h5">
<div><b>Subject:</b> Re: [Lustre-discuss] Lustre over two TCP
interfaces</div></div></div></div></div>
<div> </div></div><div><div class="h5">
<div style="font-style:normal;font-size:small;display:inline;text-decoration:none;font-family:'Calibri';font-weight:normal">
<div dir="ltr">
<div>
<div>
<div>Hi Alfonso,<br><br></div>I guess, you have two switches, with 2 interfaces
(bond0) connected to one switch and the other 2 interfaces (bond1) to the second
switch.<br><br>---<br><br></div>What you need to do is merge the switches using
a 'stacking' cable (if the switches are stackable) and create a single trunk
using 2 ports from each switch.<br>Then create a single bond on the Linux side
using all the 4 Interfaces (and have just 1 IP).<br><br></div>
<div>Use bonding mode balance-rr or 0 without LACP to get load balancing across
all the 4 NICs.<br></div>
<div> </div>If the switches aren't stackable and a single trunk cannot be
created on the switch side, then use bonding mode balance-alb or 6 on the Linux
side.<br>
<div><br>No changes need to be done to the cabling in either
case.<br><br>---<br><br></div>
<div>This way you get Load Balancing and H/A across NICs.<br></div>
<div><br><br><br>Indivar Nair<br><br><br></div></div>
<div class="gmail_extra"><br><br>
<div class="gmail_quote">On Wed, Jun 26, 2013 at 5:47 PM, Michael Shuey <span dir="ltr"><<a href="mailto:shuey@purdue.edu" target="_blank">shuey@purdue.edu</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="PADDING-LEFT:1ex;MARGIN:0px 0px 0px 0.8ex;BORDER-LEFT:#ccc 1px solid">
<div>That will probably be slow - the machine you use to proxy the IPVS
address would be a bottleneck. Out of curiosity, what problem are you
trying to solve here? Do you anticipate whole-subnet outages to be an
issue (and if so, why)?</div>
<div> </div>
<div>--<br>Mike Shuey</div>
<div>
<div><br><br>
<div class="gmail_quote">On Wed, Jun 26, 2013 at 4:53 AM, Alfonso Pardo <span dir="ltr"><<a href="mailto:alfonso.pardo@ciemat.es" target="_blank">alfonso.pardo@ciemat.es</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="PADDING-LEFT:1ex;MARGIN:0px 0px 0px 0.8ex;BORDER-LEFT:#ccc 1px solid">oooh!<br><br><br>Thanks
for you reply! May be another way is a floating IP between two interfaces
with IPVS (corosync).<br><br>-----Mensaje original----- From: Brian
O'Connor<br>Sent: Wednesday, June 26, 2013 10:15 AM<br>To: Alfonso
Pardo<br>Cc: 'Michael Shuey' ; 'WC-Discuss' ; <a href="mailto:lustre-discuss@lists.lustre.org" target="_blank">lustre-discuss@lists.lustre.<u></u>org</a><br>Subject: Re:
[Lustre-discuss] Lustre over two TCP interfaces
<div>
<div><br><br><br><br><br>On 06/26/2013 04:16 PM, Alfonso Pardo wrote:<br>
<blockquote class="gmail_quote" style="PADDING-LEFT:1ex;MARGIN:0px 0px 0px 0.8ex;BORDER-LEFT:#ccc 1px solid">But....
if I configure the OST assigning to the first interface of the<br>OSS
(bond0) and as failover OSS the second inteface of the OSS. If
the<br>bond0 network down, the client will try to connect to the failover,
that<br>is the second interface of the OSS.<br>is it
possible?<br></blockquote><br><br>I stand to be corrected, but no, I don't
think so. As I understand it<br>the failover code looks for a different
server instance, rather than a<br>different nid.<br><br>See<br><br><a href="http://lists.opensfs.org/pipermail/lustre-devel-opensfs.org/2012-August/000028.html" target="_blank">http://lists.opensfs.org/<u></u>pipermail/lustre-devel-<u></u>opensfs.org/2012-August/<u></u>000028.html</a><br>
<br><br>
<blockquote class="gmail_quote" style="PADDING-LEFT:1ex;MARGIN:0px 0px 0px 0.8ex;BORDER-LEFT:#ccc 1px solid">*From:*
Brian O'Connor <mailto:<a href="mailto:briano@sgi.com" target="_blank">briano@sgi.com</a>><br>*Sent:* Wednesday, June 26, 2013
1:09 AM<br>*To:* 'Alfonso Pardo' <mailto:<a href="mailto:alfonso.pardo@ciemat.es" target="_blank">alfonso.pardo@ciemat.<u></u>es</a>> ; 'Michael
Shuey'<br><mailto:<a href="mailto:shuey@purdue.edu" target="_blank">shuey@purdue.edu</a>><br>*Cc:* 'WC-Discuss' <mailto:<a href="mailto:WC-Discuss.Migration@intel.com" target="_blank">WC-Discuss.Migration@<u></u>intel.com</a>> ;<br>
mailto:<a href="mailto:lustre-discuss@lists.lustre.org" target="_blank">lustre-discuss@lists.<u></u>lustre.org</a><br>*Subject:* RE:
[Lustre-discuss] Lustre over two TCP interfaces<br>Unless something has
changed in the new versions of lustre, I don't<br>think lustre can do
failover between nids on the same machine.<br><br>It can choose the
available nid at mount time, but if an active nid goes<br>away after you
are mounted then the client chooses the failover nid, and<br>this must be
on a different server.<br><br>Check the archives for more discussion in
this topic :)<br><br><br><br>-----Original Message-----<br>*From: *Alfonso
Pardo [<a href="mailto:alfonso.pardo@ciemat.es" target="_blank">alfonso.pardo@ciemat.es</a><br><mailto:<a href="mailto:alfonso.pardo@ciemat.es" target="_blank">alfonso.pardo@ciemat.<u></u>es</a>>]<br>*Sent: *Tuesday,
June 25, 2013 07:23 AM Central Standard Time<br>*To: *Michael
Shuey<br>*Cc: *WC-Discuss; <a href="mailto:lustre-discuss@lists.lustre.org" target="_blank">lustre-discuss@lists.lustre.<u></u>org</a><br>*Subject: *Re:
[Lustre-discuss] Lustre over two TCP interfaces<br><br>thank
Michael,<br>This is my second step, I will change the lnet with “options
lnet<br>networks=tcp0(bond0,bond1)” because my machines has 4 nics. I have
a<br>bond0 and bond1 with LACP. I need to comunicate the clients with
two<br>network for HA network.<br>If the bond0 network is down, the
clients can reach the OSS by the<br>second network bond1.<br>If I change
the modprobe with “options
lnet<br>networks=tcp0(bond0),tcp1(<u></u>bond1)”, how the clients mount
the filesystem<br>to reach the OSS by two network?<br>*From:* Michael
Shuey <mailto:<a href="mailto:shuey@purdue.edu" target="_blank">shuey@purdue.edu</a>><br>*Sent:* Tuesday, June 25, 2013
2:14 PM<br>*To:* Alfonso Pardo <mailto:<a href="mailto:alfonso.pardo@ciemat.es" target="_blank">alfonso.pardo@ciemat.<u></u>es</a>><br>*Cc:* <a href="mailto:lustre-discuss@lists.lustre.org" target="_blank">lustre-discuss@lists.lustre.<u></u>org</a><br>
<mailto:<a href="mailto:lustre-discuss@lists.lustre.org" target="_blank">lustre-discuss@lists.<u></u>lustre.org</a>> ;
WC-Discuss<br><mailto:<a href="mailto:WC-Discuss.Migration@intel.com" target="_blank">WC-Discuss.Migration@<u></u>intel.com</a>><br>*Subject:*
Re: [Lustre-discuss] Lustre over two TCP interfaces<br>Different
interfaces need to be declared with different LNET networks -<br>something
like "networks=tcp0(eth0),tcp1(<u></u>eth1)". Of course,
that<br>assumes your clients are configured to use a mix of tcp0 and tcp1
for<br>connections (with each client only using one of the two).
This is<br>really only useful in corner cases, when you're doing something
strange;<br>if eth0 and eth1 are in the same subnet (as in your example),
this is<br>almost certainly not productive.<br>A better bet might be to
use a single LNET, and bond the two interfaces<br>together - either as an
active/passive pair, or active/active (e.g.,<br>LACP). Then you'd
declare networks=tcp0(bond0), give the bond a single<br>IP address, and
client traffic would be split across the two members in<br>the bond more
like you probably expect (given the limits of the bond<br>protocol you're
using).<br>--<br>Mike Shuey<br><br><br>On Tue, Jun 25, 2013 at 8:06 AM,
Alfonso Pardo <<a href="mailto:alfonso.pardo@ciemat.es" target="_blank">alfonso.pardo@ciemat.es</a><br><mailto:<a href="mailto:alfonso.pardo@ciemat.es" target="_blank">alfonso.pardo@ciemat.<u></u>es</a>>>
wrote:<br><br> hello friends,<br> I
need to comunicate my OSS by two ethernet TCP interfaces: eth0
and<br> eth1.<br> I have configured
this feature in my modprobe.d with:<br> “options lnet
networks=tcp0(eth0,eth1)”<br> And I can see two
interfaces with:<br> lctl --net tcp
interface_list<br> <a href="http://sa-d4-01.ceta-ciemat.es" target="_blank">sa-d4-01.ceta-ciemat.es</a> <<a href="http://sa-d4-01.ceta-ciemat.es" target="_blank">http://sa-d4-01.ceta-ciemat.<u></u>es</a>>:<br>
(<a href="http://192.168.11.15/255.255.255.0" target="_blank">192.168.11.15/255.255.255.0</a> <<a href="http://192.168.11.15/255.255.255.0" target="_blank">http://192.168.11.15/255.255.<u></u>255.0</a>>)<br>
npeer 0 nroute 2<br> <a href="http://sa-d4-01.ceta-ciemat.es" target="_blank">sa-d4-01.ceta-ciemat.es</a> <<a href="http://sa-d4-01.ceta-ciemat.es" target="_blank">http://sa-d4-01.ceta-ciemat.<u></u>es</a>>:<br>
(<a href="http://192.168.11.35/255.255.255.0" target="_blank">192.168.11.35/255.255.255.0</a> <<a href="http://192.168.11.35/255.255.255.0" target="_blank">http://192.168.11.35/255.255.<u></u>255.0</a>>)<br>
npeer 0 nroute 0<br> But, the clients only can
communicate with the first interface:<br> lctl ping
192.168.11.15<br> 12345-0@lo<br>
12345-192.168.11.15@tcp<br> lctl ping
192.168.11.35<br> failed to ping 192.168.11.35@tcp:
Input/output error<br> Any suggestions how to “enable”
the second interface?<br> thank in
advance<br><br> /Alfonso Pardo
Diaz/<br> /*System Administrator /
Researcher*/<br> /c/ Sola nº 1; 10200 TRUJILLO,
SPAIN/<br> /Tel: <a href="tel:%2B34%20927%2065%2093%2017" value="+34927659317" target="_blank">+34 927 65 93 17</a>
<tel:%2B34%20927%2065%2093%<u></u>2017> Fax: +34
927<br> 32 32 37/<br><br> CETA-Ciemat
logo <<a href="http://www.ceta-ciemat.es/" target="_blank">http://www.ceta-ciemat.es/</a>><br><br>
---------------------------- Confidencialidad: Este mensaje y
sus<br> ficheros adjuntos se dirige exclusivamente a su
destinatario y puede<br> contener información
privilegiada o confidencial. Si no es vd. el<br>
destinatario indicado, queda notificado de que la
utilización,<br> divulgación y/o copia sin autorización
está prohibida en virtud de<br> la legislación vigente.
Si ha recibido este mensaje por error, le<br> rogamos
que nos lo comunique inmediatamente respondiendo al
mensaje<br> y proceda a su destrucción. Disclaimer: This
message and its<br> attached files is intended
exclusively for its recipients and may<br> contain
confidential information. If you received this e-mail
in<br> error you are hereby notified that any
dissemination, copy or<br> disclosure of this
communication is strictly prohibited and may be<br>
unlawful. In this case, please notify us by a reply and delete
this<br> email and its contents immediately.
----------------------------<br><br>
______________________________<u></u>_________________<br>
Lustre-discuss mailing list<br> <a href="mailto:Lustre-discuss@lists.lustre.org" target="_blank">Lustre-discuss@lists.lustre.<u></u>org</a> <mailto:<a href="mailto:Lustre-discuss@lists.lustre.org" target="_blank">Lustre-discuss@lists.<u></u>lustre.org</a>><br>
<a href="http://lists.lustre.org/mailman/listinfo/lustre-discuss" target="_blank">http://lists.lustre.org/<u></u>mailman/listinfo/lustre-<u></u>discuss</a><br><br></blockquote><br><br>--
<br>Brian
O'Connor<br>------------------------------<u></u>------------------------------<u></u>-<br>SGI
Consulting<br>Email: <a href="mailto:briano@sgi.com" target="_blank">briano@sgi.com</a>, Mobile <a href="tel:%2B61%20417%20746%20452" value="+61417746452" target="_blank">+61
417 746 452</a><br>Phone: <a href="tel:%2B61%203%209963%201900" value="+61399631900" target="_blank">+61 3 9963 1900</a>, Fax: <a href="tel:%2B61%203%209963%201902" value="+61399631902" target="_blank">+61 3
9963 1902</a><br>691 Burke Road, Camberwell, Victoria,
3124<br>AUSTRALIA<br><a href="http://www.sgi.com/support/services" target="_blank">http://www.sgi.com/support/<u></u>services</a><br>------------------------------<u></u>------------------------------<u></u>-
<br></div></div></blockquote></div>
<div> </div></div></div><br>_______________________________________________<br>Lustre-discuss
mailing list<br><a href="mailto:Lustre-discuss@lists.lustre.org" target="_blank">Lustre-discuss@lists.lustre.org</a><br><a href="http://lists.lustre.org/mailman/listinfo/lustre-discuss" target="_blank">http://lists.lustre.org/mailman/listinfo/lustre-discuss</a><br>
<br></blockquote></div>
<div> </div></div></div></div></div></div></div></div>
</blockquote></div><br></div>