<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
<div>Hi Andreas,</div>
<div>
<blockquote type="cite">the network names need to be unique if the same clients are connecting to both filesystems. </blockquote>
What are complication having two lustre filesystems on the same lnet on the same IB? Does it have performance impact (broadcasts, credits, buffers)?</div>
<div><br>
</div>
<div>We have two (three) lustre fs facing clusters on the same lnet. I'm thinking if I need to change that - we have service window right now.</div>
<div><br>
</div>
<div>Initially I set up separate lnets for each lustre but as we were doing ethernet 'bridge" lnet1(ib)-rtr-eth-rtr-lnet2(ib) to remote cluster between IB networks the routing</div>
<div>get kind of complicated. As a practical matter we were able to move 1PB of data between two lustres plus IO from/to compute cluster in this configuration.</div>
<div><br>
</div>
<div>We have one lnet per IB fabric:</div>
<div><br>
</div>
<div><span class="Apple-tab-span" style="white-space:pre"></span> router(eth) -- tcp11...tcp14 -- (eth)routers - o2ib2 -- cluster2</div>
<div><font face="Menlo"> |</font></div>
<div><font face="Menlo"> <span class="Apple-tab-span" style="white-space:pre"> </span>
+------ lustre1 ----+</font></div>
<div><font face="Menlo"> | |</font></div>
<div><span style="font-family: Menlo;">cluster0 -{o2ib0} </span><span style="font-family: Menlo;">-- lustre2 --{</span><span style="font-family: Menlo;">o2ib1} - cluster1</span></div>
<div><span style="font-family: Menlo;"> | |</span></div>
<div><span style="font-family: Menlo;"> +------ lustre3 ----+</span></div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div>Right now we are merging clusters 1 and 2 and retire lustre1; </div>
<div>It can be good time to reconsider and split lnets like o2ib0 -> (o2ib20,oi2ib30) and o2ib1->(o2ib30,o2ib31).</div>
<div><br>
</div>
<div>What would be a reason for such lnet split?</div>
<div><br>
</div>
<div>Alex.</div>
<br>
<div>
<div>On Jul 13, 2016, at 8:15 PM, Dilger, Andreas <<a href="mailto:andreas.dilger@intel.com">andreas.dilger@intel.com</a>> wrote:</div>
<br class="Apple-interchange-newline">
<blockquote type="cite"><span style="font-family: Helvetica; font-size: 16px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;">It
sounds like you have two different filesystems, each using the same LNet networks "tcp0" and "o2ib0". While "tcp" is a shorthand for network "tcp0", the network names need to be unique if the same clients are connecting to both filesystems. One of the filesystems
will need to regenerate the configuration to use "tcp1" and "o2ib1" (or whatever) to allow the clients to distinguish between the different networks.</span><br style="font-family: Helvetica; font-size: 16px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;">
<br style="font-family: Helvetica; font-size: 16px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;">
<span style="font-family: Helvetica; font-size: 16px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;">Cheers,
Andreas</span></blockquote>
</div>
<br>
</body>
</html>