<div dir="ltr"><div class="gmail_default" style="font-size:small">At the moment, the Lustre servers only have a low-bandwidth (1G) management network that would be insufficient for running Lustre. The IB gateway theoretically gives me 40G Ethernet connections to my clients which also have 40G Ethernet ports. I could potentially set up a Lustre router, but would like to use the hardware I already have in place, if possible. <br><br></div><div class="gmail_default" style="font-size:small">Thanks,<br></div><div class="gmail_default" style="font-size:small">Kevin<br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Aug 1, 2016 at 7:58 AM, Stolarek, Marcin <span dir="ltr"><<a href="mailto:marcin.stolarek@delphi.com" target="_blank">marcin.stolarek@delphi.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div link="blue" vlink="purple" lang="EN-US">
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Personally I’d suggest setting up lnet router between tcp and lustre network instead of using mellanox ib 2 ethernet gateway. It may be even easier if you have
ethernet on lustre oss/mds servers but you haven’t configured lnet for tcp?<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">As far as I know this gateway works as arp proxy, which can lead you to really interesting and uncommon scenarios. This will increase the time needed to find
the solution when incident happens </span><span style="font-size:11.0pt;font-family:Wingdings;color:#1f497d">J</span><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Cheers,<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Marcin <u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> lustre-discuss [mailto:<a href="mailto:lustre-discuss-bounces@lists.lustre.org" target="_blank">lustre-discuss-bounces@lists.lustre.org</a>]
<b>On Behalf Of </b>Kevin M. Hildebrand<br>
<b>Sent:</b> Monday, August 01, 2016 1:06 PM<br>
<b>To:</b> <a href="mailto:lustre-discuss@lists.lustre.org" target="_blank">lustre-discuss@lists.lustre.org</a><br>
<b>Subject:</b> EXT SENDER-[lustre-discuss] Mounting Lustre over IB-to-Ethernet gateway<u></u><u></u></span></p>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt">Our Lustre filesystem is currently set up to use the o2ib interface only- all of the servers have<br>
options lnet networks=o2ib0(ib0)<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt">We've just added a Mellanox IB-to-Ethernet gateway and would like to be able to have clients on the Ethernet side also mount Lustre. The gateway extends the same layer-2 IP range that's being used for IPoIB
out to the Ethernet clients<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">How should I go about doing this? Since the clients don't have IB, it doesn't appear that I can use o2ib0 to mount. Do I need to add another lnet network on the servers? Something like<br>
options lnet networks=o2ib0(ib0),tcp0(ib0)? Can I have both protocols on the same interface?<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">And if I do have to add another lnet network, is there any way to do so without restarting the servers?<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Thanks,<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Kevin<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><br>
--<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Kevin Hildebrand<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">University of Maryland, College Park<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Division of IT<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</div>
</div>
**************************************************************************************** Note: If the reader of this message is not the intended recipient, or an employee or agent responsible for delivering this message to the intended recipient, you are hereby
notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by replying to the message and deleting it from your computer. Thank you.
****************************************************************************************
</div>
</blockquote></div><br></div>