<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
the lnet modules load, but when I start the lnet service it says that the network is down. I backed everything out, removed the file, and then started the lnet service again and it worked properly.</div>
<div>
<div id="appendonsend"></div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<hr tabindex="-1" style="display:inline-block; width:98%">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>From:</b> Chris Horn <hornc@cray.com><br>
<b>Sent:</b> Wednesday, October 2, 2019 2:48 PM<br>
<b>To:</b> Kurt Strosahl <strosahl@jlab.org>; lustre-discuss@lists.lustre.org <lustre-discuss@lists.lustre.org><br>
<b>Subject:</b> [EXTERNAL] Re: [lustre-discuss] Lustre rpm install creating a file that breaks lustre</font>
<div> </div>
</div>
<div lang="EN-US">
<div class="x_WordSection1">
<p class="x_MsoNormal" style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;">
Might be best to open a ticket for this. What was the nature of the failure? </p>
<p class="x_MsoNormal" style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;">
</p>
<p class="x_MsoNormal" style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;">
Chris Horn</p>
<p class="x_MsoNormal" style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;">
</p>
<div style="border:none; border-top:solid #B5C4DF 1.0pt; padding:3.0pt 0in 0in 0in">
<p class="x_MsoNormal" style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;">
<b><span style="font-size:12.0pt; color:black">From: </span></b><span style="font-size:12.0pt; color:black">lustre-discuss <lustre-discuss-bounces@lists.lustre.org> on behalf of Kurt Strosahl <strosahl@jlab.org><br>
<b>Date: </b>Wednesday, October 2, 2019 at 1:30 PM<br>
<b>To: </b>"lustre-discuss@lists.lustre.org" <lustre-discuss@lists.lustre.org><br>
<b>Subject: </b>[lustre-discuss] Lustre rpm install creating a file that breaks lustre</span></p>
</div>
<div>
<p class="x_MsoNormal" style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;">
</p>
</div>
<div>
<p class="x_MsoNormal" style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;">
<span style="font-size:12.0pt; color:black">Good Afternoon,</span></p>
</div>
<div>
<p class="x_MsoNormal" style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;">
<span style="font-size:12.0pt; color:black"> </span></p>
</div>
<div>
<p class="x_MsoNormal" style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;">
<span style="font-size:12.0pt; color:black"> While getting lustre 2.10.8 running on a RHEL 7.7 system I found that the RPM install was putting a file in /etc/modprobe.d that was preventing lnet from starting properly.</span></p>
</div>
<div>
<p class="x_MsoNormal" style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;">
<span style="font-size:12.0pt; color:black"> </span></p>
</div>
<div>
<p class="x_MsoNormal" style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;">
<span style="font-size:12.0pt; color:black">the file is ko2iblnd.conf, which contains the following...</span></p>
</div>
<div>
<p class="x_MsoNormal" style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;">
<span style="font-size:12.0pt; color:black"> </span></p>
</div>
<div>
<p class="x_MsoNormal" style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;">
<span style="font-size:12.0pt; color:black">alias ko2iblnd-opa ko2iblnd</span></p>
<div>
<p class="x_MsoNormal" style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;">
<span style="font-size:12.0pt; color:black">options ko2iblnd-opa peer_credits=128 peer_credits_hiw=64 credits=1024 concurrent_sends=256 ntx=2048 map_on_demand=32 fmr_pool_size=2048 fmr_flush_trigger=512 fmr_cache=1 conns_per_peer=4</span></p>
</div>
<div>
<p class="x_MsoNormal" style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;">
<span style="font-size:12.0pt; color:black"> </span></p>
</div>
<div>
<p class="x_MsoNormal" style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;">
<span style="font-size:12.0pt; color:black">install ko2iblnd /usr/sbin/ko2iblnd-probe</span></p>
</div>
<p class="x_MsoNormal" style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;">
<span style="font-size:12.0pt; color:black"> </span></p>
</div>
<div>
<p class="x_MsoNormal" style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;">
<span style="font-size:12.0pt; color:black">Our system is running infiniband, not omnipath. So I'm mot sure why this file is being put in place. Removing the file allows lnet to start properly.</span></p>
</div>
<div>
<p class="x_MsoNormal" style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;">
<span style="font-size:12.0pt; color:black"> </span></p>
</div>
<div>
<p class="x_MsoNormal" style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;">
<span style="font-size:12.0pt; color:black">w/r,</span></p>
</div>
<div id="x_Signature">
<div id="x_divtagdefaultwrapper">
<p><span style="font-size:10.5pt; font-family:"Courier New"; color:#333333">Kurt J. Strosahl<br>
System Administrator: Lustre, HPC<br>
Scientific Computing Group, Thomas Jefferson National Accelerator Facility</span><span style="font-size:12.0pt; color:black"></span></p>
</div>
</div>
</div>
</div>
</div>
</body>
</html>