<html><head><meta http-equiv="content-type" content="text/html; charset=utf-8"></head><body dir="auto">There's another thing that I had wanted to submit a bug report for but have gotten to it, yet… <br><br>lnetctl exports and imports in lexical order (or Python dictionary order to be exact), but some of the settings would fail if imported not according to specific order, eg: network needs to be setup before a router can be added, but lnetctl import would try setting up a router before before the network is up and fails or something similar.<div><br></div><div>I never got around after working around with a manual lnet.conf since it is pretty static…</div><div><br></div><div>Cheers,</div><div>Angelos<br>(Sent from mobile, please pardon me for typos and cursoriness.)<br><div dir="ltr"><br><blockquote type="cite">24/2/2021 19:43、Laura Hild via lustre-discuss <lustre-discuss@lists.lustre.org>のメール:<br><br></blockquote></div><blockquote type="cite"><div dir="ltr">
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
> Or you can manually build lnet.conf as lnetctl seems to have occasion<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
> problems with some of the fields exported by "lnetctl export --backup"</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
I've noticed, in particular,</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
LNetError: 122666:0:(peer.c:372:lnet_peer_ni_del_locked())<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Peer NI x.x.x.x@tcp is a gateway. Can not delete it</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
and</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
errno: -2<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
descr: "cannot add peer ni: No such file or directory"<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
not having removed the <code>peer:</code> section.</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
-Laura</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<hr tabindex="-1" style="display:inline-block; width:98%">
<div id="divRplyFwdMsg" dir="ltr"><font style="font-size:11pt" face="Calibri, sans-serif" color="#000000"><b>Od:</b> lustre-discuss <lustre-discuss-bounces@lists.lustre.org> v imenu Angelos Ching via lustre-discuss <lustre-discuss@lists.lustre.org><br>
<b>Poslano:</b> torek, 23. februar 2021 05:06<br>
<b>Za:</b> lustre-discuss@lists.lustre.org <lustre-discuss@lists.lustre.org><br>
<b>Zadeva:</b> [EXTERNAL] Re: [lustre-discuss] need to always manually add network after reboot</font>
<div> </div>
</div>
<div>
<p style="margin-top: 0px; margin-bottom: 0px;">Hi Sid,</p>
<p style="margin-top: 0px; margin-bottom: 0px;">Notice that you are using lnetctl net add to add the lnet network, which means you should be using a recent version of Lustre that depends on /etc/lnet.conf for boot time lnet configuration.</p>
<p style="margin-top: 0px; margin-bottom: 0px;">You can save the current lnet configuration using command: lnetctl export --backup > /etc/lnet.conf (make a backup of the original file first if required)<br>
</p>
<p style="margin-top: 0px; margin-bottom: 0px;">On next boot, lnet.service will load your lnet configuration from the file.</p>
<p style="margin-top: 0px; margin-bottom: 0px;">Or you can manually build lnet.conf as lnetctl seems to have occasion problems with some of the fields exported by "lnetctl export --backup"</p>
<p style="margin-top: 0px; margin-bottom: 0px;">Attaching my simple lnet.conf for your reference:</p>
<p style="margin-top: 0px; margin-bottom: 0px;"></p>
<blockquote type="cite"><font face="monospace"># cat /etc/lnet.conf<br>
ip2nets:<br>
- net-spec: o2ib<br>
ip-range:<br>
0: 10.2.8.*<br>
- net-spec: tcp<br>
ip-range:<br>
0: 10.5.9.*<br>
route:<br>
- net: o2ib<br>
gateway: 10.5.9.25@tcp<br>
hop: -1<br>
priority: 0<br>
- net: o2ib<br>
gateway: 10.5.9.24@tcp<br>
hop: -1<br>
priority: 0<br>
global:<br>
numa_range: 0<br>
max_intf: 200<br>
discovery: 1<br>
drop_asym_route: 0</font></blockquote>
Best regards,<br>
Angelos
<p style="margin-top: 0px; margin-bottom: 0px;"></p>
<div class="x_moz-cite-prefix">On 23/02/2021 13:58, Sid Young via lustre-discuss wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr"><br clear="all">
<div>
<div dir="ltr" class="x_gmail_signature">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div>G'Day all,</div>
<div>I'm finding that when I reboot any node in our new HPC, I need to keep manually adding the network using lnetctl net add --net tcp --if ens2f0</div>
<div>Then I can do an lnetctl net show and see the tcp part active...</div>
<div><br>
</div>
<div>I have options in /etc/modprobe.d/lnet.conf</div>
<div>options lnet networks=tcp<br>
</div>
<div><br>
</div>
<div>and </div>
<div><br>
</div>
<div>[root@hpc-oss-03 ~]# cat /etc/modprobe.d/lustre.conf<br>
options lnet networks="tcp(ens2f0)"<br>
options lnet ip2nets="tcp(ens2f0) 10.140.93.*<br>
</div>
<div><br>
</div>
<div>I've read the doco and tried to understand the correct parameters for a simple Lustre config so this is what I worked out is needed... but I suspect its still wrong.</div>
<div><br>
</div>
<div>Any help appreciated :)</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div>Sid Young</div>
<div><br>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<br>
<fieldset class="x_mimeAttachmentHeader"></fieldset>
<pre class="x_moz-quote-pre">_______________________________________________
lustre-discuss mailing list
<a class="x_moz-txt-link-abbreviated" href="mailto:lustre-discuss@lists.lustre.org">lustre-discuss@lists.lustre.org</a>
<a class="x_moz-txt-link-freetext" href="https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.lustre.org_listinfo.cgi_lustre-2Ddiscuss-2Dlustre.org&d=DwMDaQ&c=CJqEzB1piLOyyvZjb8YUQw&r=897kjkV-MEeU1IVizIfc5Q&m=hLa_kSVtQxFqwXUy9gxxHAHBo9kfy6QZwVZmQozlwk0&s=_1Omdg1RrZPHladfrwsA5uju-iG9SBJumOD0XguUPZw&e="><font color="red"><b>MailScanner has detected a possible fraud attempt from "urldefense.proofpoint.com" claiming to be</b></font> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org</a>
</pre>
</blockquote>
<pre class="x_moz-signature" cols="72">--
Angelos Ching
ClusterTech Limited
Tel : +852-2655-6138
Fax : +852-2994-2101
Address : Unit 211-213, Lakeside 1, 8 Science Park West Ave., Shatin, Hong Kong
Got praises or room for improvements? <a class="x_moz-txt-link-freetext" href="https://urldefense.proofpoint.com/v2/url?u=http-3A__bit.ly_TellAngelos&d=DwMDaQ&c=CJqEzB1piLOyyvZjb8YUQw&r=897kjkV-MEeU1IVizIfc5Q&m=hLa_kSVtQxFqwXUy9gxxHAHBo9kfy6QZwVZmQozlwk0&s=OeJCEZbuoxEjERYZMB2LP5fpybrBOsdumbHMZrFfOvc&e="><font color="red"><b>MailScanner has detected a possible fraud attempt from "urldefense.proofpoint.com" claiming to be</b></font> http://bit.ly/TellAngelos</a>
********************************************************************************
The information contained in this e-mail and its attachments is confidential and
intended solely for the specified addressees. If you have received this email in
error, please do not read, copy, distribute, disclose or use any information of
this email in any way and please immediately notify the sender and delete this
email. Thank you for your cooperation.
********************************************************************************</pre>
</div>
<span>_______________________________________________</span><br><span>lustre-discuss mailing list</span><br><span>lustre-discuss@lists.lustre.org</span><br><span>http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org</span><br></div></blockquote></div></body></html>