<div dir="ltr">Hi Chris,<div><br></div><div>Thanks for your help. I have collected the relevant logs according to your hints. But I need an account to open a ticket on Jira. I have sent an email to the administrator at <a href="mailto:info@whamcloud.com">info@whamcloud.com</a>. I was wondering if this is the correct way to apply for an account. I only found this email on the site.</div><div><br><div>Regards,</div><div>Chuanjun</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">Horn, Chris <<a href="mailto:chris.horn@hpe.com" target="_blank">chris.horn@hpe.com</a>> 于2023年2月18日周六 00:52写道：<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div>
<div lang="EN-US">
<div>
<p class="MsoNormal">If deleting and re-adding it restores the status to up then this sounds like a bug to me.<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">Can you enable debug tracing, reproduce the issue, and add this information to a ticket?<br>
<br>
To enable/gather debug:<br>
<br>
# lctl set_param debug=+net<br>
<reproduce issue><br>
# lctl dk > /tmp/dk.log<br>
<br>
You can create a ticket at <a href="https://jira.whamcloud.com/" target="_blank">https://jira.whamcloud.com/</a><br>
<br>
Please provide the dk.log with the ticket.<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">Thanks,<u></u><u></u></p>
<p class="MsoNormal">Chris Horn<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<div style="border-right:none;border-bottom:none;border-left:none;border-top:1pt solid rgb(181,196,223);padding:3pt 0in 0in">
<p class="MsoNormal" style="margin-bottom:12pt"><b><span style="font-size:12pt;color:black">From:
</span></b><span style="font-size:12pt;color:black">lustre-discuss <<a href="mailto:lustre-discuss-bounces@lists.lustre.org" target="_blank">lustre-discuss-bounces@lists.lustre.org</a>> on behalf of
</span><span style="font-size:12pt;font-family:"MS Gothic";color:black">腐朽</span><span style="font-size:12pt;font-family:"PingFang TC",sans-serif;color:black">银</span><span style="font-size:12pt;color:black"> via lustre-discuss <<a href="mailto:lustre-discuss@lists.lustre.org" target="_blank">lustre-discuss@lists.lustre.org</a>><br>
<b>Date: </b>Friday, February 17, 2023 at 2:53 AM<br>
<b>To: </b><a href="mailto:lustre-discuss@lists.lustre.org" target="_blank">lustre-discuss@lists.lustre.org</a> <<a href="mailto:lustre-discuss@lists.lustre.org" target="_blank">lustre-discuss@lists.lustre.org</a>><br>
<b>Subject: </b>[lustre-discuss] LNet nid down after some thing changed the NICs<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal">Hi,<u></u><u></u></p>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">I encountered a problem when using Lustre Client on k8s with kubenet. Very happy if you could help me.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">My LNet configuration is:<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">net:<br>
- net type: lo<br>
local NI(s):<br>
- nid: 0@lo<br>
status: up<br>
- net type: tcp<br>
local NI(s):<br>
- nid: 10.224.0.5@tcp<br>
status: up<br>
interfaces:<br>
0: eth0<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">It works. But after I deploy or delete a pod on the node. The nid goes down like:<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">- nid: 10.224.0.5@tcp<br>
status: down<br>
interfaces:<br>
0: eth0<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">k8s uses veth pairs, so it will add or delete network interfaces when deploying or deleting pods. But it doesn't touch the eth0 NIC. I can fix it by deleting the tcp net by `lnetctl net del` and re-add it by `lnetctl net add`. But I need
to do this every time after a pod is scheduled to this node.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">My node OS is Ubuntu 18.04 5.4.0-1101-azure. The Lustre Client is built by myself from 2.15.1. Is this an expected LNet behavior or I got something wrong? I re-build and tested it several times and got the same problem.<u></u><u></u></p>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Regards,<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Chuanjun<u></u><u></u></p>
</div>
</div>
</div>
</div>
</div>
</div></blockquote></div></div>