<div dir="ltr">I do not want to hijack this thread but just checking here before I start another new thread. I am getting similar messages randomly. The IP involved here is one Client IP. Getting messages from multiple OSS about multiple OST at the same time and stops. These types of messages appear occasionally on multiple OSS, and all these are related to one client at a time. Wondering if it is one client related issue as this FS has 100s of clients and only one client reports at a time. Unfortunately, there is no easy way for me to figure out if the specified client had an access issue around the time frame mentioned in the log (no access to clients). <div><br></div><div>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures">Dec<span class="gmail-Apple-converted-space"> </span>4 18:05:27 oss010 kernel: LustreError: 137-5: fs-OST00b0_UUID: not available for connect from <client ip>@tcp1 (no target). If you are running an HA pair check that the target is mounted on the other server.</span></p></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, 4 Dec 2023 at 05:27, Andreas Dilger via lustre-discuss <<a href="mailto:lustre-discuss@lists.lustre.org">lustre-discuss@lists.lustre.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div dir="auto">
It wasn't clear from your rail which message(s) are you concerned about? These look like normal mount message(s) to me.
<div><br>
</div>
<div>The "error" is pretty normal, it just means there were multiple services starting at once and one wasn't yet ready for the other. </div>
<div><span style="color:rgb(0,0,0);font-family:Calibri,Helvetica,sans-serif;font-size:12pt"><br>
</span></div>
<div> <span style="color:rgb(0,0,0);font-family:Calibri,Helvetica,sans-serif;font-size:12pt">LustreError: 137-5: lustrevm-MDT0000_UUID: not available for connect</span></div>
<div><span style="color:rgb(0,0,0);font-family:Calibri,Helvetica,sans-serif;font-size:12pt"> from 0@lo (no target). If you are running an HA pair check that the target</span></div>
<div><span style="color:rgb(0,0,0);font-family:Calibri,Helvetica,sans-serif;font-size:12pt"> is mounted on the other serve</span><span style="color:rgb(0,0,0);font-family:Calibri,Helvetica,sans-serif;font-size:12pt">r.</span>
<div><br>
</div>
<div>It probably makes sense to quiet this message right at mount time to avoid this. </div>
<div><br id="m_756778258079893211lineBreakAtBeginningOfSignature">
<div dir="ltr">Cheers, Andreas</div>
<div dir="ltr"><br>
<blockquote type="cite">On Dec 1, 2023, at 10:24, Audet, Martin via lustre-discuss <<a href="mailto:lustre-discuss@lists.lustre.org" target="_blank">lustre-discuss@lists.lustre.org</a>> wrote:<br>
<br>
</blockquote>
</div>
<blockquote type="cite">
<div dir="ltr">
<div id="m_756778258079893211divtagdefaultwrapper" style="font-size:12pt;color:rgb(0,0,0);font-family:Calibri,Helvetica,sans-serif" dir="ltr">
<p><span style="font-size:12pt">Hello Lustre community,</span><br>
</p>
<p><br>
</p>
<p>Have someone ever seen messages like these on in "/var/log/messages" on a Lustre server ?<br>
<br>
</p>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<p></p>
<div>Dec 1 11:26:30 vlfs kernel: Lustre: Lustre: Build Version: 2.15.4_RC1</div>
<div>Dec 1 11:26:30 vlfs kernel: LDISKFS-fs (sdd): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc</div>
<div>Dec 1 11:26:30 vlfs kernel: LDISKFS-fs (sdc): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc</div>
<div>Dec 1 11:26:30 vlfs kernel: LDISKFS-fs (sdb): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc</div>
<div>Dec 1 11:26:36 vlfs kernel: LustreError: 137-5: lustrevm-MDT0000_UUID: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server.</div>
<div>Dec 1 11:26:36 vlfs kernel: Lustre: lustrevm-OST0001: Imperative Recovery not enabled, recovery window 300-900</div>
<div>Dec 1 11:26:36 vlfs kernel: Lustre: lustrevm-OST0001: deleting orphan objects from 0x0:227 to 0x0:513</div>
<div><br>
</div>
<p></p>
</blockquote>
This happens on every boot on a Lustre server named vlfs (a AlmaLinux 8.9 VM hosted on a VMware) playing the role of both MGS and OSS (it hosts an MDT two OST using "virtual" disks). We chose LDISKFS and not ZFS. Note that this happens at every boot, well before
the clients (AlmaLinux 9.3 or 8.9 VMs) connect and even when the clients are powered off. <span style="font-size:12pt">The network connecting the clients and the </span><span style="font-size:12pt">server </span><span style="font-size:12pt">is a "virtual"
10GbE network (of course there is no virtual IB). Also we had the same messages previously with Lustre 2.15.3 using an AlmaLinux 8.8 server and AlmaLinux 8.8 / 9.2 clients (also using VMs). Note also that we compile ourselves the Lustre RPMs from the sources
from the git repository. We also chose to use a patched kernel. Our build procedure for RPMs seems to work well because our real cluster run fine on CentOS 7.9 with Lustre 2.12.9 and IB (MOFED) networking.<br>
<br>
So has anyone seen these messages ?<br>
<br>
Are they problematic ? If yes, how do we avoid them ?<br>
<br>
We would like to make sure our small test system using VMs works well before we upgrade our real cluster.</span>
<div><span style="font-size:12pt"><br>
</span></div>
<div><span style="font-size:12pt">Thanks in advance !</span></div>
<div><span style="font-size:12pt"><br>
</span></div>
<div><span style="font-size:12pt">Martin Audet</span></div>
<div><span style="font-size:12pt"> </span></div>
</div>
<span>_______________________________________________</span><br>
<span>lustre-discuss mailing list</span><br>
<span><a href="mailto:lustre-discuss@lists.lustre.org" target="_blank">lustre-discuss@lists.lustre.org</a></span><br>
<span><a href="http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org" target="_blank">http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org</a></span><br>
</div>
</blockquote>
</div>
</div>
</div>
_______________________________________________<br>
lustre-discuss mailing list<br>
<a href="mailto:lustre-discuss@lists.lustre.org" target="_blank">lustre-discuss@lists.lustre.org</a><br>
<a href="http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org" rel="noreferrer" target="_blank">http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org</a><br>
</blockquote></div>