<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
Can you please try 2.12.4, it was just released yesterday and has a number of fixes.<br class="">
<div><br class="">
<blockquote type="cite" class="">
<div class="">On Feb 12, 2020, at 13:36, Kevin M. Hildebrand <<a href="mailto:kevin@umd.edu" class="">kevin@umd.edu</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div dir="ltr" class="">
<div class="gmail_default" style="font-size:small">I just updated some of my clients to RHEL 7.7, Lustre 2.12.3, MOFED 4.7.</div>
<div class="gmail_default" style="font-size:small">Server version is 2.10.8.<br class="">
</div>
<div class="gmail_default" style="font-size:small"><br class="">
</div>
<div class="gmail_default" style="font-size:small">I'm now getting errors mounting the filesystem on the client. In fact, I can't even do an 'lctl ping' to any of the servers without getting an I/O error.</div>
<div class="gmail_default" style="font-size:small"><br class="">
</div>
<div class="gmail_default" style="font-size:small">Debug logs show this message when I attempt an lctl ping:<br class="">
</div>
<div class="gmail_default" style="font-size:small">00000800:00020000:0.0:1581538955.090767:0:20471:0:(o2iblnd.c:941:kiblnd_create_conn()) Can't create QP: -12, send_wr: 32634, recv_wr: 254, send_sge: 2, recv_sge: 1</div>
<div class="gmail_default" style="font-size:small"><br class="">
</div>
<div class="gmail_default" style="font-size:small"># lctl list_nids<br class="">
10.11.80.65@o2ib3</div>
<div class="gmail_default" style="font-size:small"># lctl ping 10.11.80.50@o2ib3<br class="">
failed to ping 10.11.80.50@o2ib3: Input/output error</div>
<div class="gmail_default" style="font-size:small"><br class="">
</div>
<div class="gmail_default" style="font-size:small">Interestingly, if I do an 'lctl ping' to the client _from_ the server, the ping succeeds, and from that point on pings from client _to_ server work fine until the client is rebooted or lnet is reloaded.<br class="">
</div>
<div class="gmail_default" style="font-size:small"><br class="">
</div>
<div class="gmail_default" style="font-size:small">ko2iblnd parameters match on clients and servers, namely:<br class="">
options ko2iblnd peer_credits=128 peer_credits_hiw=64 credits=1024 concurrent_sends=256 ntx=2048 map_on_demand=32 fmr_pool_size=2048 fmr_flush_trigger=512 fmr_cache=1
<br class="">
</div>
<div class="gmail_default" style="font-size:small"><br class="">
</div>
<div class="gmail_default" style="font-size:small">Anyone have any thoughts?</div>
<div class="gmail_default" style="font-size:small"><br class="">
</div>
<div class="gmail_default" style="font-size:small">Thanks,</div>
<div class="gmail_default" style="font-size:small">Kevin</div>
<div class="gmail_default" style="font-size:small"><br class="">
</div>
<div class="gmail_default" style="font-size:small">--</div>
<div class="gmail_default" style="font-size:small">Kevin Hildebrand</div>
<div class="gmail_default" style="font-size:small">University of Maryland</div>
<div class="gmail_default" style="font-size:small">Division of IT<br class="">
</div>
</div>
_______________________________________________<br class="">
lustre-discuss mailing list<br class="">
<a href="mailto:lustre-discuss@lists.lustre.org" class="">lustre-discuss@lists.lustre.org</a><br class="">
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org<br class="">
</div>
</blockquote>
</div>
<br class="">
<div class="">
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div>Cheers, Andreas</div>
<div>--</div>
<div>Andreas Dilger</div>
<div>Principal Lustre Architect</div>
<div>Whamcloud</div>
<div><br class="">
</div>
<div><br class="">
</div>
<div><br class="">
</div>
</div>
</div>
</div>
</div>
</div>
<br class="Apple-interchange-newline">
<br class="Apple-interchange-newline">
</div>
<br class="">
</body>
</html>