<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;
font-family:"Calibri",sans-serif;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Thanks – I do not have different type IB within one fabric. But with this info I found a few nodes that showed that error, but they are not matching the errors
I see on the server.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Btw - I got the problem resolved on one FS after upgrading to Lustre 2.11
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><a name="_____replyseparator"></a><b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"> Raj [mailto:rajgautam@gmail.com]
<br>
<b>Sent:</b> Thursday, June 07, 2018 10:36 AM<br>
<b>To:</b> Hebenstreit, Michael <michael.hebenstreit@intel.com><br>
<b>Cc:</b> White, Cliff <cliff.white@intel.com>; lustre-discuss <lustre-discuss@lists.lustre.org><br>
<b>Subject:</b> Re: [lustre-discuss] server_bulk_callback errors until server reboots<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">I seen the error when we had mix of FDR (using mlx4) and EDR(using mlx5) devices in lustre network. server_bulk_callback should have the corresponding client_bulk_callback in client.
<br>
<br>
<a href="http://wiki.lustre.org/Infiniband_Configuration_Howto">http://wiki.lustre.org/Infiniband_Configuration_Howto</a>
<o:p></o:p></p>
<div>
<div>
<p class="MsoNormal">On Thu, Jun 7, 2018 at 11:24 AM Hebenstreit, Michael <<a href="mailto:michael.hebenstreit@intel.com">michael.hebenstreit@intel.com</a>> wrote:<o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt">
<p class="MsoNormal">No, clients do not show any issues. <br>
<br>
-----Original Message-----<br>
From: White, Cliff <br>
Sent: Thursday, June 07, 2018 9:26 AM<br>
To: Hebenstreit, Michael <<a href="mailto:michael.hebenstreit@intel.com" target="_blank">michael.hebenstreit@intel.com</a>>; lustre-discuss <<a href="mailto:lustre-discuss@lists.lustre.org" target="_blank">lustre-discuss@lists.lustre.org</a>><br>
Subject: Re: [lustre-discuss] server_bulk_callback errors until server reboots<br>
<br>
<br>
<span style="font-family:"Tahoma",sans-serif"></span>On 6/7/18, 7:00 AM, "lustre-discuss on behalf of Hebenstreit, Michael" <<a href="mailto:lustre-discuss-bounces@lists.lustre.org" target="_blank">lustre-discuss-bounces@lists.lustre.org</a> on behalf of
<a href="mailto:michael.hebenstreit@intel.com" target="_blank">michael.hebenstreit@intel.com</a>> wrote:<br>
<br>
Hello<br>
<br>
I have now 2 Lustre systems that suddenly show this error - on a single OST the kernel log is filling with messages
<br>
<br>
[58858.365663] LustreError: 123642:0:(events.c:447:server_bulk_callback()) event type 3, status -61, desc ffff880524f7e000<br>
[58865.328317] LustreError: 123640:0:(events.c:447:server_bulk_callback()) event type 5, status -61, desc ffff880cab4ec800<br>
[58865.340792] LustreError: 123641:0:(events.c:447:server_bulk_callback()) event type 5, status -61, desc ffff880524f7c600<br>
[58865.353167] LustreError: 123640:0:(events.c:447:server_bulk_callback()) event type 3, status -61, desc ffff880cab4ec800<br>
[58865.365503] LustreError: 123641:0:(events.c:447:server_bulk_callback()) event type 3, status -61, desc ffff880524f7c600<br>
<br>
until the server reboots. Clients are on 2.11/RH7.5, servers are on <a href="http://2.7.19.10/RH7.4" target="_blank">
2.7.19.10/RH7.4</a> . Has anyone experienced this before?<br>
<br>
There should be some corresponding error messages on your clients, have you checked there?
<br>
cliffw<br>
<br>
Thanks<br>
Michael<br>
<br>
------------------------------------------------------------------------<br>
Michael Hebenstreit Senior Cluster Architect<br>
Intel Corporation, MS: RR1-105/H14 Core and Visual Compute Group (DCE)<br>
<a href="https://maps.google.com/?q=4100+Sara+Road&entry=gmail&source=g">4100 Sara Road</a> Tel.: +1 505-794-3144
<br>
Rio Rancho, NM 87124<br>
UNITED STATES E-mail: <a href="mailto:michael.hebenstreit@intel.com" target="_blank">
michael.hebenstreit@intel.com</a><br>
<br>
<br>
<br>
_______________________________________________<br>
lustre-discuss mailing list<br>
<a href="mailto:lustre-discuss@lists.lustre.org" target="_blank">lustre-discuss@lists.lustre.org</a><br>
<a href="http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org" target="_blank">
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org</a><br>
<br>
<br>
_______________________________________________<br>
lustre-discuss mailing list<br>
<a href="mailto:lustre-discuss@lists.lustre.org" target="_blank">lustre-discuss@lists.lustre.org</a><br>
<a href="http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org" target="_blank">http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org</a><o:p></o:p></p>
</blockquote>
</div>
</div>
</body>
</html>