<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<meta name="Generator" content="Microsoft Word 14 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.emailquote, li.emailquote, div.emailquote
{mso-style-name:emailquote;
mso-margin-top-alt:auto;
margin-right:0cm;
mso-margin-bottom-alt:auto;
margin-left:1.0pt;
border:none;
padding:0cm;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:70.85pt 70.85pt 70.85pt 70.85pt;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:1134715607;
mso-list-template-ids:1316771816;}
ol
{margin-bottom:0cm;}
ul
{margin-bottom:0cm;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="FR" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"">Hi,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif""><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif"">The two LNet networks need separate IPoIB networks.<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:35.4pt"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif"">o2ib0 - 10.0.0.0/24<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:35.4pt"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif"">o2ib1 - 10.1.0.0/24<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif""><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif"">Nodes with 2 physical interfaces (like you servers) have ib0 on o2ib0 and ib1 on o2ib1<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif"">Nodes with 1 physical interface (like your clients) have 2 logical interfaces: ib0 on o2ib0 and ib0:0 on o2ib1.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif""><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif"">Grgoire.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif""><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">De :</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> lustre-discuss-bounces@lists.lustre.org [mailto:lustre-discuss-bounces@lists.lustre.org]
<b>De la part de</b> mages, brian<br>
<b>Envoy :</b> mardi 12 fvrier 2013 21:34<br>
<b> :</b> lustre-discuss@lists.lustre.org<br>
<b>Objet :</b> [Lustre-discuss] Multirail IB Configuration Issue<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Calibri","sans-serif"">Hi,<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Calibri","sans-serif"">I’m having difficulty getting one of my clients to work with a multirail IB configuration. Here’s what I’ve got:<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Calibri","sans-serif"">
<b>Host OS Version Lustre Version Function Storage Interface ib0 Interface ib1</b><o:p></o:p></span></p>
</div>
<ol start="1" type="1">
<li class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;mso-list:l0 level1 lfo1">
<span style="font-size:10.0pt;font-family:"Calibri","sans-serif"">bmr1-s7 CentOS 5.7 2.1.1 MGS,MDS,OSS1 mdt,mdt2,ost1->6,ost13->18 192.168.1.25/24 192.168.1.35/24<o:p></o:p></span></li><li class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;mso-list:l0 level1 lfo1">
<span style="font-size:10.0pt;font-family:"Calibri","sans-serif"">bmr1-s8 CentOS 5.7 2.1.1 OSS2 ost7->12,ost19->24 192.168.1.26/24 192.168.1.36/24<o:p></o:p></span></li><li class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;mso-list:l0 level1 lfo1">
<span style="font-size:10.0pt;font-family:"Calibri","sans-serif"">bmr1-s5 CentOS 5.7 2.1.1 OSS3 ost25->30 192.168.1.20/24 192.168.1.30/24<o:p></o:p></span></li><li class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;mso-list:l0 level1 lfo1">
<span style="font-size:10.0pt;font-family:"Calibri","sans-serif"">bmr1-s6 CentOS 5.7 2.1.1 OSS4 ost31->36 192.168.1.21/24 192.168.1.31/24<o:p></o:p></span></li><li class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;mso-list:l0 level1 lfo1">
<span style="font-size:10.0pt;font-family:"Calibri","sans-serif"">bmr2-s9 CentOS 5.7 2.1.1 Client n/a 192.168.1.209/24<o:p></o:p></span></li></ol>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Calibri","sans-serif"">The “/lustre” filesystem consists of mdt and ost1->12 (using bmr1-s7 and bmr1-s8).<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Calibri","sans-serif"">The “/lustre2” filesystem consists of mdt2 and ost13->36 (using bmr1-s7, bmr1-s8, bmr1-s5, and bmr1-s6).<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Calibri","sans-serif"">On each OSS, half the OSTs are available only on ib0 and the other half only on ib1.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Calibri","sans-serif"">From bmr1-s5 and bmr1-s6 (using as clients), I can successfully mount and access “/lustre”. I can also successfully mount “/lustre2”.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Calibri","sans-serif"">From bmr2-s9, I can neither mount “/lustre” nor “/lustre2”. Originally, the issue with bmr2-s9 was that it was running 1.8.6-wc1 (server on CentOS 5.6). Since this config
(i.e., multirail) wasn’t supported on that version, I upgraded to 2.1.1. Originally, I tried installing and testing the 2.1.1 client without success. Then, since it had worked with the 2.1.1 server on both bmr1-s5 and bmr1-s6, I thought I’d try that next.
Unfortunately, it still didn’t work.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Calibri","sans-serif"">1a) Here’s what I see on the client when I try to mount “/lustre”:<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">[root@bmr2-s9 ~]# mount -t lustre
<a href="mailto:192.168.1.25@o2ib:/lustre">192.168.1.25@o2ib:/lustre</a> /mnt/lustre</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">mount.lustre: mount
<a href="mailto:192.168.1.25@o2ib:/lustre">192.168.1.25@o2ib:/lustre</a> at /mnt/lustre failed: No such file or directory</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">Is the MGS specification correct?</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">Is the filesystem name correct?</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">If upgrading, is the copied client log valid? (see upgrade docs)</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">[root@bmr2-s9 ~]#</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Calibri","sans-serif"">1b) Here’s an excerpt from “/var/log/messages” on the client (after executing the above command):<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">Feb 12 15:00:54 bmr2-s9 kernel: Lustre: 5512:0:(sec.c:1474:sptlrpc_import_sec_adapt()) import
<a href="mailto:MGC192.168.1.25@o2ib-%3eMGC192.168.1.25@o2ib_0">MGC192.168.1.25@o2ib->MGC192.168.1.25@o2ib_0</a> netid 50000: select flavor null</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">Feb 12 15:00:54 bmr2-s9 kernel: Lustre:
<a href="mailto:MGC192.168.1.25@o2ib">MGC192.168.1.25@o2ib</a>: Reactivating import</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">Feb 12 15:00:54 bmr2-s9 kernel: LustreError: 5523:0:(ldlm_lib.c:357:client_obd_setup()) can't add initial connection</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">Feb 12 15:00:54 bmr2-s9 kernel: LustreError: 5523:0:(obd_config.c:522:class_setup()) setup lustre-OST0001-osc-ffff81045d783c00 failed (-2)</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">Feb 12 15:00:54 bmr2-s9 kernel: LustreError: 5523:0:(obd_config.c:1361:class_config_llog_handler()) Err -2 on cfg command:</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">Feb 12 15:00:54 bmr2-s9 kernel: Lustre: cmd=cf003 0:lustre-OST0001-osc 1:lustre-OST0001_UUID 2:192.168.1.35@o2ib1
</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">Feb 12 15:00:54 bmr2-s9 kernel: LustreError: 15c-8:
<a href="mailto:MGC192.168.1.25@o2ib">MGC192.168.1.25@o2ib</a>: The configuration from log 'lustre-client' failed (-2). This may be the result of communication errors between this node and the MGS, a bad configuration, or other errors. See the syslog for more
information.</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">Feb 12 15:00:54 bmr2-s9 kernel: LustreError: 5512:0:(llite_lib.c:950:ll_fill_super()) Unable to process log: -2</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">Feb 12 15:00:54 bmr2-s9 kernel: LustreError: 4923:0:(lov_obd.c:927:lov_cleanup()) lov tgt 0 not cleaned! deathrow=0, lovrc=1</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">Feb 12 15:00:54 bmr2-s9 kernel: LustreError: 5512:0:(obd_config.c:567:class_cleanup()) Device 5 not setup</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">Feb 12 15:00:54 bmr2-s9 kernel: LustreError: 5512:0:(ldlm_request.c:1172:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">Feb 12 15:00:54 bmr2-s9 kernel: LustreError: 5512:0:(ldlm_request.c:1799:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">Feb 12 15:00:54 bmr2-s9 kernel: Lustre: client ffff81045d783c00 umount complete</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">Feb 12 15:00:54 bmr2-s9 kernel: LustreError: 5512:0:(obd_mount.c:2164:lustre_fill_super()) Unable to mount (-2)</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Calibri","sans-serif"">1c) Here’s an excerpt from “/var/log/messages” on the server (after executing the above command):<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">Feb 12 15:00:54 bmr1-s7 kernel: Lustre: 25911:0:(ldlm_lib.c:877:target_handle_connect()) MGS: connection from
<a href="mailto:2e13dea0-ec9c-0fbd-0f95-7b16246f2626@192.168.1.209@o2ib">2e13dea0-ec9c-0fbd-0f95-7b16246f2626@192.168.1.209@o2ib</a> t0 exp 0000000000000000 cur 1360699254 last 0</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">Feb 12 15:00:54 bmr1-s7 kernel: Lustre: 25911:0:(sec.c:1474:sptlrpc_import_sec_adapt()) import MGS->NET_0x50000c0a801d1_UUID netid 50000: select flavor null</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Calibri","sans-serif"">2a) Here’s what I see on the client when I try to mount “/lustre” (using the other interface):<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">[root@bmr2-s9 ~]# mount -t lustre
<a href="mailto:192.168.1.25@o2ib:/lustre">192.168.1.25@o2ib:/lustre</a> /mnt/lustre</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">mount.lustre: mount
<a href="mailto:192.168.1.25@o2ib:/lustre">192.168.1.25@o2ib:/lustre</a> at /mnt/lustre failed: No such file or directory</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">Is the MGS specification correct?</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">Is the filesystem name correct?</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">If upgrading, is the copied client log valid? (see upgrade docs)</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">[root@bmr2-s9 ~]# mount -t lustre
<a href="mailto:192.168.1.35@o2ib:/lustre">192.168.1.35@o2ib:/lustre</a> /mnt/lustre</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">mount.lustre: mount
<a href="mailto:192.168.1.35@o2ib:/lustre">192.168.1.35@o2ib:/lustre</a> at /mnt/lustre failed: Invalid argument</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">This may have multiple causes.</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">Is 'lustre' the correct filesystem name?</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">Are the mount options correct?</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">Check the syslog for more info.</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">[root@bmr2-s9 ~]#</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Calibri","sans-serif"">2b) Here’s an excerpt from “/var/log/messages” on the client (after executing the above command):<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">Feb 12 15:06:57 bmr2-s9 kernel: Lustre: 5580:0:(sec.c:1474:sptlrpc_import_sec_adapt()) import
<a href="mailto:MGC192.168.1.35@o2ib-%3eMGC192.168.1.35@o2ib_0">MGC192.168.1.35@o2ib->MGC192.168.1.35@o2ib_0</a> netid 50000: select flavor null</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">Feb 12 15:06:57 bmr2-s9 kernel: Lustre: 4948:0:(client.c:1778:ptlrpc_expire_one_request()) @@@ Request x1426793186721863 sent from
<a href="mailto:MGC192.168.1.35@o2ib">MGC192.168.1.35@o2ib</a> to NID <a href="mailto:192.168.1.35@o2ib">
192.168.1.35@o2ib</a> has failed due to network error: [sent 1360699617] [real_sent 1360699617] [current 1360699617] [deadline 5s] [delay -5s] req@ffff81043b76e400 x1426793186721863/t0(0) o-1-><a href="mailto:MGS@MGC192.168.1.35@o2ib_0:26/25">MGS@MGC192.168.1.35@o2ib_0:26/25</a>
lens 368/512 e 0 to 1 dl 1360699622 ref 1 fl Rpc:XN/ffffffff/ffffffff rc 0/-1</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">Feb 12 15:06:57 bmr2-s9 kernel: Lustre: 4948:0:(client.c:1778:ptlrpc_expire_one_request()) Skipped 1 previous similar message</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">Feb 12 15:06:57 bmr2-s9 kernel: LustreError: 3074:0:(o2iblnd_cb.c:2615:kiblnd_rejected())
<a href="mailto:192.168.1.35@o2ib">192.168.1.35@o2ib</a> rejected: o2iblnd fatal error</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">Feb 12 15:06:57 bmr2-s9 kernel: LustreError: 3074:0:(o2iblnd_cb.c:2615:kiblnd_rejected()) Skipped 1 previous similar message</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">Feb 12 15:07:03 bmr2-s9 kernel: LustreError: 5580:0:(client.c:1049:ptlrpc_import_delay_req()) @@@ send limit expired req@ffff81043b76e000 x1426793186721864/t0(0) o-1-><a href="mailto:MGS@MGC192.168.1.35@o2ib_0:26/25">MGS@MGC192.168.1.35@o2ib_0:26/25</a>
lens 296/352 e 0 to 0 dl 0 ref 2 fl Rpc:W/ffffffff/ffffffff rc 0/-1</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">Feb 12 15:07:03 bmr2-s9 kernel: LustreError: 5580:0:(client.c:1049:ptlrpc_import_delay_req()) Skipped 6 previous similar messages</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">Feb 12 15:07:22 bmr2-s9 kernel: Lustre: 4949:0:(import.c:526:import_select_connection())
<a href="mailto:MGC192.168.1.35@o2ib">MGC192.168.1.35@o2ib</a>: tried all connections, increasing latency to 5s</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">Feb 12 15:07:22 bmr2-s9 kernel: Lustre: 4948:0:(client.c:1778:ptlrpc_expire_one_request()) @@@ Request x1426793186721868 sent from
<a href="mailto:MGC192.168.1.35@o2ib">MGC192.168.1.35@o2ib</a> to NID <a href="mailto:192.168.1.35@o2ib">
192.168.1.35@o2ib</a> has failed due to network error: [sent 1360699642] [real_sent 1360699642] [current 1360699642] [deadline 10s] [delay -10s] req@ffff810430e30800 x1426793186721868/t0(0) o-1-><a href="mailto:MGS@MGC192.168.1.35@o2ib_0:26/25">MGS@MGC192.168.1.35@o2ib_0:26/25</a>
lens 368/512 e 0 to 1 dl 1360699652 ref 1 fl Rpc:XN/ffffffff/ffffffff rc 0/-1</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">Feb 12 15:07:22 bmr2-s9 kernel: LustreError: 3074:0:(o2iblnd_cb.c:2615:kiblnd_rejected())
<a href="mailto:192.168.1.35@o2ib">192.168.1.35@o2ib</a> rejected: o2iblnd fatal error</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">Feb 12 15:07:24 bmr2-s9 kernel: LustreError: 5591:0:(client.c:1049:ptlrpc_import_delay_req()) @@@ send limit expired req@ffff81045d7ce800 x1426793186721867/t0(0) o-1-><a href="mailto:MGS@MGC192.168.1.35@o2ib_0:26/25">MGS@MGC192.168.1.35@o2ib_0:26/25</a>
lens 296/352 e 0 to 0 dl 0 ref 2 fl Rpc:W/ffffffff/ffffffff rc 0/-1</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">Feb 12 15:07:24 bmr2-s9 kernel: LustreError: 5591:0:(client.c:1049:ptlrpc_import_delay_req()) Skipped 1 previous similar message</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">Feb 12 15:07:47 bmr2-s9 kernel: Lustre: 4949:0:(import.c:526:import_select_connection())
<a href="mailto:MGC192.168.1.35@o2ib">MGC192.168.1.35@o2ib</a>: tried all connections, increasing latency to 10s</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">Feb 12 15:07:47 bmr2-s9 kernel: Lustre: 4948:0:(client.c:1778:ptlrpc_expire_one_request()) @@@ Request x1426793186721872 sent from
<a href="mailto:MGC192.168.1.35@o2ib">MGC192.168.1.35@o2ib</a> to NID <a href="mailto:192.168.1.35@o2ib">
192.168.1.35@o2ib</a> has failed due to network error: [sent 1360699667] [real_sent 1360699667] [current 1360699667] [deadline 15s] [delay -15s] req@ffff810444576c00 x1426793186721872/t0(0) o-1-><a href="mailto:MGS@MGC192.168.1.35@o2ib_0:26/25">MGS@MGC192.168.1.35@o2ib_0:26/25</a>
lens 368/512 e 0 to 1 dl 1360699682 ref 1 fl Rpc:XN/ffffffff/ffffffff rc 0/-1</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">Feb 12 15:07:47 bmr2-s9 kernel: LustreError: 3074:0:(o2iblnd_cb.c:2615:kiblnd_rejected())
<a href="mailto:192.168.1.35@o2ib">192.168.1.35@o2ib</a> rejected: o2iblnd fatal error</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">Feb 12 15:07:54 bmr2-s9 kernel: LustreError: 156-2: The client profile 'lustre-client' could not be read from the MGS. Does that filesystem exist?</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">Feb 12 15:07:54 bmr2-s9 kernel: Lustre: client ffff81045f465800 umount complete</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">Feb 12 15:07:54 bmr2-s9 kernel: LustreError: 5580:0:(obd_mount.c:2164:lustre_fill_super()) Unable to mount (-22)</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Calibri","sans-serif"">2c) Here’s an excerpt from “/var/log/messages” on the server (after executing the above command):<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">Feb 12 15:06:57 bmr1-s7 kernel: LustreError: 9274:0:(o2iblnd_cb.c:2247:kiblnd_passive_connect()) Can't accept
<a href="mailto:192.168.1.209@o2ib">192.168.1.209@o2ib</a> on <a href="mailto:192.168.1.25@o2ib">
192.168.1.25@o2ib</a> (ib1:1:192.168.1.35): bad dst nid <a href="mailto:192.168.1.35@o2ib">
192.168.1.35@o2ib</a></span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">Feb 12 15:06:57 bmr1-s7 kernel: LustreError: 9274:0:(o2iblnd_cb.c:2247:kiblnd_passive_connect()) Skipped 2 previous similar messages</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">Feb 12 15:07:22 bmr1-s7 kernel: LustreError: 9274:0:(o2iblnd_cb.c:2247:kiblnd_passive_connect()) Can't accept
<a href="mailto:192.168.1.209@o2ib">192.168.1.209@o2ib</a> on <a href="mailto:192.168.1.25@o2ib">
192.168.1.25@o2ib</a> (ib1:1:192.168.1.35): bad dst nid <a href="mailto:192.168.1.35@o2ib">
192.168.1.35@o2ib</a></span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Calibri","sans-serif"">3) Here’s what one of the MDTs looks like (the other is similarly configured):<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">[root@bmr1-s7 ~]# tunefs.lustre --dryrun --writeconf /dev/sdp</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">checking for existing Lustre data: found CONFIGS/mountdata</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">Reading CONFIGS/mountdata</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New""> </span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New""> Read previous values:</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">Target: lustre-MDT0000</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">Index: 0</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">Lustre FS: lustre</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">Mount type: ldiskfs</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">Flags: 0x5</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New""> (MDT MGS )</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">Persistent mount opts: iopen_nopriv,user_xattr,errors=remount-ro</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">Parameters:
<a href="mailto:mgsnode=192.168.1.25@o2ib,192.168.1.35@o2ib1,10.244.78.88@tcp,192.168.1.25@tcp1">
mgsnode=192.168.1.25@o2ib,192.168.1.35@o2ib1,10.244.78.88@tcp,192.168.1.25@tcp1</a></span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New""> Permanent disk data:</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">Target: lustre-MDT0000</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">Index: 0</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">Lustre FS: lustre</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">Mount type: ldiskfs</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">Flags: 0x105</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New""> (MDT MGS writeconf )</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">Persistent mount opts: iopen_nopriv,user_xattr,errors=remount-ro</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">Parameters:
<a href="mailto:mgsnode=192.168.1.25@o2ib,192.168.1.35@o2ib1,10.244.78.88@tcp,192.168.1.25@tcp1">
mgsnode=192.168.1.25@o2ib,192.168.1.35@o2ib1,10.244.78.88@tcp,192.168.1.25@tcp1</a></span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">exiting before disk write.</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">[root@bmr1-s7 ~]#</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Calibri","sans-serif"">4) Here’s what one of the OSTs looks like (the others are similarly configured):<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">[root@bmr1-s7 ~]# tunefs.lustre --dryrun --writeconf /dev/sdf</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">checking for existing Lustre data: found CONFIGS/mountdata</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">Reading CONFIGS/mountdata</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New""> </span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New""> Read previous values:</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">Target: lustre-OST0000</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">Index: 0</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">Lustre FS: lustre</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">Mount type: ldiskfs</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">Flags: 0x2</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New""> (OST )</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">Persistent mount opts: errors=remount-ro,extents,mballoc</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">Parameters:
<a href="mailto:mgsnode=192.168.1.25@o2ib,192.168.1.35@o2ib1,10.244.78.88@tcp,192.168.1.25@tcp1">
mgsnode=192.168.1.25@o2ib,192.168.1.35@o2ib1,10.244.78.88@tcp,192.168.1.25@tcp1</a> network=o2ib0</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New""> </span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New""> </span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New""> Permanent disk data:</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">Target: lustre-OST0000</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">Index: 0</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">Lustre FS: lustre</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">Mount type: ldiskfs</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">Flags: 0x102</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New""> (OST writeconf )</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">Persistent mount opts: errors=remount-ro,extents,mballoc</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">Parameters:
<a href="mailto:mgsnode=192.168.1.25@o2ib,192.168.1.35@o2ib1,10.244.78.88@tcp,192.168.1.25@tcp1">
mgsnode=192.168.1.25@o2ib,192.168.1.35@o2ib1,10.244.78.88@tcp,192.168.1.25@tcp1</a> network=o2ib0</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New""> </span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">exiting before disk write.</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">[root@bmr1-s7 ~]#</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Calibri","sans-serif"">I’d appreciate any help or direction on a potential resolution. Let me know what additional information is needed, if any. Hopefully, I’m just missing something simple.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Calibri","sans-serif"">Thanks in advance,<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Calibri","sans-serif"">…Brian<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""> <o:p></o:p></span></p>
</div>
</div>
</body>
</html>