<html><head></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; color: rgb(0, 0, 0); "><div><div><div style="font-family: Calibri, sans-serif; font-size: 14px; ">Hi Vijesh.</div><div style="font-family: Calibri, sans-serif; font-size: 14px; "><br></div><div style="font-family: Calibri, sans-serif; font-size: 14px; ">You are probably facing a called "split brain" issue. It may happen due heartbeat communication problems.</div><div style="font-family: Calibri, sans-serif; font-size: 14px; ">One common reason is an issue with your heartbeat. What sort of heartbeat are you using?</div><div style="font-family: Calibri, sans-serif; font-size: 14px; "><br></div><div style="font-family: Calibri, sans-serif; font-size: 14px; ">Some time ago we had a problem when the OSS were overloaded and the heartbeat becomes unresponsive. This would cause a "false split brain" scenario.</div><div style="font-family: Calibri, sans-serif; font-size: 14px; ">Basically all the two nodes within your HA pair stonith itself since there was no answer from heartbeat device. </div><div style="font-family: Calibri, sans-serif; font-size: 14px; "><br></div><div style="font-family: Calibri, sans-serif; font-size: 14px; ">I guess you should take a look and start monitoring your oss nodes to understand if the message logged makes sense (very likely). How's the memory configuration of your OSS nodes? What OS? How your zone reclaim mode looks like?</div><div style="font-family: Calibri, sans-serif; font-size: 14px; "><br></div><div style="font-family: Calibri, sans-serif; font-size: 14px; ">Regards,</div><div style="font-family: Calibri, sans-serif; font-size: 14px; ">Carlos</div><div style="font-family: Calibri, sans-serif; font-size: 14px; "> </div><div style="font-family: Calibri, sans-serif; font-size: 14px; "><br></div><div style="font-family: Calibri, sans-serif; font-size: 14px; "><span class="Apple-style-span" style="font-size: 13px; font-family: Arial, helvetica, sans-serif; "><b>--</b></span></div><div><div style="font-family: Calibri, sans-serif; font-size: 14px; "><span class="Apple-style-span" style="font-size: 13px; font-family: Arial, helvetica, sans-serif; "><b>Carlos Thomaz</b> | HPC Systems Architect</span><span class="Apple-style-span" style="font-size: 13px; font-family: Arial, helvetica, sans-serif; "> </span></div><div style="font-family: Calibri, sans-serif; font-size: 14px; "><span class="Apple-style-span" style="font-size: 13px; font-family: Arial, helvetica, sans-serif; ">Mobile: +1 (303) 519-0578 </span></div><div style="font-family: Calibri, sans-serif; font-size: 14px; "><span class="Apple-style-span" style="font-size: 13px; font-family: Arial, helvetica, sans-serif; "><div>cthomaz@ddn.com | Skype ID: carlosthomaz</div><p></p><div>DataDirect Networks, Inc.</div><div>9960 Federal Dr., Ste 100 Colorado Springs, CO 80921</div><div><a href="http://www.ddn.com/">ddn.com</a> | Twitter: <a href="http://twitter.com/ddn_limitless">@ddn_limitless</a> | 1.800.TERABYTE</div></span></div></div></div></div><div style="font-family: Calibri, sans-serif; font-size: 14px; "><br></div><span id="OLK_SRC_BODY_SECTION" style="font-size: 14px; font-family: Calibri, sans-serif; "><div style="font-family:Calibri; font-size:11pt; text-align:left; color:black; BORDER-BOTTOM: medium none; BORDER-LEFT: medium none; PADDING-BOTTOM: 0in; PADDING-LEFT: 0in; PADDING-RIGHT: 0in; BORDER-TOP: #b5c4df 1pt solid; BORDER-RIGHT: medium none; PADDING-TOP: 3pt"><span style="font-weight:bold">From: </span> VIJESH EK <<a href="mailto:ekvijesh@gmail.com">ekvijesh@gmail.com</a>><br><span style="font-weight:bold">Date: </span> Sun, 22 Jan 2012 22:33:20 -0800<br><span style="font-weight:bold">To: </span> "<a href="mailto:lustre-discuss@lists.lustre.org">lustre-discuss@lists.lustre.org</a>" <<a href="mailto:lustre-discuss@lists.lustre.org">lustre-discuss@lists.lustre.org</a>><br><span style="font-weight:bold">Subject: </span> [Lustre-discuss] OSS Nodes Fencing issue in HPC<br></div><div><br></div><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Calibri; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; font-size: medium; "><div><font color="#000066"><b>" slow start_page_write 57s due to heavy IO load "</b></font></div></span><br class="Apple-interchange-newline"></span></body></html>