<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
p.msonormal0, li.msonormal0, div.msonormal0
{mso-style-name:msonormal;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="#0563C1" vlink="#954F72">
<div class="WordSection1">
<p class="MsoNormal">Hello,<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">I agree as others have stated that we would not expect the loss of a router to significantly affect the I/O destined for filesystems served by other routers, nor would we expect the I/O destined for non-routed filesystems to be affected.
However, I can say that we have seen bugs in this area in the past where the loss of a remote filesystem (the servers, not the routers serving that filesystem) did affect access to other filesystems. If I recall correctly the issue was that resources were
being consumed on the routers in trying to communicate with the lost filesystem. That resource consumption caused I/O destined for other filesystems to get backed up. I’m not aware of any outstanding issues like this, and I’ll stress that that sort of behavior
would certainly be considered a bug. So please let us know if you see any issues.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Regarding check_routers_before_use, this parameter affects how the LNet router checker behaves upon startup. The router checker on an LNet peer works by periodically sending an LNet ping to each known router. If a peer receives a response
from the router within a timeout period then the router is considered alive, otherwise it is considered dead and routes hosted by that router are removed from the routing table (until it starts responding to the pings). By default, all routers are initially
considered to be up (alive), and all routes are immediately eligible for sends. When check_routers_before_use is enabled (set to “1”) all routers are instead initially considered down (dead), and all routes must first respond to an LNet level ping before the
route becomes eligible for sends.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">The use of this parameter should not affect the scenarios you describe. Traffic destined for local networks is not affected by the up or down (alive or dead) states of routers.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Chris Horn<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:12.0pt;color:black">From: </span></b><span style="font-size:12.0pt;color:black">lustre-discuss <lustre-discuss-bounces@lists.lustre.org> on behalf of Makia Minich <makia@systemfabricworks.com><br>
<b>Date: </b>Wednesday, May 9, 2018 at 8:51 AM<br>
<b>To: </b>"lustre-discuss@lists.lustre.org" <lustre-discuss@lists.lustre.org><br>
<b>Subject: </b>[lustre-discuss] LNET Routing Question<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p class="MsoNormal"><a name="_MailOriginalBody">Hello all, <o:p></o:p></a></p>
<div>
<p class="MsoNormal"><span style="mso-bookmark:_MailOriginalBody"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="mso-bookmark:_MailOriginalBody">I have an LNET routing question. I’ve attached a quick diagram of the current setup; but basically I have two core networks (one infiniband and one ethernet) with a set of LNET routers in between.
There is storage and clients on both sides of these routers and all clients need to see all/most storage. All connections, configurations, etc are all working.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="mso-bookmark:_MailOriginalBody"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="mso-bookmark:_MailOriginalBody">The question is, if an LNET router goes down (which does cause some amount of reconnect or remapping for any clients attempting to use those routes) would this cause any issues or delays for
a client’s connection to non-routed storage? Put slightly different, if a job on the ethernet clients is actively using ethernet storage and the lnet routers go down, will job be affected? What about a new job just launching when that lnet router is down?<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="mso-bookmark:_MailOriginalBody"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="mso-bookmark:_MailOriginalBody">In addition, what does “check_routers_before_use” actually do and does it change the scenarios I mentioned? (e.g. If an ethernet client has “check_routers_before_use” would every file request
start with a ping to the routers even if it’s not leaving it’s core network?)<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="mso-bookmark:_MailOriginalBody"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="mso-bookmark:_MailOriginalBody">Thanks!<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="mso-bookmark:_MailOriginalBody"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="mso-bookmark:_MailOriginalBody"><img width="320" height="125" style="width:3.3333in;height:1.302in" id="FDB4E672-82E0-4B46-AF91-8819ED29FDAA" src="cid:image001.png@01D3F2A7.A1E86AA0" alt="cid:8DEBE4C1-F290-40B3-B226-C983C1A17279@unknownnet.local"><o:p></o:p></span></p>
<div>
<div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="mso-bookmark:_MailOriginalBody"><span style="color:black">—<o:p></o:p></span></span></p>
</div>
<div>
<p class="MsoNormal"><span style="mso-bookmark:_MailOriginalBody"><span style="color:black"><o:p> </o:p></span></span></p>
</div>
<div>
<p class="MsoNormal"><span style="mso-bookmark:_MailOriginalBody"><span style="color:black">Makia Minich<o:p></o:p></span></span></p>
</div>
<div>
<p class="MsoNormal"><span style="mso-bookmark:_MailOriginalBody"><span style="color:black">Principal Architect<o:p></o:p></span></span></p>
</div>
<div>
<p class="MsoNormal"><span style="mso-bookmark:_MailOriginalBody"><span style="color:black">System Fabric Works<br>
"Fabric Computing that Works”<br>
<br>
"Oh, I don't know. I think everything is just as it should be, y'know?”<br>
- Frank Fairfield<o:p></o:p></span></span></p>
</div>
</div>
</div>
</div>
</div>
<span style="mso-bookmark:_MailOriginalBody"></span>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
</body>
</html>