<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
</head>
<body>
<br>
Neil,<br>
<br>
I am not the person at Cray for this, but if SUSE does take an interest in this, Cray would probably be interested in weighing in and contributing info if not actually code. In fact, other HPC vendors like HPE(by which I mostly mean the old SGI) or IBM might
as well. NUMA optimization is a persistent fascination in our area of the industry...<br>
<br>
- Patrick<br>
<br>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> lustre-devel <lustre-devel-bounces@lists.lustre.org> on behalf of NeilBrown <neilb@suse.com><br>
<b>Sent:</b> Tuesday, June 26, 2018 9:44:37 PM<br>
<b>To:</b> Doug Oucharek<br>
<b>Cc:</b> Amir Shehata; Lustre Development List<br>
<b>Subject:</b> Re: [lustre-devel] [PATCH v3 07/26] staging: lustre: libcfs: NUMA support</font>
<div> </div>
</div>
<div class="BodyFragment"><font size="2"><span style="font-size:11pt;">
<div class="PlainText">On Mon, Jun 25 2018, Doug Oucharek wrote:<br>
<br>
> Some background on this NUMA change:<br>
><br>
> First off, this is just a first step to a bigger set of changes which include changes to the Lustre utilities. This was done as part of the Multi-Rail feature. One of the systems that feature is meant to support is the SGI UV system (now HPE) which has
a massive number of NUMA nodes connected by a NUMA Link. There are multiple fabric cards spread throughout the system and Multi-Rail needs to know which fabric cards are nearest to the NUMA node we are running on. To do that, the “distance” between NUMA
nodes needs to be configured.<br>
><br>
> This patch is preparing the infrastructure for the Multi-Rail feature to support configuring NUMA node distances. Technically, this patch should be landing with the Multi-Rail feature (still to be pushed) for it to make proper sense.<br>
><br>
<br>
Thanks a lot for the background.<br>
<br>
If these NUMA nodes have a 'distance' between them, and if lustre can<br>
benefit from knowing the distance, then is seems likely that other code<br>
might also benefit. In that case it would be best if the distance were<br>
encoded in some global state information so that lustre and any other<br>
subsystem can extract it.<br>
<br>
Do you know if there is any work underway by anyone to make this<br>
information generally available? If there is, we should make sure that<br>
lustre works in a compatible way so that once that work lands, lustre<br>
can use it directly and not need extra configuration.<br>
If no such work is underway, then it would be really good if something<br>
were done in that direction. If no-one here is able to work on this, I<br>
can ask around in SUSE and see if anyone here knows anything relevant.<br>
<br>
Thanks,<br>
NeilBrown<br>
</div>
</span></font></div>
</body>
</html>