<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
</head>
<body>
<br>
I will ask about source and plans to publish it, Neil - Im almost certain it has not been publicly released yet and Im not actually sure the relicensing is really complete. So Id be shocked and possibly mildly concerned if you found the source online today.<br>
<br>
Understood about the rest, and thanks! I have no idea which DVS uses, and I understand that as an out of tree user, we are in the end responsible for it. DVS has in fact had this relationship with the master branch for a long time - Andreas and company are
accommodating about interface changes, but in the end, we are responsible for keeping in sync.<br>
<br>
I would say about the acceptor and sock lnd: I believe Lustre assumes some IP transport is available for configuration, but does NOT necessarily use it for primary communication. Fabrics - Like infiniband or Cray Aries - are more or less always configured
to provide IP transport, to enable the panoply of tools and apps that rely on it. But they perform better if their native protocols are used, which is of course what the other LNDs do.<br>
<br>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> NeilBrown <neilb@suse.com><br>
<b>Sent:</b> Wednesday, June 27, 2018 8:59:05 PM<br>
<b>To:</b> Patrick Farrell; Andreas Dilger; James Simmons<br>
<b>Cc:</b> Oleg Drokin; Lustre Development List<br>
<b>Subject:</b> Re: [lustre-devel] [PATCH 00/24] lustre - more cleanups including module reduction.</font>
<div> </div>
</div>
<div class="BodyFragment"><font size="2"><span style="font-size:11pt;">
<div class="PlainText">On Wed, Jun 27 2018, Patrick Farrell wrote:<br>
<br>
> Neil,<br>
><br>
> We do indeed have such functionality (its called DVS and its<br>
> basically a high speed file system projection framework, ala NFS but<br>
> faster), so the ability to build lnet separately is valuable to us.<br>
> While it is being open sourced under the GPL, I dont think theres<br>
> any intention to try to upstream it. The current code isnt even<br>
> usable off of Cray systems as it depends on info from user space (that<br>
> is provided, in the end, from Cray proprietary hardware) to keep its<br>
> connection/routing tables up to date. Thats supposedly in the<br>
> pipeline to get fixed, but its still pretty far from generally<br>
> usable.<br>
><br>
> But wed still really appreciate it if lnet stayed separate. Dont<br>
> know if thats enough for you - I know sometimes *small* stuff is done<br>
> for out of tree users. Hopefully this meets that standard.<br>
><br>
<br>
Ahh - DVS. That answers a question I just asked in another email.<br>
My google-skills don't seem to be up to locating the source code though<br>
:-(<br>
<br>
While I wouldn't knowingly break an interface used by some out-of-tree<br>
code without good reason, it is hard to avoid if you don't know what the<br>
out-of-tree code does. It can be very tempting to remove something that<br>
isn't being used, but that can certainly hurt out-of-tree code<br>
sometimes.<br>
<br>
A particular example I'm exploring at present is the dual data paths in<br>
LNet. Or maybe it is dual types of Memory Descriptors.<br>
There is 'kiov' which uses kernel-virtual addresses and 'iovec' which<br>
uses page+offset.<br>
The kiov option isn't used in the client code and it seems likely that<br>
the server-side code could be converted to use iovec without problems.<br>
<br>
I'd like to remove the kiov as I wouldn't be able to justify its<br>
existence when submitting the client-only code upstream. But I don't<br>
want to remove the option of having an alternate MD type if it really is<br>
significantly more efficient in some context.<br>
If I know whether DVS used kiov or iovec - and in what way - that would<br>
help me to know if I might break something, and to be able to assess the<br>
cost.<br>
<br>
In my mind, the "standard" that you mention is always about<br>
practicality. Code needs to be maintainable - easy to understand and<br>
hard to break. If the LNet interface is clean and well documented in<br>
the kernel, then I don't see why we would not at least attempt to<br>
preserve it.<br>
<br>
Thanks,<br>
NeilBrown<br>
</div>
</span></font></div>
</body>
</html>