[lustre-devel] [PATCH 00/24] lustre - more cleanups including module reduction.

[Cory Spitz]

I know that Doug follows this list closely, but I'm going to flag him here directly, because I know that he has an opinion about kiov / iovec (and a 3rd if I recall correctly).

Doug told me that (possibly) Al Viro had a proposal or submitted code to simply the kiov usage and that Doug was in favor of it.  Let's choose to make the right architectural call for Lustre/LNet and let Cray worry about keeping our out-of-tree code working.  As Patrick said, Cray pays the price of keeping DVS off to the side (we're wising up, which is why it is now GPL... but don't expect us to publish it on the Internet necessarily... baby-steps).

Doug, can you weigh-in on this case?

Thanks,
-Cory

-- 

On 6/27/18, 8:59 PM, "lustre-devel on behalf of NeilBrown" <lustre-devel-bounces at lists.lustre.org on behalf of neilb at suse.com> wrote:

    On Wed, Jun 27 2018, Patrick Farrell wrote:
    
    > Neil,
    >
    > We do indeed have such functionality (it’s called DVS and it’s
    > basically a high speed file system projection framework, ala NFS but
    > faster), so the ability to build lnet separately is valuable to us.
    > While it is being open sourced under the GPL, I don’t think there’s
    > any intention to try to upstream it.  The current code isn’t even
    > usable off of Cray systems as it depends on info from user space (that
    > is provided, in the end, from Cray proprietary hardware) to keep its
    > connection/routing tables up to date.  That’s supposedly in the
    > pipeline to get fixed, but it’s still pretty far from generally
    > usable.
    >
    > But we’d still really appreciate it if lnet stayed separate.  Don’t
    > know if that’s enough for you - I know sometimes *small* stuff is done
    > for out of tree users.  Hopefully this meets that standard.
    >
    
    Ahh - DVS.  That answers a question I just asked in another email.
    My google-skills don't seem to be up to locating the source code though
    :-(
    
    While I wouldn't knowingly break an interface used by some out-of-tree
    code without good reason, it is hard to avoid if you don't know what the
    out-of-tree code does.  It can be very tempting to remove something that
    isn't being used, but that can certainly hurt out-of-tree code
    sometimes.
    
    A particular example I'm exploring at present is the dual data paths in
    LNet.  Or maybe it is dual types of Memory Descriptors.
    There is 'kiov' which uses kernel-virtual addresses and 'iovec' which
    uses page+offset.
    The kiov option isn't used in the client code and it seems likely that
    the server-side code could be converted to use iovec without problems.
    
    I'd like to remove the kiov as I wouldn't be able to justify its
    existence when submitting the client-only code upstream.  But I don't
    want to remove the option of having an alternate MD type if it really is
    significantly more efficient in some context.
    If I know whether DVS used kiov or iovec - and in what way - that would
    help me to know if I might break something, and to be able to assess the
    cost.
    
    In my mind, the "standard" that you mention is always about
    practicality.   Code needs to be maintainable - easy to understand and
    hard to break.  If the LNet interface is clean and well documented in
    the kernel, then I don't see why we would not at least attempt to
    preserve it.
    
    Thanks,
    NeilBrown