[lustre-discuss] LNET ports and connections

[Degremont, Aurelien]

Thanks! That's really interesting.
Do you have a code pointer that could show where the code will establish this connection if missing?

Le 18/02/2020 23:34, « NeilBrown » <neilb at suse.com> a écrit :

    
    It is not true that:
       LNET will established connections only if asked for by upper layers.
    
    or at least, not in the sense that the upper layers ask for a
    connection.
    Lustre knows nothing about connections.  Even LNet doesn't really know
    about connections. It is only at the socklnd level that connections mean
    much.
    
    Lustre and LNet are message-passing protocols.
    Lustre asks LNet to send a message to a given peer, and gives some
    details of the sort of reply to expect.
    LNet chooses a route and thus a network interface, and asked the LND to
    send the message.
    The socklnd LND will see if it already has a TCP connection.  If it
    does, it will use it.  If not, it will create one.
    
    So yes : it is exactly:
      possible that the server in this case opens the connection itself
      without waiting for the client to reconnect?
    
    NeilBrown
    
    
    On Tue, Feb 18 2020, Aurelien Degremont wrote:
    
    > Thanks for your reply.
    > I think I have a good enough understanding of LNET itself. My question was more about how LNET is being used by Lustre itself.
    >
    > LNET will established connections only if asked for by upper layers. 
    > When I was talking about client and server, I was talking about how Lustre was using it.
    >
    > As far as I understood, Lustre server only contact clients when they need to send LDLM callbacks.
    > They do so through the socket already opened by the client (reverse import).
    > What happened if the socket is closed is what I'm not sure. I though the server is rather waiting for the client to reconnect and if not, is more or less evicting it.
    > Could it be possible that the server in this case opens the connection itself without waiting for the client to reconnect?
    >
    >
    > Aurélien
    >
    > Le 18/02/2020 05:42, « NeilBrown » <neilb at suse.com> a écrit :
    >
    >     
    >     LNet is a peer-to-peer protocol, it has no concept of client and server.
    >     If one host needs to send a message to another but doesn't already have
    >     a connection, it creates a new connection.
    >     I don't yet know enough specifics of the lustre protocol to be certain
    >     of the circumstances when a lustre server will need to initiate a message
    >     to a client, but I imagine that recalling a lock might be one.
    >     
    >     I think you should assume that any LNet node might receive a connection
    >     from any other LNet node (for which they share an LNet network), and
    >     that the connection could come from any port between 512 and 1023
    >     (LNET_ACCEPTOR_MIN_PORT to LNET_ACCEPTOR_MAX_PORT).
    >     
    >     NeilBrown
    >     
    >     
    >     
    >     On Mon, Feb 17 2020, Degremont, Aurelien wrote:
    >     
    >     > Hi all,
    >     >
    >     > From what I've understood so far, LNET listens on port 988 by default and peers connect to it using 1021-1023 TCP ports as source ports.
    >     > At Lustre level, servers listen on 988 and clients connect to them using the same source ports 1021-1023.
    >     > So only accepting connections to port 988 on server side sounded pretty safe to me. However, I've seen connections from 1021-1023 to 988, from server hosts to client hosts sometimes.
    >     > I can't understand what mechanism could trigger these connections. Did I miss something?
    >     >
    >     > Thanks
    >     >
    >     > Aurélien
    >     >
    >     > _______________________________________________
    >     > lustre-discuss mailing list
    >     > lustre-discuss at lists.lustre.org
    >     > http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
    >