[Lustre-devel] client side reply handling
Liang Zhen
Zhen.Liang at Sun.COM
Tue Dec 15 18:55:11 PST 2009
Peter,
It depends on how buffers are attached to portal, ptlrpc has three cases
of this:
1. If buffer is for incoming request (not for RDMA, so it's not unique
match)
the MD is constructed with lnet_md_t::threshold = LNET_MD_THRESH_INF,
so it will be unlinked only for two cases:
a) user call LNetMDUnlink explicitly
b) the buffer is exhausted (lnet_md_exhausted() return 1 in LNet)
Once LNet get a request for that buffer, it will change offset of
the buffer so
it wouldn't be overwritten by the next match, it's possible that upper
layer is handing
the request, and the buffer is still on match list
2. If buffer is for bulk (it's unique)
the MD is constructed with lnet_md_t::threshold = 1, and
LNetMDAttach(...LNET_UNLINK...)
so LNet will auto-unlink the buffer from match list once it got a
match, I think
this is the case you want
3. If the buffer is for reply (it's unique as well)
the MD is constructed with lnet_md_t::threshold = LNET_MD_THRESH_INF,
so it's same as 1), the buffer wouldn't be unlinked unless user call
LNetMDUnlink
or LNetMEUnlink, so safety of reply buffer can be guaranteed only if user is
sure the buffer is unlinked.
Regards
Liang
Peter Braam wrote:
> No - buffers can and probably should (because, for example,
> re-delivery may leaver buffer state undefined) be unlinked from the
> match list before passing the buffer up to any other layer.
> Portals/LNET certainly can do this.
>
> The question is why isn't this done, Eric, Isaac, Andreas?
>
> Peter
>
> On Tue, Dec 15, 2009 at 9:28 AM, Yuriy Umanets
> <yuriy.umanets at clusterstor.com <mailto:yuriy.umanets at clusterstor.com>>
> wrote:
>
>
>
> 2009/12/9 Peter Braam <peter.braam at clusterstor.com
> <mailto:peter.braam at clusterstor.com>>
>
> Hi,
>
> hi Peter,
>
>
> I wonder if LNET doesn't have an atomic operation that unlinks
> the packet from the delivery process upon receiving a packet.
> Iirc it does. Why wouldn't one use that?
>
> That is probably because it still may be in use by upper layers
> (ptlrpc, etc), i.e. it is up to upper layer to decide should its
> buffers be still linked for access or not.
>
> Thanks.
>
>
> Peter
>
> _______________________________________________
> Lustre-devel mailing list
> Lustre-devel at lists.lustre.org
> <mailto:Lustre-devel at lists.lustre.org>
> http://lists.lustre.org/mailman/listinfo/lustre-devel
>
>
>
>
> --
> umka
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> Lustre-devel mailing list
> Lustre-devel at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-devel
>
More information about the lustre-devel
mailing list