[lustre-devel] Why can you set concurrent_sends < peer_credits ?
alexey.lyashkov at seagate.com
Wed Aug 19 11:58:49 PDT 2015
In current code it's may be same. Let me finish some bugs before look to
code. But with "good" code - you should have a different cost (credits
count) for different messages. 1Mb bulk transfer should eats more credits
than simple 4kb message. So you "should" have a two limits first one for
parallel send with zero/low cost messages and second one for large messages.
But as i understand credit based flow control don't work now - i see
several situations when server have a negative number a credits, which
indicate we have sending queue more than limits and parallel sends limits
will work in that situation..
It's know bug for me but need large time to create a network model to
create a correct credits distribution.
On Wed, Aug 19, 2015 at 9:24 PM, Chris Horn <hornc at cray.com> wrote:
> They sure look pretty related to me. In kiblnd_post_tx_locked() we return
> EAGAIN if ibc_nsends_posted == IBLND_CONCURRENT_SENDS. ibc_nsends_posted is
> incremented on every send. So it looks like you couldn’t ever send more
> than concurrent_sends messages to a single peer, which is exactly what
> peer_credits is supposed to govern, no? What am I missing?
> Chris Horn
> On Aug 19, 2015, at 12:28 PM, Alexey Lyashkov <alexey.lyashkov at seagate.com>
> concurrent_sends is measurement about parallel operations, but credits is
> flow control artifact.
> each operations may eats different number credits and calculated in per
> link and per destination basic, so it's completely different attributes.
> On Wed, Aug 19, 2015 at 7:31 PM, Chris Horn <hornc at cray.com> wrote:
>> A thread on HPDD-discuss made me think about this question. AFAICT, the
>> o2iblnd driver code will not let you have more that concurrent_sends
>> messages in flight at the same time(in fact, we LASSERT on this fact in
>> kiblnd_check_sends). Thus peer_credits is effectively limited by
>> concurrent_sends anyways. What’s the reasoning behind allowing peer_credits
>> to be larger than concurrent_sends?
>> Chris Horn
>> lustre-devel mailing list
>> lustre-devel at lists.lustre.org
> Alexey Lyashkov *·* Technical lead for a Morpheus team
> Seagate Technology, LLC
Alexey Lyashkov *·* Technical lead for a Morpheus team
Seagate Technology, LLC
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the lustre-devel