[Lustre-devel] Query to understand the Lustre request/reply message

[Alexey Lyashkov]

On Oct 14, 2010, at 08:18, Nicolas Williams wrote:

> On Thu, Oct 14, 2010 at 06:38:16AM +0300, Alexey Lyashkov wrote:
>> On Oct 14, 2010, at 03:28, Nicolas Williams wrote:
>>> Yes, it's possible to add buffers to requests.  It's not possible to add
>>> buffers to _replies_ to existing RPCs unless you know the client expects
>>> those additional buffers -- existing clients expect a given maxsize for
>>> each reply, and if your reply is bigger then it will get dropped.
>> It is wrong for last ~1year.
>> ~1year ago i add code to ptlrpc layer which a adjust buffer for reply, and resend a request.
> 
> Ah, I didn't know that was in 1.8.
that is added near of 1.8.1 and easy to check by grep -rn rq_reply_truncate in ptlrpc directory.

Severity   : normal
Bugzilla   : 19526
Description: can't stat file in some situation.
Details    : improve initialize osc date when target is added to mds and
             ability to resend too big getattr request is client isn't have info
             about ost.


>  Are there interop issues (with older
> clients) though with sending larger replies than expected?
I not clearly understand that question, but 

main propose of that change - problem with LOV EA buffer size for files with ACL (look to some conf-sanity tests).
In some situation - MDS can have larger LOV EA buffer, when client expected 
(some files with wide striping have a reference to OST which removed from a cluster, or configuration lost, or new OST added but OST isn't connected to client or bad call shrink reply buffer, or other ... - you can find more references in bugzilla.)
in that case MDS have send larger buffer to client.
older client have infinity loop on connect  or in stat syscall (because of messages without rq_no_resend flag)
new client have a resend message - and adjust maximal size of LOV EA after got a valid reply, to avoid that's problem in future.



--------------------------------------
Alexey Lyashkov
alexey.lyashkov at clusterstor.com