[Lustre-discuss] buffering

[burlen]

Andreas Dilger wrote:
> On 2010-08-11, at 23:36, burlen wrote:
>   
>> I am interested in how write()s are buffered in Lustre on the cleint, 
>> server, and network in between. Specifically I'd like to understand what 
>> happens during writes when large number of clients are making large 
>> writes to all of the OSTs on an OSS, and the buffers are inadequate to 
>> handle the outgoing/incoming data.
>>     
>
> Lustre doesn't buffer dirty pages on the OSS, only on the client.  The clients are granted a "reserve" of space in each OST filesystem to ensure there is enough free space for any cached writes that they do.
>
>   
Thanks for your answer.

If I understand the way write() typically works on Linux, during a large 
write(), too large to be buffered in the page cache, once the page cache 
is full dirty pages would be flushed to disk. the data transfer would 
block at that point until the dirty pages are written to disk, whence 
the data transfer would resume into the resulting free pages. But in 
Lustre I assume that once the client's page cache is full, the dirty 
pages are sent over the network to the OSS where they are written to 
disk. In that case, does the network layer effectively act like a 
buffer? So that the client may resume the data transfer into the page 
cache before the former set dirty pages actually hit the disk? Or does 
the data transfer block until dirty pages actually reach the disk?

Thanks
Burlen