[Lustre-discuss] Read ahead / prefetching

Andreas Dilger andreas.dilger at oracle.com
Thu Jul 29 21:41:25 PDT 2010


On 2010-07-29, at 14:02, Arifa Nisar wrote:
> Thanks for the response. If an access pattern triggers page aligned strided
> access then how many pages will be readahead? Are there some environment
> variables which may give insight about readahead behavior? 

Because Lustre is tuned for RPC-sized IO, it will always readahead at least a full RPC at a time (by default 1MB), unless the application is reading larger chunks than this, then it reads ahead in units of the IO size aligned to RPC-sized boundaries.

> -----Original Message-----
> From: Andreas Dilger [mailto:andreas.dilger at oracle.com] 
> Sent: Wednesday, July 28, 2010 5:43 PM
> To: Arifa Nisar
> Cc: lustre-discuss at lists.lustre.org
> Subject: Re: [Lustre-discuss] Read ahead / prefetching
> 
> On 2010-07-28, at 10:43, Arifa Nisar wrote:
>> I am trying to educate myself on prefetching/readahead algorithm for
>> Lustre's read. For a starter I only have two simple questions.
>> 
>> 1 - Does Lustre detect linear or random I/O pattern or it always triggers
>> readahead?
> 
> The current algorithm detects sequential and strided read patterns.
> 
>> 2 - If readahead is triggered, how many pages are read in addition to what
>> is necessary? 
> 
> It is variable based on how long the same pattern was repeating and the
> number of pages per request.
> 
> I would be happy to have you look at the strided readahead detection code
> more closely, since I suspect it would not correctly detect stride patterns
> with odd byte size and offset vs. stride patterns with full-page size and
> offset.
> 
> We disussed a while ago to rewrite the readahead code to be layered better
> on the client, but haven't had time to work on it yet.
> 
> 
> Cheers, Andreas
> --
> Andreas Dilger
> Lustre Technical Lead
> Oracle Corporation Canada Inc.
> 


Cheers, Andreas
--
Andreas Dilger
Lustre Technical Lead
Oracle Corporation Canada Inc.




More information about the lustre-discuss mailing list