[Lustre-discuss] Read ahead / prefetching

[Arifa Nisar]

Hi Andrea,

How can I know for sure that my Lustre's deployment has this feature or not?
Is it available in version 1.6.x or its only for 1.8.x? 

Thanks,
Arifa.

-----Original Message-----
From: Andreas Dilger [mailto:andreas.dilger at oracle.com] 
Sent: Friday, July 30, 2010 1:32 AM
To: Arifa Nisar
Cc: lustre-discuss at lists.lustre.org
Subject: Re: [Lustre-discuss] Read ahead / prefetching

If the size of the read calls is 1MB then after 2 reads at 4MB strides the
3rd read would trigger readahead initially for the next stride (1MB+1MB). If
the same stride pattern continues the readahead size will grow an additional
1MB each time up to the limit. 

Cheers, Andreas

On 2010-07-29, at 22:53, Arifa Nisar <a-nisar at u.northwestern.edu> wrote:

> Consider a case where stripe count = 4, stripe size = 1MB and all I/O
> accesses are aligned with the stripe boundary. If a client reads every 4th
> full stripe (accessing same OST always) then one stripe will be readahead
> from that OST in every call?
> 
> Thanks,
> Arifa.
> 
> -----Original Message-----
> From: Andreas Dilger [mailto:andreas.dilger at oracle.com] 
> Sent: Thursday, July 29, 2010 11:41 PM
> To: Arifa Nisar
> Cc: lustre-discuss at lists.lustre.org
> Subject: Re: [Lustre-discuss] Read ahead / prefetching
> 
> On 2010-07-29, at 14:02, Arifa Nisar wrote:
>> Thanks for the response. If an access pattern triggers page aligned
> strided
>> access then how many pages will be readahead? Are there some environment
>> variables which may give insight about readahead behavior? 
> 
> Because Lustre is tuned for RPC-sized IO, it will always readahead at
least
> a full RPC at a time (by default 1MB), unless the application is reading
> larger chunks than this, then it reads ahead in units of the IO size
aligned
> to RPC-sized boundaries.
> 
>> -----Original Message-----
>> From: Andreas Dilger [mailto:andreas.dilger at oracle.com] 
>> Sent: Wednesday, July 28, 2010 5:43 PM
>> To: Arifa Nisar
>> Cc: lustre-discuss at lists.lustre.org
>> Subject: Re: [Lustre-discuss] Read ahead / prefetching
>> 
>> On 2010-07-28, at 10:43, Arifa Nisar wrote:
>>> I am trying to educate myself on prefetching/readahead algorithm for
>>> Lustre's read. For a starter I only have two simple questions.
>>> 
>>> 1 - Does Lustre detect linear or random I/O pattern or it always
triggers
>>> readahead?
>> 
>> The current algorithm detects sequential and strided read patterns.
>> 
>>> 2 - If readahead is triggered, how many pages are read in addition to
> what
>>> is necessary? 
>> 
>> It is variable based on how long the same pattern was repeating and the
>> number of pages per request.
>> 
>> I would be happy to have you look at the strided readahead detection code
>> more closely, since I suspect it would not correctly detect stride
> patterns
>> with odd byte size and offset vs. stride patterns with full-page size and
>> offset.
>> 
>> We disussed a while ago to rewrite the readahead code to be layered
better
>> on the client, but haven't had time to work on it yet.
>> 
>> 
>> Cheers, Andreas
>> --
>> Andreas Dilger
>> Lustre Technical Lead
>> Oracle Corporation Canada Inc.
>> 
> 
> 
> Cheers, Andreas
> --
> Andreas Dilger
> Lustre Technical Lead
> Oracle Corporation Canada Inc.
>