[lustre-discuss] OSS Large File I/O using O_DIRECT

[John Bauer]

Andreas,

Thank you for your reply.  I am not asking about the Hybrid I/O 
feature.  A bit more on that later.  I am asking about the oti_dio_pages 
( Direct I/O ) path through the OSS.  With some experimentation last 
night I was able to confirm that an OSS uses Direct I/O when the 
incoming client request is 8MB or greater.  On the system I am running 
on ( clients version 2.14 and server version as reported by pminfo is 
lustre.sys.version=2 lustre.sys.build=0_ddn218 ) it appears that the 
reads and writes use the server cache when the user requests are less 
that 8MB and using non-rotational OSTs.  The OSS could have been 
configured this way.  I will check with admin next week.

Back to Hybrid I/O.  It concerns me that there is no mechanism is user 
space that will allow a client application to specify legacy client-side 
buffering on a per-file basis, independent of request size.  I envision 
plenty of cases where a client application is running dedicated  on a 
1TB memory client node, which leaves 500 MB for Lustre caching, and the 
application is doing large request I/O to a file that is under 500MB.  
The client application would now be subject to Hybrid's direct I/O even 
though the entire file could be buffer cached on the client.  Am I 
misunderstanding something here?

Thanks again,

John

On 1/9/2026 7:02 PM, Andreas Dilger wrote:
> Hi John,
> are you asking about the Hybrid IO feature in 2.16+ Lustre releases,
> or something else?
>
>   There is a client-side "llite.*.hybrid_io" parameter that can enable/disable Hybrid IO on a client completely, and the
> llite.*.hybrid_io_write_threshold_bytes and
> llite.*.hybrid_io_read_threshold_bytes can be used to tune the
> IO size threshold where IO changes between buffered and direct.
>
> Applications can of course use open(O_DIRECT) to use DIO instead of
> buffered IO.
>
> As for the server-side non-buffered IO path, this is controlled by
> the osd-ldiskfs.*.readcache_max_io_mb and .writethrough_max_io_mb
> parameters, both default to 8 MiB.  Note that flash (non-rotational)
> OSTs disable read and write cache entirely by default, since NVMe
> devices are typically fast enough to handle incoming IO.
>
> I don't think there is any way for clients to fetch these parameters
> directly, since they are more a property of the OST than the client.
>
> Cheers, Andreas
>
> On Jan 9, 2026, at 14:32, John Bauer <bauerj at iodoctors.com <mailto:bauerj at iodoctors.com>> wrote:
>> Hello all,
>> Is the a way to determine what size a client I/O request must be to trigger the OSS to use the Large File ( non-buffered ) I/O path?  Is this configurable?
>> Is there a way for a client-side application to trigger this behavior independent of the I/O size?
> ---
> Andreas Dilger
> Principal Lustre Architect
> adilger at thelustrecollective.com
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20260110/7e392e07/attachment.htm>