[lustre-discuss] Group Lock Semantics

[Patrick Farrell]

Hybrid IO is a new feature in Lustre 2.17 which automatically switches to direct IO for IO above a certain configurable size - it relies on another new Lustre trick, which is the ability to do unaligned direct IO.  In fact, unaligned direct IO support is in Lustre 2.16, so if you have that or newer (Or EXA6 from DDN), you could skip the alignment work you're describing.  (Much of the purpose of this work - unaligned direct IO and hybrid - is to make the benefits of direct IO easy to access.)

Currently we only switch for IO above a certain size, but we intend to eventually switch to direct IO for IO when lock contention is detected as well.  We simply haven't found time to do the development there yet.

If you're running a new enough version, try it, and I'll be curious to hear your feedback.  This work was aimed partly at solving the problem you're dealing with without having to use any form of relaxed consistency, which as you're experiencing is challenging to deal with.

See here for background:
https://wiki.lustre.org/images/a/a0/LUG2024-Hybrid_IO_Path_Update-Farrell.pdf
________________________________
From: Freddie Witherden <freddie at witherden.org>
Sent: Saturday, February 7, 2026 10:29 AM
To: Patrick Farrell <pfarrell at ddn.com>
Subject: Re: [lustre-discuss] Group Lock Semantics

On 07/02/2026 07:43, Patrick Farrell wrote:
> So, if you are able to switch to direct IO as you mention, group locks
> should be unnecessary and are better avoided.  Direct IO works like this
> in 2.15 and newer.  (Also, in 2.17, hybrid IO can do this switch for you
> automatically for larger IO sizes.)

It will take a bit of work to engineer for direct I/O (we need to ensure
all of our starting offsets are page aligned and special-case the final
writes so we either do RMW or if at the very end of the file do a
buffered write), but its nothing insurmountable.  Will report back in a
few days.

What do you mean by hybrid I/O here?

Regards, Freddie.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20260207/e7c3d50b/attachment-0001.htm>