<!DOCTYPE html>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<p>Oleg,</p>
<p>I recreated the plots with a shared time axis. In your comments
you mention the "writer node". Note that all the reading and
writing was done on a single compute node.</p>
<p>Andreas,</p>
<p>Concerning your comments, this is really troubling when
considering benchmarking. I first got a hint of this while
benchmarking on a system that uses quotas. I suspect the quota
daemon periodically walks the file system doing stat()s. One of
my runs had significantly better performance and I now think it
was because my directory was hit by a stat(). The performance
improvement I saw was a one-off. Could quotas cause this? <b>Benchmarkers
beware</b>, doing an<font face="Courier New, Courier, monospace">
<b>ls -l</b></font> in your benchmarking directory could
significantly alter performance. I've been benchmarking I/O for
decades and have never observed this behavior. More correctly, I
have seen odd one-off behaviors, but never figured out what to
attribute it to.</p>
<p>John</p>
<p><img moz-do-not-send="false"
src="cid:part1.XT0issOL.XU2EVdDy@iodoctors.com"
alt="https://www.dropbox.com/scl/fi/b8h88muy9umyt4zey1d0d/sim_fpa.png?rlkey=atmyesl61p40s9vmbpmjkfv5b&st=icbay2kp&dl=0"
width="1052" height="683"></p>
<p>All 6 OSC<b> cached versus time</b> are in this next plot. Top
frame w/o stat()s. Bottom frame with stat()s. It can be
determined which OSCs are associated with the first file by its
start time.</p>
<p>Notice in the top frame that the first file, even though it was
not read or written to after RTC=106 seconds, it never gave up any
of its cache, even though the second file was in dire need of the
memory for caching. Files were not closed until all reading and
writing was completed. The bottom frame, with the stat()s is much
better behaved and much faster.</p>
<p><img moz-do-not-send="false"
src="cid:part2.f33tGvDX.wutvfsja@iodoctors.com"
alt="https://www.dropbox.com/scl/fi/8xdif7vu3penwtx1bepjw/sim_osc_cached.png?rlkey=rqq9che3yjtzds8hjkg411nec&st=0c5pknhn&dl=0"
width="1052" height="683"></p>
<div class="moz-cite-prefix">On 3/24/2026 5:08 PM, Oleg Drokin
wrote:<br>
</div>
<blockquote type="cite"
cite="mid:1c15520a1d06022416722c5e0f2584e676afe050.camel@whamcloud.com">
<pre wrap="" class="moz-quote-pre">On Tue, 2026-03-24 at 21:08 +0000, Andreas Dilger via lustre-discuss
wrote:
</pre>
<blockquote type="cite">
<pre wrap="" class="moz-quote-pre">read locks (which would be counter productive), and whether it would
be possible to downgrade the DLM write locks to read locks while
preserving the cached data on the client(s).
</pre>
</blockquote>
<pre wrap="" class="moz-quote-pre">
I think we long wanted this functionality but never implemented it.
The other thing in those graphs (a bit annoying to see because the
scale differs from graph to graph), but the start-workload seems to run
faster at least on the writer node, but I am not sure if that's part of
the question of just a normal variancy of the environment.
</pre>
</blockquote>
</body>
</html>