<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=utf-8">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<p>I'm observing some undesirable caching of OSC data in the system
buffers. This is a single node, single process application.
There are 2 files of interest, <b>SCRATCH </b>and <b>SCR300</b>,
both are scratch files with stripeCount=4. The system has 128GB
of memory. Lustre maxes out at about 59GB of memory used for
caching.<br>
</p>
<p><b>SCRATCH</b>, About 22GB is written/read during the first 300
seconds of the run. No further activity to the file ( but remains
open ) until about 18,700 seconds into the run when another 22GB
is written/read. Illustrated in the top frame of the first plot
below. In the bottom frame of the first plot is the amount of
system cache used by each of the 4 OSC's associated with the file
over the course of the run ( nearly identical, as would be
expected ). Note that each the OSC's retains its 5.5GB of memory
even though nothing is happening to the file.<br>
</p>
<p><b>SCR300</b>, A 110GB file, written and repeatedly read between
the times of the above SCRATCH file's I/O.</p>
<p>What is of interest it that while SCR300 is doing all its I/O,
and its associated OSC's are fighting each other for caching
memory, the 4 OSC's for the inactive file(SCRATCH) retain their
22GB of memory. Why are the 4 OSC's for the inactive file exempt
from giving up their memory? It is very reproducible.</p>
<p>The application is MSC.Nastran, which has the capability to put
the data for SCR300 inside of SCRATCH(increasing its size to
132GB). If run in this mode, the caching behavior is much better
behaved and the job runs in 11,500 seconds, versus 19,000.
Illustrated in 3rd plot below. While this is a solution for this
case, it is not a general solution.</p>
<p>Thanks</p>
<p>John<br>
</p>
Plots for <b>SCRATCH</b><br>
<img src="cid:part1.3E422D16.52B91181@iodoctors.com" alt=""><br>
<br>
<br>
Plots for <b>SCR300</b><br>
<br>
<img src="cid:part2.C38E4127.EC4D43A5@iodoctors.com" alt=""><br>
<br>
<br>
Plots for <b>SCR300 </b>inside of <b>SCRATCH</b><br>
<br>
<img src="cid:part3.8719C9C9.F5BEA35E@iodoctors.com" alt=""><br>
<pre class="moz-signature" cols="72">--
I/O Doctors, LLC
507-766-0378
<a class="moz-txt-link-abbreviated" href="mailto:bauerj@iodoctors.com">bauerj@iodoctors.com</a></pre>
</body>
</html>