<div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr">Hello, </div><div dir="ltr"><br><div>TL;DR;</div><div>Is there a way to figure out how much memory Lustre will make available under memory pressure?</div><div><br></div><div>Details: </div><div>We are running lustre client on a machine with 128GB of memory (Centos 7) Intel Phi KNL machines and at certain situations we see that there can be ~10GB+ of memory allocated on the kernel side i.e. : </div><div><br></div><div>vvp_object_kmem 3535336 3536986 176 46 2 : tunables 0 0 0 : slabdata 76891 76891 0</div><div>ll_thread_kmem 33511 33511 344 47 4 : tunables 0 0 0 : slabdata 713 713 0</div><div>lov_session_kmem 34760 34760 592 55 8 : tunables 0 0 0 : slabdata 632 632 0</div><div>osc_extent_kmem 3549831 3551232 168 48 2 : tunables 0 0 0 : slabdata 73984 73984 0</div><div>osc_thread_kmem 14012 14116 2832 11 8 : tunables 0 0 0 : slabdata 1286 1286 0</div><div>osc_object_kmem 3546640 3548350 304 53 4 : tunables 0 0 0 : slabdata 66950 66950 0</div><div>signal_cache 3702537 3707144 1152 28 8 : tunables 0 0 0 : slabdata 132398 132398 0</div><div><br></div><div>/proc/meminfo: </div><div><div>MemAvailable: 114196044 kB</div><div>Slab: 11641808 kB</div><div>SReclaimable: 1410732 kB</div><div>SUnreclaim: 10231076 kB</div></div><div><br></div><div>After executing </div><div><pre class="gmail-code-java" style="margin-top:0px;margin-bottom:0px;padding:0px;max-height:30em;overflow:auto;white-space:pre-wrap;word-wrap:normal;color:rgb(51,51,51);font-size:12px;background-color:rgb(245,245,245)">echo 1 >/proc/sys/vm/drop_caches</pre><div>the slabinfo values don't change but when i actually generate memory pressure by: </div><div><pre class="gmail-code-java" style="margin-top:0px;margin-bottom:0px;padding:0px;max-height:30em;overflow:auto;white-space:pre-wrap;word-wrap:normal;color:rgb(51,51,51);font-size:12px;background-color:rgb(245,245,245)">java -Xmx117G -Xms117G -XX:+AlwaysPreTouch -version</pre></div><div><br></div><div>lots of memory gets freed: </div><div><div>vvp_object_kmem 127650 127880 176 46 2 : tunables 0 0 0 : slabdata 2780 2780 0</div><div>ll_thread_kmem 33558 33558 344 47 4 : tunables 0 0 0 : slabdata 714 714 0</div><div>lov_session_kmem 34815 34815 592 55 8 : tunables 0 0 0 : slabdata 633 633 0</div><div>osc_extent_kmem 128640 128880 168 48 2 : tunables 0 0 0 : slabdata 2685 2685 0</div><div>osc_thread_kmem 14038 14116 2832 11 8 : tunables 0 0 0 : slabdata 1286 1286 0</div><div>osc_object_kmem 82998 83263 304 53 4 : tunables 0 0 0 : slabdata 1571 1571 0</div><div>signal_cache 38734 44268 1152 28 8 : tunables 0 0 0 : slabdata 1581 1581 0</div></div><div><br></div><div>/proc/meminfo: </div><div><div>MemAvailable: 123146076 kB</div><div>Slab: 1959160 kB</div><div>SReclaimable: 334276 kB</div><div>SUnreclaim: 1624884 kB</div></div><div><br></div><div>The similar effect to generating memory pressure we see when executing: </div><div><pre class="gmail-code-java" style="margin-top:0px;margin-bottom:0px;padding:0px;max-height:30em;overflow:auto;white-space:pre-wrap;word-wrap:normal;color:rgb(51,51,51);font-size:12px;background-color:rgb(245,245,245)">echo 3 >/proc/sys/vm/drop_caches</pre></div><div>But this can take very long time (10 minutes). </div><div><br></div><div>So essentially on a machine using Lustre client, MemAvailable is no longer a good predictor of the amount of memory that can be allocated.</div><div>Is there a way to query Lustre and compensate for lustre cache memory that will be made available on memory pressure?</div><div><br></div><div>Regards.</div>-- <br><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><span><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><font color="#000000"><font face="arial,helvetica,sans-serif"><b>Jacek Tomaka</b></font></font><br><font color="#000000"><font face="arial,helvetica,sans-serif"><font size="2">Geophysical Software Developer</font></font></font><br></div><font color="#000000" face="arial,helvetica,sans-serif">
</font><div><span lang="EN-US"></span> <span lang="EN-US"><b><br><br></b></span></div><font color="#000000" face="arial,helvetica,sans-serif">
</font><img src="http://drive.google.com/uc?export=view&id=0B4X9ixpc-ZU_NHV0WnluaXp5ZkE"><br><br><span style="color:rgb(102,102,102)"><font size="2"><b>DownUnder GeoSolutions<br><br></b></font></span><div><span style="color:rgb(102,102,102)"></span><span style="color:rgb(102,102,102)">76 Kings Park Road<br></span></div><span style="color:rgb(102,102,102)">West Perth 6005 WA, Australia<br><i><b>tel </b></i><a href="tel:+61%208%209287%204143" value="+61892874143" target="_blank">+61 8 9287 4143</a><br><a href="mailto:jacekt@dug.com" target="_blank">jacekt@dug.com</a><br><b><a href="http://www.dug.com" target="_blank">www.dug.com</a></b></span></div></div></div></div></div></div></span></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div>