<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
The most likely candidate for effectively using Optane/NVRAM would be
<div class="">via Persistent Client Cache (PCC), which allows client-local storage to</div>
<div class="">be part of the Lustre namespace. Files can be cached on a local NVRAM</div>
<div class="">device (managed by a local filesystem like ext4, or possibly something</div>
<div class="">more experimental like NOVA for better performance) and then migrated</div>
<div class="">into the cache.</div>
<div class=""><br class="">
</div>
<div class="">Once the file is in PCC, it can be accessed via the local filesystem</div>
<div class="">operations, including DAX, for very low-latency operations. See the</div>
<div class="">presentation from LAD'19 for details:</div>
<div class=""><br class="">
</div>
<div class=""><a href="https://www.eofs.eu/_media/events/lad19/07_li_xi-nvram_pcc.pdf" class="">https://www.eofs.eu/_media/events/lad19/07_li_xi-nvram_pcc.pdf</a></div>
<div class=""><br class="">
</div>
<div class="">It should be noted that in Lustre 2.13, files in PCC are NOT resident in the</div>
<div class="">main filesystem, so if the client node goes offline then the files will not be</div>
<div class="">accessible until the client node is restarted. For some workloads this is OK</div>
<div class="">(e.g. files being generated locally with high IOPS that are occasionally</div>
<div class="">needed on other clients), but not for others. We will be improving PCC to</div>
<div class="">use FLR to mirror a copy into the client cache and still keep a copy in the</div>
<div class="">main filesystem, but that is not available yet.</div>
<div class=""><br class="">
</div>
<div class="">Cheers, Andreas</div>
<div class="">
<div><br class="">
<blockquote type="cite" class="">
<div class="">On Jan 13, 2020, at 10:03, Dave Holland <<a href="mailto:dh3@sanger.ac.uk" class="">dh3@sanger.ac.uk</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div class="">I haven't been to LUG or LAD recently, so I'm a bit out of the loop, but<br class="">
how much use is Optane finding in the Lustre world?<br class="">
<br class="">
The main obstacle I see is that it's server-local, so building a<br class="">
resilient/failover-capable system isn't straightforward.<br class="">
<br class="">
Thanks for any observations.<br class="">
<br class="">
Cheers,<br class="">
Dave<br class="">
-- <br class="">
** Dave Holland ** Systems Support -- Informatics Systems Group **<br class="">
** 01223 496923 ** Wellcome Sanger Institute, Hinxton, UK **<br class="">
<br class="">
</div>
</div>
</blockquote>
</div>
<br class="">
<div class="">
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div>Cheers, Andreas</div>
<div>--</div>
<div>Andreas Dilger</div>
<div>Principal Lustre Architect</div>
<div>Whamcloud</div>
<div><br class="">
</div>
<div><br class="">
</div>
<div><br class="">
</div>
</div>
</div>
</div>
</div>
</div>
<br class="Apple-interchange-newline">
<br class="Apple-interchange-newline">
</div>
<br class="">
</div>
</body>
</html>