[lustre-discuss] Lustre and Optane

[Dave Holland]

Hi Andreas,

thanks for that. I'll follow with interest - though we only got our
first Optane system to test rather recently.

Cheers,
Dave
-- 
** Dave Holland ** Systems Support -- Informatics Systems Group **
** 01223 496923 **    Wellcome Sanger Institute, Hinxton, UK    **


On Mon, Jan 13, 2020 at 07:56:42PM +0000, Andreas Dilger wrote:
> The most likely candidate for effectively using Optane/NVRAM would be
> 
> via Persistent Client Cache (PCC), which allows client-local storage
> to
> 
> be part of the Lustre namespace.  Files can be cached on a local
> NVRAM
> 
> device (managed by a local filesystem like ext4, or possibly
> something
> 
> more experimental like NOVA for better performance) and then migrated
> 
> into the cache.
> 
> Once the file is in PCC, it can be accessed via the local filesystem
> 
> operations, including DAX, for very low-latency operations.  See the
> 
> presentation from LAD'19 for details:
> 
> [1]https://www.eofs.eu/_media/events/lad19/07_li_xi-nvram_pcc.pdf
> [eofs.eu]
> 
> It should be noted that in Lustre 2.13, files in PCC are NOT resident
> in the
> 
> main filesystem, so if the client node goes offline then the files
> will not be
> 
> accessible until the client node is restarted.  For some workloads
> this is OK
> 
> (e.g. files being generated locally with high IOPS that are
> occasionally
> 
> needed on other clients), but not for others.  We will be improving
> PCC to
> 
> use FLR to mirror a copy into the client cache and still keep a copy
> in the
> 
> main filesystem, but that is not available yet.
> 
> Cheers, Andreas
> 
> On Jan 13, 2020, at 10:03, Dave Holland <[2]dh3 at sanger.ac.uk> wrote:
> 
> I haven't been to LUG or LAD recently, so I'm a bit out of the loop,
> but
> how much use is Optane finding in the Lustre world?
> The main obstacle I see is that it's server-local, so building a
> resilient/failover-capable system isn't straightforward.
> Thanks for any observations.
> Cheers,
> Dave
> --
> ** Dave Holland ** Systems Support -- Informatics Systems Group **
> ** 01223 496923 **    Wellcome Sanger Institute, Hinxton, UK    **
> 
> Cheers, Andreas
> --
> Andreas Dilger
> Principal Lustre Architect
> Whamcloud
> 
> References
> 
> 1. https://urldefense.proofpoint.com/v2/url?u=https-3A__www.eofs.eu_-5Fmedia_events_lad19_07-5Fli-5Fxi-2Dnvram-5Fpcc.pdf&d=DwMFAg&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=64bKjxgut4Pa0xs5b84yPg&m=H2vGJE5te4vZmLD2u-for6b7tPd0hqJ5ZGK1KCJW1PA&s=65AxT6uau9wbcX-T0aEFtfzXX-iIrAw-K1SlVCjJ4oY&e=
> 2. mailto:dh3 at sanger.ac.uk



-- 
 The Wellcome Sanger Institute is operated by Genome Research 
 Limited, a charity registered in England with number 1021457 and a 
 company registered in England with number 2742969, whose registered 
 office is 215 Euston Road, London, NW1 2BE.