<div dir="auto"><div>Thank you for clarification Andreas.</div><div dir="auto"><br></div><div dir="auto">Looks like averyone is in only for network RAID functionality rather then distributed file system part.</div><div dir="auto">Though of course RAID is distributed in some sense. Sadly even those who have replica functionality like ceph and gluster lack real distributed part. Ceph's crush maps ignore client placement rendering it useless on cluster reads, even though they have good chain write replication engine. Gluster has choose-local and latency replica selection not ready - so for example Facebook had to add write background replication functionality (called halo) to make writes go to close replicas and distribute by nodes themselves.</div><div dir="auto"><br></div><div dir="auto">I am sure such functions will benefit everyone's favourite enterprise clients unless they have everything on ssd and rdma. </div><div dir="auto"><br></div><div dir="auto">Would love to see such functionality in Lustre!<br><br><div class="gmail_quote" dir="auto"><div dir="ltr">On Tue, Nov 13, 2018, 11:19 AM Andreas Dilger <<a href="mailto:adilger@whamcloud.com">adilger@whamcloud.com</a> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">On Nov 12, 2018, at 14:05, Vlad Kopylov <<a href="mailto:vladkopy@gmail.com" target="_blank" rel="noreferrer">vladkopy@gmail.com</a>> wrote:<br>
> <br>
> Hello,<br>
> <br>
> I am getting mixed google results on either Lustre supports full HA with all it's daemons for fs at this point.<br>
> Simply put I have 3 servers in 3 buildings with fronted apps working with replicated data set at each location.<br>
> Can I have Lustre replicate data through those 3 nodes and mount FS at each location. Preferably without excessive read traffic between locations as there is 1-2ms latency involved.<br>
<br>
While this is something that would be interesting to implement in Lustre, it isn't how Lustre is deployed today.<br>
<br>
In the 2.11 release, there is the ability to mirror files across OSTs, and you could install a separate MDT and OSTs at each site and have "local" subdirectories located on the local MDT, with an OST pool defined to create files on the local OST.<br>
<br>
That said, it would probably be a lot easier to just have 3 separate Lustre filesystems and use a higher-level tool to do resync between the sites.<br>
<br>
Cheers, Andreas<br>
---<br>
Andreas Dilger<br>
Principal Lustre Architect<br>
Whamcloud<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
</blockquote></div></div></div>