<div dir="ltr">Thanks for all the great feedback and answers!<br><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Nov 27, 2017 at 7:04 AM, Mark Hahn <span dir="ltr"><<a href="mailto:hahn@mcmaster.ca" target="_blank">hahn@mcmaster.ca</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class=""><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Do you or anyone on the list know if/how flock affects Lustre performance?<br>
</blockquote>
<br></span>
I'm still puzzled by this: elasticsearch is specifically designed for each node to have a completely separate storage tree.<br>
why would there ever be any inter-node locking if your nodes happen to store onto Lustre?</blockquote><div>I didn't go into their source, but they use flock. <br>IIRC they do support storage being a shared medium, which would be nice as far as I'm concerned because I don't see why I need to hold the same dataset multiple times...<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> it seems like localflock would be perfect: not even pointless flock roundtrips to the MDS<br>
would take place.<br></blockquote><div>If it doesn't do shared datastore I may do that. <br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
I have no idea how well ES would work with Lustre - my ES clusters<br>
use local storage. </blockquote><div>We're currently storing on an NFS filesystem, though if each elastic instance has it's own copy of the data using local makes sense (we just try to be mostly diskless on everything except for storage machines) <br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> ES uses mmap extensively, which is well-supported<br>
by Lustre. I'm not so sure it would do well with Lustre's approach<br>
to IO (it doesn't do caching exactly like the normal Linux pagecache),<br>
and I wonder whether Lustre's proclivity for large block sizes might<br>
cause issues. offhand, I'd guess it might be appropriate to direct<br>
each ES node to specific OSTs (lfs setstripe).<br>
<br>
I note that ES tends to have a lot of small files. even just in terms<br>
of space utilization, that can be problematic for some Lustre configs.<br></blockquote><div>Doesn't that depend on the way you create your indexes? <br></div><div>Thanks again,<br></div><div>Eli<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
regards, mark hahn<br>
</blockquote></div><br></div></div>