[lustre-discuss] Write performance low

Patrick Farrell paf at cray.com
Sun Jan 29 13:47:56 PST 2017


In general, Lustre always writes asynchronously, unless explicitly told to do otherwise (direct I/O or O_SYNC).  I don't have much experience with memory mapped files - it's possible that they somehow force synchronous behavior...  But my point, I suppose, is that there's probably not a tunable for you.  Try - just to see - writes to a normal file (IE, not memory mapped), so you can confirm.  Someone on list will probably have a better idea about writes to mmap'ed files and what to expect.

- Patrick

From: lustre-discuss <lustre-discuss-bounces at lists.lustre.org> on behalf of sergio.rivas at um.es <sergio.rivas at um.es>
Sent: Sunday, January 29, 2017 8:15:00 AM
To: lustre-discuss at lists.lustre.org
Subject: [lustre-discuss] Write performance low

Good afternoon,

I'm currently developing a small library that allows to easily perform an mmap that targets files in storage, and so far the results have been quite positive in local tests. However, trying out the same code on our cluster that uses Lustre as PFS, I have noticed that the reads are conveniently cached but that the writes seem to be synchronously flushed to storage, hence, decreasing the perceived performance (despite this fact being technically correct from a data-consistency point of view).

My goal is to allow for the page cache of the OS to keep as much as dirty pages as possible with the purpose of aggregating write operations for free, but I'm afraid that I haven't figured out how to solve this with Lustre after playing around with some of the settings that I found within the manual.

Could you, please, point out if there is any tunable setting (e.g., under /proc) that allow me to increase the caching or avoid direct flushing?

Thank you very much in advance.

Kind Regards,

P.S.: I think my previous post didn't go through, so I'm sending it again. Excuse me if it did!

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20170129/dd63ba63/attachment.htm>

More information about the lustre-discuss mailing list