[lustre-discuss] Synchronous writes on a loaded ZFS OST

Vicker, Darby (JSC-EG311) darby.vicker-1 at nasa.gov
Tue May 8 06:15:44 PDT 2018


Yes, we experienced some similar slowness on our ZFS-based lustre FS too.  More details here:

http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/2017-April/014390.html

This also affects git repos quite a bit.  The fix suggested by Andreas in that thread worked fairly well and we continue to use it.  The same hardware in that thread is now running 2.10.  

I've quite anxious to test 2.11 and DoM but we haven't had the chance to do that yet.  I'm hopeful that will improve our small file IO.  

-----Original Message-----
From: lustre-discuss <lustre-discuss-bounces at lists.lustre.org> on behalf of Hans Henrik Happe <happe at nbi.dk>
Date: Tuesday, May 8, 2018 at 7:57 AM
To: "lustre-discuss at lists.lustre.org" <lustre-discuss at lists.lustre.org>
Subject: [lustre-discuss] Synchronous writes on a loaded ZFS OST

Hi,

We had some users experiencing slow vim (the editor) updates on our
Lustre homedirs. Turns out vim is doing some fsyncs that does not play
well with a loaded ZFS OST.

We tried testing with ioping, which does synced writes (like dd with
conv=fdatasync). When an OST is loaded (i.e. scrubbing) the ioping time
is multiple seconds (5-10). Without load we get 100-300ms, which still
is far from what a ZFS fs can deliver.

To test if a ZFS fs also would be affected we created a test fs on the
OST pool and ran ioping*. With or without a scrub running, the ping
times averaged at around 40ms.

Has anyone else experienced this? Can it be helped?

Cheers,
Hans Henrik

* Used -WWW because ioping -W runs on an unlinked file and ZFS will not
sync those to disk.
_______________________________________________
lustre-discuss mailing list
lustre-discuss at lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org



More information about the lustre-discuss mailing list