[lustre-discuss] Synchronous writes on a loaded ZFS OST

[Hans Henrik Happe]

Hi,

We had some users experiencing slow vim (the editor) updates on our
Lustre homedirs. Turns out vim is doing some fsyncs that does not play
well with a loaded ZFS OST.

We tried testing with ioping, which does synced writes (like dd with
conv=fdatasync). When an OST is loaded (i.e. scrubbing) the ioping time
is multiple seconds (5-10). Without load we get 100-300ms, which still
is far from what a ZFS fs can deliver.

To test if a ZFS fs also would be affected we created a test fs on the
OST pool and ran ioping*. With or without a scrub running, the ping
times averaged at around 40ms.

Has anyone else experienced this? Can it be helped?

Cheers,
Hans Henrik

* Used -WWW because ioping -W runs on an unlinked file and ZFS will not
sync those to disk.