[lustre-discuss] lfs_migrate rsync vs. lfs migrate and layout swap

Daniel Kobras kobras at linux.de
Sat Nov 25 14:01:25 PST 2017


> Am 20.11.2017 um 00:01 schrieb Dilger, Andreas <andreas.dilger at intel.com>:
> It would be interesting to strace your rsync vs. "lfs migrate" read/write patterns so that the copy method of "lfs migrate" can be improved to match rsync. Since they are both userspace copy actions they should be about the same performance. It may be that "lfs migrate" is using O_DIRECT to minimize client cache pollution (I don't have the code handy to check right now).  In the future we could use "copyfile()" to avoid this as well. 

lfs migrate indeed uses O_DIRECT for reading the source file. A few tests on a system running 2.10.1 yielded a 10x higher throughput with a modified lfs migrate that simply dropped the O_DIRECT flag. I’ve filed https://jira.hpdd.intel.com/browse/LU-10278 about it. (A simple patch to make O_DIRECT optional is ready, but I still need to charm the gods of the firewall to let me push to Gerrit.)

Kind regards,


More information about the lustre-discuss mailing list