[lustre-discuss] rsync not appropriate for lustre
Daniel Kobras
kobras at puzzle-itc.de
Fri Jan 24 00:51:44 PST 2020
Hi!
Am 23.01.20 um 16:33 schrieb Bernd Melchers:
> we are copying large data sets within our lustre filesystem and between
> lustre and an external nfs server. In both cases the performance is
> unexpected low and the reason seems that rsync is reading and writing in
> 32 kB Blocks, whereas our lustre would be more happy with 4 MB
> Blocks.
> rsync has an --block-size=SIZE Parameter but this adjusts only the
> checksum block size (and the maximum is 131072), not the i/o block size.
> Is there a solution to accelerate rsync?
Assuming sequential access to sufficiently large files, most reads
issued by a Lustre client with default setting should already be
RPC-sized, regardless of the size used in the read() syscall. Maybe
double-check your readahead settings?
Kind regards,
Daniel
--
Daniel Kobras
Principal Architect
Puzzle ITC Deutschland
+49 7071 14316 0
www.puzzle-itc.de
--
Puzzle ITC Deutschland GmbH
Sitz der Gesellschaft: Jurastr. 27/1, 72072
Tübingen
Eingetragen am Amtsgericht Stuttgart HRB 765802
Geschäftsführer:
Lukas Kallies, Daniel Kobras, Mark Pröhl
More information about the lustre-discuss
mailing list