[lustre-discuss] rsync not appropriate for lustre

Daniel Kobras kobras at puzzle-itc.de
Fri Jan 24 00:51:44 PST 2020


Am 23.01.20 um 16:33 schrieb Bernd Melchers:
> we are copying large data sets within our lustre filesystem and between
> lustre and an external nfs server. In both cases the performance is
> unexpected low and the reason seems that rsync is reading and writing in
> 32 kB Blocks, whereas our lustre would be more happy with 4 MB
> Blocks.
> rsync has an --block-size=SIZE Parameter but this adjusts only the
> checksum block size (and the maximum is 131072), not the i/o block size.
> Is there a solution to accelerate rsync? 

Assuming sequential access to sufficiently large files, most reads
issued by a Lustre client with default setting should already be
RPC-sized, regardless of the size used in the read() syscall. Maybe
double-check your readahead settings?

Kind regards,

Daniel Kobras
Principal Architect
Puzzle ITC Deutschland
+49 7071 14316 0

Puzzle ITC Deutschland GmbH
Sitz der Gesellschaft: Jurastr. 27/1, 72072 

Eingetragen am Amtsgericht Stuttgart HRB 765802
Lukas Kallies, Daniel Kobras, Mark Pröhl

More information about the lustre-discuss mailing list