[Lustre-discuss] Tar backup of MDT runs extremely slow, tar pauses on pointers to very large files

[Peter Grandi]

[ ... ]

> The tar backup of the MDT is taking a very long time. So far it has
> backed up 1.6GB of the 5.0GB used in nine hours. In watching the tar
> process pointers to small or average size files are backed up quickly
> and at a consistent pace. When tar encounters a pointer/inode
> belonging to a very large file (100GB+) the tar process stalls on that
> file for a very long time, as if it were trying to archive the real
> filesize amount of data rather than the pointer/inode.

If you have stripes on, a 100GiB file will have 100,000 1MiB
stripes, and each requires a chunk of metadata. The descriptor
for that file will have this potentially a very large number of
extents, scattered around the MDT block device, depending on how
slowly the file grew etc.