[Lustre-discuss] Tar backup of MDT runs extremely slow, tar pauses on pointers to very large files

[Andreas Dilger]

On 2012-05-29, at 1:28 PM, Peter Grandi wrote:
>> The tar backup of the MDT is taking a very long time. So far it has
>> backed up 1.6GB of the 5.0GB used in nine hours. In watching the tar
>> process pointers to small or average size files are backed up quickly
>> and at a consistent pace. When tar encounters a pointer/inode
>> belonging to a very large file (100GB+) the tar process stalls on that
>> file for a very long time, as if it were trying to archive the real
>> filesize amount of data rather than the pointer/inode.
> 
> If you have stripes on, a 100GiB file will have 100,000 1MiB
> stripes, and each requires a chunk of metadata. The descriptor
> for that file will have this potentially a very large number of
> extents, scattered around the MDT block device, depending on how
> slowly the file grew etc.

While that may be true for other distributed filesystems, that is
not true for Lustre at all.  The size of a Lustre object is not
fixed to a "chunk size" like 32MB or similar, but rather is
variable depending on the size of the file itself.  The number of
"stripes" (== objects) on a file is currently fixed at file
creation time, and the MDS only needs to store the location of
each stripe (at most one per OST).  The actual blocks/extents of
the objects are managed inside the OST itself and are never seen
by the client or the MDS.

Cheers, Andreas
--
Andreas Dilger                       Whamcloud, Inc.
Principal Lustre Engineer            http://www.whamcloud.com/