[Lustre-discuss] Tar backup of MDT runs extremely slow, tar pauses on pointers to very large files
adilger at whamcloud.com
Wed May 30 15:02:41 PDT 2012
On 2012-05-29, at 1:28 PM, Peter Grandi wrote:
>> The tar backup of the MDT is taking a very long time. So far it has
>> backed up 1.6GB of the 5.0GB used in nine hours. In watching the tar
>> process pointers to small or average size files are backed up quickly
>> and at a consistent pace. When tar encounters a pointer/inode
>> belonging to a very large file (100GB+) the tar process stalls on that
>> file for a very long time, as if it were trying to archive the real
>> filesize amount of data rather than the pointer/inode.
> If you have stripes on, a 100GiB file will have 100,000 1MiB
> stripes, and each requires a chunk of metadata. The descriptor
> for that file will have this potentially a very large number of
> extents, scattered around the MDT block device, depending on how
> slowly the file grew etc.
While that may be true for other distributed filesystems, that is
not true for Lustre at all. The size of a Lustre object is not
fixed to a "chunk size" like 32MB or similar, but rather is
variable depending on the size of the file itself. The number of
"stripes" (== objects) on a file is currently fixed at file
creation time, and the MDS only needs to store the location of
each stripe (at most one per OST). The actual blocks/extents of
the objects are managed inside the OST itself and are never seen
by the client or the MDS.
Andreas Dilger Whamcloud, Inc.
Principal Lustre Engineer http://www.whamcloud.com/
More information about the lustre-discuss