[Lustre-discuss] [wc-discuss] Tar backup of MDT runs extremely slow, tar pauses on pointers to very large files

[Richard Henwood]

On Tue, May 29, 2012 at 7:30 AM, Jeremy Filizetti
<jeremy.filizetti at gmail.com> wrote:
> Your probably using a version of tar which doesn't check the number of
> blocks used by the file to determine if it is entirely sparse.  Without that
> optimization you will read the entire file (of zero fill) to determine if it
> is sparse.  GNU tar >= 1.24 seems to have the code for this but lustre-tar
> on Whamcloud's site doesn't.
>

Hi All,

Whamcloud provide a modified version of tar 1.19 that includes the
'sparse files' optimization. It has been targeted at RHEL5 but is
expected to work on RHEL6 as well. I've made some notes on the this
work:

http://wiki.whamcloud.com/display/PUB/Lustre+Tools#LustreTools-lustretar

As you point out, the sparse files optimization is included since gnu
tar 1.24 (but, was not documented in the release notes if I remember
correctly). I have been talking to RedHat to include the spase files
patch and I expect RHEL 6.3 will have a suitable change in tar to
achieve this behavior.

cheers,
Richard

<snip>
-- 
Richard.Henwood at whamcloud.com
Whamcloud Inc.
tel: +1 512 410 9612