[Lustre-discuss] backup MDT question
Nirmal Seenu
nirmal at fnal.gov
Thu Apr 2 11:27:52 PDT 2009
The HUGE backup file size increase is due to the huge sparse files in
the MDT. A new future that got slipped into Lustre 1.6.7???
To reduce the size of the backup file use the "--sparse" option in your
tar command. This option only reduces the backup file size but not the
time taken to tar up your backup as tar keeps reading a whole bunch of
zeros from /dev/zero before it determines that the file is sparse.
To speed up the backup process and to reduce the backup file size, I did
the following:
1. I reduced the size of the MDT partition from 150GB to 40GB.
2. Move the MDT over to the new partition and write ZEROs into all the
unused space on the new MDT partition after mounting the MDT as type
ldiskfs and using a script like:
let i=1
while [ $i -lt 36 ] ;
do
dd if=/dev/zero of=/mnt/new-mdt/tmp/gbfile$i bs=128k count=8192
let i="$i+1"
done
3. Create the backup using the following command after making a LVM
snapshot of the mounted MDT:
dd if=/dev/lustrevol/mdtsnapshot | gzip -c > mdt_backup_file
Hope this helps!
Nirmal
More information about the lustre-discuss
mailing list