[Lustre-discuss] backup MDT question

Nirmal Seenu nirmal at fnal.gov
Thu Apr 2 11:27:52 PDT 2009


The HUGE backup file size increase is due to the huge sparse files in 
the MDT. A new future that got slipped into Lustre 1.6.7???

To reduce the size of the backup file use the "--sparse" option in your 
tar command. This option only reduces the backup file size but not the 
time taken to tar up your backup as tar keeps reading a whole bunch of 
zeros from /dev/zero before it determines that the file is sparse.

To speed up the backup process and to reduce the backup file size, I did 
the following:

1. I reduced the size of the MDT partition from 150GB to 40GB.

2. Move the MDT over to the new partition and write ZEROs into all the 
unused space on the new MDT partition after mounting the MDT as type 
ldiskfs and using a script like:

let i=1

while [ $i -lt 36 ] ;
  do
    dd if=/dev/zero of=/mnt/new-mdt/tmp/gbfile$i bs=128k count=8192
    let i="$i+1"
  done


3. Create the backup using the following command after making a LVM 
snapshot of the mounted MDT:

dd if=/dev/lustrevol/mdtsnapshot  | gzip -c > mdt_backup_file

Hope this helps!
Nirmal



More information about the lustre-discuss mailing list