[Lustre-discuss] MDT backup

[Jim Garlick]

On Tue, Feb 03, 2009 at 11:47:37PM +0530, Kalpak Shah wrote:
> On Tue, 2009-02-03 at 10:57 -0700, Nathan Dauchy wrote:
> > Brian J. Murrell wrote:
> > > On Mon, 2009-01-19 at 22:46 -0600, Alex Kulyavtsev wrote:
> > >> Hi,
> > >> what is the right way to backup MDT ?
> > > 
> > > I think this is covered in the operations manual.
> > > 
> > 
> > The manual does indeed cover backups:
> > http:// manual.lustre.org/manual/LustreManual16_HTML/BackupAndRestore.html
> > 
> > It includes this step:
> > 
> > >  3. Back up the EAs, run:
> > > getfattr -R -d -m '.*' -P . > ea.bak
> > > 
> > 
> > Is that necessary for newer versions of tar that support the "-xattrs"
> > option?  From the man page (I have GNU tar 1.15.1):
> > 
> > --xattrs	this  option  causes  tar  to  store  each  file's
> > 		extended  attributes in the archive. This option also
> > 		enables --acls and--selinux if they haven't been
> > 		set already, due to the fact that the data for those
> > 		are stored in special xattrs.
> > 
> > Also, AFAIK, cpio will archive extended attributes as well.
> > 
> > Is either 'tar --xattrs' or 'cpio' sufficient for capturing all the
> > Lustre MDT information?  Would that eliminate the need for the
> > 'getfattr' step, and cut the time for backup almost in half by requiring
> > only one pass through the filesystem?
> 
> The "tar" package shipped with distros cannot recognize "lustre."
> extended attributes and hence cannot be used for Lustre backups.
> 
> You will need to use Lustre patched tar available for download at
> http:// downloads.lustre.org/public/tools/lustre-tar/

I just did some testing with rsync-2.6.8-3.1 from rhel 5 and it appears
to support Lustre's extended attributes with its -X (--xattrs) option.

The test I ran was as follows:

See if initial copy looks correct:
- create some files in lustre (nonzero length)
- stop lustre and remount mdt with -t ldiskfs on /mnt/lustre/mdt
- create an ldiskfs file system on the MDS with -Oext_attr and mount
  on /mnt/lustre/mdt-backup
- rsync -Xav --delete /mnt/lustre/mdt/ /mnt/lustre/mdt-backup
- verify with getfattr -d -m '.*' -P that attrs are identical on all files
  in both copies

Make sure rsync updates file/dir if only attrs change
- unmount /mnt/lustre/mdt and restart lustre
- run lfs setstripe --count -1 on an existing directory in lustre
- stop lustre and remount mdt with -t ldiskfs on /mnt/lustre/mdt
- verify with getfattr that attrs differ on that dir in both copies
- repeat rsync
- verify with getfattr that attrs are identical on that dir in both copies

Verify that a restore works:
- rm -rf /mnt/lustre/mdt/*
- rsync -Xav --delete /mnt/lustre/mdt-backup/ /mnt/lustre/mdt
- unmount /mnt/lustre/mdt and restart lustre
- verify that md5sums of files created in first test match originals

Jim