[Lustre-discuss] MDT estimated time for backup

[Brian J. Murrell]

On Wed, 2009-04-29 at 10:17 -0400, Ms. Megan Larko wrote:
> Hello List,

Hi,

> 
> I still have not yet backed up my MDT onto a larger  LV.   (My users
> like availability times in the monthly range; fortunately Lustre is
> good enough to provide that to them.)

That's good to hear.

> To execute the getfattr and
> tar used to take me 3 to 8 hours.    My last attempt had tar -cf
> running for over two days which was not acceptable to my users.

Has your dataset grown relative to those two different times?  Or has
the process gotten less efficient somehow?

> What would be a reasonable time estimate to back this up?

Of course, this is completely subjective on the hardware involved.

> Is
> getfattr and tar still preferred over rsync -aS

Well, I think the getfattr/tar method are time-tested.  I can't imagine
why an rsync (or several of them, more on that below) that includes
extended attributes (-X IIRC) in the sync wouldn't work either.  I don't
think we've done any testing of the rsync method however.

> or the getfattr and
> dump method?

I'm not so sure you want to use dump.  dump effectively takes an image
of the filesystem and restore that image exactly.  tar and rsync are
more like taking the content out of the filesystem and recreating it.

As for time to back up/restore.  When I need to make a backup or move a
filesystem (generally speaking here, not specific to Lustre) I do it in
several passes using rsync.  I do one full pass while the filesystem is
still live and being used -- no downtime.  I then do another on the live
filesystem -- no downtime -- just to get an idea of the time/data delta
and then I shut the filesystem down and do a final rsync.  The down time
is then only a function of the size of dataset that has changed since my
last "live" sync.  If I don't know when my downtime is scheduled for, I
continue doing periodic live runs just to keep the source and target as
closely synced as possible until I do schedule the downtime.

How well this maps to your MDT is for you to decide.  :-)

b.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: This is a digitally signed message part
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20090429/5809b187/attachment.pgp>