[lustre-discuss] Backup software for Lustre

[Brett Lee]

Thanks for the details, Andreas!

Maybe OpenSFS can fund Zmanda so that their backup software can include the
Lustre metadata... :)

Brett
--
Protect Yourself Against Cybercrime
PDS Software Solutions LLC
https://www.TrustPDS.com <https://www.trustpds.com/>

On Tue, Mar 14, 2017 at 3:13 PM, Dilger, Andreas <andreas.dilger at intel.com>
wrote:

> To reply to this old thread, there are two different kinds of Lustre
> backup solutions:
> - file level backups that traverse the client POSIX filesystem, for which
> any number of
>   commercial solutions exist.  Making these solutions "capable of saving
> Lustre metadata"
>   boils down to two simple things - save the "lustre.lov" xattr during
> backup (at a minimum,
>   other xattrs also should be backed up), and then using mknod(2) +
> setxattr() to restore
>   the "lustre.lov" xattr before opening the file and restoring the data.
>
> - device level backups (e.g. "dd" for ldiskfs, and "zfs send/recv" for
> ZFS).
>
> Using the file level backups allows backup/restore of subsets of the
> filesystem, since many
> HPC sites have Lustre filesystems that are too large to backup
> completely.  I typically do
> not recommend to use device-level backups for the OSTs, unless doing an
> OST hardware migration,
> and even then it is probably less disruptive to do Lustre-level file
> migration off the OST
> before swapping it out.
>
> Whether file level backups are used or not, I would recommend sites always
> make periodic
> device level backups of the MDT(s).  The amount of space needed for an MDT
> backup is small
> compared to the rest of the filesystem (e.g. a few TB at most), and can
> avoid the need for
> a full filesystem restore (e.g. multi-PB of data, if a full backup exists
> at all) even
> though all the data is still available on the OSTs.
>
> The MDT device-level backup can use relatively slow SATA drives, since
> they will mostly be
> used for linear writes (or occasionally linear reads for restore), so a
> few multi-TB SATA III
> drives is sufficient for storing a rotating set of MDT device backups.  At
> 150MB/s for even
> a single SATA drive, this is about 2h/TB, which is reasonable to do once a
> week (or more often
> for smaller MDTs).
>
> While using an LVM snapshot of the ldiskfs MDT for the backup source is
> desirable for consistency
> reasons, having even an MDT backup from a mounted and in-use MDT is better
> than nothing at
> all when a problem is hit, since e2fsck can repair the in-use
> inconsistencies fairly easily,
> and Lustre can deal with inconsistencies between the MDT and OST
> reasonably (at most returning
> an -ENOENT error to the client for files that were deleted).
>
> Cheers, Andreas
>
> On Feb 7, 2017, at 12:32, Andrew Holway <andrew.holway at gmail.com> wrote:
> >
> > Would it be difficult to suspend IO and snapshot all the nodes (assuming
> ZFS). Could you be sure that your MDS and OSS are synchronised?
> >
> > On 7 February 2017 at 19:52, Mike Selway <mselway at cray.com> wrote:
> >> Hello Brett,
> >>
> >>                Actually, looking for someone who uses a commercialized
> approach (that retains user metadata and Lustre extended metadata) and not
> specifically the manual approaches of Chapter 17.
> >>
> >> Thanks!
> >> Mike
> >>
> >> Mike Selway | Sr. Tiered Storage Architect | Cray Inc.
> >> Work +1-301-332-4116 | mselway at cray.com
> >> 146 Castlemaine Ct,   Castle Rock,  CO  80104 | www.cray.com
> >>
> >>
> >>> From: Brett Lee [mailto:brettlee.lustre at gmail.com]
> >>> Sent: Monday, February 06, 2017 11:45 AM
> >>> To: Mike Selway <mselway at cray.com>
> >>> Cc: lustre-discuss at lists.lustre.org
> >>> Subject: Re: [lustre-discuss] Backup software for Lustre
> >>>
> >>> Hey Mike,
> >>>
> >>> "Chapter 17" and
> >>> http://www.intel.com/content/www/us/en/lustre/backup-and-
> restore-training.html
> >>>
> >>> both contain methods to backup & restore the entire Lustre file system.
> >>>
> >>> Are you looking for a solution that backs up only the (user) data
> files and their associated metadata (e.g. xattrs)?
> >>>
> >>> Brett
> >>> --
> >>> Protect Yourself From Cybercrime
> >>> PDS Software Solutions LLC
> >>> https://www.TrustPDS.com
> >>>
> >>>> On Mon, Feb 6, 2017 at 11:12 AM, Mike Selway <mselway at cray.com>
> wrote:
> >>>>
> >>>> Hello,
> >>>>          Anyone aware of and/or using a Backup software package to
> protect their LFS environment (not referring to the tools/scripts suggested
> in Chapter 17).
> >>>>
> >>>> Regards,
> >>>> Mike
>
> Cheers, Andreas
> --
> Andreas Dilger
> Lustre Principal Architect
> Intel Corporation
>
>
>
>
>
>
>
> _______________________________________________
> lustre-discuss mailing list
> lustre-discuss at lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20170314/f43999ba/attachment.htm>