<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
</head>
<body dir="auto">
<div>I ran a test locally with RHEL 6.8 and the included tar 1.26 using strace, and tar is properly using mknod+setxattr to restore the "lov" xattr, and the stripe count and stripe size to be preserved. </div>
<div id="AppleMailSignature"><br>
</div>
<div id="AppleMailSignature">The OST index is not preserved with the xattr restore, since that may cause imbalance if the files were backed up in a different filesystem (e.g. one with fewer OSTs). The MDS will balance OST allocation as needed for the current
OST usage. </div>
<div id="AppleMailSignature"><br>
</div>
<div id="AppleMailSignature">Could you please run your tar on RHEL 7 with strace to see if it is doing this correctly?</div>
<div id="AppleMailSignature"><br>
Cheers, Andreas</div>
<div><br>
On Mar 18, 2017, at 21:51, Brett Lee <<a href="mailto:brettlee.lustre@gmail.com">brettlee.lustre@gmail.com</a>> wrote:<br>
<br>
</div>
<blockquote type="cite">
<div>
<p>Hi Andreas, I expected that to be the case, but found out it was not. Instead, the restore restores everything - unless directed otherwise.</p>
<p>Backup == cmd + add xattrs.<br>
Restore == cmd + exclude xattrs.</p>
<p>Brett<br>
--<br>
Protect Yourself Against Cybercrime<br>
PDS Software Solutions LLC<br>
<a href="https://www.TrustPDS.com">https://www.TrustPDS.com</a></p>
<div class="gmail_quote">On Mar 18, 2017 9:28 PM, "Dilger, Andreas" <<a href="mailto:andreas.dilger@intel.com">andreas.dilger@intel.com</a>> wrote:<br type="attribution">
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="auto">
<div><span></span></div>
<div>
<div>Do you need to specify --xattrs (or similar) during the restore phase as well?<br>
<br>
Cheers, Andreas</div>
<div><br>
On Mar 17, 2017, at 15:12, Brett Lee <<a href="mailto:brettlee.lustre@gmail.com" target="_blank">brettlee.lustre@gmail.com</a>> wrote:<br>
<br>
</div>
<blockquote type="cite">
<div>
<div dir="ltr">
<div>Hi. In what I thought was a valid test, I was unable to confirm that a backup and restore retained the layouts. Perhaps my expectation or process was incorrect? The process was:<br>
<br>
1. Create 4 files, each with different stripe sizes and stripe counts (verified with getstripe).<br>
2. Back up the files using tar-1.26-31.el7.x86_64.<br>
3. Recreate a file system and restore the files.<br>
<br>
<div>Backup command: tar --xattrs -zcvf /scratch.tgz /scratch<br>
</div>
<div>Restore command: tar zxvf /scratch.tgz</div>
<br>
After restoration, getstripe showed that each file had the default stripe count (1) and stripe size (1MB).<br>
</div>
FWIW: After restoring, getfattr produced the same result for each file:<br>
# getfattr -d -m - -R <file><br>
lustre.lov=<wbr>0s0AvRCwEAAAAdAAAAAAAAAAAEAAAC<wbr>AAAAAAAQAAEAAAAFAAAAAAAAAAAAAA<wbr>AAAAAAAAAAAAAAAAA=<br>
trusted.link=0s3/<wbr>HqEQEAAAAuAAAAAAAAAAAAAAAAAAAA<wbr>ABYAAAACAAAEAAAAAAUAAAAAMS5kZA<wbr>==<br>
trusted.lma=<wbr>0sAAAAAAAAAAAABAAAAgAAAB0AAAAA<wbr>AAAA<br>
trusted.lov=<wbr>0s0AvRCwEAAAAdAAAAAAAAAAAEAAAC<wbr>AAAAAAAQAAEAAAAFAAAAAAAAAAAAAA<wbr>AAAAAAAAAAAAAAAAA=<br>
<br>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div class="gmail_extra">
<div>
<div class="m_6576155286495465332gmail_signature">
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div>Brett</div>
<div>--<br>
</div>
<div dir="ltr"><span style="font-size:small">Protect Yourself </span><span style="font-size:small"><span><span style="font-size:small">Against</span></span> Cybercrime</span><br>
<div>
<div>PDS Software Solutions LLC</div>
<div><a href="https://www.trustpds.com/" target="_blank">https://www.TrustPDS.com</a></div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<br>
<div class="gmail_quote">On Wed, Mar 15, 2017 at 5:03 AM, Dilger, Andreas <span dir="ltr">
<<a href="mailto:andreas.dilger@intel.com" target="_blank">andreas.dilger@intel.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div dir="auto">
<div>I believe Zmanda is already using GNU tar (or RHEL tar) for the actual backup storage? I that case it should already work, since we fixed tar long ago to backup and restore xattrs in a way that preserves Lustre layouts. <br>
<br>
Cheers, Andreas</div>
<div>
<div class="m_6576155286495465332gmail-h5">
<div><br>
On Mar 14, 2017, at 15:47, Brett Lee <<a href="mailto:brettlee.lustre@gmail.com" target="_blank">brettlee.lustre@gmail.com</a>> wrote:<br>
<br>
</div>
<blockquote type="cite">
<div>
<div dir="ltr">Thanks for the details, Andreas!
<div><br>
</div>
<div>Maybe OpenSFS can fund Zmanda so that their backup software can include the Lustre metadata... :)<br>
<div class="gmail_extra"><br clear="all">
<div>
<div class="m_6576155286495465332gmail-m_4365798426798410258gmail_signature">
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div>Brett</div>
<div>--<br>
</div>
<div dir="ltr"><span style="font-size:small">Protect Yourself </span><span style="font-size:small"><span><span style="font-size:small">Against</span></span> Cybercrime</span><br>
<div>
<div>PDS Software Solutions LLC</div>
<div><a href="https://www.trustpds.com/" target="_blank">https://www.TrustPDS.com</a></div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<br>
<div class="gmail_quote">On Tue, Mar 14, 2017 at 3:13 PM, Dilger, Andreas <span dir="ltr">
<<a href="mailto:andreas.dilger@intel.com" target="_blank">andreas.dilger@intel.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
To reply to this old thread, there are two different kinds of Lustre backup solutions:<br>
- file level backups that traverse the client POSIX filesystem, for which any number of<br>
commercial solutions exist. Making these solutions "capable of saving Lustre metadata"<br>
boils down to two simple things - save the "lustre.lov" xattr during backup (at a minimum,<br>
other xattrs also should be backed up), and then using mknod(2) + setxattr() to restore<br>
the "lustre.lov" xattr before opening the file and restoring the data.<br>
<br>
- device level backups (e.g. "dd" for ldiskfs, and "zfs send/recv" for ZFS).<br>
<br>
Using the file level backups allows backup/restore of subsets of the filesystem, since many<br>
HPC sites have Lustre filesystems that are too large to backup completely. I typically do<br>
not recommend to use device-level backups for the OSTs, unless doing an OST hardware migration,<br>
and even then it is probably less disruptive to do Lustre-level file migration off the OST<br>
before swapping it out.<br>
<br>
Whether file level backups are used or not, I would recommend sites always make periodic<br>
device level backups of the MDT(s). The amount of space needed for an MDT backup is small<br>
compared to the rest of the filesystem (e.g. a few TB at most), and can avoid the need for<br>
a full filesystem restore (e.g. multi-PB of data, if a full backup exists at all) even<br>
though all the data is still available on the OSTs.<br>
<br>
The MDT device-level backup can use relatively slow SATA drives, since they will mostly be<br>
used for linear writes (or occasionally linear reads for restore), so a few multi-TB SATA III<br>
drives is sufficient for storing a rotating set of MDT device backups. At 150MB/s for even<br>
a single SATA drive, this is about 2h/TB, which is reasonable to do once a week (or more often<br>
for smaller MDTs).<br>
<br>
While using an LVM snapshot of the ldiskfs MDT for the backup source is desirable for consistency<br>
reasons, having even an MDT backup from a mounted and in-use MDT is better than nothing at<br>
all when a problem is hit, since e2fsck can repair the in-use inconsistencies fairly easily,<br>
and Lustre can deal with inconsistencies between the MDT and OST reasonably (at most returning<br>
an -ENOENT error to the client for files that were deleted).<br>
<br>
Cheers, Andreas<br>
<span><br>
On Feb 7, 2017, at 12:32, Andrew Holway <<a href="mailto:andrew.holway@gmail.com" target="_blank">andrew.holway@gmail.com</a>> wrote:<br>
><br>
> Would it be difficult to suspend IO and snapshot all the nodes (assuming ZFS). Could you be sure that your MDS and OSS are synchronised?<br>
><br>
> On 7 February 2017 at 19:52, Mike Selway <<a href="mailto:mselway@cray.com" target="_blank">mselway@cray.com</a>> wrote:<br>
>> Hello Brett,<br>
>><br>
>> Actually, looking for someone who uses a commercialized approach (that retains user metadata and Lustre extended metadata) and not specifically the manual approaches of Chapter 17.<br>
>><br>
>> Thanks!<br>
>> Mike<br>
>><br>
>> Mike Selway | Sr. Tiered Storage Architect | Cray Inc.<br>
>> Work <a href="tel:%2B1-301-332-4116" value="+13013324116" target="_blank">+1-301-332-4116</a> |
<a href="mailto:mselway@cray.com" target="_blank">mselway@cray.com</a><br>
>> 146 Castlemaine Ct, Castle Rock, CO 80104 | <a href="http://www.cray.com" rel="noreferrer" target="_blank">
www.cray.com</a><br>
>><br>
>><br>
</span><span>>>> From: Brett Lee [mailto:<a href="mailto:brettlee.lustre@gmail.com" target="_blank">brettlee.lustre@gmail.<wbr>com</a>]<br>
>>> Sent: Monday, February 06, 2017 11:45 AM<br>
>>> To: Mike Selway <<a href="mailto:mselway@cray.com" target="_blank">mselway@cray.com</a>><br>
>>> Cc: <a href="mailto:lustre-discuss@lists.lustre.org" target="_blank">lustre-discuss@lists.lustre.or<wbr>g</a><br>
>>> Subject: Re: [lustre-discuss] Backup software for Lustre<br>
>>><br>
>>> Hey Mike,<br>
>>><br>
>>> "Chapter 17" and<br>
>>> <a href="http://www.intel.com/content/www/us/en/lustre/backup-and-restore-training.html" rel="noreferrer" target="_blank">
http://www.intel.com/content/w<wbr>ww/us/en/lustre/backup-and-res<wbr>tore-training.html</a><br>
>>><br>
>>> both contain methods to backup & restore the entire Lustre file system.<br>
>>><br>
>>> Are you looking for a solution that backs up only the (user) data files and their associated metadata (e.g. xattrs)?<br>
>>><br>
>>> Brett<br>
>>> --<br>
>>> Protect Yourself From Cybercrime<br>
>>> PDS Software Solutions LLC<br>
>>> <a href="https://www.TrustPDS.com" rel="noreferrer" target="_blank">https://www.TrustPDS.com</a><br>
>>><br>
>>>> On Mon, Feb 6, 2017 at 11:12 AM, Mike Selway <<a href="mailto:mselway@cray.com" target="_blank">mselway@cray.com</a>> wrote:<br>
>>>><br>
>>>> Hello,<br>
>>>> Anyone aware of and/or using a Backup software package to protect their LFS environment (not referring to the tools/scripts suggested in Chapter 17).<br>
>>>><br>
>>>> Regards,<br>
>>>> Mike<br>
<br>
</span>Cheers, Andreas<br>
--<br>
Andreas Dilger<br>
Lustre Principal Architect<br>
Intel Corporation<br>
<div class="m_6576155286495465332gmail-m_4365798426798410258HOEnZb">
<div class="m_6576155286495465332gmail-m_4365798426798410258h5"><br>
<br>
<br>
<br>
<br>
<br>
<br>
______________________________<wbr>_________________<br>
lustre-discuss mailing list<br>
<a href="mailto:lustre-discuss@lists.lustre.org" target="_blank">lustre-discuss@lists.lustre.or<wbr>g</a><br>
<a href="http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org" rel="noreferrer" target="_blank">http://lists.lustre.org/listin<wbr>fo.cgi/lustre-discuss-lustre.o<wbr>rg</a><br>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</blockquote>
</div>
</div>
</blockquote>
</body>
</html>