<div dir="ltr">Sure, happy to help. I did not see mknod+setxattr in the strace output. Included is a trimmed version of the strace output, along with a few more bits of information. Thanks!<br><br># cat /proc/fs/lustre/version <br>lustre: 2.7.19.8<br># cat /etc/redhat-release <br>CentOS Linux release 7.3.1611 (Core) <br># uname -r<br>3.10.0-514.2.2.el7_lustre.x86_64<br># rpm -qa|grep tar<br>tar-1.26-31.el7.x86_64<br># sha1sum `which tar` `which gtar`<br>ea17ec98894212b2e2285eb2dd99aad76185ea7d /usr/bin/tar<br>ea17ec98894212b2e2285eb2dd99aad76185ea7d /usr/bin/gtar<br><br>Striping was set on the four directories before creating the files.<br>mkdir -p /scratch/1; lfs setstripe -c 1 --stripe-size 128K /scratch/1; lfs getstripe /scratch/1<br>mkdir -p /scratch/2; lfs setstripe -c 2 --stripe-size 256K /scratch/2; lfs getstripe /scratch/2<br>mkdir -p /scratch/3; lfs setstripe -c 3 --stripe-size 768K /scratch/3; lfs getstripe /scratch/3<br>mkdir -p /scratch/4; lfs setstripe -c 4 --stripe-size 1M /scratch/4; lfs getstripe /scratch/4<br>After tar, all files and directories all had the default Lustre striping.<br><br># tar ztvf /scratch.tgz <br>drwxr-xr-x root/root 0 2017-03-19 10:54 scratch/<br>drwxr-xr-x root/root 0 2017-03-19 10:57 scratch/4/<br>-rw-r--r-- root/root 4194304 2017-03-19 10:57 scratch/4/4.dd<br>drwxr-xr-x root/root 0 2017-03-19 10:57 scratch/3/<br>-rw-r--r-- root/root 4194304 2017-03-19 10:57 scratch/3/3.dd<br>drwxr-xr-x root/root 0 2017-03-19 10:57 scratch/1/<br>-rw-r--r-- root/root 4194304 2017-03-19 10:57 scratch/1/1.dd<br>drwxr-xr-x root/root 0 2017-03-19 10:57 scratch/2/<br>-rw-r--r-- root/root 4194304 2017-03-19 10:57 scratch/2/2.dd<br><br># strace tar zxvf /scratch.tgz > strace.out 2>&1<br>execve("/usr/bin/tar", ["tar", "zxvf", "/scratch.tgz"], [/* 22 vars */]) = 0<br>...<br>(-cut - loading libraries)<br>...<br>fstat(1, {st_mode=S_IFREG|0644, st_size=10187, ...}) = 0<br>mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f4a63d9f000<br>write(1, "scratch/\n", 9scratch/<br>) = 9<br>mkdirat(AT_FDCWD, "scratch", 0700) = -1 EEXIST (File exists)<br>newfstatat(AT_FDCWD, "scratch", {st_mode=S_IFDIR|0755, st_size=4096, ...}, AT_SYMLINK_NOFOLLOW) = 0<br>write(1, "scratch/4/\n", 11scratch/4/<br>) = 11<br>mkdirat(AT_FDCWD, "scratch/4", 0700) = 0<br>write(1, "scratch/4/4.dd\n", 15scratch/4/4.dd<br>) = 15<br>openat(AT_FDCWD, "scratch/4/4.dd", <br>read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 10240) = 10240<br>O_WRONLY|O_CREAT|O_EXCL|O_NOCTTY|O_NONBLOCK|O_CLOEXEC, 0600) = 4<br>write(4, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 5632) = 5632<br>read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 10240) = 10240<br>write(4, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 5632) = 5632<br>...<br>(-cut)<br>...<br>write(4, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 512) = 512<br>dup2(4, 4) = 4<br>fstat(4, {st_mode=S_IFREG|0600, st_size=4194304, ...}) = 0<br>utimensat(4, NULL, {{1489935825, 0}, {1489935444, 0}}, 0) = 0<br>fchown(4, 0, 0) = 0<br>fchmod(4, 0644) = 0<br>close(4) = 0<br>write(1, "scratch/3/\n", 11scratch/3/<br>) = 11<br>newfstatat(AT_FDCWD, "scratch/4", {st_mode=S_IFDIR|0700, st_size=4096, ...}, AT_SYMLINK_NOFOLLOW) = 0<br>utimensat(AT_FDCWD, "scratch/4", {{1489935825, 0}, {1489935444, 0}}, AT_SYMLINK_NOFOLLOW) = 0<br>fchownat(AT_FDCWD, "scratch/4", 0, 0, AT_SYMLINK_NOFOLLOW) = 0<br>fchmodat(AT_FDCWD, "scratch/4", 0755) = 0<br>mkdirat(AT_FDCWD, "scratch/3", 0700) = 0<br>write(1, "scratch/3/3.dd\n", 15scratch/3/3.dd<br>) = 15<br>openat(AT_FDCWD, "scratch/3/3.dd", O_WRONLY|O_CREAT|O_EXCL|O_NOCTTY|O_NONBLOCK|O_CLOEXEC, 0600) = 4<br>write(4, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 6656) = 6656<br>read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 10240) = 10240<br>write(4, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 10240) = 10240<br>...<br>(-cut - pick up with last file...)<br>...<br>d(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 10240) = 10240<br>--- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=2476, si_status=0, si_utime=7, si_stime=0} ---<br>write(4, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 10240) = 10240<br>read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 10240) = 10240<br>write(4, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 10240) = 10240<br>read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 10240) = 10240<br>write(4, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 10240) = 10240<br>read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 10240) = 10240<br>write(4, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 10240) = 10240<br>read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 10240) = 10240<br>write(4, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 10240) = 10240<br>read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 10240) = 10240<br>write(4, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 7680) = 7680<br>dup2(4, 4) = 4<br>fstat(4, {st_mode=S_IFREG|0600, st_size=4194304, ...}) = 0<br>utimensat(4, NULL, {{1489935825, 0}, {1489935432, 0}}, 0) = 0<br>fchown(4, 0, 0) = 0<br>fchmod(4, 0644) = 0<br>close(4) = 0<br>clock_gettime(CLOCK_REALTIME, {1489935825, 628399394}) = 0<br>clock_gettime(CLOCK_REALTIME, {1489935825, 628414336}) = 0<br>close(3) = 0<br>wait4(2476, [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], 0, NULL) = 2476<br>newfstatat(AT_FDCWD, "scratch/2", {st_mode=S_IFDIR|0700, st_size=4096, ...}, AT_SYMLINK_NOFOLLOW) = 0<br>utimensat(AT_FDCWD, "scratch/2", {{1489935825, 0}, {1489935432, 0}}, AT_SYMLINK_NOFOLLOW) = 0<br>fchownat(AT_FDCWD, "scratch/2", 0, 0, AT_SYMLINK_NOFOLLOW) = 0<br>fchmodat(AT_FDCWD, "scratch/2", 0755) = 0<br>newfstatat(AT_FDCWD, "scratch", {st_mode=S_IFDIR|0755, st_size=4096, ...}, 0) = 0<br>utimensat(AT_FDCWD, "scratch", {{1489934977, 0}, {1489935261, 0}}, 0) = 0<br>fchownat(AT_FDCWD, "scratch", 0, 0, 0) = 0<br>close(1) = 0<br>munmap(0x7f4a63d9f000, 4096) = 0<br>close(2) = 0<br>exit_group(0) = ?<br>+++ exited with 0 +++<br><br><div class="gmail_extra"><br clear="all"><div><div class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div>Brett</div><div>--<br></div><div dir="ltr"><span style="font-size:small">Protect Yourself </span><span style="font-size:small"><span><span style="font-size:small">Against</span></span> Cybercrime</span><br><div><div>PDS Software Solutions LLC</div><div><a href="https://www.trustpds.com/" target="_blank">https://www.TrustPDS.com</a></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div>
<br><div class="gmail_quote">On Sun, Mar 19, 2017 at 7:39 AM, Dilger, Andreas <span dir="ltr"><<a href="mailto:andreas.dilger@intel.com" target="_blank">andreas.dilger@intel.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="auto">
<div>I ran a test locally with RHEL 6.8 and the included tar 1.26 using strace, and tar is properly using mknod+setxattr to restore the "lov" xattr, and the stripe count and stripe size to be preserved. </div>
<div id="m_4334345014962728702AppleMailSignature"><br>
</div>
<div id="m_4334345014962728702AppleMailSignature">The OST index is not preserved with the xattr restore, since that may cause imbalance if the files were backed up in a different filesystem (e.g. one with fewer OSTs). The MDS will balance OST allocation as needed for the current
OST usage. </div>
<div id="m_4334345014962728702AppleMailSignature"><br>
</div>
<div id="m_4334345014962728702AppleMailSignature">Could you please run your tar on RHEL 7 with strace to see if it is doing this correctly?</div>
<div id="m_4334345014962728702AppleMailSignature"><br>
Cheers, Andreas</div><div><div class="h5">
<div><br>
On Mar 18, 2017, at 21:51, Brett Lee <<a href="mailto:brettlee.lustre@gmail.com" target="_blank">brettlee.lustre@gmail.com</a>> wrote:<br>
<br>
</div>
<blockquote type="cite">
<div>
<p>Hi Andreas, I expected that to be the case, but found out it was not. Instead, the restore restores everything - unless directed otherwise.</p>
<p>Backup == cmd + add xattrs.<br>
Restore == cmd + exclude xattrs.</p>
<p>Brett<br>
--<br>
Protect Yourself Against Cybercrime<br>
PDS Software Solutions LLC<br>
<a href="https://www.TrustPDS.com" target="_blank">https://www.TrustPDS.com</a></p>
<div class="gmail_quote">On Mar 18, 2017 9:28 PM, "Dilger, Andreas" <<a href="mailto:andreas.dilger@intel.com" target="_blank">andreas.dilger@intel.com</a>> wrote:<br type="attribution">
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="auto">
<div><span></span></div>
<div>
<div>Do you need to specify --xattrs (or similar) during the restore phase as well?<br>
<br>
Cheers, Andreas</div>
<div><br>
On Mar 17, 2017, at 15:12, Brett Lee <<a href="mailto:brettlee.lustre@gmail.com" target="_blank">brettlee.lustre@gmail.com</a>> wrote:<br>
<br>
</div>
<blockquote type="cite">
<div>
<div dir="ltr">
<div>Hi. In what I thought was a valid test, I was unable to confirm that a backup and restore retained the layouts. Perhaps my expectation or process was incorrect? The process was:<br>
<br>
1. Create 4 files, each with different stripe sizes and stripe counts (verified with getstripe).<br>
2. Back up the files using tar-1.26-31.el7.x86_64.<br>
3. Recreate a file system and restore the files.<br>
<br>
<div>Backup command: tar --xattrs -zcvf /scratch.tgz /scratch<br>
</div>
<div>Restore command: tar zxvf /scratch.tgz</div>
<br>
After restoration, getstripe showed that each file had the default stripe count (1) and stripe size (1MB).<br>
</div>
FWIW: After restoring, getfattr produced the same result for each file:<br>
# getfattr -d -m - -R <file><br>
lustre.lov=0s0AvRCwEAAAAdAAAAA<wbr>AAAAAAEAAACAAAAAAAQAAEAAAAFAAA<wbr>AAAAAAAAAAAAAAAAAAAAAAAAAAAA=<br>
trusted.link=0s3/HqEQEAAAAuAAA<wbr>AAAAAAAAAAAAAAAAAABYAAAACAAAEA<wbr>AAAAAUAAAAAMS5kZA==<br>
trusted.lma=0sAAAAAAAAAAAABAAA<wbr>AgAAAB0AAAAAAAAA<br>
trusted.lov=0s0AvRCwEAAAAdAAAA<wbr>AAAAAAAEAAACAAAAAAAQAAEAAAAFAA<wbr>AAAAAAAAAAAAAAAAAAAAAAAAAAAAA=<br>
<br>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div class="gmail_extra">
<div>
<div class="m_4334345014962728702m_6576155286495465332gmail_signature">
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div>Brett</div>
<div>--<br>
</div>
<div dir="ltr"><span style="font-size:small">Protect Yourself </span><span style="font-size:small"><span><span style="font-size:small">Against</span></span> Cybercrime</span><br>
<div>
<div>PDS Software Solutions LLC</div>
<div><a href="https://www.trustpds.com/" target="_blank">https://www.TrustPDS.com</a></div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<br>
<div class="gmail_quote">On Wed, Mar 15, 2017 at 5:03 AM, Dilger, Andreas <span dir="ltr">
<<a href="mailto:andreas.dilger@intel.com" target="_blank">andreas.dilger@intel.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div dir="auto">
<div>I believe Zmanda is already using GNU tar (or RHEL tar) for the actual backup storage? I that case it should already work, since we fixed tar long ago to backup and restore xattrs in a way that preserves Lustre layouts. <br>
<br>
Cheers, Andreas</div>
<div>
<div class="m_4334345014962728702m_6576155286495465332gmail-h5">
<div><br>
On Mar 14, 2017, at 15:47, Brett Lee <<a href="mailto:brettlee.lustre@gmail.com" target="_blank">brettlee.lustre@gmail.com</a>> wrote:<br>
<br>
</div>
<blockquote type="cite">
<div>
<div dir="ltr">Thanks for the details, Andreas!
<div><br>
</div>
<div>Maybe OpenSFS can fund Zmanda so that their backup software can include the Lustre metadata... :)<br>
<div class="gmail_extra"><br clear="all">
<div>
<div class="m_4334345014962728702m_6576155286495465332gmail-m_4365798426798410258gmail_signature">
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div>Brett</div>
<div>--<br>
</div>
<div dir="ltr"><span style="font-size:small">Protect Yourself </span><span style="font-size:small"><span><span style="font-size:small">Against</span></span> Cybercrime</span><br>
<div>
<div>PDS Software Solutions LLC</div>
<div><a href="https://www.trustpds.com/" target="_blank">https://www.TrustPDS.com</a></div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<br>
<div class="gmail_quote">On Tue, Mar 14, 2017 at 3:13 PM, Dilger, Andreas <span dir="ltr">
<<a href="mailto:andreas.dilger@intel.com" target="_blank">andreas.dilger@intel.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
To reply to this old thread, there are two different kinds of Lustre backup solutions:<br>
- file level backups that traverse the client POSIX filesystem, for which any number of<br>
commercial solutions exist. Making these solutions "capable of saving Lustre metadata"<br>
boils down to two simple things - save the "lustre.lov" xattr during backup (at a minimum,<br>
other xattrs also should be backed up), and then using mknod(2) + setxattr() to restore<br>
the "lustre.lov" xattr before opening the file and restoring the data.<br>
<br>
- device level backups (e.g. "dd" for ldiskfs, and "zfs send/recv" for ZFS).<br>
<br>
Using the file level backups allows backup/restore of subsets of the filesystem, since many<br>
HPC sites have Lustre filesystems that are too large to backup completely. I typically do<br>
not recommend to use device-level backups for the OSTs, unless doing an OST hardware migration,<br>
and even then it is probably less disruptive to do Lustre-level file migration off the OST<br>
before swapping it out.<br>
<br>
Whether file level backups are used or not, I would recommend sites always make periodic<br>
device level backups of the MDT(s). The amount of space needed for an MDT backup is small<br>
compared to the rest of the filesystem (e.g. a few TB at most), and can avoid the need for<br>
a full filesystem restore (e.g. multi-PB of data, if a full backup exists at all) even<br>
though all the data is still available on the OSTs.<br>
<br>
The MDT device-level backup can use relatively slow SATA drives, since they will mostly be<br>
used for linear writes (or occasionally linear reads for restore), so a few multi-TB SATA III<br>
drives is sufficient for storing a rotating set of MDT device backups. At 150MB/s for even<br>
a single SATA drive, this is about 2h/TB, which is reasonable to do once a week (or more often<br>
for smaller MDTs).<br>
<br>
While using an LVM snapshot of the ldiskfs MDT for the backup source is desirable for consistency<br>
reasons, having even an MDT backup from a mounted and in-use MDT is better than nothing at<br>
all when a problem is hit, since e2fsck can repair the in-use inconsistencies fairly easily,<br>
and Lustre can deal with inconsistencies between the MDT and OST reasonably (at most returning<br>
an -ENOENT error to the client for files that were deleted).<br>
<br>
Cheers, Andreas<br>
<span><br>
On Feb 7, 2017, at 12:32, Andrew Holway <<a href="mailto:andrew.holway@gmail.com" target="_blank">andrew.holway@gmail.com</a>> wrote:<br>
><br>
> Would it be difficult to suspend IO and snapshot all the nodes (assuming ZFS). Could you be sure that your MDS and OSS are synchronised?<br>
><br>
> On 7 February 2017 at 19:52, Mike Selway <<a href="mailto:mselway@cray.com" target="_blank">mselway@cray.com</a>> wrote:<br>
>> Hello Brett,<br>
>><br>
>> Actually, looking for someone who uses a commercialized approach (that retains user metadata and Lustre extended metadata) and not specifically the manual approaches of Chapter 17.<br>
>><br>
>> Thanks!<br>
>> Mike<br>
>><br>
>> Mike Selway | Sr. Tiered Storage Architect | Cray Inc.<br>
>> Work <a href="tel:%2B1-301-332-4116" value="+13013324116" target="_blank">+1-301-332-4116</a> |
<a href="mailto:mselway@cray.com" target="_blank">mselway@cray.com</a><br>
>> 146 Castlemaine Ct, Castle Rock, CO 80104 | <a href="http://www.cray.com" rel="noreferrer" target="_blank">
www.cray.com</a><br>
>><br>
>><br>
</span><span>>>> From: Brett Lee [mailto:<a href="mailto:brettlee.lustre@gmail.com" target="_blank">brettlee.lustre@gmail.<wbr>com</a>]<br>
>>> Sent: Monday, February 06, 2017 11:45 AM<br>
>>> To: Mike Selway <<a href="mailto:mselway@cray.com" target="_blank">mselway@cray.com</a>><br>
>>> Cc: <a href="mailto:lustre-discuss@lists.lustre.org" target="_blank">lustre-discuss@lists.lustre.or<wbr>g</a><br>
>>> Subject: Re: [lustre-discuss] Backup software for Lustre<br>
>>><br>
>>> Hey Mike,<br>
>>><br>
>>> "Chapter 17" and<br>
>>> <a href="http://www.intel.com/content/www/us/en/lustre/backup-and-restore-training.html" rel="noreferrer" target="_blank">
http://www.intel.com/content/w<wbr>ww/us/en/lustre/backup-and-res<wbr>tore-training.html</a><br>
>>><br>
>>> both contain methods to backup & restore the entire Lustre file system.<br>
>>><br>
>>> Are you looking for a solution that backs up only the (user) data files and their associated metadata (e.g. xattrs)?<br>
>>><br>
>>> Brett<br>
>>> --<br>
>>> Protect Yourself From Cybercrime<br>
>>> PDS Software Solutions LLC<br>
>>> <a href="https://www.TrustPDS.com" rel="noreferrer" target="_blank">https://www.TrustPDS.com</a><br>
>>><br>
>>>> On Mon, Feb 6, 2017 at 11:12 AM, Mike Selway <<a href="mailto:mselway@cray.com" target="_blank">mselway@cray.com</a>> wrote:<br>
>>>><br>
>>>> Hello,<br>
>>>> Anyone aware of and/or using a Backup software package to protect their LFS environment (not referring to the tools/scripts suggested in Chapter 17).<br>
>>>><br>
>>>> Regards,<br>
>>>> Mike<br>
<br>
</span>Cheers, Andreas<br>
--<br>
Andreas Dilger<br>
Lustre Principal Architect<br>
Intel Corporation<br>
<div class="m_4334345014962728702m_6576155286495465332gmail-m_4365798426798410258HOEnZb">
<div class="m_4334345014962728702m_6576155286495465332gmail-m_4365798426798410258h5"><br>
<br>
<br>
<br>
<br>
<br>
<br>
______________________________<wbr>_________________<br>
lustre-discuss mailing list<br>
<a href="mailto:lustre-discuss@lists.lustre.org" target="_blank">lustre-discuss@lists.lustre.or<wbr>g</a><br>
<a href="http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org" rel="noreferrer" target="_blank">http://lists.lustre.org/listin<wbr>fo.cgi/lustre-discuss-lustre.o<wbr>rg</a><br>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</blockquote>
</div>
</div>
</blockquote>
</div></div></div>
</blockquote></div><br></div></div>