[Lustre-devel] SAM-QFS, ADM, and Lustre HSM
Nathaniel Rutman
Nathan.Rutman at Sun.COM
Mon Jan 26 13:53:27 PST 2009
Andreas Dilger wrote:
> On Jan 23, 2009 10:46 -0600, Harriet G. Coverston wrote:
>
>> SAM supports classification policy rules for files -- (1) number of
>> copies, up to 4 (2) where to put the copies on which vsn pools -
>> disk and/or tape, local and/or remote) (3) when to make the copies
>> (time based archiving). You specify the policy in the archiver.cmd
>> file. You can group files for a policy rule by pathname, owner, group,
>> size, wildcard, and access time.
>>
My point about this is that files will be stored using the FID as the
file name, so name-based policies at the copytool level are worthless.
Unless we a.) add the path/filename back to the file (EA?), and b.)
modify the SAM policy engine to use the "real" path/filename instead of
the FID.
>> This brings up the question of restore. In case of a Lustre disk
>> failure, how are you going to restore your Lustre file system?
>>
>
> ...
>
> - since the archive does not contain a copy of the namespace (it only
> has 128-bit FIDs as identifiers for the file) we would need to make
> a separate backup of the MDS filesystem (which is all namespace).
> There are already several mechanisms to do this, either using the
> ext2 "dump" program to read from the raw device, or to make an LVM
> snapshot and use e.g. tar to make a filesystem-level backup. Both
> of these need to include a backup of the extended attributes.
>
Or include the path/filename in each file, and the restore process uses
this to repopulate the filesystem.
>
>> Agree. I don't see any SAM-QFS code changes required. The Lustre
>> copytool will write to HPSS using the HPSS APIs and write to SAM-QFS
>> with a ftp or pftp interface. This is minimum changes.
>>
>
> We weren't thinking of using an FTP interface to SAM, though I guess
> this is possible. Rather we were thinking of just mounting both QFS
> and Lustre on a Linux client and using "cp" or equivalent tool.
>
Harriet already knew this, she just forgot :)
More information about the lustre-devel
mailing list