[Lustre-devel] HSM arch wiki
Nathan.Rutman at Sun.COM
Wed Nov 26 08:29:56 PST 2008
Alex Kulyavtsev wrote:
> Few questions :
> - For large existing archive of tapes (~10,000,000 files) it is
> desirable to import file metadata to lustre fs without actually
> copying files on disk.
> Import shall be done in reasonable time (hours rather than month) or
Agreed. Probably best done via a special ioctl that would create a stub
file and populate the metadata.
> - to provide bandwidth to tape it is desirable to have multiple
> migrator nodes connected to HSM. What element of proposed design
> distributes copy-out processes across migrator nodes to provide
> scalability ? Is it functionality of HSM specific copy tool or does
> lustre agent provide it ?
Lustre agents can run on multiple Lustre clients in parallel.
Coordinator distributes copyout jobs to different agents.
> - a "smart" HSM system can reorder requests to optimize tape access.
> It is common to have 2000 requests pending in queue with tens or
> hundreds IO transfers actually served. Current limit of pending
> requests is about 30,000. We found implementing of pending requests as
> processes (one copy-out tool process per request waiting for IO) is
> resource consuming and is not scalable. What is the way to serve
> ~100,000 request waiting for transfer ?
Coordinator decides when to request copyin/out jobs, and could throttle
the total number of concurrent accesses.
> - how to prestage files ? Send asynchronous request for copy-in file
> from tape without blocking on wait. It is needed to stage large data
> sets for future processing. Prestaging "file sets" is desirable.
Policy engine would request copyin of files before cache miss on open.
Policy could define file sets.
> - what proposed scanario to handle OST down ? Suppose file is present
> on one of OSTs and it went down (striping is one). My understanding is
> client will wait when OST will come back (case) and file will not
> be staged from tape automatically. IF file is not present on any OST,
> it will be staged immediately (case). Is possible to stage file
> automatically (case) to another OST and mark a copy on old OST for
> removal ?
With our V2 HSM, we will have the ability to keep more detailed layouts;
this optimization could be part of those changes.
> We discussed some of these questions with Peter, he suggested to ask
> on devel list.
We greatly appreciate it! Please ask/suggest away.
> Best regards, Alex.
> Nathaniel Rutman wrote:
>> High-level architecture page for the Lustre HSM project
>> HSM core team - this is intended to be sufficient to write a full
>> HLD/DLD from. What is it missing?
>> Lustre-devel mailing list
>> Lustre-devel at lists.lustre.org
More information about the lustre-devel