[Lustre-devel] Lustre HSM - some talking points.
Colin Ngam
Colin.Ngam at Sun.COM
Mon Feb 2 11:25:17 PST 2009
Vicky White wrote:
>
>> 6. No namepace. No namespace. Lustre pathnames can be stored as
>> Extended
>> Attributes.
>
>
> I realize you are talking about SAMQFS, but do we want to keep the
> design consistent with what we do for hpss?
>
> HPSS will support Extended Attributes in release 7.2, to be available
> September 2009, but it will be expensive to use these EAs for
> pathnames, though it can be done. The current design is to specify
> EAs in XML, and for the most efficient storage of the EA in the
> database, the containing XML document needs to be 512 bytes or fewer
> so that it can be stored inline. Larger XML objects will have to be
> stored externally as LOBs (large objects), which will make queries
> cost more.
>
> So we need to think about what that cost will be when we are
> considering repositories with millions or billions of files.
>
> Vicky
Hi Vicky,
I do not see why we need to query these EAs in normal operation. These
EAs will only be accessed when we need to perform Ultimate Disaster
Recovery - when you have lost all data on disks and all you have are tapes.
I was thinking about XML - but it is "opaque" to Object SAMQFS so, it is
up to the Lustre side. Whatever it is, the Applications -
Lustre-Restore for example, is the one that has to understand the
format. I am not a Tar Header expert - but I assume that these EAs can
go with the file in the tar ball. I do not expect to keep any of it on
line on disk cache, on the SAMFS side. I see no reason.
With respect to whether it should be consistent with HPSS - I would say
if that is all we need and it is sufficient - why not. Otherwise, let's
make it better than HPSS. It must be in SUN's best interest to sell SAMQFS?
I do apologize Vicky, are you a SUN employee?
Thanks.
colin
More information about the lustre-devel
mailing list