[Lustre-devel] Lustre HSM - some talking points.

Colin Ngam Colin.Ngam at Sun.COM
Mon Feb 2 11:25:17 PST 2009

Vicky White wrote:
>> 6.  No namepace.  No namespace.  Lustre pathnames can be stored as 
>> Extended
>> Attributes.
> I realize you are talking about SAMQFS, but do we want to keep the 
> design consistent with what we do for hpss?
> HPSS will support Extended Attributes in release 7.2, to be available 
> September 2009, but it will be expensive to use these EAs for 
> pathnames, though it can be done.   The current design is to specify 
> EAs in XML, and for the most efficient storage of the EA in the 
> database, the containing XML document needs to be 512 bytes or fewer 
> so that it can be stored inline.  Larger XML objects will have to be 
> stored externally as LOBs (large objects), which will make queries 
> cost more.
> So we need to think about what that cost will be when we are 
> considering repositories with millions or billions of files.
> Vicky

Hi Vicky,

I do not see why we need to query these EAs in normal operation.  These 
EAs will only be accessed when we need to perform Ultimate Disaster 
Recovery - when you have lost all data on disks and all you have are tapes.

I was thinking about XML - but it is "opaque" to Object SAMQFS so, it is 
up to the Lustre side.  Whatever it is, the Applications - 
Lustre-Restore for example, is the one that has to understand the 
format.  I am not a Tar Header expert - but I assume that these EAs can 
go with the file in the tar ball.  I do not expect to keep any of it on 
line on disk cache, on the SAMFS side.  I see no reason.

With respect to whether it should be consistent with HPSS - I would say 
if that is all we need and it is sufficient - why not.  Otherwise, let's 
make it better than HPSS.  It must be in SUN's best interest to sell SAMQFS?

I do apologize Vicky, are you a SUN employee?



More information about the lustre-devel mailing list