[Lustre-discuss] Lustre as a reliable f/s for an archive data centre

[John Ouellette]

Hi -- I work for an astronomical data archive centre which stores about 
300TB of data.  We are considering options for replacing our current 
data management system, and one proposed alternative uses Lustre as one 
component.  From what I have read about Lustre, it is largely targeted 
to HPC installations and not data centres, so I'm a bit worried about 
its applicability here.

Although we do have a requirement for high throughput (although not to 
really support 1000s of clients: more likely a few dozen nodes), our 
primary concerns are reliability and data integrity.  From reading the 
docs, it looks as though Lustre can be made to be very reliable, but how 
about data integrity?   Are there problems with data corruption?  If the 
MDS data is lost, is it possible to rebuild this, given only the 
file-system data?  How easy is it to back-up Lustre?  Do you back-up the 
MDS data and the OST data, or do you back-up through a Lustre client?

Thanks in advance for any answers or pointers,
John Ouellette