[lustre-discuss] Data stored in OST

[BALVERS Martin]

I have a question about the following comment:
>>> The usual practice is to use RAUD10 for the MDT(s) on "enterprise" high-endurance SSD, and RAID6 for the OST on "professional" mixed-load SSDs or "small" (1-2TB at most) "datacenter" HDDs, fronted by failover-servers.

What would be the problem with large 'datacenter' type HDD's for an OST (in raid10 for instance)?

Thanks,
Martin Balvers

-----Original Message-----
From: lustre-discuss <lustre-discuss-bounces at lists.lustre.org> On Behalf Of Peter Grandi via lustre-discuss
Sent: Monday, May 22, 2023 15:35
To: list Lustre discussion <lustre-discuss at lists.Lustre.org>
Subject: Re: [lustre-discuss] Data stored in OST

** Caution - this is an external email **

>>> On Mon, 22 May 2023 13:08:19 +0530, Nick dan via lustre-discuss <lustre-discuss at lists.lustre.org> said:

> Hi I had one doubt. In lustre, data is divided into stripes and stored 
> in multiple OSTs. So each OST will have some part of data. My question 
> is if one OST fails, will there be data loss?

This is extensively discussed in the Lustre manual with comprehensive illustrations:

  https://urldefense.com/v3/__https://doc.lustre.org/lustre_manual.xhtml*understandinglustre.storageio__;Iw!!OUGTln_Lrg!TlK5iwVueuM5ES6iTrGKJ5KfN4p2-JQvUwesXX0wXbSlcx_3apJz7K4idfYgRDm6QLhD6ZDxlJcpa5chGhmzAjwx705bBcifYw$
  https://urldefense.com/v3/__https://doc.lustre.org/lustre_manual.xhtml*pfl__;Iw!!OUGTln_Lrg!TlK5iwVueuM5ES6iTrGKJ5KfN4p2-JQvUwesXX0wXbSlcx_3apJz7K4idfYgRDm6QLhD6ZDxlJcpa5chGhmzAjwx707k4bctVQ$
  https://urldefense.com/v3/__https://doc.lustre.org/lustre_manual.xhtml*understandingfailover__;Iw!!OUGTln_Lrg!TlK5iwVueuM5ES6iTrGKJ5KfN4p2-JQvUwesXX0wXbSlcx_3apJz7K4idfYgRDm6QLhD6ZDxlJcpa5chGhmzAjwx706ej8wTvg$

The usual practice is to use RAUD10 for the MDT(s) on "enterprise" high-endurance SSD, and RAID6 for the OST on "professional" mixed-load SSDs or "small" (1-2TB at most) "datacenter" HDDs, fronted by failover-servers.

I personally think that is is best to rely on Lustre striping and the "new" PFL LFR layout (across two OST "pools"), and have each OST on a single device, and and very few OSTs per OSS, when Lustre is used as "scratch" area for an HPC cluster.

  https://urldefense.com/v3/__https://doc.lustre.org/lustre_manual.xhtml*flr__;Iw!!OUGTln_Lrg!TlK5iwVueuM5ES6iTrGKJ5KfN4p2-JQvUwesXX0wXbSlcx_3apJz7K4idfYgRDm6QLhD6ZDxlJcpa5chGhmzAjwx705i523T9Q$
_______________________________________________
lustre-discuss mailing list
lustre-discuss at lists.lustre.org
https://urldefense.com/v3/__http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org__;!!OUGTln_Lrg!TlK5iwVueuM5ES6iTrGKJ5KfN4p2-JQvUwesXX0wXbSlcx_3apJz7K4idfYgRDm6QLhD6ZDxlJcpa5chGhmzAjwx706PzOli0g$

Ce message électronique et tous les fichiers attachés qu'il contient sont confidentiels et destinés exclusivement à l'usage de la personne à laquelle ils sont adressés. Si vous avez reçu ce message par erreur, merci de le retourner à son émetteur. Les idées et opinions présentées dans ce message sont celles de son auteur, et ne représentent pas nécessairement celles de DANONE ou d'une quelconque de ses filiales. La publication, l'usage, la distribution, l'impression ou la copie non autorisée de ce message et des attachements qu'il contient sont strictement interdits. 

This e-mail and any files transmitted with it are confidential and intended solely for the use of the individual to whom it is addressed. If you have received this email in error please send it back to the person that sent it to you. Any views or opinions presented are solely those of its author and do not necessarily represent those of DANONE or any of its subsidiary companies. Unauthorized publication, use, dissemination, forwarding, printing or copying of this email and its associated attachments is strictly prohibited.