[lustre-discuss] MDT/MGS mirroring

[Bertschinger, Thomas Andrew Hjorth]

Hi Stepan, 

Thanks, Cameron, for suggesting HALO.

To be clear on its status, HALO should support most of the same features as Pacemaker for this use case. However, it is still under active development and may still have bugs or usability issues. Therefore I would not recommend it (yet) if you're risk-averse. If you're an adventurous or risk-tolerant user, it might be a better fit today. You can look at the HALO admin guide at https://github.com/lanl/halo/blob/main/docs/admin_guide.typ for an overview of how it works.

Feel free to reach out to me directly or to report an issue on the github page if you try it out and run into any questions.

If you prefer to use Pacemaker, I think these pages look fairly comprehensive in terms of describing how to set it up:

https://wiki.lustre.org/Creating_a_Framework_for_High_Availability_with_Pacemaker
https://wiki.lustre.org/Creating_Pacemaker_Resources_for_Lustre_Storage_Services

Thanks!

Thomas Bertschinger
________________________________________
From: lustre-discuss <lustre-discuss-bounces at lists.lustre.org> on behalf of harr1 via lustre-discuss <lustre-discuss at lists.lustre.org>
Sent: Monday, August 3, 2026 10:01 AM
To: lustre-discuss at lists.lustre.org
Subject: [EXTERNAL] Re: [lustre-discuss] MDT/MGS mirroring

Stepan,

You might consider using LANL's new HALO resource manager which is
simpler to manage than Pacemaker (PM). https://github.com/lanl/halo

It's still undergoing active development but may robust enough for your
purposes. We are considering moving to HALO eventually.

If you with to go with PM, there are various different ways to set it
up, what we call "centralized" or "pair-wise". Our centralized configs
run pacemakerd on just the management node, with the various servers
each running pacemaker_remote and talking to the central pacemakerd
daemon. Our pair-wise config essentially makes every node pair its own
cluster, with both nodes running pacemakerd; this method is harder to
manage, but much more scalable. With the Centralized config, we have
each Lustre server node configured with 3 resources: one for pacemakerd
(started or not), one for the zpool (MDT or OST) on the node, and one
for the Lustre mount on that node. The central node also runs a
"stonith" resource using powerman to fence nodes.

Cameron