[lustre-discuss] MGS failover problem

Marcin Dulak marcin.dulak at gmail.com
Wed Jan 11 05:53:45 PST 2017


Lustre has multi-mount protection so there should be no risk for pacemaker
trying to mount lustre on a secondary servicenode in case of failure of the
primary node.
If lustre is still mounted on the primary the mount on the secondary will
fail.
http://doc.lustre.org/lustre_manual.xhtml#managingfailover
I'm not sure about the current status of multi-mount protection on ZFS.
Normally lustre (ldiskfs) is managed by pacemaker, it is enough to
mount/umount lustre on MGS/OSS - I have never done anything to lnet in case
of failover, but let's wait for the confirmation of this from a developer.
I would not risk STONITH unless you have a well designed, redundant network
setup for corosync and you really trust your pacemaker/corosync cluster.
You may end up in a situation when your lustre over infiniband works
properly but the pacemaker cluster decides to fail - a real case here:
https://bugzilla.redhat.com/show_bug.cgi?id=1114852
There is a nice script that helps to get started
https://github.com/gc3-uzh-ch/schroedinger-lustre-ha/blob/master/make-lustre-crm-config.py

Best regards,

Marcin

On Wed, Jan 11, 2017 at 2:17 PM, Michael Di Domenico <mdidomenico4 at gmail.com
> wrote:

> On Tue, Jan 10, 2017 at 11:32 AM, Vicker, Darby (JSC-EG311)
> <darby.vicker-1 at nasa.gov> wrote:
> > One other thought comes to mind.  We are using the init.d scripts (i.e.
> /etc/init.d/{lustre,lnet} and /etc/ldev.conf.  We have lnet chkconfig’ed on
> so lnet is starting on boot on all servers.  But ‘lustre’ is chkconfig’ed
> off so that if a server reboots for whatever reason we don’t get into a
> situation where we multi-mount.  On a clean boot we have to manually mount
> the MDT/OST’s (i.e. do a “service lustre start”).  To do the failover we do
> the “/etc/init.d/lustre stop local” on the primary and “/etc/init.d/lustre
> start foreign” on the secondary to do the failover.  What is the right
> thing to do with lnet on failover?  Should it be stopped on the primary
> node before doing a failover to the secondary node?  This is the state of
> the pro
>
> I'm certainly no lustre expert, but i would suspect you want lnet to
> be stopped on the primary node if you failed over to the secondary.
> historically lustre is a STONITH based failover system.  therefore i
> would expect that if you "failed over" from one node to another the
> primary node is effectively powered off.  i can certainly believe that
> there's some code in lustre that checks lnet and if its up tries to do
> something.  which could be the source of the error messages you're
> seeing.
>
> but i'm not an expert, so i could be way off base.
> _______________________________________________
> lustre-discuss mailing list
> lustre-discuss at lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20170111/aed2f1d0/attachment.htm>


More information about the lustre-discuss mailing list