[Pacemaker] VMWare Guest as Cluster Nodes - how to do stonith

Fri May 21 09:48:39 UTC 2010

If you are talking about two guest OSes on different ESX hosts, I think
libvirt or any VIAPI variation for rebooting/poweroff the GuestOS is not a
reliable STONITH.

The reason is: STONITH is to avoid "split brain" which means network
partition. So it's very possible that network problem prevent the quorum
winner accessing the one which is suppose to be shot.
+ network problem cause vCenter cannot reach the target ESX, so VIAPI
against vCenter won't work for STONITH
+ network problem cause STONITH requestor cannot reach the target ESX host,
so it's very possible you have no chance to shoot the node by accessing
VIAPI.

So any VIAPI variation won't be a reliable STONITH in my mind.
SBD could be a good candidate for virtulization environment. But I think we
could need improve the reliability of SBD in Vmware ESX environment by
following method:
1. Using VM Guest App monitor replace software watchdog, since VM GuestApp
monitor is heartbeat between ESX and GuestOS, stopping mark or touch the
heartbeat would cause the ESX host reboot the VM. The mechanism is good to
emulate a "hardware watchdog" in virtualization environment.
2. SBD is suicide based, the quorum winner need wait sometime for sure the
target is dead. But how long we need wait? It's up to experience. I think if
STONITH initiator wait until the STONITH target clear the suicide request
after reboot, it's reliable enough to believe the system has been shoot.

Just my 2 cents.

2010/5/21 <martin.braun at icw.de>

> Hi Sebastian,
>
> > Do you have any experiences with the libvirt? It sounds very
> > promising to me. Normally a simple soap call to reboot the guest
> > would be enough to have kind of stonith.
>
> I have some experiences with libvirt, but not yet with the ESX
> hypervisordriver. However with that driver configured you should be able
> to start|stop|reset your VM with virsh commands or even with one of the
> language bindings (e.g. python). It looks very promising to me.
>
> > The whole topic is kind of a black whole in my opinion because there
> > isn't that much information available.
>
> You are right, the documentation of libvirt is not the best, but what I
> can recommend is the IRC support of the libvirt developers!  They once
> helped me out a great deal with libvirts virtualbox driver.
>
>
>    * server: irc.oftc.net
>    * port: 6667 (the usual IRC port)
>    * channel: #virt
>
> HTH,
> Martin
>
> "Koch, Sebastian" <Sebastian.Koch at netzwerk.de> wrote on 20.05.2010
> 18:18:26:
>
> > [image removed]
> >
> > Re: [Pacemaker] VMWare Guest as Cluster Nodes - how to do stonith
> >
> > Koch, Sebastian
> >
> > to:
> >
> > The Pacemaker cluster resource manager
> >
> > 20.05.2010 18:21
> >
> > Please respond to The Pacemaker cluster resource manager
> >
> > Hi Martin,
> >
> > thanks for the reply. Do you have any experience with external/
> > vmware? I read that it isn't compatible with ESX Hosts?
> >
> > Do you have any experiences with the libvirt? It sounds very
> > promising to me. Normally a simple soap call to reboot the guest
> > would be enough to have kind of stonith.
> > You are right there could be a scenario where we would need to
> > stonith the esx host but lets start with the simple things.
> >
> > The whole topic is kind of a black whole in my opinion because there
> > isn't that much information available.
> >
> >
> > Best Regards
> > Sebastian
> >
> > -----Ursprüngliche Nachricht-----
> > Von: martin.braun at icw.de [mailto:martin.braun at icw.de]
> > Gesendet: Donnerstag, 20. Mai 2010 17:59
> > An: The Pacemaker cluster resource manager
> > Betreff: Re: [Pacemaker] VMWare Guest as Cluster Nodes - how to do
> stonith
> >
> > Hi Sebastian,
> >
> >
> > > guests. Does anybody have experiences or advises on how to configure
> > > stonith for vmware guests or maybe there is another solution to build
> > > a reliable virtual cluster.
> >
> > See this thread:
> > http://www.gossamer-threads.com/lists/linuxha/pacemaker/62850?
> > search_string=Pacemaker%20in%20VMware%20guests;#62850
> >
> > I am planning to experiment with libvirt (http://libvirt.org/drvesx.html
> > ), so please let me know your proceedings when you go this way.
> >
> > The only problem I see is: what about the esx-hosts, isn't there a
> > scenario when you want to kill the host (and that could be rather
> > problematic)?
> >
> >
> > Best,
> > Martin
> >
> >
> >
> >
> > "Koch, Sebastian" <Sebastian.Koch at netzwerk.de> wrote on 20.05.2010
> > 16:13:59:
> >
> > > [image removed]
> > >
> > > [Pacemaker] VMWare Guest as Cluster Nodes - how to do stonith
> > >
> > > Koch, Sebastian
> > >
> > > to:
> > >
> > > The Pacemaker cluster resource manager
> > >
> > > 20.05.2010 16:16
> > >
> > > Please respond to The Pacemaker cluster resource manager
> > >
> > > Hi,
> > >
> > > i was researching the whole vmware / stonith problem for a couple of
> > > days and i did not find any reliable information or examples on how to
>
> > > implement stonith with virtualized debian linux on vmware esx 4
> > > guests. Does anybody have experiences or advises on how to configure
> > > stonith for vmware guests or maybe there is another solution to build
> > > a reliable virtual cluster.
> > >
> > > Any help and hints is appreciated.
> > > Thanks in advance.
> > >
> > > Regards,
> > > Sebastian_______________________________________________
> > > Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
> > > http://oss.clusterlabs.org/mailman/listinfo/pacemaker
> > >
> > > Project Home: http://www.clusterlabs.org Getting started:
> > > http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> >
>
> >
> > _______________________________________________
> > Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
> > http://oss.clusterlabs.org/mailman/listinfo/pacemaker
> >
> > Project Home: http://www.clusterlabs.org
> > Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>
>
> InterComponentWare AG:
> Vorstand: Peter Kirschbauer (Vors.), Jörg Stadler / Aufsichtsratsvors.:
> Prof. Dr. Christof Hettich
> Firmensitz: 69190 Walldorf, Altrottstraße 31 / AG Mannheim HRB 351761 /
> USt.-IdNr.: DE 198388516
> _______________________________________________
> Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>
> Project Home: http://www.clusterlabs.org
> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>

-- 
Javen Wu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.clusterlabs.org/pipermail/pacemaker/attachments/20100521/319ae774/attachment-0002.htm>