[Pacemaker] MySQL, Percona replication manager - split brain

[Digimer]

On 25/10/14 03:32 PM, Andrew wrote:
> Hi all.
>
> I use Percona as RA on cluster (nothing mission-critical, currently -
> just zabbix data); today after restarting MySQL resource (crm resource
> restart p_mysql) I've got a split brain state - MySQL for some reason
> started first at ex-slave node, ex-master starts later (possibly I've
> set too small timeout to shutdown - only 120s, but I'm not sure).
>
> After restart resource on both nodes it seems like mysql replication was
> ok - but then after ~50min it fails in split brain again for unknown
> reason (no resource restart was noticed).
>
> In 'show replication status' there is an error in table caused by unique
> index dup.
>
> So I have a questions:
> 1) Which thing causes split brain, and how to avoid it in future?

Cause:

Logs?

Prevent:

Fencing (aka stonith). This is why fencing is required.

> 2) How to resolve split brain state? Is it enough just to wait for
> failure, then - restart mysql by hand and clean row with dup index in
> slave db, and then run resource again? Or there is some automation for
> such cases?

How are you sharing data? Can you give us a better understanding of your 
setup?

-- 
Digimer
Papers and Projects: https://alteeve.ca/w/
What if the cure for cancer is trapped in the mind of a person without 
access to education?