[Pacemaker] configuration lost or cib not in sync?

[Andrew Beekhof]

On 25 Feb 2014, at 8:46 pm, Michael Böhm <dudleyperkins at gmail.com> wrote:

> Hello everybody,
> 
> today i had to stop a 2-node-cluster and bring them back up, but something did not went as expected. Apparently one of the nodes didn't have an up-to-date configuration, which was surprising for me as i thought this would be (coro)synced.
> (Nodes are Debian wheezy, pacemaker 1.1.7, corosync 1.4.2)
> 
> Here is what i've done:
> 1. put node1 in standby -> all node1-resources successfully migrated to node2
> 2. stopped pacemaker & corosync then shutdown node1
> 3. all resources were running on node2
> 4. stopped pacemaker & corosync then shutdown node2
> 
> moved the servers and tried to bring them back up:
> 
> 1. started both servers (pacemaker does not start at startup, but as i noticed corosync did)
> 2. started pacemaker on node2 (last active node before shutdown)
> 
> and here's the thing -> even though node2 had all resources when it was stopped, it now came up with only a few of them. The resources, that have been added in the last month were missing.
> 
> Looking at the files in /var/lib/heartbeat/crm/, the last cib was from over a month ago.

At some point there was probably an error that caused the disk writes to fail and get turned off.
It would probably be worth having a look in your log archives to see what the error was.

> As there were more and newer cib-files on node1 i tried my luck there:
> 
> 1. stopped pacemaker on node2
> 2. started pacemaker on node1
> 
> Node1 came up with all resources. Then i started pacemaker on node2 again and quickly there were three "new versions" of cib-files in /var/lib/heartbeat/crm/ and they were up-to-date.
> 
> So everything is good now, but i wonder if i did something wrong or in the wrong order and how can i be sure the resource-configuration is always in sync on both nodes?
> 
> Regards
> 
> 
> _______________________________________________
> Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
> 
> Project Home: http://www.clusterlabs.org
> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> Bugs: http://bugs.clusterlabs.org

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 841 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <https://lists.clusterlabs.org/pipermail/pacemaker/attachments/20140226/c523ce58/attachment-0003.sig>