[Pacemaker] Nodes not rejoining cluster

[Andrew Beekhof]

Gotta have logs.  From all 3 nodes mentioned.
Only then can we determine if the problem is at the corosync or
pacemaker layer - which is the pre-requisit for figuring out what to
do next :)

On Fri, Mar 30, 2012 at 1:30 PM, Gregg Stock <gregg at damagecontrolusa.com> wrote:
> I had a circuit breaker go out and take two of the 5 nodes in my cluster
> down. Now that their back up and running, they are not rejoining the
> cluster.
>
> Here is what I get from crm_mon -1
>
> node 1,2 and 3 itchy, scratchy and walter show the following:
> ============
> Last updated: Thu Mar 29 19:04:05 2012
> Last change: Thu Mar 29 19:04:03 2012 via cibadmin on walter
> Stack: openais
> Current DC: walter - partition with quorum
> Version: 1.1.6-3.el6-a02c0f19a00c1eb2527ad38f146ebc0834814558
> 5 Nodes configured, 5 expected votes
> 9 Resources configured.
> ============
>
> Online: [ itchy scratchy walter butthead timmy ]
>
>
> On butthead I get
>
> ============
> Last updated: Thu Mar 29 19:04:24 2012
> Last change: Thu Mar 29 18:42:09 2012 via cibadmin on itchy
> Stack: openais
> Current DC: NONE
> 5 Nodes configured, 5 expected votes
> 9 Resources configured.
> ============
>
> OFFLINE: [ itchy scratchy walter butthead timmy ]
>
>
> On Timmy, I get
>
> ============
> Last updated: Thu Mar 29 19:04:20 2012
> Last change:
> Current DC: NONE
> 0 Nodes configured, unknown expected votes
> 0 Resources configured.
> ============
>
>
> I don't have anything important running yet. so I can do a full clean up of
> everything if needed.
>
> I also get some weird behavior with timmy. I brought this node up with the
> host name as timmy.example.com and I changed the host name to timmy but when
> the cluster is offline timmy.example.com shows up as offline. I enter crm
> node delete timmy.example.com and it goes away until timmy goes offline
> again.
>
> Thanks,
> Gregg Stock
>
>
>
> _______________________________________________
> Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>
> Project Home: http://www.clusterlabs.org
> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> Bugs: http://bugs.clusterlabs.org
>