[Pacemaker] Cluster split brain on vmware VSphere
Dejan Muhamedagic
dejanmm at fastmail.fm
Mon Jun 7 12:40:19 UTC 2010
Hi,
On Mon, Jun 07, 2010 at 01:04:53PM +0200, Torresani, Roberto wrote:
> Hi list,
> sorry if this is not the right ml or if the question is already answered somewhere...if this is the case, just point me to the solution please.
>
> I have two active/passive mysql cluster on vmware vsphere running:
> - CentOS 5.5 32 bit
> - heartbeat-3.0.3-2.el5
> - corosync-1.2.2-1.1.el5
> - pacemaker-1.0.8-6.1.el5
>
> The cluster have stonith configured using an adapted version of xen0 stonith external plugins, which seems to work fine.
>
> The problem is that from time to time (every day...), especially when the virtualcenter is snapshotting the vm for make backup of them, the cluster split itself.
> Here some of the events:
>
> Jun 5 22:06:12 mysqldrupal-n1 corosync[3139]: [TOTEM ] Process pause detected for 1111 ms, flushing membership messages.
> Jun 5 22:06:12 mysqldrupal-n1 corosync[3139]: [TOTEM ] Process pause detected for 1112 ms, flushing membership messages.
> Jun 5 22:06:12 mysqldrupal-n1 corosync[3139]: [TOTEM ] A processor joined or left the membership and a new membership was formed.
Looks like there are not enough cycles for the corosync process.
> Jun 5 22:16:18 mysqldrupal-n1 crmd: [3151]: ERROR: crmd_ha_msg_filter: Another DC detected: mysqldrupal-n2 (op=noop)
>
> and both nodes start resetting one each other, till the situation stabilizes after a while.
>
> I tried to modify some parameters in the corosync.conf but without luck...
You didn't show the corosync.conf. Did you try to change the
token to a larger value, say 10000? BTW, this topic is for the
openais mailing list.
Thanks,
Dejan
> Both cluster behaves the same.
>
> Anyone of you have similar problem?
> Any idea please?
>
> In attach you can find the relevant messages log and the cib configuration.
>
> Regards,
> Roberto
> Jun 5 21:58:49 mysqldrupal-n1 crmd: [3151]: info: run_graph: ====================================================
> Jun 5 21:58:49 mysqldrupal-n1 crmd: [3151]: notice: run_graph: Transition 100 (Complete=0, Pending=0, Fired=0, Skipped=0, Incomplete=0, Source=/var/lib/pengine/pe-input-5160.bz2): Complete
> Jun 5 21:58:49 mysqldrupal-n1 crmd: [3151]: info: te_graph_trigger: Transition 100 is now complete
> Jun 5 21:58:49 mysqldrupal-n1 crmd: [3151]: info: notify_crmd: Transition 100 status: done - <null>
> Jun 5 21:58:49 mysqldrupal-n1 crmd: [3151]: info: do_state_transition: State transition S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SUCCESS cause=C_FSA_INTERNAL origin=notify_crmd ]
> Jun 5 21:58:49 mysqldrupal-n1 crmd: [3151]: info: do_state_transition: Starting PEngine Recheck Timer
> Jun 5 22:00:38 mysqldrupal-n1 cib: [3147]: info: cib_stats: Processed 1 operations (10000.00us average, 0% utilization) in the last 10min
> Jun 5 22:06:12 mysqldrupal-n1 corosync[3139]: [TOTEM ] Process pause detected for 1111 ms, flushing membership messages.
> Jun 5 22:06:12 mysqldrupal-n1 corosync[3139]: [TOTEM ] Process pause detected for 1112 ms, flushing membership messages.
> Jun 5 22:06:12 mysqldrupal-n1 corosync[3139]: [pcmk ] notice: pcmk_peer_update: Transitional membership event on ring 8336: memb=2, new=0, lost=0
> Jun 5 22:06:12 mysqldrupal-n1 corosync[3139]: [pcmk ] info: pcmk_peer_update: memb: mysqldrupal-n1 1204725952
> Jun 5 22:06:12 mysqldrupal-n1 corosync[3139]: [pcmk ] info: pcmk_peer_update: memb: mysqldrupal-n2 1221503168
> Jun 5 22:06:12 mysqldrupal-n1 corosync[3139]: [pcmk ] notice: pcmk_peer_update: Stable membership event on ring 8336: memb=2, new=0, lost=0
> Jun 5 22:06:12 mysqldrupal-n1 corosync[3139]: [pcmk ] info: pcmk_peer_update: MEMB: mysqldrupal-n1 1204725952
> Jun 5 22:06:12 mysqldrupal-n1 corosync[3139]: [pcmk ] info: pcmk_peer_update: MEMB: mysqldrupal-n2 1221503168
> Jun 5 22:06:12 mysqldrupal-n1 corosync[3139]: [TOTEM ] A processor joined or left the membership and a new membership was formed.
> Jun 5 22:06:12 mysqldrupal-n1 corosync[3139]: [MAIN ] Completed service synchronization, ready to provide service.
> Jun 5 22:10:55 mysqldrupal-n1 corosync[3139]: [TOTEM ] A processor failed, forming new configuration.
> Jun 5 22:10:55 mysqldrupal-n1 corosync[3139]: [pcmk ] notice: pcmk_peer_update: Transitional membership event on ring 8340: memb=2, new=0, lost=0
> Jun 5 22:10:55 mysqldrupal-n1 corosync[3139]: [pcmk ] info: pcmk_peer_update: memb: mysqldrupal-n1 1204725952
> Jun 5 22:10:55 mysqldrupal-n1 corosync[3139]: [pcmk ] info: pcmk_peer_update: memb: mysqldrupal-n2 1221503168
> Jun 5 22:10:55 mysqldrupal-n1 corosync[3139]: [pcmk ] notice: pcmk_peer_update: Stable membership event on ring 8340: memb=2, new=0, lost=0
> Jun 5 22:10:55 mysqldrupal-n1 corosync[3139]: [pcmk ] info: pcmk_peer_update: MEMB: mysqldrupal-n1 1204725952
> Jun 5 22:10:55 mysqldrupal-n1 corosync[3139]: [pcmk ] info: pcmk_peer_update: MEMB: mysqldrupal-n2 1221503168
> Jun 5 22:10:55 mysqldrupal-n1 corosync[3139]: [TOTEM ] A processor joined or left the membership and a new membership was formed.
> Jun 5 22:10:55 mysqldrupal-n1 corosync[3139]: [MAIN ] Completed service synchronization, ready to provide service.
> Jun 5 22:13:49 mysqldrupal-n1 crmd: [3151]: info: crm_timer_popped: PEngine Recheck Timer (I_PE_CALC) just popped!
> Jun 5 22:13:49 mysqldrupal-n1 crmd: [3151]: info: do_state_transition: State transition S_IDLE -> S_POLICY_ENGINE [ input=I_PE_CALC cause=C_TIMER_POPPED origin=crm_timer_popped ]
> Jun 5 22:13:49 mysqldrupal-n1 crmd: [3151]: info: do_state_transition: Progressed to state S_POLICY_ENGINE after C_TIMER_POPPED
> Jun 5 22:13:49 mysqldrupal-n1 crmd: [3151]: info: do_state_transition: All 2 cluster nodes are eligible to run resources.
> Jun 5 22:13:49 mysqldrupal-n1 crmd: [3151]: info: do_pe_invoke: Query 196: Requesting the current CIB: S_POLICY_ENGINE
> Jun 5 22:13:49 mysqldrupal-n1 crmd: [3151]: info: do_pe_invoke_callback: Invoking the PE: query=196, ref=pe_calc-dc-1275768829-141, seq=8316, quorate=1
> Jun 5 22:13:49 mysqldrupal-n1 pengine: [3150]: notice: unpack_config: On loss of CCM Quorum: Ignore
> Jun 5 22:13:49 mysqldrupal-n1 pengine: [3150]: info: unpack_config: Node scores: 'red' = -INFINITY, 'yellow' = 0, 'green' = 0
> Jun 5 22:13:49 mysqldrupal-n1 pengine: [3150]: info: determine_online_status: Node mysqldrupal-n2 is online
> Jun 5 22:13:49 mysqldrupal-n1 pengine: [3150]: info: determine_online_status: Node mysqldrupal-n1 is online
> Jun 5 22:13:49 mysqldrupal-n1 pengine: [3150]: notice: group_print: Resource Group: mysql-grp
> Jun 5 22:13:49 mysqldrupal-n1 pengine: [3150]: notice: native_print: MySQL-VIP (ocf::heartbeat:IPaddr2): Started mysqldrupal-n1
> Jun 5 22:13:49 mysqldrupal-n1 pengine: [3150]: notice: native_print: MySQL-DB (ocf::heartbeat:mysql): Started mysqldrupal-n1
> Jun 5 22:13:49 mysqldrupal-n1 pengine: [3150]: notice: clone_print: Clone Set: Fencing
> Jun 5 22:13:49 mysqldrupal-n1 pengine: [3150]: notice: short_print: Started: [ mysqldrupal-n2 mysqldrupal-n1 ]
> Jun 5 22:13:49 mysqldrupal-n1 pengine: [3150]: notice: LogActions: Leave resource MySQL-VIP (Started mysqldrupal-n1)
> Jun 5 22:13:49 mysqldrupal-n1 pengine: [3150]: notice: LogActions: Leave resource MySQL-DB (Started mysqldrupal-n1)
> Jun 5 22:13:49 mysqldrupal-n1 pengine: [3150]: notice: LogActions: Leave resource vmware-fencing:0 (Started mysqldrupal-n2)
> Jun 5 22:13:49 mysqldrupal-n1 pengine: [3150]: notice: LogActions: Leave resource vmware-fencing:1 (Started mysqldrupal-n1)
> Jun 5 22:13:49 mysqldrupal-n1 crmd: [3151]: info: do_state_transition: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=handle_response ]
> Jun 5 22:13:49 mysqldrupal-n1 pengine: [3150]: info: process_pe_message: Transition 101: PEngine Input stored in: /var/lib/pengine/pe-input-5161.bz2
> Jun 5 22:13:49 mysqldrupal-n1 crmd: [3151]: info: unpack_graph: Unpacked transition 101: 0 actions in 0 synapses
> Jun 5 22:13:49 mysqldrupal-n1 crmd: [3151]: info: do_te_invoke: Processing graph 101 (ref=pe_calc-dc-1275768829-141) derived from /var/lib/pengine/pe-input-5161.bz2
> Jun 5 22:13:49 mysqldrupal-n1 crmd: [3151]: info: run_graph: ====================================================
> Jun 5 22:13:49 mysqldrupal-n1 crmd: [3151]: notice: run_graph: Transition 101 (Complete=0, Pending=0, Fired=0, Skipped=0, Incomplete=0, Source=/var/lib/pengine/pe-input-5161.bz2): Complete
> Jun 5 22:13:49 mysqldrupal-n1 crmd: [3151]: info: te_graph_trigger: Transition 101 is now complete
> Jun 5 22:13:49 mysqldrupal-n1 crmd: [3151]: info: notify_crmd: Transition 101 status: done - <null>
> Jun 5 22:13:49 mysqldrupal-n1 crmd: [3151]: info: do_state_transition: State transition S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SUCCESS cause=C_FSA_INTERNAL origin=notify_crmd ]
> Jun 5 22:13:49 mysqldrupal-n1 crmd: [3151]: info: do_state_transition: Starting PEngine Recheck Timer
> Jun 5 22:15:51 mysqldrupal-n1 corosync[3139]: [TOTEM ] Process pause detected for 1301 ms, flushing membership messages.
> Jun 5 22:15:51 mysqldrupal-n1 corosync[3139]: [pcmk ] notice: pcmk_peer_update: Transitional membership event on ring 8344: memb=2, new=0, lost=0
> Jun 5 22:15:51 mysqldrupal-n1 corosync[3139]: [pcmk ] info: pcmk_peer_update: memb: mysqldrupal-n1 1204725952
> Jun 5 22:15:51 mysqldrupal-n1 corosync[3139]: [pcmk ] info: pcmk_peer_update: memb: mysqldrupal-n2 1221503168
> Jun 5 22:15:51 mysqldrupal-n1 corosync[3139]: [pcmk ] notice: pcmk_peer_update: Stable membership event on ring 8344: memb=2, new=0, lost=0
> Jun 5 22:15:51 mysqldrupal-n1 corosync[3139]: [pcmk ] info: pcmk_peer_update: MEMB: mysqldrupal-n1 1204725952
> Jun 5 22:15:51 mysqldrupal-n1 corosync[3139]: [pcmk ] info: pcmk_peer_update: MEMB: mysqldrupal-n2 1221503168
> Jun 5 22:15:51 mysqldrupal-n1 corosync[3139]: [TOTEM ] A processor joined or left the membership and a new membership was formed.
> Jun 5 22:15:51 mysqldrupal-n1 corosync[3139]: [MAIN ] Completed service synchronization, ready to provide service.
> Jun 5 22:16:18 mysqldrupal-n1 corosync[3139]: [pcmk ] notice: pcmk_peer_update: Transitional membership event on ring 8352: memb=1, new=0, lost=1
> Jun 5 22:16:18 mysqldrupal-n1 corosync[3139]: [pcmk ] info: pcmk_peer_update: memb: mysqldrupal-n1 1204725952
> Jun 5 22:16:18 mysqldrupal-n1 corosync[3139]: [pcmk ] info: pcmk_peer_update: lost: mysqldrupal-n2 1221503168
> Jun 5 22:16:18 mysqldrupal-n1 corosync[3139]: [pcmk ] notice: pcmk_peer_update: Stable membership event on ring 8352: memb=2, new=1, lost=0
> Jun 5 22:16:18 mysqldrupal-n1 corosync[3139]: [pcmk ] info: pcmk_peer_update: NEW: mysqldrupal-n2 1221503168
> Jun 5 22:16:18 mysqldrupal-n1 corosync[3139]: [pcmk ] info: pcmk_peer_update: MEMB: mysqldrupal-n1 1204725952
> Jun 5 22:16:18 mysqldrupal-n1 corosync[3139]: [pcmk ] info: pcmk_peer_update: MEMB: mysqldrupal-n2 1221503168
> Jun 5 22:16:18 mysqldrupal-n1 corosync[3139]: [TOTEM ] A processor joined or left the membership and a new membership was formed.
> Jun 5 22:16:18 mysqldrupal-n1 corosync[3139]: [MAIN ] Completed service synchronization, ready to provide service.
> Jun 5 22:16:18 mysqldrupal-n1 crmd: [3151]: ERROR: crmd_ha_msg_filter: Another DC detected: mysqldrupal-n2 (op=noop)
> Jun 5 22:16:18 mysqldrupal-n1 crmd: [3151]: info: do_state_transition: State transition S_IDLE -> S_ELECTION [ input=I_ELECTION cause=C_FSA_INTERNAL origin=crmd_ha_msg_filter ]
> Jun 5 22:16:18 mysqldrupal-n1 crmd: [3151]: info: update_dc: Unset DC mysqldrupal-n1
> Jun 5 22:16:18 mysqldrupal-n1 cib: [3147]: info: cib_process_diff: Diff 0.456.1 -> 0.456.2 not applied to 0.455.9: current "epoch" is less than required
> Jun 5 22:16:18 mysqldrupal-n1 cib: [3147]: WARN: cib_server_process_diff: Not requesting full refresh in slave mode.
> Jun 5 22:16:18 mysqldrupal-n1 cib: [3147]: info: cib_process_diff: Diff 0.456.2 -> 0.456.3 not applied to 0.455.9: current "epoch" is less than required
> Jun 5 22:16:18 mysqldrupal-n1 cib: [3147]: WARN: cib_server_process_diff: Not requesting full refresh in slave mode.
> Jun 5 22:16:18 mysqldrupal-n1 cib: [3147]: info: cib_process_diff: Diff 0.456.3 -> 0.456.4 not applied to 0.455.9: current "epoch" is less than required
> Jun 5 22:16:18 mysqldrupal-n1 cib: [3147]: WARN: cib_server_process_diff: Not requesting full refresh in slave mode.
> Jun 5 22:16:18 mysqldrupal-n1 cib: [3147]: info: cib_process_diff: Diff 0.456.4 -> 0.457.1 not applied to 0.455.9: current "epoch" is less than required
> Jun 5 22:16:18 mysqldrupal-n1 cib: [3147]: WARN: cib_server_process_diff: Not requesting full refresh in slave mode.
> Jun 5 22:16:18 mysqldrupal-n1 crmd: [3151]: info: do_state_transition: State transition S_ELECTION -> S_INTEGRATION [ input=I_ELECTION_DC cause=C_FSA_INTERNAL origin=do_election_check ]
> Jun 5 22:16:18 mysqldrupal-n1 crmd: [3151]: info: do_dc_takeover: Taking over DC status for this partition
> Jun 5 22:16:18 mysqldrupal-n1 cib: [3147]: info: cib_process_readwrite: We are now in R/O mode
> Jun 5 22:16:18 mysqldrupal-n1 cib: [3147]: info: cib_process_request: Operation complete: op cib_slave_all for section 'all' (origin=local/crmd/197, version=0.455.9): ok (rc=0)
> Jun 5 22:16:18 mysqldrupal-n1 cib: [3147]: info: cib_process_readwrite: We are now in R/W mode
> Jun 5 22:16:18 mysqldrupal-n1 cib: [3147]: info: cib_process_request: Operation complete: op cib_master for section 'all' (origin=local/crmd/198, version=0.455.9): ok (rc=0)
> Jun 5 22:16:18 mysqldrupal-n1 cib: [3147]: info: cib_process_request: Operation complete: op cib_modify for section cib (origin=local/crmd/199, version=0.455.9): ok (rc=0)
> Jun 5 22:16:18 mysqldrupal-n1 cib: [3147]: info: cib_process_request: Operation complete: op cib_modify for section crm_config (origin=local/crmd/201, version=0.455.9): ok (rc=0)
> Jun 5 22:16:18 mysqldrupal-n1 cib: [3147]: info: cib_process_request: Operation complete: op cib_modify for section crm_config (origin=local/crmd/203, version=0.455.9): ok (rc=0)
> Jun 5 22:16:18 mysqldrupal-n1 crmd: [3151]: info: do_dc_join_offer_all: join-4: Waiting on 2 outstanding join acks
> Jun 5 22:16:18 mysqldrupal-n1 crmd: [3151]: info: ais_dispatch: Membership 8352: quorum retained
> Jun 5 22:16:18 mysqldrupal-n1 crmd: [3151]: info: crm_ais_dispatch: Setting expected votes to 2
> Jun 5 22:16:19 mysqldrupal-n1 cib: [3147]: info: cib_process_request: Operation complete: op cib_modify for section crm_config (origin=local/crmd/206, version=0.455.9): ok (rc=0)
> Jun 5 22:16:19 mysqldrupal-n1 crmd: [3151]: info: config_query_callback: Checking for expired actions every 900000ms
> Jun 5 22:16:19 mysqldrupal-n1 crmd: [3151]: info: config_query_callback: Sending expected-votes=2 to corosync
> Jun 5 22:16:19 mysqldrupal-n1 crmd: [3151]: info: update_dc: Set DC to mysqldrupal-n1 (3.0.1)
> Jun 5 22:16:19 mysqldrupal-n1 crmd: [3151]: info: check_join_state: do_dc_join_filter_offer: Membership changed since join started: 8316 -> 8352
> Jun 5 22:16:19 mysqldrupal-n1 crmd: [3151]: info: update_dc: Unset DC mysqldrupal-n1
> Jun 5 22:16:19 mysqldrupal-n1 crmd: [3151]: info: join_make_offer: Making join offers based on membership 8352
> Jun 5 22:16:19 mysqldrupal-n1 crmd: [3151]: info: do_dc_join_offer_all: join-5: Waiting on 2 outstanding join acks
> Jun 5 22:16:19 mysqldrupal-n1 crmd: [3151]: info: ais_dispatch: Membership 8352: quorum retained
> Jun 5 22:16:19 mysqldrupal-n1 crmd: [3151]: info: crm_ais_dispatch: Setting expected votes to 2
> Jun 5 22:16:19 mysqldrupal-n1 cib: [3147]: info: cib_process_request: Operation complete: op cib_modify for section crm_config (origin=local/crmd/209, version=0.455.9): ok (rc=0)
> Jun 5 22:16:19 mysqldrupal-n1 crmd: [3151]: info: update_dc: Set DC to mysqldrupal-n1 (3.0.1)
> Jun 5 22:16:19 mysqldrupal-n1 crmd: [3151]: info: do_state_transition: State transition S_INTEGRATION -> S_FINALIZE_JOIN [ input=I_INTEGRATED cause=C_FSA_INTERNAL origin=check_join_state ]
> Jun 5 22:16:19 mysqldrupal-n1 crmd: [3151]: info: do_state_transition: All 2 cluster nodes responded to the join offer.
> Jun 5 22:16:19 mysqldrupal-n1 crmd: [3151]: info: do_dc_join_finalize: join-5: Syncing the CIB from mysqldrupal-n2 to the rest of the cluster
> Jun 5 22:16:19 mysqldrupal-n1 cib: [3147]: info: cib_replace_notify: Replaced: 0.455.9 -> 0.457.1 from mysqldrupal-n2
> Jun 5 22:16:19 mysqldrupal-n1 cib: [3147]: info: log_data_element: cib:diff: - <cib dc-uuid="mysqldrupal-n1" epoch="455" num_updates="9" >
> Jun 5 22:16:19 mysqldrupal-n1 attrd: [3149]: info: do_cib_replaced: Sending full refresh
> Jun 5 22:16:19 mysqldrupal-n1 cib: [3147]: info: log_data_element: cib:diff: - <status >
> Jun 5 22:16:19 mysqldrupal-n1 attrd: [3149]: info: attrd_trigger_update: Sending flush op to all hosts for: probe_complete (true)
> Jun 5 22:16:19 mysqldrupal-n1 cib: [3147]: info: log_data_element: cib:diff: - <node_state join="member" id="mysqldrupal-n1" >
> Jun 5 22:16:19 mysqldrupal-n1 cib: [3147]: info: log_data_element: cib:diff: - <transient_attributes id="mysqldrupal-n1" __crm_diff_marker__="removed:top" >
> Jun 5 22:16:19 mysqldrupal-n1 cib: [3147]: info: log_data_element: cib:diff: - <instance_attributes id="status-mysqldrupal-n1" >
> Jun 5 22:16:19 mysqldrupal-n1 cib: [3147]: info: log_data_element: cib:diff: - <nvpair id="status-mysqldrupal-n1-probe_complete" name="probe_complete" value="true" />
> Jun 5 22:16:19 mysqldrupal-n1 cib: [3147]: info: log_data_element: cib:diff: - </instance_attributes>
> Jun 5 22:16:19 mysqldrupal-n1 cib: [3147]: info: log_data_element: cib:diff: - </transient_attributes>
> Jun 5 22:16:19 mysqldrupal-n1 cib: [3147]: info: log_data_element: cib:diff: - <lrm id="mysqldrupal-n1" __crm_diff_marker__="removed:top" >
> Jun 5 22:16:19 mysqldrupal-n1 cib: [3147]: info: log_data_element: cib:diff: - <lrm_resources >
> Jun 5 22:16:19 mysqldrupal-n1 cib: [3147]: info: log_data_element: cib:diff: - <lrm_resource id="MySQL-VIP" type="IPaddr2" class="ocf" provider="heartbeat" >
> Jun 5 22:16:19 mysqldrupal-n1 cib: [3147]: info: log_data_element: cib:diff: - <lrm_rsc_op id="MySQL-VIP_monitor_0" operation="monitor" crm-debug-origin="build_active_RAs" crm_feature_set="3.0.1" transition-key="6:6:7:80c0a647-0bb1-4e91-afe4-4aab2c861383" transition-magic="0:7;6:6:7:80c0a647-0bb1-4e91-afe4-4aab2c861383" call-id="2" rc-code="7" op-status="0" interval="0" last-run="1275682876" last-rc-change="1275682876" exec-time="240" queue-time="0" op-digest="eea53129ce7eee7cbc958e71a2b1badf" />
> Jun 5 22:16:19 mysqldrupal-n1 cib: [3147]: info: log_data_element: cib:diff: - <lrm_rsc_op id="MySQL-VIP_start_0" operation="start" crm-debug-origin="build_active_RAs" crm_feature_set="3.0.1" transition-key="5:3:0:a2a09eab-c909-4992-b9ad-10ac6c06f9f6" transition-magic="0:0;5:3:0:a2a09eab-c909-4992-b9ad-10ac6c06f9f6" call-id="6" rc-code="0" op-status="0" interval="0" last-run="1275683191" last-rc-change="1275683191" exec-time="230" queue-time="10" op-digest="eea53129ce7eee7cbc958e71a2b1badf" />
> Jun 5 22:16:19 mysqldrupal-n1 cib: [3147]: info: log_data_element: cib:diff: - <lrm_rsc_op id="MySQL-VIP_monitor_25000" operation="monitor" crm-debug-origin="build_active_RAs" crm_feature_set="3.0.1" transition-key="6:4:0:a2a09eab-c909-4992-b9ad-10ac6c06f9f6" transition-magic="0:0;6:4:0:a2a09eab-c909-4992-b9ad-10ac6c06f9f6" call-id="7" rc-code="0" op-status="0" interval="25000" last-run="1275683271" last-rc-change="1275683196" exec-time="100" queue-time="0" op-digest="b0d22fd23cd8c837118fdd66bf24a2f9" />
> Jun 5 22:16:19 mysqldrupal-n1 cib: [3147]: info: log_data_element: cib:diff: - </lrm_resource>
> Jun 5 22:16:19 mysqldrupal-n1 cib: [3147]: info: log_data_element: cib:diff: - <lrm_resource id="vmware-fencing:1" type="external/unitn.vmware" class="stonith" >
> Jun 5 22:16:19 mysqldrupal-n1 cib: [3147]: info: log_data_element: cib:diff: - <lrm_rsc_op id="vmware-fencing:1_monitor_0" operation="monitor" crm-debug-origin="build_active_RAs" crm_feature_set="3.0.1" transition-key="8:6:7:80c0a647-0bb1-4e91-afe4-4aab2c861383" transition-magic="0:7;8:6:7:80c0a647-0bb1-4e91-afe4-4aab2c861383" call-id="4" rc-code="7" op-status="0" interval="0" last-run="1275682876" last-rc-change="1275682876" exec-time="20" queue-time="0" op-digest="bb73a94b5273ddbfd07a1757f8057faa" />
> Jun 5 22:16:19 mysqldrupal-n1 cib: [3147]: info: log_data_element: cib:diff: - <lrm_rsc_op id="vmware-fencing:1_start_0" operation="start" crm-debug-origin="build_active_RAs" crm_feature_set="3.0.1" transition-key="20:6:0:80c0a647-0bb1-4e91-afe4-4aab2c861383" transition-magic="0:0;20:6:0:80c0a647-0bb1-4e91-afe4-4aab2c861383" call-id="5" rc-code="0" op-status="0" interval="0" last-run="1275682876" last-rc-change="1275682876" exec-time="270" queue-time="10" op-digest="bb73a94b5273ddbfd07a1757f8057faa" />
> Jun 5 22:16:19 mysqldrupal-n1 cib: [3147]: info: log_data_element: cib:diff: - </lrm_resource>
> Jun 5 22:16:19 mysqldrupal-n1 cib: [3147]: info: log_data_element: cib:diff: - <lrm_resource id="MySQL-DB" type="mysql" class="ocf" provider="heartbeat" >
> Jun 5 22:16:19 mysqldrupal-n1 cib: [3147]: info: log_data_element: cib:diff: - <lrm_rsc_op id="MySQL-DB_monitor_0" operation="monitor" crm-debug-origin="build_active_RAs" crm_feature_set="3.0.1" transition-key="7:6:7:80c0a647-0bb1-4e91-afe4-4aab2c861383" transition-magic="0:7;7:6:7:80c0a647-0bb1-4e91-afe4-4aab2c861383" call-id="3" rc-code="7" op-status="0" interval="0" last-run="1275682876" last-rc-change="1275682876" exec-time="260" queue-time="0" op-digest="b0b72c35d0ac375641c8163d68c1259f" />
> Jun 5 22:16:19 mysqldrupal-n1 cib: [3147]: info: log_data_element: cib:diff: - <lrm_rsc_op id="MySQL-DB_start_0" operation="start" crm-debug-origin="build_active_RAs" crm_feature_set="3.0.1" transition-key="7:4:0:a2a09eab-c909-4992-b9ad-10ac6c06f9f6" transition-magic="0:0;7:4:0:a2a09eab-c909-4992-b9ad-10ac6c06f9f6" call-id="8" rc-code="0" op-status="0" interval="0" last-run="1275683192" last-rc-change="1275683192" exec-time="7170" queue-time="0" op-digest="b0b72c35d0ac375641c8163d68c1259f" />
> Jun 5 22:16:19 mysqldrupal-n1 cib: [3147]: info: log_data_element: cib:diff: - <lrm_rsc_op id="MySQL-DB_monitor_25000" operation="monitor" crm-debug-origin="build_active_RAs" crm_feature_set="3.0
> node mysqldrupal-n1 \
> attributes standby="off"
> node mysqldrupal-n2 \
> attributes standby="off"
> primitive MySQL-DB ocf:heartbeat:mysql \
> params binary="/usr/bin/mysqld_safe" config="/etc/my.cnf" datadir="/var/lib/mysql" user="mysql" group="mysql" \
> log="/var/log/mysqld.log" pid="/var/run/mysqld/mysqld.pid" socket="/var/lib/mysql/mysql.sock" test_user="XXXXX" test_table="XXXXX" test_passwd="XXXXXX" \
> op monitor interval="25s" timeout="30s" \
> op start interval="0" timeout="120s" \
> op stop interval="0" timeout="120s" \
> meta migration-threshold="3" failure-timeout="60s" target-role="Started"
> primitive MySQL-VIP ocf:heartbeat:IPaddr2 \
> params ip="XXXXXX" cidr_netmask="24" \
> meta target-role="Started" \
> op start interval="0" timeout="90s" \
> op stop interval="0" timeout="100s" \
> op monitor interval="25s" timeout="30s"
> primitive vmware-fencing stonith:external/unitn.vmware \
> params hostlist="mysqldrupal-n1 mysqldrupal-n2"
> group mysql-grp MySQL-VIP MySQL-DB \
> meta is-managed="true"
> clone Fencing vmware-fencing
> property $id="cib-bootstrap-options" \
> dc-version="1.0.8-9881a7350d6182bae9e8e557cf20a3cc5dac3ee7" \
> cluster-infrastructure="openais" \
> expected-quorum-votes="2" \
> stonith-enabled="true" \
> no-quorum-policy="ignore"
> rsc_defaults $id="rsc-options" \
> resource-stickiness="100"
> _______________________________________________
> Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>
> Project Home: http://www.clusterlabs.org
> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> Bugs: http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker
More information about the Pacemaker
mailing list