[Pacemaker] Apache won't start on RHEL 5.3 2-node cluster
Andrew Beekhof
andrew at beekhof.net
Thu Sep 16 16:26:47 UTC 2010
Did you enable the server-status functionality in httpd.conf?
On Thu, Sep 16, 2010 at 4:44 PM, Brandon DeYoung
<brandon.deyoung at gmail.com> wrote:
> Hi Pacemaker mailing list,
> I'm following the "Clusters from Scratch" guide except I'm using RHEL 5.3
> instead of Fedora. Though I doubt that is relevant for this problem, these
> RHEL 5.3 "machines" are really xen instances. I've looked through the mail
> archives and have not been able to find anyone with the exact same problem.
>
> I have 2 nodes, pace01 and pace02. On both nodes I can start and stop apache
> with init scripts and bring up the pages in a browser.
>
> After adding this configure statement:
>
> crm configure primitive WebSite ocf:heartbeat:apache \
> params configfile=/etc/httpd/conf/httpd.conf \
> op monitor interval=60
>
> This is where I am:
>
> [root at pace01 ~]# crm_mon -fo1
> ============
> Last updated: Thu Sep 16 10:32:56 2010
> Stack: openais
> Current DC: pace01 - partition with quorum
> Version: 1.0.9-89bd754939df5150de7cd76835f98fe90851b677
> 2 Nodes configured, 2 expected votes
> 2 Resources configured.
> ============
>
> Online: [ pace01 pace02 ]
>
> ClusterIP (ocf::heartbeat:IPaddr2): Started pace01
>
> Operations:
> * Node pace02:
> WebSite: migration-threshold=1000000 fail-count=1000000
> + (4) start: rc=1 (unknown error)
> + (5) stop: rc=0 (ok)
> * Node pace01:
> ClusterIP: migration-threshold=1000000
> + (3) start: rc=0 (ok)
> + (4) monitor: interval=30000ms rc=0 (ok)
> WebSite: migration-threshold=1000000 fail-count=1000000
> + (6) start: rc=1 (unknown error)
> + (7) stop: rc=0 (ok)
>
> Failed actions:
> WebSite_start_0 (node=pace02, call=4, rc=1, status=complete): unknown
> error
> WebSite_start_0 (node=pace01, call=6, rc=1, status=complete): unknown
> error
>
>
> ----------------------------------------------------------------------------------------------------------------------------
>
> [root at pace01 ~]# crm configure show
> node pace01
> node pace02
> primitive ClusterIP ocf:heartbeat:IPaddr2 \
> params ip="172.29.84.154" cidr_netmask="32" \
> op monitor interval="30"
> primitive WebSite ocf:heartbeat:apache \
> params configfile="/etc/httpd/conf/httpd.conf" \
> op monitor interval="60"
> property $id="cib-bootstrap-options" \
> dc-version="1.0.9-89bd754939df5150de7cd76835f98fe90851b677" \
> cluster-infrastructure="openais" \
> expected-quorum-votes="2" \
> stonith-enabled="false" \
> no-quorum-policy="ignore"
> rsc_defaults $id="rsc-options" \
> resource-stickiness="100"
> --------------------------------------------------------------------------------------------------------------------------
>
> [root at pace01 ~]# grep -i lrmd.*WebSite /var/log/messages
> Sep 15 14:00:47 pace01 lrmd: [32149]: info: rsc:WebSite:5: probe
> Sep 15 14:00:52 pace01 lrmd: [32149]: info: rsc:WebSite:6: start
> Sep 15 14:00:55 pace01 lrmd: [32149]: info: rsc:WebSite:7: stop
> Sep 15 14:00:56 pace01 lrmd: [32149]: info: RA output: (WebSite:stop:stderr)
> /usr/lib/ocf/resource.d//heartbeat/apache: line 437: kill: (32467) - No such
> p
>
> ------------------------------------------------------------------------------------------------------------------------------
>
> [root at pace01 ~]# crm_verify -LV
> crm_verify[14253]: 2010/09/16_10:34:17 WARN: unpack_rsc_op: Processing
> failed op WebSite_start_0 on pace02: unknown error (1)
> crm_verify[14253]: 2010/09/16_10:34:17 WARN: unpack_rsc_op: Processing
> failed op WebSite_start_0 on pace01: unknown error (1)
> crm_verify[14253]: 2010/09/16_10:34:17 WARN: common_apply_stickiness:
> Forcing WebSite away from pace01 after 1000000 failures (max=1000000)
> crm_verify[14253]: 2010/09/16_10:34:17 WARN: common_apply_stickiness:
> Forcing WebSite away from pace02 after 1000000 failures (max=1000000)
>
>
> Sorry if this next bit is unnecessarily verbose, but here is ~15 seconds of
> /var/log/messages from both nodes when apache is added to the pacemaker
> config:
>
>
> #################################
> ########PACE02##################
> #################################
>
> Sep 15 14:00:43 pace02 cib: [26337]: info: write_cib_contents: Archived
> previous version as /var/lib/heartbeat/crm/cib-88.raw
> Sep 15 14:00:43 pace02 cib: [26337]: info: write_cib_contents: Wrote version
> 0.115.0 of the CIB to disk (digest: fa2d713cb48a259b425133cec51cd077)
> Sep 15 14:00:43 pace02 cib: [26337]: info: retrieveCib: Reading cluster
> configuration from: /var/lib/heartbeat/crm/cib.VBA3sR (digest:
> /var/lib/heartbeat/crm/cib.w0QMZR)
> Sep 15 14:00:43 pace02 crmd: [26283]: info: update_dc: Unset DC pace01
> Sep 15 14:00:43 pace02 crmd: [26283]: info: do_state_transition: State
> transition S_NOT_DC -> S_PENDING [ input=I_PENDING cause=C_FSA_INTERNAL
> origin=do_election_count_vote ]
> Sep 15 14:00:44 pace02 crmd: [26283]: info: update_dc: Set DC to pace01
> (3.0.1)
> Sep 15 14:00:45 pace02 crmd: [26283]: info: do_state_transition: State
> transition S_PENDING -> S_NOT_DC [ input=I_NOT_DC cause=C_HA_MESSAGE
> origin=do_cl_join_finalize_respond ]
> Sep 15 14:00:45 pace02 attrd: [26281]: info: attrd_local_callback: Sending
> full refresh (origin=crmd)
> Sep 15 14:00:45 pace02 attrd: [26281]: info: attrd_trigger_update: Sending
> flush op to all hosts for: fail-count-WebSite (<null>)
> Sep 15 14:00:45 pace02 attrd: [26281]: info: attrd_trigger_update: Sending
> flush op to all hosts for: probe_complete (true)
> Sep 15 14:00:45 pace02 attrd: [26281]: info: attrd_trigger_update: Sending
> flush op to all hosts for: terminate (<null>)
> Sep 15 14:00:45 pace02 attrd: [26281]: info: attrd_trigger_update: Sending
> flush op to all hosts for: last-failure-WebSite (<null>)
> Sep 15 14:00:45 pace02 attrd: [26281]: info: attrd_trigger_update: Sending
> flush op to all hosts for: shutdown (<null>)
> Sep 15 14:00:47 pace02 crmd: [26283]: info: do_lrm_rsc_op: Performing
> key=7:6:7:b7d1a9ca-8a97-4fd9-88b8-b8d303c3ebbf op=WebSite_monitor_0 )
> Sep 15 14:00:47 pace02 lrmd: [26280]: info: rsc:WebSite:3: probe
> Sep 15 14:00:47 pace02 apache[26338]: INFO: apache not running
> Sep 15 14:00:47 pace02 crmd: [26283]: info: process_lrm_event: LRM operation
> WebSite_monitor_0 (call=3, rc=7, cib-update=11, confirmed=true) not running
> Sep 15 14:00:47 pace02 crmd: [26283]: info: do_lrm_rsc_op: Performing
> key=10:6:0:b7d1a9ca-8a97-4fd9-88b8-b8d303c3ebbf op=WebSite_start_0 )
> Sep 15 14:00:47 pace02 lrmd: [26280]: info: rsc:WebSite:4: start
> Sep 15 14:00:48 pace02 apache[26373]: INFO: apache not running
> Sep 15 14:00:48 pace02 apache[26373]: INFO: waiting for apache
> /etc/httpd/conf/httpd.conf to come up
> Sep 15 14:00:49 pace02 crmd: [26283]: info: process_lrm_event: LRM operation
> WebSite_start_0 (call=4, rc=1, cib-update=12, confirmed=true) unknown error
> Sep 15 14:00:49 pace02 attrd: [26281]: info: attrd_ais_dispatch: Update
> relayed from pace01
> Sep 15 14:00:49 pace02 attrd: [26281]: info: attrd_trigger_update: Sending
> flush op to all hosts for: fail-count-WebSite (INFINITY)
> Sep 15 14:00:49 pace02 attrd: [26281]: info: attrd_perform_update: Sent
> update 40: fail-count-WebSite=INFINITY
> Sep 15 14:00:49 pace02 attrd: [26281]: info: attrd_ais_dispatch: Update
> relayed from pace01
> Sep 15 14:00:49 pace02 attrd: [26281]: info: attrd_trigger_update: Sending
> flush op to all hosts for: last-failure-WebSite (1284573649)
> Sep 15 14:00:49 pace02 attrd: [26281]: info: attrd_perform_update: Sent
> update 43: last-failure-WebSite=1284573649
> Sep 15 14:00:50 pace02 crmd: [26283]: info: do_lrm_rsc_op: Performing
> key=1:8:0:b7d1a9ca-8a97-4fd9-88b8-b8d303c3ebbf op=WebSite_stop_0 )
> Sep 15 14:00:50 pace02 lrmd: [26280]: info: rsc:WebSite:5: stop
> Sep 15 14:00:52 pace02 lrmd: [26280]: info: RA output: (WebSite:stop:stderr)
> /usr/lib/ocf/resource.d//heartbeat/apache: line 437: kill: (26406) - No such
> process
> Sep 15 14:00:52 pace02 apache[26438]: INFO: Killing apache PID 26406
> Sep 15 14:00:52 pace02 apache[26438]: INFO: apache stopped.
> Sep 15 14:00:52 pace02 crmd: [26283]: info: process_lrm_event: LRM operation
> WebSite_stop_0 (call=5, rc=0, cib-update=13, confirmed=true) ok
>
> #################################
> ########PACE01##################
> #################################
>
> Sep 15 14:00:42 pace01 cibadmin: [32366]: info: Invoked: cibadmin -Ql
> Sep 15 14:00:42 pace01 cibadmin: [32367]: info: Invoked: cibadmin -Ql
> Sep 15 14:00:43 pace01 crm_shadow: [32387]: info: Invoked: crm_shadow -c
> __crmshell.32345
> Sep 15 14:00:43 pace01 cibadmin: [32388]: info: Invoked: cibadmin -p -U
> Sep 15 14:00:43 pace01 crm_shadow: [32390]: info: Invoked: crm_shadow -C
> __crmshell.32345 --force
> Sep 15 14:00:43 pace01 cib: [32148]: info: cib_replace_notify: Replaced:
> 0.113.7 -> 0.115.1 from <null>
> Sep 15 14:00:43 pace01 cib: [32148]: info: log_data_element: cib:diff: -
> <cib epoch="113" num_updates="7" admin_epoch="0" />
> Sep 15 14:00:43 pace01 cib: [32148]: info: log_data_element: cib:diff: +
> <cib epoch="115" num_updates="1" admin_epoch="0" >
> Sep 15 14:00:43 pace01 cib: [32148]: info: log_data_element: cib:diff: +
> <configuration >
> Sep 15 14:00:43 pace01 cib: [32148]: info: log_data_element: cib:diff: +
> <resources >
> Sep 15 14:00:43 pace01 cib: [32148]: info: log_data_element: cib:diff:
> + <primitive class="ocf" id="WebSite" provider="heartbeat"
> type="apache" __crm_diff_marker__="added:top" >
> Sep 15 14:00:43 pace01 cib: [32148]: info: log_data_element: cib:diff:
> + <instance_attributes id="WebSite-instance_attributes" >
> Sep 15 14:00:43 pace01 cib: [32148]: info: log_data_element: cib:diff:
> + <nvpair id="WebSite-instance_attributes-configfile"
> name="configfile" value="/etc/httpd/conf/httpd.conf" />
> Sep 15 14:00:43 pace01 cib: [32148]: info: log_data_element: cib:diff:
> + </instance_attributes>
> Sep 15 14:00:43 pace01 cib: [32148]: info: log_data_element: cib:diff:
> + <operations >
> Sep 15 14:00:43 pace01 cib: [32148]: info: log_data_element: cib:diff:
> + <op id="WebSite-monitor-60" interval="60" name="monitor" />
> Sep 15 14:00:43 pace01 cib: [32148]: info: log_data_element: cib:diff:
> + </operations>
> Sep 15 14:00:43 pace01 crmd: [32152]: info: abort_transition_graph:
> need_abort:59 - Triggered transition abort (complete=1) : Non-status change
> Sep 15 14:00:43 pace01 attrd: [32150]: info: do_cib_replaced: Sending full
> refresh
> Sep 15 14:00:43 pace01 cib: [32148]: info: log_data_element: cib:diff:
> + </primitive>
> Sep 15 14:00:43 pace01 crmd: [32152]: info: need_abort: Aborting on change
> to epoch
> Sep 15 14:00:43 pace01 attrd: [32150]: info: attrd_trigger_update: Sending
> flush op to all hosts for: fail-count-WebSite (<null>)
> Sep 15 14:00:43 pace01 cib: [32148]: info: log_data_element: cib:diff: +
> </resources>
> Sep 15 14:00:43 pace01 crmd: [32152]: info: do_state_transition: State
> transition S_IDLE -> S_POLICY_ENGINE [ input=I_PE_CALC cause=C_FSA_INTERNAL
> origin=abort_transition_graph ]
> Sep 15 14:00:43 pace01 cib: [32148]: info: log_data_element: cib:diff: +
> </configuration>
> Sep 15 14:00:43 pace01 crmd: [32152]: info: do_state_transition: All 2
> cluster nodes are eligible to run resources.
> Sep 15 14:00:43 pace01 cib: [32148]: info: log_data_element: cib:diff: +
> </cib>
> Sep 15 14:00:43 pace01 crmd: [32152]: info: do_pe_invoke: Query 64:
> Requesting the current CIB: S_POLICY_ENGINE
> Sep 15 14:00:43 pace01 cib: [32148]: info: cib_process_request: Operation
> complete: op cib_replace for section 'all' (origin=local/crm_shadow/2,
> version=0.115.1): ok (rc=0)
> Sep 15 14:00:43 pace01 crmd: [32152]: info: do_state_transition: State
> transition S_POLICY_ENGINE -> S_ELECTION [ input=I_ELECTION
> cause=C_FSA_INTERNAL origin=do_cib_replaced ]
> Sep 15 14:00:43 pace01 cib: [32148]: info: cib_process_request: Operation
> complete: op cib_modify for section nodes (origin=local/crmd/62,
> version=0.115.1): ok (rc=0)
> Sep 15 14:00:43 pace01 crm_shadow: [32392]: info: Invoked: crm_shadow -D
> __crmshell.32345 --force
> Sep 15 14:00:43 pace01 crmd: [32152]: info: update_dc: Unset DC pace01
> Sep 15 14:00:43 pace01 attrd: [32150]: info: attrd_trigger_update: Sending
> flush op to all hosts for: probe_complete (true)
> Sep 15 14:00:43 pace01 cib: [32393]: info: write_cib_contents: Archived
> previous version as /var/lib/heartbeat/crm/cib-81.raw
> Sep 15 14:00:43 pace01 cibadmin: [32394]: info: Invoked: cibadmin -Ql
> Sep 15 14:00:43 pace01 crmd: [32152]: info: do_state_transition: State
> transition S_ELECTION -> S_INTEGRATION [ input=I_ELECTION_DC
> cause=C_FSA_INTERNAL origin=do_election_check ]
> Sep 15 14:00:43 pace01 attrd: [32150]: info: attrd_trigger_update: Sending
> flush op to all hosts for: terminate (<null>)
> Sep 15 14:00:44 pace01 cib: [32393]: info: write_cib_contents: Wrote version
> 0.115.0 of the CIB to disk (digest: e7374d53ff60b0a3e814f26c61104159)
> Sep 15 14:00:44 pace01 crmd: [32152]: info: do_dc_takeover: Taking over DC
> status for this partition
> Sep 15 14:00:44 pace01 attrd: [32150]: info: attrd_trigger_update: Sending
> flush op to all hosts for: last-failure-WebSite (<null>)
> Sep 15 14:00:44 pace01 cib: [32393]: info: retrieveCib: Reading cluster
> configuration from: /var/lib/heartbeat/crm/cib.on269y (digest:
> /var/lib/heartbeat/crm/cib.oCdSPA)
> Sep 15 14:00:44 pace01 cib: [32148]: info: cib_process_readwrite: We are now
> in R/O mode
> Sep 15 14:00:44 pace01 cib: [32148]: info: cib_process_request: Operation
> complete: op cib_slave_all for section 'all' (origin=local/crmd/65,
> version=0.115.1): ok (rc=0)
> Sep 15 14:00:44 pace01 cib: [32148]: info: cib_process_readwrite: We are now
> in R/W mode
> Sep 15 14:00:44 pace01 cib: [32148]: info: cib_process_request: Operation
> complete: op cib_master for section 'all' (origin=local/crmd/66,
> version=0.115.1): ok (rc=0)
> Sep 15 14:00:44 pace01 cib: [32148]: info: cib_process_request: Operation
> complete: op cib_modify for section cib (origin=local/crmd/67,
> version=0.115.1): ok (rc=0)
> Sep 15 14:00:44 pace01 cib: [32148]: WARN: G_SIG_dispatch: Dispatch function
> for SIGCHLD was delayed 110 ms (> 100 ms) before being called (GSource:
> 0x92b9810)
> Sep 15 14:00:44 pace01 cib: [32148]: info: G_SIG_dispatch: started at
> 2020165924 should have started at 2020165913
> Sep 15 14:00:44 pace01 cib: [32148]: WARN: G_SIG_dispatch: Dispatch function
> for SIGCHLD took too long to execute: 70 ms (> 30 ms) (GSource: 0x92b9810)
> Sep 15 14:00:44 pace01 cib: [32148]: info: cib_process_request: Operation
> complete: op cib_modify for section crm_config (origin=local/crmd/69,
> version=0.115.1): ok (rc=0)
> Sep 15 14:00:44 pace01 cib: [32148]: info: cib_process_request: Operation
> complete: op cib_modify for section crm_config (origin=local/crmd/71,
> version=0.115.1): ok (rc=0)
> Sep 15 14:00:44 pace01 attrd: [32150]: info: attrd_trigger_update: Sending
> flush op to all hosts for: shutdown (<null>)
> Sep 15 14:00:44 pace01 crmd: [32152]: info: do_dc_join_offer_all: join-4:
> Waiting on 2 outstanding join acks
> Sep 15 14:00:44 pace01 crmd: [32152]: info: ais_dispatch: Membership 272:
> quorum retained
> Sep 15 14:00:44 pace01 crmd: [32152]: info: crm_ais_dispatch: Setting
> expected votes to 2
> Sep 15 14:00:44 pace01 cib: [32148]: info: cib_process_request: Operation
> complete: op cib_modify for section crm_config (origin=local/crmd/74,
> version=0.115.1): ok (rc=0)
> Sep 15 14:00:44 pace01 crmd: [32152]: info: config_query_callback: Checking
> for expired actions every 900000ms
> Sep 15 14:00:44 pace01 crmd: [32152]: info: config_query_callback: Sending
> expected-votes=2 to corosync
> Sep 15 14:00:44 pace01 crmd: [32152]: info: update_dc: Set DC to pace01
> (3.0.1)
> Sep 15 14:00:44 pace01 crmd: [32152]: info: ais_dispatch: Membership 272:
> quorum retained
> Sep 15 14:00:45 pace01 crmd: [32152]: info: crm_ais_dispatch: Setting
> expected votes to 2
> Sep 15 14:00:45 pace01 cib: [32148]: info: cib_process_request: Operation
> complete: op cib_modify for section crm_config (origin=local/crmd/77,
> version=0.115.1): ok (rc=0)
> Sep 15 14:00:45 pace01 crmd: [32152]: info: do_state_transition: State
> transition S_INTEGRATION -> S_FINALIZE_JOIN [ input=I_INTEGRATED
> cause=C_FSA_INTERNAL origin=check_join_state ]
> Sep 15 14:00:45 pace01 crmd: [32152]: info: do_state_transition: All 2
> cluster nodes responded to the join offer.
> Sep 15 14:00:45 pace01 crmd: [32152]: info: do_dc_join_finalize: join-4:
> Syncing the CIB from pace01 to the rest of the cluster
> Sep 15 14:00:45 pace01 cib: [32148]: info: cib_process_request: Operation
> complete: op cib_sync for section 'all' (origin=local/crmd/78,
> version=0.115.1): ok (rc=0)
> Sep 15 14:00:45 pace01 cib: [32148]: info: cib_process_request: Operation
> complete: op cib_modify for section nodes (origin=local/crmd/79,
> version=0.115.1): ok (rc=0)
> Sep 15 14:00:45 pace01 crmd: [32152]: info: do_dc_join_ack: join-4: Updating
> node state to member for pace02
> Sep 15 14:00:45 pace01 cib: [32148]: info: cib_process_request: Operation
> complete: op cib_modify for section nodes (origin=local/crmd/80,
> version=0.115.1): ok (rc=0)
> Sep 15 14:00:45 pace01 crmd: [32152]: info: do_dc_join_ack: join-4: Updating
> node state to member for pace01
> Sep 15 14:00:45 pace01 cib: [32148]: info: cib_process_request: Operation
> complete: op cib_delete for section //node_state[@uname='pace02']/lrm
> (origin=local/crmd/81, version=0.115.2): ok (rc=0)
> Sep 15 14:00:45 pace01 crmd: [32152]: info: erase_xpath_callback: Deletion
> of "//node_state[@uname='pace02']/lrm": ok (rc=0)
> Sep 15 14:00:45 pace01 cib: [32148]: info: cib_process_request: Operation
> complete: op cib_delete for section //node_state[@uname='pace01']/lrm
> (origin=local/crmd/83, version=0.115.4): ok (rc=0)
> Sep 15 14:00:45 pace01 crmd: [32152]: info: do_state_transition: State
> transition S_FINALIZE_JOIN -> S_POLICY_ENGINE [ input=I_FINALIZED
> cause=C_FSA_INTERNAL origin=check_join_state ]
> Sep 15 14:00:45 pace01 crmd: [32152]: info: do_state_transition: All 2
> cluster nodes are eligible to run resources.
> Sep 15 14:00:45 pace01 cib: [32148]: info: cib_process_request: Operation
> complete: op cib_modify for section nodes (origin=local/crmd/85,
> version=0.115.5): ok (rc=0)
> Sep 15 14:00:45 pace01 crmd: [32152]: info: do_dc_join_final: Ensuring DC,
> quorum and node attributes are up-to-date
> Sep 15 14:00:45 pace01 crmd: [32152]: info: crm_update_quorum: Updating
> quorum status to true (call=87)
> Sep 15 14:00:45 pace01 attrd: [32150]: info: attrd_local_callback: Sending
> full refresh (origin=crmd)
> Sep 15 14:00:45 pace01 cib: [32148]: info: cib_process_request: Operation
> complete: op cib_modify for section cib (origin=local/crmd/87,
> version=0.115.5): ok (rc=0)
> Sep 15 14:00:45 pace01 crmd: [32152]: info: abort_transition_graph:
> do_te_invoke:191 - Triggered transition abort (complete=1) : Peer Cancelled
> Sep 15 14:00:45 pace01 attrd: [32150]: info: attrd_trigger_update: Sending
> flush op to all hosts for: fail-count-WebSite (<null>)
> Sep 15 14:00:45 pace01 crmd: [32152]: info: do_pe_invoke: Query 88:
> Requesting the current CIB: S_POLICY_ENGINE
> Sep 15 14:00:45 pace01 attrd: [32150]: info: attrd_trigger_update: Sending
> flush op to all hosts for: probe_complete (true)
> Sep 15 14:00:45 pace01 crmd: [32152]: info: erase_xpath_callback: Deletion
> of "//node_state[@uname='pace01']/lrm": ok (rc=0)
> Sep 15 14:00:46 pace01 attrd: [32150]: info: attrd_trigger_update: Sending
> flush op to all hosts for: terminate (<null>)
> Sep 15 14:00:46 pace01 crmd: [32152]: info: process_graph_event: Action
> ClusterIP_monitor_0/5 (0:7;5:3:7:6bbbad8b-ed26-4764-b444-d11cc1eae9de)
> initiated by a different transitioner
> Sep 15 14:00:46 pace01 attrd: [32150]: info: attrd_trigger_update: Sending
> flush op to all hosts for: last-failure-WebSite (<null>)
> Sep 15 14:00:46 pace01 crmd: [32152]: info: abort_transition_graph:
> process_graph_event:456 - Triggered transition abort (complete=1,
> tag=lrm_rsc_op, id=ClusterIP_monitor_0,
> magic=0:7;5:3:7:6bbbad8b-ed26-4764-b444-d11cc1eae9de, cib=0.115.5) : Foreign
> event
> Sep 15 14:00:46 pace01 attrd: [32150]: info: attrd_trigger_update: Sending
> flush op to all hosts for: shutdown (<null>)
> Sep 15 14:00:46 pace01 crmd: [32152]: info: process_graph_event: Action
> ClusterIP_start_0/7 (0:0;7:5:0:6bbbad8b-ed26-4764-b444-d11cc1eae9de)
> initiated by a different transitioner
> Sep 15 14:00:46 pace01 crmd: [32152]: info: abort_transition_graph:
> process_graph_event:456 - Triggered transition abort (complete=1,
> tag=lrm_rsc_op, id=ClusterIP_start_0,
> magic=0:0;7:5:0:6bbbad8b-ed26-4764-b444-d11cc1eae9de, cib=0.115.5) : Foreign
> event
> Sep 15 14:00:46 pace01 crmd: [32152]: info: process_graph_event: Action
> ClusterIP_monitor_30000/8 (0:0;8:5:0:6bbbad8b-ed26-4764-b444-d11cc1eae9de)
> initiated by a different transitioner
> Sep 15 14:00:46 pace01 crmd: [32152]: info: abort_transition_graph:
> process_graph_event:456 - Triggered transition abort (complete=1,
> tag=lrm_rsc_op, id=ClusterIP_monitor_30000,
> magic=0:0;8:5:0:6bbbad8b-ed26-4764-b444-d11cc1eae9de, cib=0.115.5) : Foreign
> event
> Sep 15 14:00:46 pace01 crmd: [32152]: info: do_pe_invoke_callback: Invoking
> the PE: query=88, ref=pe_calc-dc-1284573646-36, seq=272, quorate=1
> Sep 15 14:00:46 pace01 pengine: [32151]: notice: unpack_config: On loss of
> CCM Quorum: Ignore
> Sep 15 14:00:46 pace01 crmd: [32152]: info: do_pe_invoke: Query 89:
> Requesting the current CIB: S_POLICY_ENGINE
> Sep 15 14:00:46 pace01 pengine: [32151]: info: unpack_config: Node scores:
> 'red' = -INFINITY, 'yellow' = 0, 'green' = 0
> Sep 15 14:00:46 pace01 crmd: [32152]: info: do_pe_invoke: Query 90:
> Requesting the current CIB: S_POLICY_ENGINE
> Sep 15 14:00:46 pace01 pengine: [32151]: info: determine_online_status: Node
> pace02 is online
> Sep 15 14:00:46 pace01 crmd: [32152]: info: do_pe_invoke: Query 91:
> Requesting the current CIB: S_POLICY_ENGINE
> Sep 15 14:00:46 pace01 pengine: [32151]: info: determine_online_status: Node
> pace01 is online
> Sep 15 14:00:46 pace01 crmd: [32152]: info: do_pe_invoke_callback: Invoking
> the PE: query=91, ref=pe_calc-dc-1284573646-37, seq=272, quorate=1
> Sep 15 14:00:46 pace01 pengine: [32151]: notice: native_print:
> ClusterIP (ocf::heartbeat:IPaddr2): Started pace01
> Sep 15 14:00:46 pace01 pengine: [32151]: notice: native_print: WebSite
> (ocf::heartbeat:apache): Stopped
> Sep 15 14:00:46 pace01 pengine: [32151]: notice: RecurringOp: Start
> recurring monitor (60s) for WebSite on pace02
> Sep 15 14:00:46 pace01 pengine: [32151]: notice: LogActions: Leave resource
> ClusterIP (Started pace01)
> Sep 15 14:00:46 pace01 pengine: [32151]: notice: LogActions: Start
> WebSite (pace02)
> Sep 15 14:00:46 pace01 crmd: [32152]: info: handle_response: pe_calc
> calculation pe_calc-dc-1284573646-36 is obsolete
> Sep 15 14:00:46 pace01 pengine: [32151]: info: process_pe_message:
> Transition 5: PEngine Input stored in: /var/lib/pengine/pe-input-3231.bz2
> Sep 15 14:00:46 pace01 pengine: [32151]: notice: unpack_config: On loss of
> CCM Quorum: Ignore
> Sep 15 14:00:46 pace01 pengine: [32151]: info: unpack_config: Node scores:
> 'red' = -INFINITY, 'yellow' = 0, 'green' = 0
> Sep 15 14:00:46 pace01 pengine: [32151]: info: determine_online_status: Node
> pace02 is online
> Sep 15 14:00:46 pace01 pengine: [32151]: info: determine_online_status: Node
> pace01 is online
> Sep 15 14:00:46 pace01 pengine: [32151]: notice: native_print:
> ClusterIP (ocf::heartbeat:IPaddr2): Started pace01
> Sep 15 14:00:47 pace01 pengine: [32151]: notice: native_print: WebSite
> (ocf::heartbeat:apache): Stopped
> Sep 15 14:00:47 pace01 pengine: [32151]: notice: RecurringOp: Start
> recurring monitor (60s) for WebSite on pace02
> Sep 15 14:00:47 pace01 pengine: [32151]: notice: LogActions: Leave resource
> ClusterIP (Started pace01)
> Sep 15 14:00:47 pace01 pengine: [32151]: notice: LogActions: Start
> WebSite (pace02)
> Sep 15 14:00:47 pace01 crmd: [32152]: info: do_state_transition: State
> transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS
> cause=C_IPC_MESSAGE origin=handle_response ]
> Sep 15 14:00:47 pace01 pengine: [32151]: info: process_pe_message:
> Transition 6: PEngine Input stored in: /var/lib/pengine/pe-input-3232.bz2
> Sep 15 14:00:47 pace01 crmd: [32152]: info: unpack_graph: Unpacked
> transition 6: 7 actions in 7 synapses
> Sep 15 14:00:47 pace01 crmd: [32152]: info: do_te_invoke: Processing graph 6
> (ref=pe_calc-dc-1284573646-37) derived from
> /var/lib/pengine/pe-input-3232.bz2
> Sep 15 14:00:47 pace01 crmd: [32152]: info: te_rsc_command: Initiating
> action 5: monitor WebSite_monitor_0 on pace01 (local)
> Sep 15 14:00:47 pace01 crmd: [32152]: info: do_lrm_rsc_op: Performing
> key=5:6:7:b7d1a9ca-8a97-4fd9-88b8-b8d303c3ebbf op=WebSite_monitor_0 )
> Sep 15 14:00:47 pace01 lrmd: [32149]: info: rsc:WebSite:5: probe
> Sep 15 14:00:47 pace01 crmd: [32152]: info: te_rsc_command: Initiating
> action 7: monitor WebSite_monitor_0 on pace02
> Sep 15 14:00:47 pace01 crmd: [32152]: info: match_graph_event: Action
> WebSite_monitor_0 (7) confirmed on pace02 (rc=0)
> Sep 15 14:00:47 pace01 crmd: [32152]: info: te_rsc_command: Initiating
> action 6: probe_complete probe_complete on pace02 - no waiting
> Sep 15 14:00:47 pace01 apache[32399]: INFO: apache not running
> Sep 15 14:00:47 pace01 crmd: [32152]: info: process_lrm_event: LRM operation
> WebSite_monitor_0 (call=5, rc=7, cib-update=92, confirmed=true) not running
> Sep 15 14:00:47 pace01 crmd: [32152]: info: match_graph_event: Action
> WebSite_monitor_0 (5) confirmed on pace01 (rc=0)
> Sep 15 14:00:47 pace01 crmd: [32152]: info: te_rsc_command: Initiating
> action 4: probe_complete probe_complete on pace01 (local) - no waiting
> Sep 15 14:00:47 pace01 crmd: [32152]: info: te_pseudo_action: Pseudo action
> 3 fired and confirmed
> Sep 15 14:00:47 pace01 crmd: [32152]: info: te_rsc_command: Initiating
> action 10: start WebSite_start_0 on pace02
> Sep 15 14:00:49 pace01 crmd: [32152]: WARN: status_from_rc: Action 10
> (WebSite_start_0) on pace02 failed (target: 0 vs. rc: 1): Error
> Sep 15 14:00:49 pace01 crmd: [32152]: WARN: update_failcount: Updating
> failcount for WebSite on pace02 after failed start: rc=1 (update=INFINITY,
> time=1284573649)
> Sep 15 14:00:49 pace01 crmd: [32152]: info: abort_transition_graph:
> match_graph_event:272 - Triggered transition abort (complete=0,
> tag=lrm_rsc_op, id=WebSite_start_0,
> magic=0:1;10:6:0:b7d1a9ca-8a97-4fd9-88b8-b8d303c3ebbf, cib=0.115.8) : Event
> failed
> Sep 15 14:00:49 pace01 crmd: [32152]: info: update_abort_priority: Abort
> priority upgraded from 0 to 1
> Sep 15 14:00:49 pace01 crmd: [32152]: info: update_abort_priority: Abort
> action done superceeded by restart
> Sep 15 14:00:49 pace01 crmd: [32152]: info: match_graph_event: Action
> WebSite_start_0 (10) confirmed on pace02 (rc=4)
> Sep 15 14:00:49 pace01 crmd: [32152]: info: run_graph:
> ====================================================
> Sep 15 14:00:49 pace01 crmd: [32152]: notice: run_graph: Transition 6
> (Complete=6, Pending=0, Fired=0, Skipped=1, Incomplete=0,
> Source=/var/lib/pengine/pe-input-3232.bz2): Stopped
> Sep 15 14:00:49 pace01 crmd: [32152]: info: te_graph_trigger: Transition 6
> is now complete
> Sep 15 14:00:49 pace01 crmd: [32152]: info: do_state_transition: State
> transition S_TRANSITION_ENGINE -> S_POLICY_ENGINE [ input=I_PE_CALC
> cause=C_FSA_INTERNAL origin=notify_crmd ]
> Sep 15 14:00:49 pace01 crmd: [32152]: info: do_state_transition: All 2
> cluster nodes are eligible to run resources.
> Sep 15 14:00:49 pace01 crmd: [32152]: info: do_pe_invoke: Query 93:
> Requesting the current CIB: S_POLICY_ENGINE
> Sep 15 14:00:49 pace01 crmd: [32152]: info: do_pe_invoke_callback: Invoking
> the PE: query=93, ref=pe_calc-dc-1284573649-43, seq=272, quorate=1
> Sep 15 14:00:49 pace01 pengine: [32151]: notice: unpack_config: On loss of
> CCM Quorum: Ignore
> Sep 15 14:00:49 pace01 crmd: [32152]: info: abort_transition_graph:
> te_update_diff:146 - Triggered transition abort (complete=1,
> tag=transient_attributes, id=pace02, magic=NA, cib=0.115.9) : Transient
> attribute: update
> Sep 15 14:00:49 pace01 pengine: [32151]: info: unpack_config: Node scores:
> 'red' = -INFINITY, 'yellow' = 0, 'green' = 0
> Sep 15 14:00:49 pace01 crmd: [32152]: info: abort_transition_graph:
> te_update_diff:146 - Triggered transition abort (complete=1,
> tag=transient_attributes, id=pace02, magic=NA, cib=0.115.10) : Transient
> attribute: update
> Sep 15 14:00:49 pace01 pengine: [32151]: info: determine_online_status: Node
> pace02 is online
> Sep 15 14:00:49 pace01 crmd: [32152]: info: do_pe_invoke: Query 94:
> Requesting the current CIB: S_POLICY_ENGINE
> Sep 15 14:00:49 pace01 pengine: [32151]: WARN: unpack_rsc_op: Processing
> failed op WebSite_start_0 on pace02: unknown error (1)
> Sep 15 14:00:49 pace01 crmd: [32152]: info: do_pe_invoke: Query 95:
> Requesting the current CIB: S_POLICY_ENGINE
> Sep 15 14:00:49 pace01 pengine: [32151]: info: determine_online_status: Node
> pace01 is online
> Sep 15 14:00:49 pace01 crmd: [32152]: info: do_pe_invoke_callback: Invoking
> the PE: query=95, ref=pe_calc-dc-1284573649-44, seq=272, quorate=1
> Sep 15 14:00:49 pace01 pengine: [32151]: notice: native_print:
> ClusterIP (ocf::heartbeat:IPaddr2): Started pace01
> Sep 15 14:00:49 pace01 pengine: [32151]: notice: native_print: WebSite
> (ocf::heartbeat:apache): Started pace02 FAILED
> Sep 15 14:00:50 pace01 pengine: [32151]: notice: RecurringOp: Start
> recurring monitor (60s) for WebSite on pace02
> Sep 15 14:00:50 pace01 pengine: [32151]: notice: LogActions: Leave resource
> ClusterIP (Started pace01)
> Sep 15 14:00:50 pace01 pengine: [32151]: notice: LogActions: Recover
> resource WebSite (Started pace02)
> Sep 15 14:00:50 pace01 crmd: [32152]: info: handle_response: pe_calc
> calculation pe_calc-dc-1284573649-43 is obsolete
> Sep 15 14:00:50 pace01 pengine: [32151]: info: process_pe_message:
> Transition 7: PEngine Input stored in: /var/lib/pengine/pe-input-3233.bz2
> Sep 15 14:00:50 pace01 pengine: [32151]: notice: unpack_config: On loss of
> CCM Quorum: Ignore
> Sep 15 14:00:50 pace01 pengine: [32151]: info: unpack_config: Node scores:
> 'red' = -INFINITY, 'yellow' = 0, 'green' = 0
> Sep 15 14:00:50 pace01 pengine: [32151]: info: determine_online_status: Node
> pace02 is online
> Sep 15 14:00:50 pace01 pengine: [32151]: WARN: unpack_rsc_op: Processing
> failed op WebSite_start_0 on pace02: unknown error (1)
> Sep 15 14:00:50 pace01 pengine: [32151]: info: determine_online_status: Node
> pace01 is online
> Sep 15 14:00:50 pace01 pengine: [32151]: notice: native_print:
> ClusterIP (ocf::heartbeat:IPaddr2): Started pace01
> Sep 15 14:00:50 pace01 pengine: [32151]: notice: native_print: WebSite
> (ocf::heartbeat:apache): Started pace02 FAILED
> Sep 15 14:00:50 pace01 pengine: [32151]: info: get_failcount: WebSite has
> failed INFINITY times on pace02
> Sep 15 14:00:50 pace01 pengine: [32151]: WARN: common_apply_stickiness:
> Forcing WebSite away from pace02 after 1000000 failures (max=1000000)
> Sep 15 14:00:50 pace01 pengine: [32151]: notice: RecurringOp: Start
> recurring monitor (60s) for WebSite on pace01
> Sep 15 14:00:50 pace01 pengine: [32151]: notice: LogActions: Leave resource
> ClusterIP (Started pace01)
> Sep 15 14:00:50 pace01 pengine: [32151]: notice: LogActions: Move resource
> WebSite (Started pace02 -> pace01)
> Sep 15 14:00:50 pace01 crmd: [32152]: info: do_state_transition: State
> transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS
> cause=C_IPC_MESSAGE origin=handle_response ]
> Sep 15 14:00:50 pace01 pengine: [32151]: info: process_pe_message:
> Transition 8: PEngine Input stored in: /var/lib/pengine/pe-input-3234.bz2
> Sep 15 14:00:50 pace01 crmd: [32152]: info: unpack_graph: Unpacked
> transition 8: 4 actions in 4 synapses
> Sep 15 14:00:50 pace01 crmd: [32152]: info: do_te_invoke: Processing graph 8
> (ref=pe_calc-dc-1284573649-44) derived from
> /var/lib/pengine/pe-input-3234.bz2
> Sep 15 14:00:50 pace01 crmd: [32152]: info: te_rsc_command: Initiating
> action 1: stop WebSite_stop_0 on pace02
> Sep 15 14:00:52 pace01 crmd: [32152]: info: match_graph_event: Action
> WebSite_stop_0 (1) confirmed on pace02 (rc=0)
> Sep 15 14:00:52 pace01 crmd: [32152]: info: te_rsc_command: Initiating
> action 9: start WebSite_start_0 on pace01 (local)
> Sep 15 14:00:52 pace01 crmd: [32152]: info: do_lrm_rsc_op: Performing
> key=9:8:0:b7d1a9ca-8a97-4fd9-88b8-b8d303c3ebbf op=WebSite_start_0 )
> Sep 15 14:00:52 pace01 lrmd: [32149]: info: rsc:WebSite:6: start
> Sep 15 14:00:52 pace01 crmd: [32152]: info: te_pseudo_action: Pseudo action
> 3 fired and confirmed
> Sep 15 14:00:52 pace01 apache[32434]: INFO: apache not running
> Sep 15 14:00:52 pace01 apache[32434]: INFO: waiting for apache
> /etc/httpd/conf/httpd.conf to come up
> Sep 15 14:00:53 pace01 crmd: [32152]: info: process_lrm_event: LRM operation
> WebSite_start_0 (call=6, rc=1, cib-update=96, confirmed=true) unknown error
> Sep 15 14:00:53 pace01 crmd: [32152]: WARN: status_from_rc: Action 9
> (WebSite_start_0) on pace01 failed (target: 0 vs. rc: 1): Error
> Sep 15 14:00:53 pace01 crmd: [32152]: WARN: update_failcount: Updating
> failcount for WebSite on pace01 after failed start: rc=1 (update=INFINITY,
> time=1284573653)
> Sep 15 14:00:53 pace01 crmd: [32152]: info: abort_transition_graph:
> match_graph_event:272 - Triggered transition abort (complete=0,
> tag=lrm_rsc_op, id=WebSite_start_0,
> magic=0:1;9:8:0:b7d1a9ca-8a97-4fd9-88b8-b8d303c3ebbf, cib=0.115.12) : Event
> failed
> Sep 15 14:00:53 pace01 crmd: [32152]: info: update_abort_priority: Abort
> priority upgraded from 0 to 1
> Sep 15 14:00:53 pace01 crmd: [32152]: info: update_abort_priority: Abort
> action done superceeded by restart
> Sep 15 14:00:53 pace01 crmd: [32152]: info: match_graph_event: Action
> WebSite_start_0 (9) confirmed on pace01 (rc=4)
> Sep 15 14:00:53 pace01 crmd: [32152]: info: run_graph:
> ====================================================
> Sep 15 14:00:53 pace01 crmd: [32152]: notice: run_graph: Transition 8
> (Complete=3, Pending=0, Fired=0, Skipped=1, Incomplete=0,
> Source=/var/lib/pengine/pe-input-3234.bz2): Stopped
> Sep 15 14:00:53 pace01 crmd: [32152]: info: te_graph_trigger: Transition 8
> is now complete
> Sep 15 14:00:53 pace01 crmd: [32152]: info: do_state_transition: State
> transition S_TRANSITION_ENGINE -> S_POLICY_ENGINE [ input=I_PE_CALC
> cause=C_FSA_INTERNAL origin=notify_crmd ]
> Sep 15 14:00:53 pace01 crmd: [32152]: info: do_state_transition: All 2
> cluster nodes are eligible to run resources.
> Sep 15 14:00:53 pace01 attrd: [32150]: info: attrd_trigger_update: Sending
> flush op to all hosts for: fail-count-WebSite (INFINITY)
> Sep 15 14:00:53 pace01 crmd: [32152]: info: do_pe_invoke: Query 97:
> Requesting the current CIB: S_POLICY_ENGINE
> Sep 15 14:00:53 pace01 attrd: [32150]: info: attrd_perform_update: Sent
> update 74: fail-count-WebSite=INFINITY
> Sep 15 14:00:53 pace01 crmd: [32152]: info: do_pe_invoke_callback: Invoking
> the PE: query=97, ref=pe_calc-dc-1284573653-47, seq=272, quorate=1
> Sep 15 14:00:53 pace01 pengine: [32151]: notice: unpack_config: On loss of
> CCM Quorum: Ignore
> Sep 15 14:00:53 pace01 attrd: [32150]: info: attrd_trigger_update: Sending
> flush op to all hosts for: last-failure-WebSite (1284573653)
> Sep 15 14:00:54 pace01 crmd: [32152]: info: abort_transition_graph:
> te_update_diff:146 - Triggered transition abort (complete=1,
> tag=transient_attributes, id=pace01, magic=NA, cib=0.115.13) : Transient
> attribute: update
> Sep 15 14:00:54 pace01 pengine: [32151]: info: unpack_config: Node scores:
> 'red' = -INFINITY, 'yellow' = 0, 'green' = 0
> Sep 15 14:00:54 pace01 attrd: [32150]: info: attrd_perform_update: Sent
> update 77: last-failure-WebSite=1284573653
> Sep 15 14:00:54 pace01 crmd: [32152]: info: abort_transition_graph:
> te_update_diff:146 - Triggered transition abort (complete=1,
> tag=transient_attributes, id=pace01, magic=NA, cib=0.115.14) : Transient
> attribute: update
> Sep 15 14:00:54 pace01 pengine: [32151]: info: determine_online_status: Node
> pace02 is online
> Sep 15 14:00:54 pace01 crmd: [32152]: info: do_pe_invoke: Query 98:
> Requesting the current CIB: S_POLICY_ENGINE
> Sep 15 14:00:54 pace01 pengine: [32151]: WARN: unpack_rsc_op: Processing
> failed op WebSite_start_0 on pace02: unknown error (1)
> Sep 15 14:00:54 pace01 crmd: [32152]: info: do_pe_invoke: Query 99:
> Requesting the current CIB: S_POLICY_ENGINE
> Sep 15 14:00:54 pace01 pengine: [32151]: info: determine_online_status: Node
> pace01 is online
> Sep 15 14:00:54 pace01 crmd: [32152]: info: do_pe_invoke_callback: Invoking
> the PE: query=99, ref=pe_calc-dc-1284573654-48, seq=272, quorate=1
> Sep 15 14:00:54 pace01 pengine: [32151]: WARN: unpack_rsc_op: Processing
> failed op WebSite_start_0 on pace01: unknown error (1)
> Sep 15 14:00:54 pace01 pengine: [32151]: notice: native_print:
> ClusterIP (ocf::heartbeat:IPaddr2): Started pace01
> Sep 15 14:00:54 pace01 pengine: [32151]: notice: native_print: WebSite
> (ocf::heartbeat:apache): Started pace01 FAILED
> Sep 15 14:00:54 pace01 pengine: [32151]: info: get_failcount: WebSite has
> failed INFINITY times on pace02
> Sep 15 14:00:54 pace01 pengine: [32151]: WARN: common_apply_stickiness:
> Forcing WebSite away from pace02 after 1000000 failures (max=1000000)
> Sep 15 14:00:54 pace01 pengine: [32151]: notice: RecurringOp: Start
> recurring monitor (60s) for WebSite on pace01
> Sep 15 14:00:54 pace01 pengine: [32151]: notice: LogActions: Leave resource
> ClusterIP (Started pace01)
> Sep 15 14:00:54 pace01 pengine: [32151]: notice: LogActions: Recover
> resource WebSite (Started pace01)
> Sep 15 14:00:54 pace01 crmd: [32152]: info: handle_response: pe_calc
> calculation pe_calc-dc-1284573653-47 is obsolete
> Sep 15 14:00:54 pace01 pengine: [32151]: info: process_pe_message:
> Transition 9: PEngine Input stored in: /var/lib/pengine/pe-input-3235.bz2
> Sep 15 14:00:54 pace01 pengine: [32151]: notice: unpack_config: On loss of
> CCM Quorum: Ignore
> Sep 15 14:00:54 pace01 pengine: [32151]: info: unpack_config: Node scores:
> 'red' = -INFINITY, 'yellow' = 0, 'green' = 0
> Sep 15 14:00:54 pace01 pengine: [32151]: info: determine_online_status: Node
> pace02 is online
> Sep 15 14:00:54 pace01 pengine: [32151]: WARN: unpack_rsc_op: Processing
> failed op WebSite_start_0 on pace02: unknown error (1)
> Sep 15 14:00:54 pace01 pengine: [32151]: info: determine_online_status: Node
> pace01 is online
> Sep 15 14:00:54 pace01 pengine: [32151]: WARN: unpack_rsc_op: Processing
> failed op WebSite_start_0 on pace01: unknown error (1)
> Sep 15 14:00:54 pace01 pengine: [32151]: notice: native_print:
> ClusterIP (ocf::heartbeat:IPaddr2): Started pace01
> Sep 15 14:00:55 pace01 pengine: [32151]: notice: native_print: WebSite
> (ocf::heartbeat:apache): Started pace01 FAILED
> Sep 15 14:00:55 pace01 pengine: [32151]: info: get_failcount: WebSite has
> failed INFINITY times on pace01
> Sep 15 14:00:55 pace01 pengine: [32151]: WARN: common_apply_stickiness:
> Forcing WebSite away from pace01 after 1000000 failures (max=1000000)
> Sep 15 14:00:55 pace01 pengine: [32151]: info: get_failcount: WebSite has
> failed INFINITY times on pace02
> Sep 15 14:00:55 pace01 pengine: [32151]: WARN: common_apply_stickiness:
> Forcing WebSite away from pace02 after 1000000 failures (max=1000000)
> Sep 15 14:00:55 pace01 pengine: [32151]: info: native_color: Resource
> WebSite cannot run anywhere
> Sep 15 14:00:55 pace01 pengine: [32151]: notice: LogActions: Leave resource
> ClusterIP (Started pace01)
> Sep 15 14:00:55 pace01 pengine: [32151]: notice: LogActions: Stop resource
> WebSite (pace01)
> Sep 15 14:00:55 pace01 crmd: [32152]: info: do_state_transition: State
> transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS
> cause=C_IPC_MESSAGE origin=handle_response ]
> Sep 15 14:00:55 pace01 crmd: [32152]: info: unpack_graph: Unpacked
> transition 10: 2 actions in 2 synapses
> Sep 15 14:00:55 pace01 pengine: [32151]: info: process_pe_message:
> Transition 10: PEngine Input stored in: /var/lib/pengine/pe-input-3236.bz2
> Sep 15 14:00:55 pace01 crmd: [32152]: info: do_te_invoke: Processing graph
> 10 (ref=pe_calc-dc-1284573654-48) derived from
> /var/lib/pengine/pe-input-3236.bz2
> Sep 15 14:00:55 pace01 crmd: [32152]: info: te_rsc_command: Initiating
> action 2: stop WebSite_stop_0 on pace01 (local)
> Sep 15 14:00:55 pace01 crmd: [32152]: info: do_lrm_rsc_op: Performing
> key=2:10:0:b7d1a9ca-8a97-4fd9-88b8-b8d303c3ebbf op=WebSite_stop_0 )
> Sep 15 14:00:55 pace01 lrmd: [32149]: info: rsc:WebSite:7: stop
> Sep 15 14:00:56 pace01 lrmd: [32149]: info: RA output: (WebSite:stop:stderr)
> /usr/lib/ocf/resource.d//heartbeat/apache: line 437: kill: (32467) - No such
> process
> Sep 15 14:00:56 pace01 apache[32529]: INFO: Killing apache PID 32467
> Sep 15 14:00:56 pace01 apache[32529]: INFO: apache stopped.
> Sep 15 14:00:57 pace01 crmd: [32152]: info: process_lrm_event: LRM operation
> WebSite_stop_0 (call=7, rc=0, cib-update=100, confirmed=true) ok
> Sep 15 14:00:57 pace01 crmd: [32152]: info: match_graph_event: Action
> WebSite_stop_0 (2) confirmed on pace01 (rc=0)
> Sep 15 14:00:57 pace01 crmd: [32152]: info: te_pseudo_action: Pseudo action
> 3 fired and confirmed
> Sep 15 14:00:57 pace01 crmd: [32152]: info: run_graph:
> ====================================================
> Sep 15 14:00:57 pace01 crmd: [32152]: notice: run_graph: Transition 10
> (Complete=2, Pending=0, Fired=0, Skipped=0, Incomplete=0,
> Source=/var/lib/pengine/pe-input-3236.bz2): Complete
> Sep 15 14:00:57 pace01 crmd: [32152]: info: te_graph_trigger: Transition 10
> is now complete
> Sep 15 14:00:57 pace01 crmd: [32152]: info: notify_crmd: Transition 10
> status: done - <null>
> Sep 15 14:00:57 pace01 crmd: [32152]: info: do_state_transition: State
> transition S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SUCCESS
> cause=C_FSA_INTERNAL origin=notify_crmd ]
> Sep 15 14:00:57 pace01 crmd: [32152]: info: do_state_transition: Starting
> PEngine Recheck Timer
>
>
> Thanks in advance for your help!
> ~Brandon
>
>
>
>
> _______________________________________________
> Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>
> Project Home: http://www.clusterlabs.org
> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> Bugs:
> http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker
>
>
More information about the Pacemaker
mailing list