[Pacemaker] Need help in starting resources -

Thu Feb 17 15:52:17 UTC 2011

okay, you are right. inthe syslog of lxds07 I did a grep on "crmd" and "lrmd":

Feb 17 15:32:42 lxds07 crmd: [1083]: info: do_pe_invoke: Waiting for the PE to connect
Feb 17 16:23:50 lxds07 crmd: [1083]: info: handle_shutdown_request: Creating shutdown request for lxds05 (state=S_POLICY_ENGINE)
Feb 17 16:23:50 lxds07 crmd: [1083]: info: do_pe_invoke: Waiting for the PE to connect
Feb 17 16:23:50 lxds07 crmd: [1083]: info: abort_transition_graph: te_update_diff:146 - Triggered transition abort (complete=1, tag=transient_attributes, id=lxds05, magic=NA, cib=0.3590.9) : Transient attribute: update
Feb 17 16:23:50 lxds07 crmd: [1083]: info: do_pe_invoke: Waiting for the PE to connect
Feb 17 16:24:53 lxds07 crmd: [1083]: notice: ais_dispatch: Membership 2484: quorum lost
Feb 17 16:24:53 lxds07 crmd: [1083]: info: ais_status_callback: status: lxds05 is now lost (was member)
Feb 17 16:24:53 lxds07 crmd: [1083]: info: crm_update_peer: Node lxds05: id=1821026496 state=lost (new) addr=r(0) ip(192.168.138.108)  votes=1 born=2480 seen=2480 proc=00000000000000000000000000013312
Feb 17 16:24:53 lxds07 crmd: [1083]: info: erase_node_from_join: Removed node lxds05 from join calculations: welcomed=0 itegrated=0 finalized=0 confirmed=1
Feb 17 16:24:53 lxds07 crmd: [1083]: info: crm_update_quorum: Updating quorum status to false (call=209)
Feb 17 16:24:53 lxds07 cib: [1079]: info: cib_process_request: Operation complete: op cib_modify for section nodes (origin=local/crmd/207, version=0.3590.9): ok (rc=0)
Feb 17 16:24:53 lxds07 cib: [1079]: info: cib_process_request: Operation complete: op cib_modify for section cib (origin=local/crmd/209, version=0.3591.1): ok (rc=0)
Feb 17 16:24:53 lxds07 crmd: [1083]: info: crm_ais_dispatch: Setting expected votes to 2
Feb 17 16:24:53 lxds07 crmd: [1083]: info: do_pe_invoke: Waiting for the PE to connect
Feb 17 16:24:53 lxds07 crmd: [1083]: WARN: match_down_event: No match for shutdown action on lxds05
Feb 17 16:24:53 lxds07 crmd: [1083]: info: te_update_diff: Stonith/shutdown of lxds05 not matched
Feb 17 16:24:53 lxds07 crmd: [1083]: info: abort_transition_graph: te_update_diff:191 - Triggered transition abort (complete=1, tag=node_state, id=lxds05, magic=NA, cib=0.3590.10) : Node failure
Feb 17 16:24:53 lxds07 crmd: [1083]: info: abort_transition_graph: need_abort:59 - Triggered transition abort (complete=1) : Non-status change
Feb 17 16:24:53 lxds07 crmd: [1083]: info: need_abort: Aborting on change to have-quorum
Feb 17 16:24:53 lxds07 crmd: [1083]: info: do_pe_invoke: Waiting for the PE to connect
Feb 17 16:24:53 lxds07 crmd: [1083]: WARN: register_fsa_input_adv: do_pe_invoke stalled the FSA with pending inputs
Feb 17 16:24:53 lxds07 cib: [1079]: info: cib_process_request: Operation complete: op cib_modify for section crm_config (origin=local/crmd/211, version=0.3591.1): ok (rc=0)
Feb 17 16:32:10 lxds07 crmd: [1083]: notice: ais_dispatch: Membership 2488: quorum acquired
Feb 17 16:32:10 lxds07 crmd: [1083]: info: ais_status_callback: status: lxds05 is now member (was lost)
Feb 17 16:32:10 lxds07 crmd: [1083]: info: crm_update_peer: Node lxds05: id=1821026496 state=member (new) addr=r(0) ip(192.168.138.108)  votes=1 born=2480 seen=2488 proc=00000000000000000000000000013312
Feb 17 16:32:10 lxds07 crmd: [1083]: info: crm_update_quorum: Updating quorum status to true (call=216)
Feb 17 16:32:10 lxds07 cib: [1079]: info: cib_process_request: Operation complete: op cib_delete for section //node_state[@uname='lxds05']/lrm (origin=local/crmd/212, version=0.3591.2): ok (rc=0)
Feb 17 16:32:10 lxds07 cib: [1079]: info: cib_process_request: Operation complete: op cib_delete for section //node_state[@uname='lxds05']/transient_attributes (origin=local/crmd/213, version=0.3591.3): ok (rc=0)
Feb 17 16:32:10 lxds07 cib: [1079]: info: cib_process_request: Operation complete: op cib_modify for section nodes (origin=local/crmd/214, version=0.3591.3): ok (rc=0)
Feb 17 16:32:10 lxds07 cib: [1079]: info: cib_process_request: Operation complete: op cib_modify for section cib (origin=local/crmd/216, version=0.3592.1): ok (rc=0)
Feb 17 16:32:10 lxds07 crmd: [1083]: info: crm_ais_dispatch: Setting expected votes to 2
Feb 17 16:32:10 lxds07 crmd: [1083]: info: do_pe_invoke: Waiting for the PE to connect
Feb 17 16:32:10 lxds07 crmd: [1083]: info: erase_xpath_callback: Deletion of "//node_state[@uname='lxds05']/lrm": ok (rc=0)
Feb 17 16:32:10 lxds07 crmd: [1083]: info: abort_transition_graph: te_update_diff:157 - Triggered transition abort (complete=1, tag=transient_attributes, id=lxds05, magic=NA, cib=0.3591.3) : Transient attribute: removal
Feb 17 16:32:10 lxds07 crmd: [1083]: info: erase_xpath_callback: Deletion of "//node_state[@uname='lxds05']/transient_attributes": ok (rc=0)
Feb 17 16:32:10 lxds07 crmd: [1083]: info: abort_transition_graph: need_abort:59 - Triggered transition abort (complete=1) : Non-status change
Feb 17 16:32:10 lxds07 crmd: [1083]: info: need_abort: Aborting on change to have-quorum
Feb 17 16:32:10 lxds07 crmd: [1083]: info: ais_dispatch: Membership 2488: quorum retained
Feb 17 16:32:10 lxds07 cib: [1079]: info: cib_process_request: Operation complete: op cib_modify for section crm_config (origin=local/crmd/218, version=0.3592.1): ok (rc=0)
Feb 17 16:32:10 lxds07 cib: [1079]: info: cib_process_request: Operation complete: op cib_modify for section nodes (origin=local/crmd/219, version=0.3592.1): ok (rc=0)
Feb 17 16:32:10 lxds07 crmd: [1083]: info: crm_ais_dispatch: Setting expected votes to 2
Feb 17 16:32:10 lxds07 crmd: [1083]: info: do_pe_invoke: Waiting for the PE to connect
Feb 17 16:32:10 lxds07 crmd: [1083]: WARN: register_fsa_input_adv: do_pe_invoke stalled the FSA with pending inputs
Feb 17 16:32:10 lxds07 crmd: [1083]: info: do_pe_invoke: Waiting for the PE to connect
Feb 17 16:32:10 lxds07 crmd: [1083]: info: do_pe_invoke: Waiting for the PE to connect
Feb 17 16:32:10 lxds07 cib: [1079]: info: cib_process_request: Operation complete: op cib_modify for section crm_config (origin=local/crmd/222, version=0.3592.1): ok (rc=0)
Feb 17 16:32:11 lxds07 crmd: [1083]: info: do_pe_invoke: Waiting for the PE to connect
Feb 17 16:32:12 lxds07 crmd: [1083]: info: do_state_transition: State transition S_POLICY_ENGINE -> S_INTEGRATION [ input=I_NODE_JOIN cause=C_HA_MESSAGE origin=route_message ]
Feb 17 16:32:12 lxds07 crmd: [1083]: info: update_dc: Unset DC lxds07
Feb 17 16:32:12 lxds07 crmd: [1083]: info: do_dc_join_offer_all: A new node joined the cluster
Feb 17 16:32:12 lxds07 crmd: [1083]: info: join_make_offer: Making join offers based on membership 2488
Feb 17 16:32:12 lxds07 crmd: [1083]: info: do_dc_join_offer_all: join-6: Waiting on 2 outstanding join acks
Feb 17 16:32:12 lxds07 crmd: [1083]: info: do_pe_invoke: Waiting for the PE to connect
Feb 17 16:32:12 lxds07 crmd: [1083]: info: update_dc: Set DC to lxds07 (3.0.1)
Feb 17 16:32:12 lxds07 crmd: [1083]: info: do_pe_invoke: Waiting for the PE to connect
Feb 17 16:32:12 lxds07 crmd: [1083]: info: do_pe_invoke: Waiting for the PE to connect
Feb 17 16:32:12 lxds07 crmd: [1083]: info: do_pe_invoke: Waiting for the PE to connect
Feb 17 16:32:12 lxds07 crmd: [1083]: info: do_pe_invoke: Waiting for the PE to connect
Feb 17 16:32:12 lxds07 crmd: [1083]: WARN: register_fsa_input_adv: do_pe_invoke stalled the FSA with pending inputs
Feb 17 16:32:12 lxds07 crmd: [1083]: info: do_state_transition: State transition S_INTEGRATION -> S_FINALIZE_JOIN [ input=I_INTEGRATED cause=C_FSA_INTERNAL origin=check_join_state ]
Feb 17 16:32:12 lxds07 crmd: [1083]: info: do_state_transition: All 2 cluster nodes responded to the join offer.
Feb 17 16:32:12 lxds07 crmd: [1083]: info: do_dc_join_finalize: join-6: Syncing the CIB from lxds07 to the rest of the cluster
Feb 17 16:32:12 lxds07 crmd: [1083]: info: do_pe_invoke: Waiting for the PE to connect
Feb 17 16:32:12 lxds07 cib: [1079]: info: cib_process_request: Operation complete: op cib_sync for section 'all' (origin=local/crmd/224, version=0.3592.1): ok (rc=0)
Feb 17 16:32:12 lxds07 cib: [1079]: info: cib_process_request: Operation complete: op cib_modify for section nodes (origin=local/crmd/225, version=0.3592.1): ok (rc=0)
Feb 17 16:32:12 lxds07 cib: [1079]: info: cib_process_request: Operation complete: op cib_modify for section nodes (origin=local/crmd/226, version=0.3592.1): ok (rc=0)
Feb 17 16:32:12 lxds07 cib: [1079]: info: cib_process_request: Operation complete: op cib_delete for section //node_state[@uname='lxds05']/transient_attributes (origin=lxds05/crmd/6, version=0.3592.1): ok (rc=0)
Feb 17 16:32:12 lxds07 crmd: [1083]: info: do_pe_invoke: Waiting for the PE to connect
Feb 17 16:32:12 lxds07 crmd: [1083]: info: do_dc_join_ack: join-6: Updating node state to member for lxds05
Feb 17 16:32:12 lxds07 crmd: [1083]: info: do_pe_invoke: Waiting for the PE to connect
Feb 17 16:32:12 lxds07 crmd: [1083]: info: do_dc_join_ack: join-6: Updating node state to member for lxds07
Feb 17 16:32:12 lxds07 crmd: [1083]: info: do_pe_invoke: Waiting for the PE to connect
Feb 17 16:32:12 lxds07 cib: [1079]: info: cib_process_request: Operation complete: op cib_delete for section //node_state[@uname='lxds05']/lrm (origin=local/crmd/227, version=0.3592.1): ok (rc=0)
Feb 17 16:32:12 lxds07 crmd: [1083]: info: erase_xpath_callback: Deletion of "//node_state[@uname='lxds05']/lrm": ok (rc=0)
Feb 17 16:32:12 lxds07 crmd: [1083]: info: do_state_transition: State transition S_FINALIZE_JOIN -> S_POLICY_ENGINE [ input=I_FINALIZED cause=C_FSA_INTERNAL origin=check_join_state ]
Feb 17 16:32:12 lxds07 crmd: [1083]: info: do_state_transition: All 2 cluster nodes are eligible to run resources.
Feb 17 16:32:12 lxds07 crmd: [1083]: info: do_dc_join_final: Ensuring DC, quorum and node attributes are up-to-date
Feb 17 16:32:12 lxds07 crmd: [1083]: info: crm_update_quorum: Updating quorum status to true (call=233)
Feb 17 16:32:12 lxds07 crmd: [1083]: info: abort_transition_graph: do_te_invoke:191 - Triggered transition abort (complete=1) : Peer Cancelled
Feb 17 16:32:12 lxds07 attrd: [1081]: info: attrd_local_callback: Sending full refresh (origin=crmd)
Feb 17 16:32:12 lxds07 crmd: [1083]: info: do_pe_invoke: Waiting for the PE to connect
Feb 17 16:32:12 lxds07 crmd: [1083]: WARN: register_fsa_input_adv: do_pe_invoke stalled the FSA with pending inputs
Feb 17 16:32:12 lxds07 crmd: [1083]: info: do_pe_invoke: Waiting for the PE to connect
Feb 17 16:32:12 lxds07 crmd: [1083]: info: abort_transition_graph: te_update_diff:267 - Triggered transition abort (complete=1, tag=lrm_rsc_op, id=apache2_monitor_0, magic=0:7;24:3:7:5f4f5692-97ac-43d6-8be7-15d1fa0f67d8, cib=0.3592.3) : Resource op removal
Feb 17 16:32:12 lxds07 crmd: [1083]: info: do_pe_invoke: Waiting for the PE to connect
Feb 17 16:32:12 lxds07 cib: [1079]: info: cib_process_request: Operation complete: op cib_delete for section //node_state[@uname='lxds07']/lrm (origin=local/crmd/229, version=0.3592.3): ok (rc=0)
Feb 17 16:32:12 lxds07 crmd: [1083]: info: erase_xpath_callback: Deletion of "//node_state[@uname='lxds07']/lrm": ok (rc=0)
Feb 17 16:32:12 lxds07 crmd: [1083]: info: te_update_diff: Detected LRM refresh - 13 resources updated: Skipping all resource events
Feb 17 16:32:12 lxds07 crmd: [1083]: info: abort_transition_graph: te_update_diff:227 - Triggered transition abort (complete=1, tag=diff, id=(null), magic=NA, cib=0.3592.4) : LRM Refresh
Feb 17 16:32:12 lxds07 crmd: [1083]: info: do_pe_invoke: Waiting for the PE to connect
Feb 17 16:32:12 lxds07 cib: [1079]: info: cib_process_request: Operation complete: op cib_modify for section nodes (origin=local/crmd/231, version=0.3592.4): ok (rc=0)
Feb 17 16:32:12 lxds07 cib: [1079]: info: cib_process_request: Operation complete: op cib_modify for section cib (origin=local/crmd/233, version=0.3592.4): ok (rc=0)
Feb 17 16:35:46 lxds07 crmd: [1083]: info: do_pe_invoke: Waiting for the PE to connect
Feb 17 16:35:46 lxds07 crmd: [1083]: info: do_pe_invoke: Waiting for the PE to connect
Feb 17 16:35:46 lxds07 crmd: [1083]: info: do_pe_invoke: Waiting for the PE to connect
Feb 17 16:35:46 lxds07 crmd: [1083]: info: abort_transition_graph: te_update_diff:146 - Triggered transition abort (complete=1, tag=transient_attributes, id=lxds05, magic=NA, cib=0.3592.5) : Transient attribute: update
Feb 17 16:35:46 lxds07 crmd: [1083]: info: do_pe_invoke: Waiting for the PE to connect

lrmd:

Feb 17 15:34:49 lxds07 lrmd: [1080]: info: rsc:syslogng:1:27: monitor
Feb 17 16:03:12 lxds07 lrmd: [1080]: info: rsc:drbd_disk1:1:51: monitor
Feb 17 16:03:15 lxds07 lrmd: [1080]: info: rsc:fs_drbd2:61: monitor
Feb 17 16:03:15 lxds07 lrmd: [1080]: info: rsc:fs_drbd1:59: monitor
Feb 17 16:03:15 lxds07 lrmd: [1080]: info: rsc:fs_drbd:57: monitor
Feb 17 16:03:19 lxds07 lrmd: [1080]: info: rsc:apache2:65: monitor
Feb 17 16:03:21 lxds07 lrmd: [1080]: info: rsc:drbd_disk:1:50: monitor
Feb 17 16:03:23 lxds07 lrmd: [1080]: info: rsc:drbd_disk2:0:55: monitor
Feb 17 16:03:23 lxds07 lrmd: [1080]: info: rsc:ip1:63: monitor
Feb 17 16:03:33 lxds07 lrmd: [1080]: info: rsc:mailto:69: monitor
Feb 17 16:03:34 lxds07 lrmd: [1080]: info: rsc:nagios:67: monitor
Feb 17 16:32:41 lxds07 lrmd: [1080]: info: rsc:pingd:1:20: monitor
Feb 17 16:34:57 lxds07 lrmd: [1080]: info: rsc:syslogng:1:27: monitor
Feb 17 16:49:02 lxds07 crmd: [1083]: WARN: log_data_element: do_lrm_invoke: bad input <create_request_adv origin="send_lrm_rsc_op" t="crmd" version="3.0.1" subt="request" reference="lrm_delete-crm_resource-1297957742-1" crm_task="lrm_delete" crm_sys_to="lrmd" crm_sys_from="1758_crm_resource" crm_host_to="lxds07" src="lxds05" seq="6" >
Feb 17 16:49:02 lxds07 crmd: [1083]: info: send_direct_ack: ACK'ing resource op drbd_disk:0_delete_60000 from 0:0:crm-resource-1758: lrm_invoke-lrmd-1297957742-54
Feb 17 16:49:03 lxds07 crmd: [1083]: info: send_direct_ack: ACK'ing resource op drbd_disk:1_delete_60000 from 0:0:crm-resource-1760: lrm_invoke-lrmd-1297957743-55