[Pacemaker] Pacemaker installed to custom location
Andrew Beekhof
andrew at beekhof.net
Thu Apr 25 20:29:12 EDT 2013
On 26/04/2013, at 12:12 AM, James Masson <james.masson at opencredo.com> wrote:
>
> Hi list,
>
> I'm trying to build and run pacemaker from a custom location.
>
>
> #################
> # cluster-glue
> tar xf pacemaker/cluster-glue-1.0.11+.tar.gz
> (
> cd Reusable-Cluster-Components-glue--8347e8c9b94f
> ./autogen.sh
> ./configure --prefix=${BOSH_INSTALL_TARGET} --enable-fatal-warnings=no --with-daemon-group=vcap --with-daemon-user=vcap --with-ocf-root=${BOSH_INSTALL_TARGET}/usr/lib/ocf/resource.d/pacemaker
> make
> make install
> )
>
>
> # libqb
> tar xf pacemaker/libqb-0.14.4.tar.gz
> (
> cd libqb-0.14.4
> ./autogen.sh
> ./configure --prefix=${BOSH_INSTALL_TARGET}
> make
> make install
> )
>
> # corosync
> tar xzf pacemaker/corosync-2.3.0.tar.gz
> (
> cd corosync-2.3.0
> export PKG_CONFIG_PATH="${BOSH_INSTALL_TARGET}/lib/pkgconfig/"
> export LDFLAGS="-L${BOSH_INSTALL_TARGET}/lib -L${BOSH_INSTALL_TARGET}/lib/heartbeat -L${BOSH_INSTALL_TARGET}/lib/stonith -L${BOSH_INSTALL_TARGET}/lib/pkgconfig"
> export CFLAGS="-I${BOSH_INSTALL_TARGET}/include/heartbeat -I${BOSH_INSTALL_TARGET}/include "
> ./autogen.sh
> # ./configure --prefix=${BOSH_INSTALL_TARGET} --disable-nss --with-socket-dir=/var/vcap/sys/run/cluster-stack --sysconfdir=/var/vcap/jobs/cluster-stack/etc
> ./configure --prefix=${BOSH_INSTALL_TARGET} --disable-nss
> make
> make install
> )
>
> # pacemaker
> tar xf pacemaker/Pacemaker-1.1.9.tar.gz
> (
> cd pacemaker-Pacemaker-1.1.9
> export PKG_CONFIG_PATH="${BOSH_INSTALL_TARGET}/lib/pkgconfig/"
> export LDFLAGS="-L${BOSH_INSTALL_TARGET}/lib -L${BOSH_INSTALL_TARGET}/lib/heartbeat -L${BOSH_INSTALL_TARGET}/lib/stonith -L${BOSH_INSTALL_TARGET}/lib/pkgconfig"
> export CFLAGS="-I${BOSH_INSTALL_TARGET}/include/heartbeat -I${BOSH_INSTALL_TARGET}/include "
> ./autogen.sh
> ./configure --prefix=${BOSH_INSTALL_TARGET} --without-snmp --with-corosync -with-ais --with-cs-quorum --with-ais-prefix=${BOSH_INSTALL_TARGET}
> make
> make install
> )
> ######################
>
> I've tried this with the latest versions, and with recompiling the current Ubuntu versions of these packages - the result is the same.
>
> The packages compile correctly - in this case into the directory /var/vcap/packages/cluster-stack
>
> The contents of that directory is then shipped to other machines to run.
>
> ld.so.conf is updated with "/var/vcap/packages/cluster-stack/lib/"
>
> Corosync starts up fine.
>
> Pacemakerd does not - the result is.
Try turning up the debug to see why the cib isn't happy:
> Apr 25 13:54:10 [10482] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 pacemakerd: error: pcmk_child_exit: Child process cib exited (pid=10484, rc=100)
> Apr 25 13:54:10 [10482] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 pacemakerd: warning: pcmk_child_exit: Pacemaker child process cib no longer
>
> ###################
> Apr 25 13:53:52 [10461] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 corosync notice [MAIN ] Corosync Cluster Engine ('UNKNOWN'): started and ready to provide service.
> Apr 25 13:53:52 [10461] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 corosync info [MAIN ] Corosync built-in features: pie relro bindnow
> Apr 25 13:53:52 [10461] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 corosync notice [TOTEM ] Initializing transport (UDP/IP Unicast).
> Apr 25 13:53:52 [10461] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 corosync notice [TOTEM ] Initializing transmit/receive security (NSS) crypto: none hash: none
> Apr 25 13:53:52 [10461] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 corosync notice [TOTEM ] The network interface [10.0.4.50] is now up.
> Apr 25 13:53:52 [10461] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 corosync notice [SERV ] Service engine loaded: corosync configuration map access [0]
> Apr 25 13:53:52 [10461] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 corosync info [QB ] server name: cmap
> Apr 25 13:53:52 [10461] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 corosync notice [SERV ] Service engine loaded: corosync configuration service [1]
> Apr 25 13:53:52 [10461] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 corosync info [QB ] server name: cfg
> Apr 25 13:53:52 [10461] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 corosync notice [SERV ] Service engine loaded: corosync cluster closed process group service v1.01 [2]
> Apr 25 13:53:52 [10461] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 corosync info [QB ] server name: cpg
> Apr 25 13:53:52 [10461] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 corosync notice [SERV ] Service engine loaded: corosync profile loading service [4]
> Apr 25 13:53:52 [10461] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 corosync notice [SERV ] Service engine loaded: corosync cluster quorum service v0.1 [3]
> Apr 25 13:53:52 [10461] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 corosync info [QB ] server name: quorum
> Apr 25 13:53:52 [10461] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 corosync notice [TOTEM ] adding new UDPU member {10.0.4.50}
> Apr 25 13:53:52 [10461] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 corosync notice [TOTEM ] adding new UDPU member {10.0.4.51}
> Apr 25 13:53:52 [10461] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 corosync notice [TOTEM ] A processor joined or left the membership and a new membership (10.0.4.50:44) was formed.
> Apr 25 13:53:52 [10461] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 corosync notice [TOTEM ] A processor joined or left the membership and a new membership (10.0.4.50:48) was formed.
> Apr 25 13:53:52 [10461] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 corosync notice [MAIN ] Completed service synchronization, ready to provide service.
> Apr 25 13:54:10 [10482] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 pacemakerd: info: read_config: User configured file based logging and explicitly disabled syslog.
> Apr 25 13:54:10 [10482] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 pacemakerd: notice: main: Starting Pacemaker 1.1.9 (Build: 2a917dd): ncurses libqb-logging libqb-ipc lha-fencing nagios corosync-native
> Apr 25 13:54:10 [10482] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 pacemakerd: info: main: Maximum core file size is: 18446744073709551615
> Apr 25 13:54:10 [10482] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 pacemakerd: info: qb_ipcs_us_publish: server name: pacemakerd
> Apr 25 13:54:10 [10482] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 pacemakerd: notice: corosync_node_name: Unable to get node name for nodeid 0
> Apr 25 13:54:10 [10482] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 pacemakerd: notice: get_local_node_name: Defaulting to uname -n for the local corosync node name
> Apr 25 13:54:10 [10482] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 pacemakerd: notice: update_node_processes: 0x61cd50 Node 839122954 now known as fcde02a2-cc41-4c58-b6d2-b7bb0bada436, was:
> Apr 25 13:54:10 [10482] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 pacemakerd: info: start_child: Forked child 10484 for process cib
> Apr 25 13:54:10 [10482] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 pacemakerd: info: start_child: Forked child 10485 for process stonith-ng
> Apr 25 13:54:10 [10482] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 pacemakerd: info: start_child: Forked child 10486 for process lrmd
> Apr 25 13:54:10 [10482] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 pacemakerd: info: start_child: Forked child 10487 for process attrd
> Apr 25 13:54:10 [10482] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 pacemakerd: info: start_child: Forked child 10488 for process pengine
> Apr 25 13:54:10 [10482] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 pacemakerd: info: start_child: Forked child 10489 for process crmd
> Apr 25 13:54:10 [10482] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 pacemakerd: info: main: Starting mainloop
> Apr 25 13:54:10 [10485] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 stonith-ng: info: crm_log_init: Changed active directory to /var/vcap/data/packages/cluster-stack/0.12-dev.1/var/lib/heartbeat/cores/root
> Apr 25 13:54:10 [10485] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 stonith-ng: info: get_cluster_type: Verifying cluster type: 'corosync'
> Apr 25 13:54:10 [10485] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 stonith-ng: info: get_cluster_type: Assuming an active 'corosync' cluster
> Apr 25 13:54:10 [10485] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 stonith-ng: notice: crm_cluster_connect: Connecting to cluster infrastructure: corosync
> Apr 25 13:54:10 [10486] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 lrmd: info: crm_log_init: Changed active directory to /var/vcap/data/packages/cluster-stack/0.12-dev.1/var/lib/heartbeat/cores/root
> Apr 25 13:54:10 [10486] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 lrmd: info: qb_ipcs_us_publish: server name: lrmd
> Apr 25 13:54:10 [10486] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 lrmd: info: main: Starting
> Apr 25 13:54:10 [10482] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 pacemakerd: error: pcmk_child_exit: Child process cib exited (pid=10484, rc=100)
> Apr 25 13:54:10 [10482] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 pacemakerd: warning: pcmk_child_exit: Pacemaker child process cib no longer wishes to be respawned. Shutting ourselves down.
> Apr 25 13:54:10 [10482] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 pacemakerd: notice: pcmk_shutdown_worker: Shuting down Pacemaker
> Apr 25 13:54:10 [10482] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 pacemakerd: notice: stop_child: Stopping crmd: Sent -15 to process 10489
> Apr 25 13:54:10 [10485] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 stonith-ng: info: crm_get_peer: Node <null> now has id: 839122954
> Apr 25 13:54:10 [10485] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 stonith-ng: info: crm_update_peer_proc: init_cpg_connection: Node (null)[839122954] - corosync-cpg is now online
> Apr 25 13:54:10 [10482] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 pacemakerd: notice: pcmk_child_exit: Child process crmd terminated with signal 15 (pid=10489, core=0)
> Apr 25 13:54:10 [10482] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 pacemakerd: notice: stop_child: Stopping pengine: Sent -15 to process 10488
> Apr 25 13:54:10 [10482] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 pacemakerd: info: pcmk_child_exit: Child process pengine exited (pid=10488, rc=0)
> Apr 25 13:54:10 [10482] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 pacemakerd: notice: stop_child: Stopping attrd: Sent -15 to process 10487
> Apr 25 13:54:10 [10482] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 pacemakerd: error: pcmk_child_exit: Child process attrd exited (pid=10487, rc=100)
> Apr 25 13:54:10 [10482] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 pacemakerd: warning: pcmk_child_exit: Pacemaker child process attrd no longer wishes to be respawned. Shutting ourselves down.
> Apr 25 13:54:10 [10482] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 pacemakerd: notice: stop_child: Stopping lrmd: Sent -15 to process 10486
> Apr 25 13:54:10 [10486] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 lrmd: info: crm_signal_dispatch: Invoking handler for signal 15: Terminated
> Apr 25 13:54:10 [10486] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 lrmd: info: lrmd_shutdown: Terminating with 0 clients
> Apr 25 13:54:10 [10486] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 lrmd: info: qb_ipcs_us_withdraw: withdrawing server sockets
> Apr 25 13:54:10 [10486] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 lrmd: info: crm_xml_cleanup: Cleaning up memory from libxml2
> Apr 25 13:54:10 [10482] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 pacemakerd: info: pcmk_child_exit: Child process lrmd exited (pid=10486, rc=0)
> Apr 25 13:54:10 [10482] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 pacemakerd: notice: stop_child: Stopping stonith-ng: Sent -15 to process 10485
> Apr 25 13:54:10 [10485] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 stonith-ng: notice: corosync_node_name: Unable to get node name for nodeid 839122954
> Apr 25 13:54:10 [10485] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 stonith-ng: notice: get_local_node_name: Defaulting to uname -n for the local corosync node name
> Apr 25 13:54:10 [10485] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 stonith-ng: info: init_cs_connection_once: Connection to 'corosync': established
> Apr 25 13:54:10 [10485] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 stonith-ng: info: crm_get_peer: Node 839122954 is now known as fcde02a2-cc41-4c58-b6d2-b7bb0bada436
> Apr 25 13:54:10 [10485] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 stonith-ng: info: crm_get_peer: Node 839122954 has uuid 839122954
> Apr 25 13:54:10 [10485] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 stonith-ng: info: crm_ipc_connect: Could not establish cib_rw connection: Connection refused (111)
> Apr 25 13:54:11 [10485] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 stonith-ng: info: crm_ipc_connect: Could not establish cib_rw connection: Connection refused (111)
> Apr 25 13:54:13 [10485] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 stonith-ng: info: crm_ipc_connect: Could not establish cib_rw connection: Connection refused (111)
> Apr 25 13:54:16 [10485] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 stonith-ng: info: crm_ipc_connect: Could not establish cib_rw connection: Connection refused (111)
> Apr 25 13:54:20 [10485] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 stonith-ng: info: crm_ipc_connect: Could not establish cib_rw connection: Connection refused (111)
> Apr 25 13:54:20 [10485] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 stonith-ng: error: setup_cib: Could not connect to the CIB service: Transport endpoint is not connected (-107)
> Apr 25 13:54:20 [10485] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 stonith-ng: info: qb_ipcs_us_publish: server name: stonith-ng
> Apr 25 13:54:20 [10485] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 stonith-ng: info: main: Starting stonith-ng mainloop
> Apr 25 13:54:20 [10485] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 stonith-ng: info: crm_signal_dispatch: Invoking handler for signal 15: Terminated
> Apr 25 13:54:20 [10485] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 stonith-ng: info: stonith_shutdown: Terminating with 0 clients
> Apr 25 13:54:20 [10485] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 stonith-ng: info: qb_ipcs_us_withdraw: withdrawing server sockets
> Apr 25 13:54:20 [10485] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 stonith-ng: info: main: Done
> Apr 25 13:54:20 [10485] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 stonith-ng: info: crm_xml_cleanup: Cleaning up memory from libxml2
> Apr 25 13:54:20 [10482] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 pacemakerd: info: pcmk_child_exit: Child process stonith-ng exited (pid=10485, rc=0)
> Apr 25 13:54:20 [10482] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 pacemakerd: notice: pcmk_shutdown_worker: Shutdown complete
> Apr 25 13:54:20 [10482] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 pacemakerd: notice: pcmk_shutdown_worker: Attempting to inhibit respawning after fatal error
> Apr 25 13:54:20 [10482] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 pacemakerd: info: crm_xml_cleanup: Cleaning up memory from libxml2
> #################################
>
> Corosync config is this
>
>
> ################
> # Please read the openais.conf.5 manual page
> #
> #
>
> totem {
> version: 2
>
> crypto_cipher: none
> crypto_hash: none
>
> interface {
> ringnumber: 0
> bindnetaddr: 10.0.4.0
> mcastport: 5405
> ttl: 1
> }
> transport: udpu
> }
>
>
>
> nodelist {
>
> node {
> ring0_addr: 10.0.4.50
> }
>
> node {
> ring0_addr: 10.0.4.51
> }
>
>
> }
>
>
> service {
> # Load the Pacemaker Cluster Resource Manager
> name: pacemaker
> ver: 1
> }
>
> logging {
> fileline: off
> to_stderr: yes
> to_logfile: on
> logfile: /var/vcap/sys/log/cluster-stack/corosync.log
> to_syslog: no
> syslog_facility: daemon
> debug: off
> timestamp: on
> logger_subsys {
> subsys: AMF
> debug: off
> tags: enter|leave|trace1|trace2|trace3|trace4|trace6
> }
> }
> ####################################
>
>
> I have a working corosync/pacemaker setup using a similar configuration with a local source install, so I'm pretty sure the problem is due to the relocation of the package tree.
>
> Any ideas what I've missed?
>
> thanks
>
> James M
>
> _______________________________________________
> Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>
> Project Home: http://www.clusterlabs.org
> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> Bugs: http://bugs.clusterlabs.org
More information about the Pacemaker
mailing list