[Pacemaker] Pacemaker installed to custom location

James Masson james.masson at opencredo.com
Thu Apr 25 10:12:47 EDT 2013 

Hi list,

I'm trying to build and run pacemaker from a custom location.


#################
# cluster-glue
tar xf pacemaker/cluster-glue-1.0.11+.tar.gz
(
   cd Reusable-Cluster-Components-glue--8347e8c9b94f
   ./autogen.sh
   ./configure --prefix=${BOSH_INSTALL_TARGET} 
--enable-fatal-warnings=no --with-daemon-group=vcap 
--with-daemon-user=vcap 
--with-ocf-root=${BOSH_INSTALL_TARGET}/usr/lib/ocf/resource.d/pacemaker
   make
   make install
)


# libqb
tar xf pacemaker/libqb-0.14.4.tar.gz
(
   cd libqb-0.14.4
   ./autogen.sh
   ./configure --prefix=${BOSH_INSTALL_TARGET}
   make
   make install
)

# corosync
tar xzf pacemaker/corosync-2.3.0.tar.gz
(
   cd corosync-2.3.0
   export PKG_CONFIG_PATH="${BOSH_INSTALL_TARGET}/lib/pkgconfig/"
   export LDFLAGS="-L${BOSH_INSTALL_TARGET}/lib 
-L${BOSH_INSTALL_TARGET}/lib/heartbeat 
-L${BOSH_INSTALL_TARGET}/lib/stonith -L${BOSH_INSTALL_TARGET}/lib/pkgconfig"
   export CFLAGS="-I${BOSH_INSTALL_TARGET}/include/heartbeat 
-I${BOSH_INSTALL_TARGET}/include "
   ./autogen.sh
#  ./configure --prefix=${BOSH_INSTALL_TARGET} --disable-nss 
--with-socket-dir=/var/vcap/sys/run/cluster-stack 
--sysconfdir=/var/vcap/jobs/cluster-stack/etc
   ./configure --prefix=${BOSH_INSTALL_TARGET} --disable-nss
   make
   make install
)

# pacemaker
tar xf pacemaker/Pacemaker-1.1.9.tar.gz
(
   cd pacemaker-Pacemaker-1.1.9
   export PKG_CONFIG_PATH="${BOSH_INSTALL_TARGET}/lib/pkgconfig/"
   export LDFLAGS="-L${BOSH_INSTALL_TARGET}/lib 
-L${BOSH_INSTALL_TARGET}/lib/heartbeat 
-L${BOSH_INSTALL_TARGET}/lib/stonith -L${BOSH_INSTALL_TARGET}/lib/pkgconfig"
   export CFLAGS="-I${BOSH_INSTALL_TARGET}/include/heartbeat 
-I${BOSH_INSTALL_TARGET}/include "
   ./autogen.sh
   ./configure --prefix=${BOSH_INSTALL_TARGET}   --without-snmp 
--with-corosync -with-ais --with-cs-quorum 
--with-ais-prefix=${BOSH_INSTALL_TARGET}
   make
   make install
)
######################

I've tried this with the latest versions, and with recompiling the 
current Ubuntu versions of these packages - the result is the same.

The packages compile correctly - in this case into the directory 
/var/vcap/packages/cluster-stack

The contents of that directory is then shipped to other machines to run.

ld.so.conf is updated with "/var/vcap/packages/cluster-stack/lib/"

Corosync starts up fine.

Pacemakerd does not - the result is.

###################
Apr 25 13:53:52 [10461] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 corosync 
notice  [MAIN  ] Corosync Cluster Engine ('UNKNOWN'): started and ready 
to provide service.
Apr 25 13:53:52 [10461] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 corosync 
info    [MAIN  ] Corosync built-in features: pie relro bindnow
Apr 25 13:53:52 [10461] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 corosync 
notice  [TOTEM ] Initializing transport (UDP/IP Unicast).
Apr 25 13:53:52 [10461] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 corosync 
notice  [TOTEM ] Initializing transmit/receive security (NSS) crypto: 
none hash: none
Apr 25 13:53:52 [10461] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 corosync 
notice  [TOTEM ] The network interface [10.0.4.50] is now up.
Apr 25 13:53:52 [10461] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 corosync 
notice  [SERV  ] Service engine loaded: corosync configuration map 
access [0]
Apr 25 13:53:52 [10461] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 corosync 
info    [QB    ] server name: cmap
Apr 25 13:53:52 [10461] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 corosync 
notice  [SERV  ] Service engine loaded: corosync configuration service [1]
Apr 25 13:53:52 [10461] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 corosync 
info    [QB    ] server name: cfg
Apr 25 13:53:52 [10461] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 corosync 
notice  [SERV  ] Service engine loaded: corosync cluster closed process 
group service v1.01 [2]
Apr 25 13:53:52 [10461] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 corosync 
info    [QB    ] server name: cpg
Apr 25 13:53:52 [10461] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 corosync 
notice  [SERV  ] Service engine loaded: corosync profile loading service [4]
Apr 25 13:53:52 [10461] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 corosync 
notice  [SERV  ] Service engine loaded: corosync cluster quorum service 
v0.1 [3]
Apr 25 13:53:52 [10461] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 corosync 
info    [QB    ] server name: quorum
Apr 25 13:53:52 [10461] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 corosync 
notice  [TOTEM ] adding new UDPU member {10.0.4.50}
Apr 25 13:53:52 [10461] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 corosync 
notice  [TOTEM ] adding new UDPU member {10.0.4.51}
Apr 25 13:53:52 [10461] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 corosync 
notice  [TOTEM ] A processor joined or left the membership and a new 
membership (10.0.4.50:44) was formed.
Apr 25 13:53:52 [10461] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 corosync 
notice  [TOTEM ] A processor joined or left the membership and a new 
membership (10.0.4.50:48) was formed.
Apr 25 13:53:52 [10461] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 corosync 
notice  [MAIN  ] Completed service synchronization, ready to provide 
service.
Apr 25 13:54:10 [10482] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 pacemakerd: 
     info: read_config: 	User configured file based logging and 
explicitly disabled syslog.
Apr 25 13:54:10 [10482] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 pacemakerd: 
   notice: main: 	Starting Pacemaker 1.1.9 (Build: 2a917dd):  ncurses 
libqb-logging libqb-ipc lha-fencing nagios  corosync-native
Apr 25 13:54:10 [10482] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 pacemakerd: 
     info: main: 	Maximum core file size is: 18446744073709551615
Apr 25 13:54:10 [10482] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 pacemakerd: 
     info: qb_ipcs_us_publish: 	server name: pacemakerd
Apr 25 13:54:10 [10482] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 pacemakerd: 
   notice: corosync_node_name: 	Unable to get node name for nodeid 0
Apr 25 13:54:10 [10482] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 pacemakerd: 
   notice: get_local_node_name: 	Defaulting to uname -n for the local 
corosync node name
Apr 25 13:54:10 [10482] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 pacemakerd: 
   notice: update_node_processes: 	0x61cd50 Node 839122954 now known as 
fcde02a2-cc41-4c58-b6d2-b7bb0bada436, was:
Apr 25 13:54:10 [10482] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 pacemakerd: 
     info: start_child: 	Forked child 10484 for process cib
Apr 25 13:54:10 [10482] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 pacemakerd: 
     info: start_child: 	Forked child 10485 for process stonith-ng
Apr 25 13:54:10 [10482] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 pacemakerd: 
     info: start_child: 	Forked child 10486 for process lrmd
Apr 25 13:54:10 [10482] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 pacemakerd: 
     info: start_child: 	Forked child 10487 for process attrd
Apr 25 13:54:10 [10482] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 pacemakerd: 
     info: start_child: 	Forked child 10488 for process pengine
Apr 25 13:54:10 [10482] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 pacemakerd: 
     info: start_child: 	Forked child 10489 for process crmd
Apr 25 13:54:10 [10482] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 pacemakerd: 
     info: main: 	Starting mainloop
Apr 25 13:54:10 [10485] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 stonith-ng: 
     info: crm_log_init: 	Changed active directory to 
/var/vcap/data/packages/cluster-stack/0.12-dev.1/var/lib/heartbeat/cores/root
Apr 25 13:54:10 [10485] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 stonith-ng: 
     info: get_cluster_type: 	Verifying cluster type: 'corosync'
Apr 25 13:54:10 [10485] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 stonith-ng: 
     info: get_cluster_type: 	Assuming an active 'corosync' cluster
Apr 25 13:54:10 [10485] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 stonith-ng: 
   notice: crm_cluster_connect: 	Connecting to cluster infrastructure: 
corosync
Apr 25 13:54:10 [10486] fcde02a2-cc41-4c58-b6d2-b7bb0bada436       lrmd: 
     info: crm_log_init: 	Changed active directory to 
/var/vcap/data/packages/cluster-stack/0.12-dev.1/var/lib/heartbeat/cores/root
Apr 25 13:54:10 [10486] fcde02a2-cc41-4c58-b6d2-b7bb0bada436       lrmd: 
     info: qb_ipcs_us_publish: 	server name: lrmd
Apr 25 13:54:10 [10486] fcde02a2-cc41-4c58-b6d2-b7bb0bada436       lrmd: 
     info: main: 	Starting
Apr 25 13:54:10 [10482] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 pacemakerd: 
    error: pcmk_child_exit: 	Child process cib exited (pid=10484, rc=100)
Apr 25 13:54:10 [10482] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 pacemakerd: 
  warning: pcmk_child_exit: 	Pacemaker child process cib no longer 
wishes to be respawned. Shutting ourselves down.
Apr 25 13:54:10 [10482] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 pacemakerd: 
   notice: pcmk_shutdown_worker: 	Shuting down Pacemaker
Apr 25 13:54:10 [10482] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 pacemakerd: 
   notice: stop_child: 	Stopping crmd: Sent -15 to process 10489
Apr 25 13:54:10 [10485] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 stonith-ng: 
     info: crm_get_peer: 	Node <null> now has id: 839122954
Apr 25 13:54:10 [10485] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 stonith-ng: 
     info: crm_update_peer_proc: 	init_cpg_connection: Node 
(null)[839122954] - corosync-cpg is now online
Apr 25 13:54:10 [10482] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 pacemakerd: 
   notice: pcmk_child_exit: 	Child process crmd terminated with signal 
15 (pid=10489, core=0)
Apr 25 13:54:10 [10482] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 pacemakerd: 
   notice: stop_child: 	Stopping pengine: Sent -15 to process 10488
Apr 25 13:54:10 [10482] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 pacemakerd: 
     info: pcmk_child_exit: 	Child process pengine exited (pid=10488, rc=0)
Apr 25 13:54:10 [10482] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 pacemakerd: 
   notice: stop_child: 	Stopping attrd: Sent -15 to process 10487
Apr 25 13:54:10 [10482] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 pacemakerd: 
    error: pcmk_child_exit: 	Child process attrd exited (pid=10487, rc=100)
Apr 25 13:54:10 [10482] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 pacemakerd: 
  warning: pcmk_child_exit: 	Pacemaker child process attrd no longer 
wishes to be respawned. Shutting ourselves down.
Apr 25 13:54:10 [10482] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 pacemakerd: 
   notice: stop_child: 	Stopping lrmd: Sent -15 to process 10486
Apr 25 13:54:10 [10486] fcde02a2-cc41-4c58-b6d2-b7bb0bada436       lrmd: 
     info: crm_signal_dispatch: 	Invoking handler for signal 15: Terminated
Apr 25 13:54:10 [10486] fcde02a2-cc41-4c58-b6d2-b7bb0bada436       lrmd: 
     info: lrmd_shutdown: 	Terminating with  0 clients
Apr 25 13:54:10 [10486] fcde02a2-cc41-4c58-b6d2-b7bb0bada436       lrmd: 
     info: qb_ipcs_us_withdraw: 	withdrawing server sockets
Apr 25 13:54:10 [10486] fcde02a2-cc41-4c58-b6d2-b7bb0bada436       lrmd: 
     info: crm_xml_cleanup: 	Cleaning up memory from libxml2
Apr 25 13:54:10 [10482] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 pacemakerd: 
     info: pcmk_child_exit: 	Child process lrmd exited (pid=10486, rc=0)
Apr 25 13:54:10 [10482] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 pacemakerd: 
   notice: stop_child: 	Stopping stonith-ng: Sent -15 to process 10485
Apr 25 13:54:10 [10485] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 stonith-ng: 
   notice: corosync_node_name: 	Unable to get node name for nodeid 839122954
Apr 25 13:54:10 [10485] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 stonith-ng: 
   notice: get_local_node_name: 	Defaulting to uname -n for the local 
corosync node name
Apr 25 13:54:10 [10485] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 stonith-ng: 
     info: init_cs_connection_once: 	Connection to 'corosync': established
Apr 25 13:54:10 [10485] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 stonith-ng: 
     info: crm_get_peer: 	Node 839122954 is now known as 
fcde02a2-cc41-4c58-b6d2-b7bb0bada436
Apr 25 13:54:10 [10485] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 stonith-ng: 
     info: crm_get_peer: 	Node 839122954 has uuid 839122954
Apr 25 13:54:10 [10485] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 stonith-ng: 
     info: crm_ipc_connect: 	Could not establish cib_rw connection: 
Connection refused (111)
Apr 25 13:54:11 [10485] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 stonith-ng: 
     info: crm_ipc_connect: 	Could not establish cib_rw connection: 
Connection refused (111)
Apr 25 13:54:13 [10485] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 stonith-ng: 
     info: crm_ipc_connect: 	Could not establish cib_rw connection: 
Connection refused (111)
Apr 25 13:54:16 [10485] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 stonith-ng: 
     info: crm_ipc_connect: 	Could not establish cib_rw connection: 
Connection refused (111)
Apr 25 13:54:20 [10485] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 stonith-ng: 
     info: crm_ipc_connect: 	Could not establish cib_rw connection: 
Connection refused (111)
Apr 25 13:54:20 [10485] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 stonith-ng: 
    error: setup_cib: 	Could not connect to the CIB service: Transport 
endpoint is not connected (-107)
Apr 25 13:54:20 [10485] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 stonith-ng: 
     info: qb_ipcs_us_publish: 	server name: stonith-ng
Apr 25 13:54:20 [10485] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 stonith-ng: 
     info: main: 	Starting stonith-ng mainloop
Apr 25 13:54:20 [10485] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 stonith-ng: 
     info: crm_signal_dispatch: 	Invoking handler for signal 15: Terminated
Apr 25 13:54:20 [10485] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 stonith-ng: 
     info: stonith_shutdown: 	Terminating with  0 clients
Apr 25 13:54:20 [10485] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 stonith-ng: 
     info: qb_ipcs_us_withdraw: 	withdrawing server sockets
Apr 25 13:54:20 [10485] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 stonith-ng: 
     info: main: 	Done
Apr 25 13:54:20 [10485] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 stonith-ng: 
     info: crm_xml_cleanup: 	Cleaning up memory from libxml2
Apr 25 13:54:20 [10482] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 pacemakerd: 
     info: pcmk_child_exit: 	Child process stonith-ng exited (pid=10485, 
rc=0)
Apr 25 13:54:20 [10482] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 pacemakerd: 
   notice: pcmk_shutdown_worker: 	Shutdown complete
Apr 25 13:54:20 [10482] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 pacemakerd: 
   notice: pcmk_shutdown_worker: 	Attempting to inhibit respawning after 
fatal error
Apr 25 13:54:20 [10482] fcde02a2-cc41-4c58-b6d2-b7bb0bada436 pacemakerd: 
     info: crm_xml_cleanup: 	Cleaning up memory from libxml2
#################################

Corosync config is this


################
# Please read the openais.conf.5 manual page
#
#

totem {
         version: 2

         crypto_cipher: none
         crypto_hash: none

         interface {
                 ringnumber: 0
                 bindnetaddr: 10.0.4.0
                 mcastport: 5405
                 ttl: 1
         }
         transport: udpu
}



nodelist {

                 node {
                         ring0_addr: 10.0.4.50
                 }

                 node {
                         ring0_addr: 10.0.4.51
                 }


}


service {
	# Load the Pacemaker Cluster Resource Manager
	name: pacemaker
	ver:  1
}

logging {
         fileline: off
         to_stderr: yes
         to_logfile: on
	logfile: /var/vcap/sys/log/cluster-stack/corosync.log
         to_syslog: no
	syslog_facility: daemon
         debug: off
         timestamp: on
         logger_subsys {
                 subsys: AMF
                 debug: off
                 tags: enter|leave|trace1|trace2|trace3|trace4|trace6
         }
}
####################################


I have a working corosync/pacemaker setup using a similar configuration 
with a local source install, so I'm pretty sure the problem is due to 
the relocation of the package tree.

Any ideas what I've missed?

thanks

James M

More information about the Pacemaker mailing list