Hello,<div>I am new to pacemaker and struggling with the somewhat limited documentation. I looked through the archives and didn't find anything that matched my problem. I have brand new pacemaker setup running on CentOS 5.5. I am using the below config file to start up mysql which is also a brand new build. Right now the cluster is only running on one node while I try to isolate this problem. This is a brand new cib file as well. The cluster starts up but then every 30 seconds or so I see it restart mysql. If I stop heartbeat and bring up mysql by itself it starts up just fine. Its driving me batty so I thought I would post it here and see if someone was able to help. What I see in syslog from heartbeat is:</div>
<div><br></div><div><div>ul 22 15:34:09 sipl-mysql-109 lrmd: [11182]: info: rsc:d_mysql:69: start</div><div>Jul 22 15:34:11 sipl-mysql-109 lrmd: [11182]: info: RA output: (ip_db:start:stderr) ARPING 10.200.131.9 from 10.200.131.9 eth0 Sent 5 probes (5 broadcast(s)) Received 0 response(s) </div>
<div>Jul 22 15:34:13 sipl-mysql-109 mysql[14915]: [15086]: INFO: MySQL started</div><div>Jul 22 15:34:13 sipl-mysql-109 crmd: [11185]: info: process_lrm_event: LRM operation d_mysql_start_0 (call=69, rc=0, cib-update=105, confirmed=true) ok</div>
<div>Jul 22 15:34:13 sipl-mysql-109 crmd: [11185]: info: match_graph_event: Action d_mysql_start_0 (6) confirmed on sipl-mysql-109 (rc=0)</div><div>Jul 22 15:34:13 sipl-mysql-109 crmd: [11185]: info: te_rsc_command: Initiating action 1: monitor d_mysql_monitor_10000 on sipl-mysql-109 (local)</div>
<div>Jul 22 15:34:13 sipl-mysql-109 crmd: [11185]: info: do_lrm_rsc_op: Performing key=1:14:0:989206b7-461a-42db-a2a7-7b447bd6c5b3 op=d_mysql_monitor_10000 )</div><div>Jul 22 15:34:13 sipl-mysql-109 lrmd: [11182]: info: rsc:d_mysql:70: monitor</div>
<div>Jul 22 15:34:13 sipl-mysql-109 crmd: [11185]: info: te_rsc_command: Initiating action 8: start ip_db_start_0 on sipl-mysql-109 (local)</div><div>Jul 22 15:34:13 sipl-mysql-109 crmd: [11185]: info: do_lrm_rsc_op: Performing key=8:14:0:989206b7-461a-42db-a2a7-7b447bd6c5b3 op=ip_db_start_0 )</div>
<div>Jul 22 15:34:13 sipl-mysql-109 lrmd: [11182]: info: rsc:ip_db:71: start</div><div>Jul 22 15:34:13 sipl-mysql-109 crmd: [11185]: info: process_lrm_event: LRM operation d_mysql_monitor_10000 (call=70, rc=7, cib-update=106, confirmed=false) not running</div>
<div>Jul 22 15:34:13 sipl-mysql-109 crmd: [11185]: WARN: status_from_rc: Action 1 (d_mysql_monitor_10000) on sipl-mysql-109 failed (target: 0 vs. rc: 7): Error</div><div>Jul 22 15:34:13 sipl-mysql-109 crmd: [11185]: WARN: update_failcount: Updating failcount for d_mysql on sipl-mysql-109 after failed monitor.</div>
</div><div><br></div><div><br></div><div>The output of crm configure show is:</div><div><br></div><div><div>primitive d_mysql ocf:heartbeat:mysql \</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>op start interval="0" timeout="120" \</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>op stop interval="0" timeout="120" \</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>op monitor interval="10" timeout="30" depth="0" param binary="/usr/bin/mysqld_safe" config="/etc/my.cnf" datadir="/var/lib/mysql" user="mysql" pid="/var/run/mysqld/mysql.pid" socket="/var/lib/mysql/mysql.sock"</div>
<div>primitive ip_db ocf:heartbeat:IPaddr2 \</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>params ip="10.200.131.9" cidr_netmask="32" \</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>op monitor interval="30s" nic="eth0"</div>
<div>group sv_db d_mysql ip_db</div><div>property $id="cib-bootstrap-options" \</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>stonith-enabled="false" \</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>no-quorum-policy="ignore" \</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>start-failure-is-fatal="false" \</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>expected-quorum-votes="2" \</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>dc-version="1.0.9-89bd754939df5150de7cd76835f98fe90851b677" \</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>cluster-infrastructure="Heartbeat"</div>
<div>rsc_defaults $id="rsc_defaults-options" \</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>migration-threshold="20" \</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>failure-timeout="20"</div>
</div><div><br></div><div>My versions are as follows:</div><div><br></div><div><div>[root@sipl-mysql-109 rc0.d]# rpm -qa | egrep "coro|pacemaker|heart"</div><div>corosynclib-1.2.5-1.3.el5</div><div>corosync-1.2.5-1.3.el5</div>
<div>corosync-1.2.5-1.3.el5</div><div>heartbeat-3.0.3-2.3.el5</div><div>pacemaker-1.0.9.1-1.11.el5</div><div>pacemaker-1.0.9.1-1.11.el5</div><div>corosynclib-1.2.5-1.3.el5</div><div>heartbeat-libs-3.0.3-2.3.el5</div><div>
heartbeat-3.0.3-2.3.el5</div><div>pacemaker-libs-1.0.9.1-1.11.el5</div><div>heartbeat-libs-3.0.3-2.3.el5</div><div>pacemaker-libs-1.0.9.1-1.11.el5</div><div><br></div><div><div>rpm -qa | grep resource</div><div>resource-agents-1.0.3-2.6.el5</div>
</div><div><br></div><div>[root@sipl-mysql-109 rc0.d]# cat /etc/redhat-release </div><div>CentOS release 5.5 (Final)</div><div><br></div><div>[root@sipl-mysql-109 rc0.d]# uname -r</div><div>2.6.18-194.8.1.el5</div><div><br>
</div><div>[root@sipl-mysql-109 rc0.d]# mysql -V</div><div>mysql Ver 14.14 Distrib 5.1.48, for unknown-linux-gnu (x86_64) using readline 5.1</div></div><div><br></div><div>My <a href="http://ha.cf">ha.cf</a> looks like:</div>
<div><br></div><div><div>autojoin none</div><div>mcast eth0 227.0.0.10 694 1 0</div><div>warntime 5</div><div>deadtime 15</div><div>initdead 60</div><div>keepalive 5</div><div>auto_failback off</div><div>node sipl-mysql-109</div>
<div>node sipl-mysql-209</div><div>crm on </div></div><div><br></div><div><br></div><div>Mysql show the following in it's error log:</div><div><br></div><div><div>100722 15:33:57 [Note] Plugin 'FEDERATED' is disabled.</div>
<div>100722 15:33:57 InnoDB: Started; log sequence number 0 44233</div><div>100722 15:33:57 [Note] Event Scheduler: Loaded 0 events</div><div>100722 15:33:57 [Note] /usr/sbin/mysqld: ready for connections.</div><div>Version: '5.1.48-community-log' socket: '/var/lib/mysql/mysql.sock' port: 3306 MySQL Community Server (GPL)</div>
<div>100722 15:34:01 [Note] /usr/sbin/mysqld: Normal shutdown</div><div><br></div><div>100722 15:34:01 [Note] Event Scheduler: Purging the queue. 0 events</div><div>100722 15:34:01 InnoDB: Starting shutdown...</div><div>
100722 15:34:02 InnoDB: Shutdown completed; log sequence number 0 44233</div><div>100722 15:34:02 [Note] /usr/sbin/mysqld: Shutdown complete</div><div><br></div><div>100722 15:34:02 mysqld_safe mysqld from pid file /var/run/mysql/mysqld.pid ended</div>
<div>100722 15:34:03 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql</div><div>100722 15:34:03 [Warning] '--skip-locking' is deprecated and will be removed in a future release. Please use '--skip-external-locking' instead.</div>
<div>100722 15:34:03 [Note] Plugin 'FEDERATED' is disabled.</div><div>100722 15:34:03 InnoDB: Started; log sequence number 0 44233</div><div>100722 15:34:03 [Note] Event Scheduler: Loaded 0 events</div><div>100722 15:34:03 [Note] /usr/sbin/mysqld: ready for connections.</div>
<div>Version: '5.1.48-community-log' socket: '/var/lib/mysql/mysql.sock' port: 3306 MySQL Community Server (GPL)</div><div><br></div></div><div>Any help would be greatly appreciated. Thanks in advance.</div>
<div>F.</div><div><br></div>