[Pacemaker] Percona Replication Manager - Adding a new slave
Mark Stunnenberg
mark at e-rave.nl
Thu Mar 8 14:26:39 UTC 2012
Continuing on the mailing list (I posted it on the site since it was
more of a addition to your blog post):
---
That's the whole reason for me to add that post on your blog, the
functionality you describe is not working.
I had a test setup with 3 servers, which was working nicely.
Then I removed 1 server and replaced it with a fresh server. (so the
clone-max is already on 3)
- Removed the "old" node from the config.
- I installed a server, mysql-server, made a binary copy.
- Started the slave to test replication from master.
- Installed corosync/pacemaker/mysql-RA on new server.
- Stopped mysql.
- Started corosync.
What happened was that the mysql RA did a CHANGE MASTER TO on my new
slave, with the data it read from the clusters' "replication_info"
variable. But the thing is, that data is set by the current master when
it became a master. That has been over a month ago, so the master log
files aren't there anymore, the "master log pos" and "master log file"
are not consistent with the snapshot I made from the master to create a
new slave. So I ended up with a new slave and the error that it couldn't
find mysql-bin.000002 (that's the binlog that was active when the
current master became one)
What I did was this:
- Created a new slave (again)
- Let it replicate from master to test
- Stop slave and check it's current master log file and log pos
- Use the following command to set the correct values in the
"replication_info" variable:
crm_attribute --type crm_config --name replication_info -s
mysql_replication -v "192.168.5.96|mysql-bin.000022|1029552725"
Now start corosync on the new slave, slave resources are started and the
CHANGE MASTER TO will set the correct values and replication will start.
So every time I want to add a new slave to the cluster, before you start
corosync on it, you need to set the "replication_info" variable to
reflect the currently added slave's position in the master log file.
That's not really a big problem, it's just something that should be
noted ;)
If I try the method you describe, the slave wants to start replicating
from an invalid position (because of an incorrect "replication_info".
in the code I read:
mysql_start() {
..snip..
if ocf_is_ms; then
# We're configured as a stateful resource. We must start as
# slave by default. At this point we don't know if the CRM has
# already promoted a master. So, we simply start in read only
# mode.
set_read_only on
# Now, let's see whether there is a master. We might be a new
# node that is just joining the cluster, and the CRM may have
# promoted a master before.
master_host=`echo $OCF_RESKEY_CRM_meta_notify_master_uname`
if [ "$master_host" -a "$master_host" != `uname -n` ]; then
ocf_log info "Changing MySQL configuration to replicate
from $master_host."
set_master
start_slave
if [ $? -ne 0 ]; then
ocf_log err "Failed to start slave"
return $OCF_ERR_GENERIC
fi
else
ocf_log info "No MySQL master present - clearing
replication state"
unset_master
fi
.. snip ..
}
The cluster cannot know what to set the "replication_info" to, since it
doesn't know where the slave is and if it's correct, so it needs to be
done manually.
Correct me if I'm completely wrong here! :)
Regards,
Mark Stunnenberg
---
Hi Mark,
the discussion move to the pacemaker mailing list. I'll answer the
slave since it is a common question. Adding a new slave is very easy,
if the currnet master is the same as the one in the replication_info,
binlog file and position are ignored. So what you do is the following.
- Setup replication with the master
- Stop mysql
- Start Pacemaker on the new server and increase clone-max by 1. This
will start MySQL and it will be a slave of the current master.
Please ask your next question to the pacemaker mailing list:
pacemaker at oss.clusterlabs.org
Regards,
Yves
On 12-03-07 01:11 PM, Mark Stunnenberg wrote:
> It's been pretty quiet here for a while now :)
>
> I've to add something I ran into while I had some maintenance planned
> on the cluster. What I wanted to do is add another server to the
> cluster.
>
> The "replication_info" config setting is only changed when a new
> master is elected. So how to add a new slave?
>
> Create a new slave by dumping/binary copying or any other way your
> normally prepare a slave. Now change the "replication_info" manually
> in corosync to the new binlog and logpos position from when you took
> the dump/copy.
>
> From within corosync itself you'll get an error when you try to edit
> it, so use the command that the mysql RA uses, which is:
> crm_attribute --type crm_config --name replication_info -s
> mysql_replication -v "||"
>
> Now start corosync on your slave and let it join the cluster, it will
> now correctly start slaving from the position you made the snapshot.
>
--
Regards,
Yves
skype: y.trudeau
aim: ytrudeaumysql
gtalk/Jabber: trudeau.y at gmail.com
email: yves at percona.com
Phone: (408) 457-0699
Flat-rate 24x7 support for MySQL<http://tinyurl.com/mysql-support>
More information about the Pacemaker
mailing list