[Pacemaker] Unexpected master/slave behavior

[David Mohr]

Hi everyone,
we are trying to understand how pacemaker deals with master/slave
resources. After reading the documentation we believe we understand how
location and colocation work. Yet in practice we don't get the results that
we expect. We experimented with a minimal setup:

* 2 nodes,
* drbd in a primary/secondary setup,
* an IP address that's supposed to be up on the drbd master.

Our goal was to learn how to get crm to move the ip and the drbd master
from one node to the other. In the end this is supposed to enable
connectivity based resource placement (e.g. through pingd).


The configuration:

# crm configure show
node s1b \
        attributes standby="off"
node s1a \
        attributes standby="off"
primitive drbd0 ocf:linbit:drbd \
        params drbd_resource="vm1" drbdconf="/etc/drbd.conf" \
        op start interval="0" timeout="240" \
        op stop interval="0" timeout="100" \
        op monitor interval="59s" role="Master" timeout="30s" \
        op monitor interval="60s" role="Slave" timeout="30s" \
        meta is-managed="true"
primitive master_ip ocf:heartbeat:IPaddr2 \
        params ip="10.0.0.10" nic="bond0" \
        meta is-managed="true"
ms ms_drbd0 drbd0 \
        meta master-max="1" master-node-max="1" clone-max="2"
clone-node-max="1" notify="true" target-role="Master"
colocation col_test inf: master_ip ms_drbd0:Master
property \
        cluster-infrastructure="Heartbeat" \
        stonith-enabled="false"

For these experiments s1a is the drbd primary, and that's also where
master_ip is running.


First we tried to move the master_ip directly. The docs say that due to
the colocation drbd0:master should take master_ip's preference into
account, and thus switch s1a to be the primary node before moving
master_ip.

# crm resource migrate master_ip s1b
# crm configure show | grep 'cli-'
location cli-prefer-master_ip master_ip \
        rule $id="cli-prefer-rule-master_ip" inf: #uname eq s1b

Unfortunately this rule had no visible effect, s1a remains as the drbd
primary and runs master_ip.


The second try was to add a rule that forces s1a to become the primary:
# crm configure \
    location master_on_s1b ms_drbd0 \
        rule $id="test-rule" $role="Master" inf: #uname eq s1b

However again nothing happens at all. Both the drbd primary and master_ip
stay on s1a.

When working with -inf everything is working as expected, e.g. 
# crm resource migrate master_ip
will move the resource to s1b.

But instead we want to work with positive scores, since it allows more
fine-grained control. As far as our understanding _both_ of the above rules
should move master_ip to s1b. What are we doing wrong? We tried to look
through the log files, but were not able to get to the root of the problem.
We also tried various pacemaker versions: 1.0.8+hg15494, 1.0.9 and 1.0.9.1.
We're trying to avoid upgrading to the unstable tree. Is it a bug or a
misunderstanding on our side?

Any help would be appreciated!

Thanks,
~David