[Pacemaker] Invisible dependency (at least to me)

[Robert Dahlem]

Hi,

I have this configuration (minus the stonith resources and stuff on
other nodes for brevity):

================================================================================
node korfwf01
node korfwf02
rsc_template template-DRBD ocf:linbit:drbd \
        op start interval="0" timeout="240" \
        op promote interval="0" timeout="105" \
        op demote interval="0" timeout="105" \
        op notify interval="0" timeout="90" \
        op stop interval="0" timeout="100" \
        op monitor role="Slave" timeout="20" interval="20" \
        op monitor role="Master" timeout="20" interval="10"
rsc_template template-FS ocf:heartbeat:Filesystem \
        op start timeout="60s" interval="0" \
        op stop timeout="60s" interval="0" \
        op notify timeout="60s" interval="0" \
        op monitor timeout="40s" interval="20s"
rsc_template template-HAPROXY ocf:KORDOBA:haproxy \
        op start timeout="20s" interval="0" on-fail="stop" \
        op stop timeout="20s" interval="0" on-fail="block" \
        op monitor timeout="20s" interval="10s"
rsc_template template-IP ocf:heartbeat:IPaddr \
        op monitor interval="5s" timeout="20s" \
        op start on-fail="stop" interval="0" \
        op stop on-fail="block" interval="0"
rsc_template template-SQUID ocf:KORDOBA:squid \
        op start timeout="60s" on-fail="stop" interval="0" \
        op stop timeout="20s" on-fail="block" interval="0" \
        op monitor timeout="30s" interval="10s"
primitive ALL-ffm ocf:heartbeat:Dummy
primitive DRBD-ffm @template-DRBD \
        params drbd_resource="ffm"
primitive FS-ffm @template-FS \
        params device="/dev/drbd/by-res/ffm/0" \
        directory="/ffm" fstype="ext3"
primitive HAPROXY-ffm @template-HAPROXY \
        params configfile="/ffm/haproxy/haproxy-ffm.cfg" port="8090"
primitive IP-DMZ-ffm @template-IP \
        params ip="192.168.205.8"
primitive IP-Intra-ffm @template-IP \
        params ip="10.133.128.4"
primitive SQUID-ffm @template-SQUID \
        params configfile="/ffm/squid/squid-ffm.conf" port="8080"
group group-IP-ffm IP-Intra-ffm IP-DMZ-ffm
ms ms-DRBD-ffm DRBD-ffm \
        meta master-max="1" master-node-max="1" \
        clone-max="2" clone-node-max="1" notify="true"
location loc-ALL-ffm-korfwf01 ALL-ffm 2: korfwf01
location loc-ALL-ffm-korfwf02 ALL-ffm 1: korfwf02
colocation coloc-FS-ffm-follows-ms-DRBD-ffm inf: \
        FS-ffm ms-DRBD-ffm:Master
colocation coloc-HAPROXY-ffm-follows-FS-ffm inf: HAPROXY-ffm FS-ffm
colocation coloc-SQUID-ffm-follows-HAPROXY-ffm inf: \
        SQUID-ffm HAPROXY-ffm
colocation coloc-group-IP-ffm-follows-ALL-ffm inf: group-IP-ffm ALL-ffm
colocation coloc-ms-DRBD-ffm-follows-group-IP-ffm inf: \
        ms-DRBD-ffm:Master group-IP-ffm
order ord-ALL-ffm-before-group-IP-ffm inf: ALL-ffm group-IP-ffm
order ord-DRBD-ffm-before-FS-ffm inf: ms-DRBD-ffm:promote FS-ffm:start
order ord-FS-ffm-before-HAPROXY-ffm inf: FS-ffm HAPROXY-ffm
order ord-HAPROXY-ffm-before-SQUID-ffm inf: HAPROXY-ffm SQUID-ffm
order ord-group-IP-ffm-before-DRBD-ffm inf: \
        group-IP-ffm ms-DRBD-ffm:promote
property $id="cib-bootstrap-options" \
        dc-version="1.1.10-f3eeaf4" \
        cluster-infrastructure="classic openais (with plugin)" \
        expected-quorum-votes="4" \
        no-quorum-policy="freeze" \
        last-lrm-refresh="1405095139" \
        stop-all-resources="false" \
        maintenance-mode="false" \
        stonith-enabled="true"
rsc_defaults $id="rsc-options" \
        resource-stickiness="100"
================================================================================

The configuration builds this dependency tree:

ALL-ffm
  group-IP-ffm IP-Intra-ffm IP-DMZ-ffm
    ms-DRBD-ffm
      FS-ffm
        HAPROXY-ffm
          SQUID-ffm

ALL-ffm is a dummy resource allowing me to move all depending resources
from one node to another.

group-IP-ffm is a group containing several IP addresses.

DRBD and FS are doing the obious, HAPROXY is a load balancer selecting
between several parent proxies and SQUID is - well - a squid proxy.

Now I wanted to add a proxy-ARP script. It depends on group-IP-ffm but
has no dependencies to other resources. New tree should be:

ALL-ffm
  group-IP-ffm IP-Intra-ffm IP-DMZ-ffm
    PROXYARP-ffm
    ms-DRBD-ffm
      FS-ffm
        HAPROXY-ffm
          SQUID-ffm

I added:
================================================================================
rsc_template template-PROXYARP ocf:KORDOBA:proxy_arp \
        op start timeout="240s" on-fail="stop" interval="0" \
        op stop timeout="20s" interval="0" on-fail="block" \
        op monitor timeout="20s" interval="120s"
primitive PROXYARP-ffm @template-PROXYARP \
        params configfile="/ffm/proxy_arp/proxy_arp-ffm"
colocation coloc-ms-PROXYARP-ffm-follows-group-IP-ffm inf: \
        PROXYARP-ffm group-IP-ffm
order ord-group-IP-ffm-before-PROXYARP-ffm inf: \
        group-IP-ffm PROXYARP-ffm
================================================================================

Unfortunately I made the mistake to place the configuration file in /ffm
so actually PROXYARP-ffm depends on FS-ffm, but this is not configured
in pacemaker.

In the first place this had no effect because the file system /ffm was
already mounted when I added the resource PROXYARP-ffm. But when I tried
to move ALL-ffm from node korfwf01 to node korfwf02 I ended up with this
situation:

================================================================================
Full list of resources:

 ALL-ffm        (ocf::heartbeat:Dummy): Started korfwf02
 Resource Group: group-IP-ffm
     IP-Intra-ffm       (ocf::heartbeat:IPaddr):        Started korfwf02
     IP-DMZ-ffm (ocf::heartbeat:IPaddr):        Started korfwf02
 PROXYARP-ffm   (ocf::KORDOBA:proxy_arp):       Stopped
 Master/Slave Set: ms-DRBD-ffm [DRBD-ffm]
     Slaves: [ korfwf01 korfwf02 ]
 FS-ffm (ocf::heartbeat:Filesystem):    Stopped
 HAPROXY-ffm    (ocf::KORDOBA:haproxy): Stopped
 SQUID-ffm      (ocf::KORDOBA:squid):   Stopped

Failed actions:
    PROXYARP-ffm_start_0 on korfwf02 'not configured' (6): call=123,
status=complete, last-rc-change='Mon Jul 14 10:54:54 2014', queued=0ms,
exec=13ms
================================================================================

The start failure of PROXYARP-ffm is expected because I have the
configuration file in the wrong location and validate-all notices that.

But the start failure of PROXYARP-ffm also seems to have a negative
effect on ms-DRBD-ffm (no promote done) although there is no dependency
between those two. With all other resources depending on
ms-DRBD-ffm:Master they do not get started although I expected them to
get started independent of PROXYARP-ffm.

Could someone please shed some light on this?

Kind regards,
Robert