[Pacemaker] co-location of STONITH resources

[Riccardo Murri]

Hello,

we are using IPMI to fence off nodes in the cluster (yes, we know it's
a bad idea but we do not yet have proper STONITH HW so we have to
sitck with it).

We would like to co-locate STONITH with a functional ethernet
interface on the IPMI network (`eth0.617`).  So we have the following
rules::

    primitive ipmi_net_up ethmonitor \
            params interface=eth0.617 name=ipmi_net_up \
            op monitor interval=5s timeout=60s \
            op start interval=0 timeout=60s \
            op stop interval=0

    clone ipmi_net_up_clone ipmi_net_up \
            meta globally-unique=false ordered=false notify=false
interleave=true clone-node-max=1

    colocation stonith-lustre-oss1-with-ipmi inf: stonith-lustre-oss1
ipmi_net_up_clone
    colocation stonith-lustre-oss2-with-ipmi inf: stonith-lustre-oss2
ipmi_net_up_clone
    ...
    colocation stonith-lustre-oss8-with-ipmi inf: stonith-lustre-oss8
ipmi_net_up_clone

Now the ethmonitor probe is successful::

    Jun 25 18:34:59 lustre-mds1 crmd[3255]:   notice:
process_lrm_event: Operation ipmi_net_up_monitor_5000: ok
(node=lustre-mds1.ften.es.hpcn.uzh.ch, call=725, rc=0, cib-update=392,
confirmed=false)

However, STONITH resources appear as "Stopped", and `crm_simulate`
says they cannot be located anywhere::

    # crm_simulate -sL | fgrep stonith
     stonith-lustre-oss1    (stonith:fence_ipmilan):        Stopped
     stonith-lustre-oss2    (stonith:fence_ipmilan):        Stopped
     stonith-lustre-oss3    (stonith:fence_ipmilan):        Stopped
     stonith-lustre-oss4    (stonith:fence_ipmilan):        Stopped
     stonith-lustre-oss5    (stonith:fence_ipmilan):        Stopped
     stonith-lustre-oss6    (stonith:fence_ipmilan):        Stopped
     stonith-lustre-oss7    (stonith:fence_ipmilan):        Stopped
     stonith-lustre-oss8    (stonith:fence_ipmilan):        Stopped
    native_color: stonith-lustre-oss1 allocation score on
lustre-mds1.ften.es.hpcn.uzh.ch: -INFINITY
    native_color: stonith-lustre-oss1 allocation score on
lustre-mds2.ften.es.hpcn.uzh.ch: -INFINITY
    native_color: stonith-lustre-oss1 allocation score on
lustre-oss1.ften.es.hpcn.uzh.ch: -INFINITY
    native_color: stonith-lustre-oss1 allocation score on
lustre-oss2.ften.es.hpcn.uzh.ch: -INFINITY
    native_color: stonith-lustre-oss1 allocation score on
lustre-oss3.ften.es.hpcn.uzh.ch: -INFINITY
    native_color: stonith-lustre-oss1 allocation score on
lustre-oss4.ften.es.hpcn.uzh.ch: -INFINITY
    native_color: stonith-lustre-oss1 allocation score on
lustre-oss5.ften.es.hpcn.uzh.ch: -INFINITY
    native_color: stonith-lustre-oss1 allocation score on
lustre-oss6.ften.es.hpcn.uzh.ch: -INFINITY
    native_color: stonith-lustre-oss1 allocation score on
lustre-oss7.ften.es.hpcn.uzh.ch: -INFINITY
    native_color: stonith-lustre-oss1 allocation score on
lustre-oss8.ften.es.hpcn.uzh.ch: -INFINITY
    native_color: stonith-lustre-oss2 allocation score on
lustre-mds1.ften.es.hpcn.uzh.ch: -INFINITY
    native_color: stonith-lustre-oss2 allocation score on
lustre-mds2.ften.es.hpcn.uzh.ch: -INFINITY
    native_color: stonith-lustre-oss2 allocation score on
lustre-oss1.ften.es.hpcn.uzh.ch: -INFINITY
    ...

What are we doing wrong?

Thanks,
Riccardo