[Pacemaker] co-location of STONITH resources

[Digimer]

On 25/06/14 12:45 PM, Riccardo Murri wrote:
> Hello,
>
> we are using IPMI to fence off nodes in the cluster (yes, we know it's
> a bad idea but we do not yet have proper STONITH HW so we have to
> sitck with it).

IPMI is just fine as a fence method. It's probably the most commonly 
used one, actually. The only trick is to understand that IPMI draws its 
power from the host, so it will fail if power is entirely lost to the 
node. For that, I use stonith topology to order IPMI -> switched PDUs.

http://clusterlabs.org/wiki/Fencing_topology

> We would like to co-locate STONITH with a functional ethernet
> interface on the IPMI network (`eth0.617`).  So we have the following
> rules::

Unless I am wrong (and I am not a pacemaker expert), you no longer need 
to tell stonith where to run (on pacemaker 1.1.10+, anyway).

>      primitive ipmi_net_up ethmonitor \
>              params interface=eth0.617 name=ipmi_net_up \
>              op monitor interval=5s timeout=60s \
>              op start interval=0 timeout=60s \
>              op stop interval=0
>
>      clone ipmi_net_up_clone ipmi_net_up \
>              meta globally-unique=false ordered=false notify=false
> interleave=true clone-node-max=1
>
>      colocation stonith-lustre-oss1-with-ipmi inf: stonith-lustre-oss1
> ipmi_net_up_clone
>      colocation stonith-lustre-oss2-with-ipmi inf: stonith-lustre-oss2
> ipmi_net_up_clone
>      ...
>      colocation stonith-lustre-oss8-with-ipmi inf: stonith-lustre-oss8
> ipmi_net_up_clone
>
> Now the ethmonitor probe is successful::
>
>      Jun 25 18:34:59 lustre-mds1 crmd[3255]:   notice:
> process_lrm_event: Operation ipmi_net_up_monitor_5000: ok
> (node=lustre-mds1.ften.es.hpcn.uzh.ch, call=725, rc=0, cib-update=392,
> confirmed=false)
>
> However, STONITH resources appear as "Stopped", and `crm_simulate`
> says they cannot be located anywhere::
>
>      # crm_simulate -sL | fgrep stonith
>       stonith-lustre-oss1    (stonith:fence_ipmilan):        Stopped
>       stonith-lustre-oss2    (stonith:fence_ipmilan):        Stopped
>       stonith-lustre-oss3    (stonith:fence_ipmilan):        Stopped
>       stonith-lustre-oss4    (stonith:fence_ipmilan):        Stopped
>       stonith-lustre-oss5    (stonith:fence_ipmilan):        Stopped
>       stonith-lustre-oss6    (stonith:fence_ipmilan):        Stopped
>       stonith-lustre-oss7    (stonith:fence_ipmilan):        Stopped
>       stonith-lustre-oss8    (stonith:fence_ipmilan):        Stopped
>      native_color: stonith-lustre-oss1 allocation score on
> lustre-mds1.ften.es.hpcn.uzh.ch: -INFINITY
>      native_color: stonith-lustre-oss1 allocation score on
> lustre-mds2.ften.es.hpcn.uzh.ch: -INFINITY
>      native_color: stonith-lustre-oss1 allocation score on
> lustre-oss1.ften.es.hpcn.uzh.ch: -INFINITY
>      native_color: stonith-lustre-oss1 allocation score on
> lustre-oss2.ften.es.hpcn.uzh.ch: -INFINITY
>      native_color: stonith-lustre-oss1 allocation score on
> lustre-oss3.ften.es.hpcn.uzh.ch: -INFINITY
>      native_color: stonith-lustre-oss1 allocation score on
> lustre-oss4.ften.es.hpcn.uzh.ch: -INFINITY
>      native_color: stonith-lustre-oss1 allocation score on
> lustre-oss5.ften.es.hpcn.uzh.ch: -INFINITY
>      native_color: stonith-lustre-oss1 allocation score on
> lustre-oss6.ften.es.hpcn.uzh.ch: -INFINITY
>      native_color: stonith-lustre-oss1 allocation score on
> lustre-oss7.ften.es.hpcn.uzh.ch: -INFINITY
>      native_color: stonith-lustre-oss1 allocation score on
> lustre-oss8.ften.es.hpcn.uzh.ch: -INFINITY
>      native_color: stonith-lustre-oss2 allocation score on
> lustre-mds1.ften.es.hpcn.uzh.ch: -INFINITY
>      native_color: stonith-lustre-oss2 allocation score on
> lustre-mds2.ften.es.hpcn.uzh.ch: -INFINITY
>      native_color: stonith-lustre-oss2 allocation score on
> lustre-oss1.ften.es.hpcn.uzh.ch: -INFINITY
>      ...
>
> What are we doing wrong?
>
> Thanks,
> Riccardo


-- 
Digimer
Papers and Projects: https://alteeve.ca/w/
What if the cure for cancer is trapped in the mind of a person without 
access to education?