[Pacemaker] iscsi clone + xen migration

[laurent+pacemaker at u-picardie.fr]

Hi there,

I have a 3 nodes cluster setup like this :

# crm configure show
node cardhu
node dualamd1
node dualamd3
primitive dom0-iscsi1-cnx1 ocf:heartbeat:iscsi \
        params portal="10.151.1.72" target="testfrontal-01:test01"
primitive domU-test01 ocf:heartbeat:Xen \
        meta allow-migrate="true" \
        params xmfile="/etc/xen/test01.cfg" allow_migrate="true"
group dom0-iscsi1 dom0-iscsi1-cnx1
clone clone-dom0-iscsi1 dom0-iscsi1 \
        meta clone-max="2" clone-node-max="1" globally-unique="false"

colocation domU-test01-iscsi1-colocation1 +inf: domU-test01 clone-dom0-iscsi1
order domU-test01-iscsi1-order1 0: clone-dom0-iscsi1 domU-test01

property $id="cib-bootstrap-options" \
        dc-version="1.0.4-e8cf4c8509db6ef1b2b21eb7ca8c94b094757e26" \
        cluster-infrastructure="openais" \
        expected-quorum-votes="3075295522" stonith-enabled="false" \
        start-failure-is-fatal="false" \
        default-resource-stickiness="5000" \ 
        symmetric-cluster="true" no-quorum-policy="ignore"

iscsi clones run on dualamd1 and dualamd3, xen resource runs on
dualamd3.
If i issue a "crm resource migrate domU-test01 dualamd1" it works as
expected (while there's an error in the logs)

excerts of the log for this operation:
pengine: [28017]: notice: print_list: #011Started: [ dualamd3 dualamd1 ]
pengine: [28017]: info: complex_migrate_reload: Migrating domU-test01 from dualamd3 to dualamd1
pengine: [28017]: info: complex_migrate_reload: Repairing domU-test01-iscsi1-colocation1: domU-test01 == clone-dom0-iscsi1 (1000000)
pengine: [28017]: ERROR: complex_migrate_reload: Ordering domU-test01_migrate_from_0 before all_stopped (stop)
pengine: [28017]: notice: LogActions: Migrate resource domU-test01#011(Started dualamd3 -> dualamd1)
pengine: [28017]: notice: LogActions: Leave resource dom0-iscsi1-cnx1:0#011(Started dualamd3)
pengine: [28017]: notice: LogActions: Leave resource dom0-iscsi1-cnx1:1#011(Started dualamd1)


The domU-test01 is now running on dualamd1, and I have unmigrated
domU-test01.

If I issue a "crm resource migrate domU-test01" (no destination given)
something weird happens.
It selects the correct node for the migration (for me dualamd3
is the only good choice since it is already running the iscsi clone)

But it also tries to move one clone instance for no apparent reason
and the colocation now prevents the xen resource to be migrated.

This is a excerpt of the logs:
pengine: [28017]: notice: clone_print: Clone Set: clone-dom0-iscsi1
pengine: [28017]: notice: print_list: #011Started: [ dualamd3 dualamd1 ]
pengine: [28017]: notice: at_stack_bottom: Cannot migrate domU-test01 due to colocation activity (last was clone-dom0-iscsi1)
pengine: [28017]: notice: LogActions: Move resource domU-test01#011(Started dualamd1 -> dualamd3)
pengine: [28017]: notice: LogActions: Leave resource dom0-iscsi1-cnx1:0#011(Started dualamd3)
pengine: [28017]: notice: LogActions: Move resource dom0-iscsi1-cnx1:1#011(Started dualamd1 -> cardhu)

Is it because of my rules, am I missing something ?
Should I add rules to make sure the the xen resource is "always"
migrated to a node that is already running the clone ?

Thanks in advance.

-- 
Laurent