[Pacemaker] Concurrent start of kvm virtual machines

[Karasun Karasun]

Hi,
I use combination Pacemaker 1-1-9971ebba4494 with corosync 1.4.2 (build
from source) in two node setup and it looks like everything works great.
My cluster contain two DRBD resources in Master-Slave configuration. On
each of DRBD resources are placed some virtual disk images for KVM virtual
machines.
The idea of this cluster is to run 3 KVM virtual machines on node1 and 5
KVM virtual machines on node2. In case of failure of node1, node2 should
start DRBD resource dedicated to node1 and after that,it should start all 3
KVM virtual machines which were on node1. Of course it works in my setup,
which is attached at the end.
I configured virtual machines to starts sequentially in group with their
DRBD resource (e.g. group group_node1 srv_drbd0 kvm1 kvm2 kvm3), as a
result I get sequence of actions:
1. srv_drbd0 starts and in case of success kvm1 can start
2. kvm1 starts and in case of success kvm2 can start
3. kvm2 starts and in case of success kvm3 can start
4. kvm3 starts

My problem is that starting cluster takes long time and I need concurrent
start of KVM virtual machines after DRBD resource is available, just like
in this scenario:
1. srv_drbd0 starts and in case of success kvm1, kvm2, kvm3 can start
2. kvm1, kvm2, kvm3 start concurrently
Is it possible to configure pacemaker cluster in that way?

My cluster configuration (stonith is off during tests):

node node1
> node node2
> primitive kvm1 ocf:heartbeat:VirtualDomain \
>         params config="/etc/libvirt/qemu/kvm1.xml"
> hypervisor="qemu:///system" \
>         op start interval="0" timeout="90s" \
>         op stop interval="0" timeout="90s" \
>         op monitor interval="10" timeout="30" \
>         utilization cpu="2" hv_memory="2048"
> primitive kvm2 ocf:heartbeat:VirtualDomain \
>         params config="/etc/libvirt/qemu/kvm2.xml"
> hypervisor="qemu:///system" \
>         op start interval="0" timeout="90s" \
>         op stop interval="0" timeout="90s" \
>         op monitor interval="10" timeout="30" \
>         utilization cpu="2" hv_memory="512"
> primitive kvm3 ocf:heartbeat:VirtualDomain \
>         params config="/etc/libvirt/qemu/kvm3.xml"
> hypervisor="qemu:///system" \
>         op start interval="0" timeout="90s" \
>         op stop interval="0" timeout="90s" \
>         op monitor interval="10" timeout="30" \
>         utilization cpu="2" hv_memory="512"
> primitive drbd_r0 ocf:linbit:drbd \
>         params drbd_resource="r0" \
>         op monitor interval="29s" role="Master" \
>         op monitor interval="31s" role="Slave" \
>         op start interval="0" timeout="240" \
>         op stop interval="0" timeout="100"
> primitive drbd_r1 ocf:linbit:drbd \
>         params drbd_resource="r1" \
>         op monitor interval="29s" role="Master" \
>         op monitor interval="31s" role="Slave" \
>         op start interval="0" timeout="240" \
>         op stop interval="0" timeout="100"
> primitive kvm4 ocf:heartbeat:VirtualDomain \
>         params config="/etc/libvirt/qemu/kvm4.xml"
> hypervisor="qemu:///system" \
>         op start interval="0" timeout="90s" \
>         op stop interval="0" timeout="90s" \
>         op monitor interval="10" timeout="30" \
>         utilization cpu="2" hv_memory="512"
> primitive kvm5 ocf:heartbeat:VirtualDomain \
>         params config="/etc/libvirt/qemu/kvm5.xml"
> hypervisor="qemu:///system" \
>         op start interval="0" timeout="90s" \
>         op stop interval="0" timeout="90s" \
>         op monitor interval="10" timeout="30" \
>         utilization cpu="2" hv_memory="2048"
> primitive kvm6 ocf:heartbeat:VirtualDomain \
>         params config="/etc/libvirt/qemu/kvm6.xml"
> hypervisor="qemu:///system" \
>         op start interval="0" timeout="90s" \
>         op stop interval="0" timeout="90s" \
>         op monitor interval="10" timeout="30" \
>         utilization cpu="2" hv_memory="512"
> primitive kvm7 ocf:heartbeat:VirtualDomain \
>         params config="/etc/libvirt/qemu/kvm7.xml"
> hypervisor="qemu:///system" \
>         op start interval="0" timeout="90s" \
>         op stop interval="0" timeout="90s" \
>         op monitor interval="10" timeout="30" \
>         utilization cpu="2" hv_memory="512"
> primitive kvm8 ocf:heartbeat:VirtualDomain \
>         params config="/etc/libvirt/qemu/kvm8.xml"
> hypervisor="qemu:///system" \
>         op start interval="0" timeout="90s" \
>         op stop interval="0" timeout="90s" \
>         op monitor interval="10" timeout="30" \
>         utilization cpu="2" hv_memory="2048"
> primitive srv_drbd0 ocf:heartbeat:Filesystem \
>         params device="/dev/drbd0" directory="/srv/drbd0" fstype="ext4"
> primitive srv_drbd1 ocf:heartbeat:Filesystem \
>         params device="/dev/drbd1" directory="/srv/drbd1" fstype="ext4"
> group group_node1 srv_drbd0 kvm1 kvm2 kvm3
> group group_node2 srv_drbd1 kvm4 kvm5 kvm6 kvm7 kvm8
> ms ms_drbd_r0 drbd_r0 \
>         meta clone-max="2" notify="true" master-max="1"
> master-node-max="1" clone-node-max="1"
> ms ms_drbd_r1 drbd_r1 \
>         meta clone-max="2" notify="true" master-max="1"
> master-node-max="1" clone-node-max="1"
> location cli-prefer-group_node1 group_node1 \
>         rule $id="cli-prefer-rule-group_node1" inf: #uname eq node1
> location cli-prefer-group_node2 group_node2 \
>         rule $id="cli-prefer-rule-group_node2" inf: #uname eq node2
> colocation colocation_node1 inf: group_node1 ms_drbd_r0:Master
> colocation colocation_node2 inf: group_node2 ms_drbd_r1:Master
> order order_node1 inf: ms_drbd_r0:promote group_node1:start
> order order_node2 inf: ms_drbd_r1:promote group_node2:start
> property $id="cib-bootstrap-options" \
>         dc-version="1.1.6-9971ebba4494012a93c03b40a2c58ec0eb60f50c" \
>         cluster-infrastructure="openais" \
>         expected-quorum-votes="2" \
>         stonith-enabled="false" \
>         no-quorum-policy="ignore" \
>         last-lrm-refresh="1352497364"
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.clusterlabs.org/pipermail/pacemaker/attachments/20121203/b48fd6e7/attachment-0002.html>