[Pacemaker] Searching for a viable Debian solution

[Paul Gear]

Hi folks,

Over the last several days i've been reading, asking questions (special 
thanks to beekhof, tserong, fghass and the other kind folks on #linux-ha 
putting up with my questions), and experimenting with my test setup, and 
i'm yet to find a viable combination of options for creating a VM 
cluster on Debian.

The preferred components of my setup are:
- Debian lenny
- Xen VMs
- OCFS2 on an iSCSI NAS for shared storage
- whatever openais/corosync/pacemaker combination will do the job
- I'd like to have live migration of VMs but it's not essential

The point of my setup is to have a test cluster with which i can play 
(and run my own home network) without breaking the ones i have to 
support at customers.  I work with small businesses, educational 
institutions, and non-profits, none of whom have the means to support a 
fully working test configuration, or to buy time from big name 
consultants who can employ full-time clustering specialists.

My experiments thus far have been with corosync & pacemaker from 
http://people.debian.org/~madkiss/ha/, and with libvirt from 
lenny-backports.

When i tried the latest versions of corosync & pacemaker in the above 
repo, i was able to get the cluster running appropriately, but when i 
tried to add the DLM clone resource, i got this error: "controld[3526]: 
ERROR: Setup problem: Couldn't find utility dlm_controld.pcmk".  So it 
seems that OCFS2 is not viable with the current state of the 
corosync/openais/pacemaker stack.  I've investigated a couple of 
alternatives to my preferred list, including:

- cLVM - requires the Red Hat cluster stack and conflicts with the 
current corosync/openais/pacemaker versions.  This would be my preferred 
solution if it were possible to use LVs from the same VG on different 
nodes simultaneously.  I'm not sure whether this is possible, because 
the cLVM documentation is almost non-existent, and i couldn't find 
answers to any of my big questions.

- libvirt with iSCSI as the backend for the storage pool.  I've tried 
this and it looks promising, but it appears non-functional on iSCSI 
storage backends at the moment.  I get this message when i try to create 
a volume: "libvirtd: ... error : storageVolumeCreateXML:1301 : this 
function is not supported by the hypervisor: storage pool does not 
support volume creation"

I substituted KVM for Xen, and the result was the same.

So, the question remains: what is a viable HA stack for Debian with 
virtualization and shared storage?  I'm happy to switch technologies 
where it's necessary; here are the things i would be willing to try if 
necessary:

- Different clustered file storage setup.  What are the possibilities? 
GFS2?  DRBD seems feasible, but it doesn't actually solve anything in 
the above equation, so i'd rather keep the iSCSI NAS.

- Switching distros?  Possible - my order of preference: Debian squeeze, 
Debian sid, Ubuntu lucid, openSUSE 11.2, CentOS 5.4.

- Switching cluster stacks?  I've worked with clusters on HP-UX, 
NetWare, OES/Linux 1, heartbeat 1, and heartbeat 2, so i'm sure one more 
change won't kill me, but the more similar in feel it is to pacemaker or 
heartbeat 2, the better.  If using non-integrated OCFS2 cluster 
membership and an older openais or heartbeat version is viable, i'm 
happy to look at that as well, although obviously i'd rather go forward 
than backward.

- Switching VM technologies?  I guess KVM, VirtualBox, or OpenVZ would 
be viable, but again, it doesn't seem to solve much in the current scenario.

What's the path of least resistance here?

Thanks in advance,
Paul