[Pacemaker] Asymmetric cluster resource locations

[Nick Morrison]

Hello,

I've been browsing around the history of the list, and it looks like this topic has been covered by people with lots more experience in clustering than I.  For that reason, I thought I should ask to be sure, just in case my problem is actually an easy one to solve.

My desired architecture is perhaps a little weird.

I'm trying to find a path-of-least-resistance to configuring our four Dell servers into a reasonably-High-Availability cluster.  I want to avoid using any active/active block devices because I don't want to have to think about split brain recovery.

The architecture.  Call them host-a1 and host-a2, and host-b1 and host-b2.

. Each of these is an primary/secondary DRBD pair (managed by pacemaker)
. Each of the pairs runs failover NFS (managed by pacemaker)
. host-a[12] export /data/pool-a
. host-b[12] export /data/pool-b
. All four hosts run virtual machines, with virtual disk images stored in /data/pool-a or /data/pool-b
. The one limitation on virtual machines is that they can't run on their own pool's secondary storage, as it's not accessible
. It's not accessible because trying to make an NFS server mount its own export was failing horribly
  . (otherwise I'd mount the same virt-a:/data/pool-a on all four machines and life would be easy)

Crazy?  Maybe.  I haven't seen anyone design an architecture like this.  Perhaps everyone is braver than me about GFS2/OCFS2.  I'm willing to accept a few seconds of IO-Wait while the NFS server switches over.

a) am I missing anything?

b) the reason I wrote to the list:

I'm constraining my resources per server pair with -inf: away from the other nodes and 2: to the nodes I want them on.  Except, sometimes I'll find that my NFS-DRBDFS-POOL-A ocf:heartbeat:Filesystem resource that's supposed to be running on host-a1 (along with the DRBD master for pool-a) is suddenly running on host-b2, and setting it to "unmanaged".

I think that's because pacemaker is running a monitor operation on all of the nodes, finding "a filesystem" mounted in /data/pool-a (which it is - via NFS, from the POOL-A pair) and so assuming that someone's gone and started up the service without telling it.  Hence unmanaged.

My question is : am I right? and, what's the best way to deal with this scenario?

I thought of running two clusters.  This works ok, EXCEPT, I lose the ability to have my virtual machines managed by pacemaker also, because it would have to know about all four nodes.  Maybe I could run three clusters, node-a1+nodea2 for drbd/nfs, nodeb1+nodeb2 for drbd/nfs, and node.* for virtual machines.

Or maybe I just manage my virtual machines by hand.

Your thoughts are appreciated.  If you've come across any good reading on the topic, I'd love to have the link :-)


Cheers and beers,

Nick!