[Pacemaker] Alternative communication engine to corosync (etcd/consul/zookeeper/doozerd)

[Lars Marowsky-Bree]

On 2014-06-20T11:32:58, Patrick Hemmer <pacemaker at feystorm.net> wrote:

> > I rather doubt it.  k/v stores and CPG are not very alike from where I'm sitting.
> No, they are not alike, but you could implement something looking like CPG.
> When a key is created, that's a CPG message. They support atomic
> operations (collision checking), so no 2 nodes could update a key at the
> same time. So while it's a different underlying system, the end result
> is the same, ordered messages.
> >> I think pacemaker might be able to use a key/value store natively.
> But I wouldn't even bother with hacking a KVS into something like CPG if
> it's not needed. I would do it such that the CIB is stored as keys and
> values natively. I would even think this is more efficient. I'm not sure
> how the CIB is transmitted between nodes, but I think it easiest to just
> set a single key when you want to update something like a resource's
> last-rc-change value.

That's actually the most likely entry point; almost all of pacemaker's
state and configuration flows through the CIB, and transitions are
triggered appropriately.

Yes, there are exceptions, like fencing messages, but the bulk of the
information flow is through the CIB - and implementing CPG-over-K/V to
then implement a hierarchical distributed database like the CIB on top
seems like a broken design.

(Things like OCFS2/GFS2/cLVM2 would also probably benefit from adopting
their interfaces to the K/V world natively.)

The one thing HA clusters like those build with pacemaker typically do
require though is strong consistency and coherence. That is not, as far
as I understand, a traditional property of these K/V stores.

Also, pacemaker implements its own local resource manager, and it's
questionable if that is still really required on modern Linux versions.

I'd assume that such a rearchitecture would actually look quite
different from what Pacemaker/corosync deliver today. I'd assume that it
might indeed be better to start "from scratch", and investigate a
combination of docker/etc/systemd/fleet.

(Looking at fleet, I can see a number of things that are missing from it
before I'd trust it with my data/services. But the fundamental design
appears sound.)


Regards,
    Lars

-- 
Architect Storage/HA
SUSE LINUX Products GmbH, GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer, HRB 21284 (AG Nürnberg)
"Experience is the name everyone gives to their mistakes." -- Oscar Wilde