[Pacemaker] Multi-site support in pacemaker (tokens, deadman, CTR)

[Tim Serong]

On 4/28/2011 at 11:06 PM, Florian Haas <florian.haas at linbit.com> wrote: 
> On 2011-04-27 20:55, Lars Marowsky-Bree wrote: 
> > On 2011-04-26T23:34:16, Yan Gao <ygao at novell.com> wrote: 
> >> And the cibs between different sites would still be synchronized? 
> >  
> > The idea is that there would be - perhaps as part of the CTR daemon - a 
> > process that would replicate (manually triggered, periodically, or 
> > automatically) the configuration details of resources associated with a 
> > given ticket (which are easily determined since they depend on it) to 
> > the other sites that are eligible for the ticket. 
> >  
> > Initially, I'd be quite happy if there was a "replicate now" button to 
> > push or script to call - admins may actually have good reasons not to 
> > immediately replicate everywhere, anyway. 
> >  
> > It's conceivable that there would need to be some mangling as 
> > configuration is replicated; e.g., path names and IP addresses may be 
> > different. We _could_ express this using our CIB syntax already 
> > (instance attribute sets take rules, and it'd probably be easy enough 
> > to extend this matching to select on ticket ownership), and perhaps that 
> > is good enough, since I'd imagine there would actually be quite little 
> > to modify. 
> >  
> > (Having many differences would make the configuration very complex to 
> > manage and understand; hence, we want a syntax that makes it easy to 
> > have a few different values, and annoying to have many ;-) 
>  
> As I understood it we had essentially reached consensus in Boston that 
> CIB replication would best be achieved by a pair of complementary 
> resource agents. I don't think we had a name then, but I'll call them 
> Publisher and Subscriber for the purposes of this discussion. 
>  
> The idea would be that Publisher exposes the <configuration/> section of 
> the CIB via a network daemon, preferably one that uses encryption. 
> Suppose this is something like lighttpd with SSL/TLS support. This would 
> be a simple primitive running exactly once in the Pacemaker cluster, and 
> only if that cluster holds the "ticket". 

Hawk just about does that (exposes bits of the CIB via HTTPS), although
admittedly it'd be overkill for just exposing the configuration section
for machine processing.

A stunningly trivial implementation is, simply, in lighttpd.conf:

  cgi.assign = ( "/config" => "" )

Then, create a shell script called "config" in lighttpd's document root
directory, containing:

  #!/bin/sh
  
  echo "Content-type: text/xml"
  echo
  /usr/sbin/cibadmin -Q --scope constraints

Not so much with the security, but it works...

>  
> Subscriber, by contrast, subscribes to this stream and will usually 
> mangle configuration in some shape or form, preferably configurable 
> through an RA parameter. What was discussed in Boston is that in an 
> initial step, Subscriber could simply take an XSLT script, apply it to 
> the CIB stream with xsltproc, and then update its local CIB with the result. 
>  
> Subscriber would be the only resource (besides STONITH resources and 
> Slaves of master/slave sets) that can be active in a cluster that does 
> not hold the "ticket". 
>  
> Comments? 
>  
> Cheers, 
> Florian 
>  



-- 
Tim Serong <tserong at novell.com>
Senior Clustering Engineer, OPS Engineering, Novell Inc.