[Pacemaker] Pacemaker remote nodes, naming, and attributes

[David Vossel]

----- Original Message -----
> From: "David Vossel" <dvossel at redhat.com>
> To: "The Pacemaker cluster resource manager" <pacemaker at oss.clusterlabs.org>
> Sent: Wednesday, June 19, 2013 4:47:58 PM
> Subject: Re: [Pacemaker] Pacemaker remote nodes, naming, and attributes
> 
> ----- Original Message -----
> > From: "Lindsay Todd" <rltodd.ml1 at gmail.com>
> > To: "The Pacemaker cluster resource manager"
> > <Pacemaker at oss.clusterlabs.org>
> > Sent: Wednesday, June 19, 2013 4:11:58 PM
> > Subject: [Pacemaker] Pacemaker remote nodes, naming, and attributes
> > 
> > I built a set of rpms for pacemaker 1.1.0-rc4 and updated my test cluster
> > (hopefully won't be a "test" cluster forever), as well as my VMs running
> > pacemaker-remote. The OS everywhere is Scientific Linux 6.4. I am wanting
> > to
> > set some attributes on remote nodes, which I can use to control where
> > services run.
> > 
> > The first deviation I note from the documentation is the naming of the
> > remote
> > nodes. I see:
> > 
> > 
> > 
> > 
> > Last updated: Wed Jun 19 16:50:39 2013
> > Last change: Wed Jun 19 16:19:53 2013 via cibadmin on cvmh04
> > Stack: cman
> > Current DC: cvmh02 - partition with quorum
> > Version: 1.1.10rc4-1.el6.ccni-d19719c
> > 8 Nodes configured, unknown expected votes
> > 49 Resources configured.
> > 
> > 
> > Online: [ cvmh01 cvmh02 cvmh03 cvmh04 db02:vm-db02 ldap01:vm-ldap01
> > ldap02:vm-ldap02 swbuildsl6:vm-swbuildsl6 ]
> > 
> > Full list of resources:
> > 
> > and so forth. The "remote-node" names are simply the hostname, so the
> > vm-db02
> > VirtualDomain resource has a remote-node name of db02. The "Pacemaker
> > Remote" manual suggests this should be displayed as "db02", not
> > "db02:vm-db02", although I can see how the latter format would be useful.
> 
> Yep, this got changed since the documentation was published.  We wanted
> people to be able to recognize which remote-node went with which resource
> easily.
> 
> > 
> > So now let's set an attribute on this remote node. What name do I use? How
> > about:
> > 
> > 
> > 
> > 
> > # crm_attribute --node "db02:vm-db02" \
> > --name "service_postgresql" \
> > --update "true"
> > Could not map name=db02:vm-db02 to a UUID
> > Please choose from one of the matches above and suppy the 'id' with
> > --attr-id
> > 
> > Perhaps not the most informative output, but obviously it fails. Let's try
> > the unqualified name:
> > 
> > 
> > 
> > 
> > # crm_attribute --node "db02" \
> > --name "service_postgresql" \
> > --update "true"
> > Remote-nodes do not maintain permanent attributes,
> > 'service_postgresql=true'
> > will be removed after db02 reboots.
> > Error setting service_postgresql=true (section=status, set=status-db02): No
> > such device or address
> > Error performing operation: No such device or address

I just tested this and ran into the same errors you did.  Turns out this happens when the remote-node's status section is empty.  If you start a resource on the node and then set the attribute it will work... obviously this is a bug. I'm working on a fix.

> > So a little more informative, but still it fails. It probably isn't a
> > surprise that using "crm node" doesn't work too well either (with the
> > unqualified name, it creates a "db02" node marked as unclean).
> > 
> > Well, that bit about attributes on remote nodes isn't too surprising,
> > although I wasn't sure until I tried it. Once I have an invocation that
> > works, I'm thinking maybe a resource agent could help by looking for a
> > local
> > file of node attributes and making the appropriate settings. Or maybe there
> > is another way to do this? Meanwhile I need a command to set attributes
> > that
> > works!
> 
> From what I remember, transient attributes work but permanent ones don't.
> I'll verify this and explain why that's the case in another later.

yep, I can confirm that remote-nodes only allow transient node attributes.

-- Vossel

> > 
> > Since I haven't gotten far enough yet to see for myself, I've also wondered
> > a
> > few things:
> > 
> > 
> >     * How do remote nodes impact the size of the largest cluster that can
> >     be
> >     managed? I can see having many VMs with services on them. (I also have
> >     VMs that are not remote nodes.)
> 
> VM remote-nodes scale much differently than cluster nodes running corosync.
> The expectation is that VM remote-nodes should have much less impact on the
> messaging layer than adding an actual cluster node.
> 
> >     * Do remote nodes affect quorum calculations at all?
> 
> no
> 
> > 
> > Thanks for the help.
> 
> thanks for the feedback :)
> 
> -- Vossel
> 
> > /Lindsay
> > 
> > _______________________________________________
> > Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
> > http://oss.clusterlabs.org/mailman/listinfo/pacemaker
> > 
> > Project Home: http://www.clusterlabs.org
> > Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> > Bugs: http://bugs.clusterlabs.org
> > 
>