[Pacemaker] CMAN integration questions

[Vladislav Bogdanov]

23.12.2010 14:14, Andrew Beekhof wrote:
> On Thu, Dec 23, 2010 at 10:41 AM, Vladislav Bogdanov
> <bubble at hoster-ok.com> wrote:
>> Hi Andrew,
>>
>> It was a big surprise for me to see all pacemaker-specific bits removed
>> from dlm and gfs2 in cluster-3.1.0, so there is currently no way to use
>> pacemaker on f13 with dlm/gfs2/clvm but without cman.
>>
>> So, would you please bring some light on details of integration with cman?
>>
>> Especially I need to understand how pacemaker integrates with cman's
>> fencing/dlm subsystem:
>> *) Do I need to configure fencing in both cman and pacemaker?
> 
> No.  Just in Pacemaker.
> fenced spins waiting for Pacemaker to make an API call that tells it
> that fencing completed, at which point the dlm can continue.
> 
> David (author of fenced and the dlm) and I discussed this at length
> and we are in agreement that is the right (and safe) intermediate
> step.

Aha, fencing works this way too, thanks. I thought that cman should be a
fencing master.

Although I caught one problem when node were fenced (actually started
after manual forced poweroff) but fenced daemon still thought that it is
in a 'fencing' state. pacemeker did its job (tengine_stonith_notify:
Notifing CMAN that 'v02-b' is now fenced) but cman (actually master
fenced instance) never got it. And this seems to be related to bugreport
I referred to - it seems like all nodes left fenced:default group just
before that. Oh, all that dlm-related things are so unstable... :(
fenced tried to fence node itself, but didn't succeed just because it
doesn't have any fence agents configured.

...

>> And one more question/proposal about CMAN/DLM/GFS2:
>> now it is possible to use DLM/GFS2 on nodes without pacemaker installed.
>> I mean, if I configure additional node in cman but have no pacemaker
>> started on that node, then I'm still able to mount GFS2 on that node.
>> One minor problem is that rest of pacemaker cluster waits for that node
>> to start pacemaker too.
> 
> Did you try this?

Only cman membership yet. Trying to make it once more, but without
success yet... Will try more a little bit later, after solving that
problem with fencing domain.
>From what I saw, pacemaker (correctly) shows that node as 'pending'.
And allocates clone instances for it. I'd want to get rid of that instances.

> Its no different to running just corosync on a node.
Yes, but cman plugins are now mandatory for dlm/gfs.

BTW is supposed that pacemaker should not care about dlm_controld and
friends anymore? Or it will receive enough information from cman about
whole DLM subsystem?

> 
>> So all clone resources are extended with one
>> more instance which "will never be started".  On the other hand I see in
>> pacemaker sources, that there are two types of nodes: member and ping,
>> and all resource processing is done only for nodes which are members.
>> Would it be too hard to add one more node type, f.e. "arbiter" (it
>> participates in cman cluster so it influences quorum/fencing), which is
>> only valid for CMAN clusters and is not supposed to run any resources?
>> Then clones will not try to extend on that arbiter nodes, fewer
>> resources, less computations, cleaner 'crm status' output.
>>
>> Could you please comment on this?
> 
> Before trying thought experiments, its best to get the starting point
> correct :-)

Ahm, what exactly do you mean? :)

> 
>>
>> Thanks,
>> Vladislav
>>
>> _______________________________________________
>> Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>
>> Project Home: http://www.clusterlabs.org
>> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>> Bugs: http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker
>>
> 
> _______________________________________________
> Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
> 
> Project Home: http://www.clusterlabs.org
> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> Bugs: http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker