[Pacemaker] Node removal in corosync-based cluster

[Vladislav Bogdanov]

31.05.2011 12:32, Dejan Muhamedagic wrote:
> On Mon, May 30, 2011 at 05:27:41PM +0300, Vladislav Bogdanov wrote:
>> Hi,
>>
>> 30.05.2011 17:12, Dejan Muhamedagic wrote:
>>> Hi,
>>>
>>> On Sun, May 29, 2011 at 11:58:17PM +0300, Vladislav Bogdanov wrote:
>>>> Hi all.
>>>>
>>>> I've got a task to remove some nodes from cluster to save some power and
>>>> found that it is not sufficient to follow
>>>> http://www.clusterlabs.org/doc/en-US/Pacemaker/1.1/html/Pacemaker_Explained/s-node-delete.html#s-del-ais
>>>
>>> Did you try just "crm node delete ..."? That should do whatever's
>>> necessary to remove a node (on which pacemaker be down).
>>
>> What extra commands does it run comparing to Pacemaker Explained?
> 
> I must say that I don't know, easy to check in the code, but it
> should do whatever's necessary. If it doesn't then we need to fix
> it. The idea is that users don't need to know about cibadmin,
> crm_node, and such stuff.

The problem is not how to delete node, this works with cibadmin, and
should work with crm shell too. Node is deleted and everything is fine
until pacemaker restart on any remaining node. But after such restart
node re-appears in cib. That should be some artifact in pacemaker (e.g.
not all node-specific data is deleted from some list and that list is
then consulted for a list of nodes). qourum-votes are not touched btw,
they show correct number. Hm. Not sure if clone instances are allocated.
Probably not.
Problem is quite minor, I wrote that just to note that it exists.

Best,
Vladislav

> 
> Thanks,
> 
> Dejan
> 
>>> I can vaguely recall this case, but cannot remember anymore what
>>> was the outcome.
>>>
>>> Thanks,
>>>
>>> Dejan
>>>
>>>> After pacemaker is then restarted on any another remaining node, removed
>>>> node comes back in a CIB (with OFFLINE status). Even removal of that
>>>> node from corosync's objdb with
>>>> corosync-objctl -d runtime.totem.pg.mrp.srp.members.COROSYNC_ID
>>>> does not help.
>>>>
>>>> The only way to completely remove node I found is to stop pacemaker on
>>>> all cluster nodes and start it again then. So there should be some
>>>> 'ghost' data in a running pacemaker instance which is not deleted after
>>>> removal of node data and status from a CIB.
>>>>
>>>> I hope this could be easily fixed, just a note that it does not fully
>>>> work now (1.1.5 with some patches from both 1.1 and devel).
>>>>
>>>> Best,
>>>> Vladislav
>>>>
>>>> _______________________________________________
>>>> Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
>>>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>>>
>>>> Project Home: http://www.clusterlabs.org
>>>> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>>>> Bugs: http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker
>>>
>>> _______________________________________________
>>> Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
>>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>>
>>> Project Home: http://www.clusterlabs.org
>>> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>>> Bugs: http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker
>>
>>
>> _______________________________________________
>> Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>
>> Project Home: http://www.clusterlabs.org
>> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>> Bugs: http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker
> 
> _______________________________________________
> Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
> 
> Project Home: http://www.clusterlabs.org
> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> Bugs: http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker