[Pacemaker] node1 fencing itself after node2 being fenced

[Vladislav Bogdanov]

Hi Fabio,

19.02.2014 12:32, Fabio M. Di Nitto wrote:
> On 2/19/2014 9:39 AM, Fabio M. Di Nitto wrote:
>> On 2/18/2014 9:24 PM, Asgaroth wrote:
>>>>
>>>> Just a guess. Do you have startup fencing enabled in dlm-controld (I actually
>>>> do not remember if it is applicable to cman's version, but it exists in dlm-4) or
>>>> cman?
>>>> If yes, then that may play its evil game, because imho it is not intended to
>>>> use with pacemaker which has its own startup fencing policy (if you redirect
>>>> fencing to pacemaker).
>>>>
>>>
>>> I can't seem to find the option to enable/disable startup fencing in either dlm_controld or cman.
>>
>> 3 things:
>>
>> 1) add <logging debug="on"/> to cluster.conf
>> 2) reproduce the issue
>> 3) collect sosreports and/or crm_reports and send them over.
>>    if security/privacy is a concern, send them privately to me and
>>    Andrew.
>>
>> The thread has gone for a while with many different suggestions, but
>> there is a lot of confusion around rhel6/7, agents/init and so on.
>>
>> Just to remove some of the misconceptions:
>>
>> clvmd in RHEL6 is only supported via init script.
>> dlm config should not be changed to avoid fencing at startup.
>>
>> There are other timers involved on clvmd startup that are directly
>> related to the storage. We might be hitting those ones. No logs, no help.
>>
>> At this point, in 6, pacemaker is not involved if not for fencing
>> operations.
> 
> Just to clarify this last sentence:
> 
> "At this stage of the boot process, in 6, pacemaker is not involved
> directly. If node3 is attempting to fence another node, then it would
> block on pacemaker, since fencing is proxy´d to pcmk. But node3 should
> have no reason to fence, only showing some underlying issues."

Doesn't it check on startup (or on lockspace creation) that fencing is
available?

And, what happens with that when the whole cluster boots up?
If I understand correctly, dlm should block all operations, including
requests from clvmd and thus all lvm tools, until pacemaker is started
locally. But clvmd init script runs {vg,lv}display after clvmd is
started, and it should block, introducing the deadlock. Am I missing
something?

> 
> Fabio
> 
>>
>> In RHEL7:
>>
>> clvmd/dlm will be/are managed _only_ by pacemaker as resources. This
>> integration is only 99.9% done. We recently discovered 3 corner cases
>> that needs fixing via the agents that David V. is pushing around (after
>> all 7 is not released yet ;))

Fabio, could you please list that cases, as it is really valuable for
some people to know them? And for some reason I'm not able to locate
that patch David talks about, probably it is still not pushed to github.


Best,
Vladislav

>>
>> Cheers
>> Fabio
>>
>>>
>>> "dlm_controld -h" doesn’t list an option to enable/disable start up fencing.
>>> I had a quick read of the cman man page and I also don’t see any option mentioning startup fencing.
>>>
>>> Would you mind pointing me in the direction of the parameter to disable this in cman/dlm_controld please.
>>>
>>> PS: I am redirecting all fencing operations to pacemaker using the following directive:
>>> 	<method name="pcmk-redirect">
>>
>>
>>
>>>
>>> Thanks
>>>
>>>
>>> _______________________________________________
>>> Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
>>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>>
>>> Project Home: http://www.clusterlabs.org
>>> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>>> Bugs: http://bugs.clusterlabs.org
>>>
>>
>>
>> _______________________________________________
>> Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>
>> Project Home: http://www.clusterlabs.org
>> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>> Bugs: http://bugs.clusterlabs.org
>>
> 
> 
> _______________________________________________
> Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
> 
> Project Home: http://www.clusterlabs.org
> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> Bugs: http://bugs.clusterlabs.org
>