[Pacemaker] Signal hangup handling for pacemaker and corosync

Thu Jul 24 04:46:26 UTC 2014

24.07.2014 03:39, Andrew Beekhof wrote:
> 
> On 23 Jul 2014, at 2:46 pm, Vladislav Bogdanov <bubble at hoster-ok.com> wrote:
> 
>> 23.07.2014 05:56, Andrew Beekhof wrote:
>>>
>>> On 21 Jul 2014, at 3:45 pm, Vladislav Bogdanov <bubble at hoster-ok.com> wrote:
>>>
>>>> 21.07.2014 08:36, Andrew Beekhof wrote:
>>>>>
>>>>> On 21 Jul 2014, at 2:50 pm, Vladislav Bogdanov <bubble at hoster-ok.com> wrote:
>>>>>
>>>>>> 21.07.2014 06:28, Andrew Beekhof wrote:
>>>>>>>
>>>>>>> On 15 Jul 2014, at 8:45 pm, Arjun Pandey <apandepublic at gmail.com> wrote:
>>>>>>>
>>>>>>>> On Tue, Jul 15, 2014 at 3:36 PM, Andrew Beekhof <andrew at beekhof.net> wrote:
>>>>>>>>>
>>>>>>>>> On 15 Jul 2014, at 8:00 pm, Arjun Pandey <apandepublic at gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> Right. Actually the issue i am facing is that i am starting the
>>>>>>>>>> pacemaker service remotely from a wrapper and thus pacemakerd dies
>>>>>>>>>> when the wrapper exits.nohup solves the problem but then HUP cannot be
>>>>>>>>>> used by pacemaker. Is this workaround ok ?
>>>>>>>>>
>>>>>>>>> I guess. How are you starting pacemaker?  Usually its with some variant of 'service pacemaker start'.
>>>>>>>> I am using 'service pacemaker start'. However this is being called
>>>>>>>> from my script. So when the script exits pacemaker gets SIGHUP.
>>>>>>>
>>>>>>> Release testing starts clusters as:
>>>>>>>
>>>>>>> ssh -l root somenode -- service pacemaker start
>>>>>>
>>>>>> It could depend on what "service" is.
>>>>>> It would either schedule systemd to run job (el7/fc18+), or just run
>>>>>> init script itself (el6). In latter case, if process didn't detach from
>>>>>> its controlling terminal when that terminal gone away, it will be sent a
>>>>>> SIGHUP.
>>>>>
>>>>> Except we test rhel6 the same way...
>>>>
>>>> I understand. This issue is from "sometimes happens on some systems"
>>>> folder. I recall I had problems ages ago with a daemon run from rc.local
>>>> sometimes exists with HUP. 'sleep 1' after its launch was the easiest fix.
>>>
>>> How about: https://github.com/beekhof/pacemaker/commit/95175f5
>>
>> That doesn't hurt, but could be just not enough, as pacemakerd does not
>> daemonize itself, but is put into background by shell means. Thus, when
>> you add signal handler, pacemakerd already runs some time in the
>> background. If terminal (ssh session) disconnects before signal handler
>> is installed, then process exits anyways.
> 
> Just moving it earlier would seem the simplest option

Yes, but it still remains racy. Strictly speaking, handler should be
installed in the child before the parent process exits. And you cannot
control this when shell does the fork.

> 
>> I'd suggest to add '-d' option and daemonize (double-fork or fork+setsid
>> plus common daemonization cleanups) if it is set after signal handlers
>> are installed but before main loop is run.
>> Also, SIGTTIN and SIGTTOU could be added to ignore list for a daemon mode.
> 
> The meaning of those two aren't making sense to my brain today.
> 
>>
>>>
>>>>
>>>>>
>>>>>> I'd recommend adding HUP handler (f.e. ignore) or/and detach
>>>>>> (setsid()) right before daemonizing.
>>>>>>
>>>>>>>
>>>>>>> And I've never seen the behaviour you speak of.
>>>>>>> How is what you're doing different?
>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> I was checking out the current pacemaker code.setsid is called for
>>>>>>>>>> each child process.However if we do this for main process to then it
>>>>>>>>>> will also be detached from the terminal.
>>>>>>>>>>
>>>>>>>>>> Regards
>>>>>>>>>> Arjun
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Tue, Jul 15, 2014 at 3:01 PM, Andrew Beekhof <andrew at beekhof.net> wrote:
>>>>>>>>>>>
>>>>>>>>>>> On 15 Jul 2014, at 7:13 pm, Arjun Pandey <apandepublic at gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi Andrew
>>>>>>>>>>>>
>>>>>>>>>>>> AFAIK linux daemons don't terminate on SIGHUP.
>>>>>>>>>>>
>>>>>>>>>>> Read the man page, POSIX specifies that the default action is 'term' ie. 'terminate'.
>>>>>>>>>>>
>>>>>>>>>>>> They typically reload
>>>>>>>>>>>> configuration on receiving this signal.Eg- rsyslogd. I thought it was
>>>>>>>>>>>> safe to make this assumption here as well.
>>>>>>>>>>>
>>>>>>>>>>> Not anywhere as it turns out
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Regards
>>>>>>>>>>>> Arjun
>>>>>>>>>>>>
>>>>>>>>>>>> On Tue, Jul 15, 2014 at 2:15 PM, Andrew Beekhof <andrew at beekhof.net> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> On 15 Jul 2014, at 6:19 pm, Arjun Pandey <apandepublic at gmail.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi all
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I am running pacemaker version 1.1.10-14.el6 on CentOS 6. On setting
>>>>>>>>>>>>>> up cluster if I send SIGHUP to either pacemaker or corosync services ,
>>>>>>>>>>>>>> they die.
>>>>>>>>>>>>>> Is this a bug ? What is the intension behind this behavior?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Standard default I believe.
>>>>>>>>>>>>> Have you run 'man 7 signal' lately?
>>>>>>>>>>>>>
>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>> Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
>>>>>>>>>>>>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>>>>>>>>>>>>
>>>>>>>>>>>>> Project Home: http://www.clusterlabs.org
>>>>>>>>>>>>> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>>>>>>>>>>>>> Bugs: http://bugs.clusterlabs.org
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>> Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
>>>>>>>>>>>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>>>>>>>>>>>
>>>>>>>>>>>> Project Home: http://www.clusterlabs.org
>>>>>>>>>>>> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>>>>>>>>>>>> Bugs: http://bugs.clusterlabs.org
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
>>>>>>>>>>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>>>>>>>>>>
>>>>>>>>>>> Project Home: http://www.clusterlabs.org
>>>>>>>>>>> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>>>>>>>>>>> Bugs: http://bugs.clusterlabs.org
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
>>>>>>>>>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>>>>>>>>>
>>>>>>>>>> Project Home: http://www.clusterlabs.org
>>>>>>>>>> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>>>>>>>>>> Bugs: http://bugs.clusterlabs.org
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
>>>>>>>>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>>>>>>>>
>>>>>>>>> Project Home: http://www.clusterlabs.org
>>>>>>>>> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>>>>>>>>> Bugs: http://bugs.clusterlabs.org
>>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
>>>>>>>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>>>>>>>
>>>>>>>> Project Home: http://www.clusterlabs.org
>>>>>>>> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>>>>>>>> Bugs: http://bugs.clusterlabs.org
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
>>>>>>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>>>>>>
>>>>>>> Project Home: http://www.clusterlabs.org
>>>>>>> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>>>>>>> Bugs: http://bugs.clusterlabs.org
>>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
>>>>>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>>>>>
>>>>>> Project Home: http://www.clusterlabs.org
>>>>>> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>>>>>> Bugs: http://bugs.clusterlabs.org
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
>>>>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>>>>
>>>>> Project Home: http://www.clusterlabs.org
>>>>> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>>>>> Bugs: http://bugs.clusterlabs.org
>>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
>>>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>>>
>>>> Project Home: http://www.clusterlabs.org
>>>> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>>>> Bugs: http://bugs.clusterlabs.org
>>>
>>>
>>>
>>> _______________________________________________
>>> Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
>>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>>
>>> Project Home: http://www.clusterlabs.org
>>> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>>> Bugs: http://bugs.clusterlabs.org
>>>
>>
>>
>> _______________________________________________
>> Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>
>> Project Home: http://www.clusterlabs.org
>> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>> Bugs: http://bugs.clusterlabs.org
> 
> 
> 
> _______________________________________________
> Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
> 
> Project Home: http://www.clusterlabs.org
> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> Bugs: http://bugs.clusterlabs.org
> 