[Pacemaker] How to deal with unix signals in a glib mainloop (was: [Problem] The attrd does not sometimes stop.)

[Lars Ellenberg]

On Tue, Jan 17, 2012 at 12:13:37AM +0100, Lars Ellenberg wrote:
> On Tue, Jan 17, 2012 at 09:52:35AM +1100, Andrew Beekhof wrote:
> > 
> > Ok, done:
> > 
> > https://github.com/beekhof/pacemaker/commit/2a6b296
> > 
> > If I'm adding voodoo, I at least want the reason well documented so it
> > can be removed again if the reason goes away.
> 
> That about sums it up, then ;-)

But as having to do this was just "too ugly to be true",
I dug a little deeper...

The way to do this is obviously to use the glib api ;-)
http://developer.gnome.org/glib/2.30/glib-UNIX-specific-utilities-and-integration.html#g-unix-signal-add-full

(Since glib 2.30, yay; if you don't have that yet, read on anyways)

What it does internally, and what other people have also done for a long
time  to solve this and similar problems, is:

Add to the main context a "wakeup pipe",
which is an eventfd if available,
or an actual pipe if not.
If it is a pipe, set those file descriptors non-blocking.
And, of course, add the evenfd (or the read end of the pipe)
to the poll loop (with default priority, btw,
which is good enough to have the poll terminate).

That is done internally when creating the main context.
http://git.gnome.org/browse/glib/tree/glib/gmain.c#n548
http://git.gnome.org/browse/glib/tree/glib/gwakeup.c#n138

(the line numbers are correct for glib master as of today,
 which should correspond to 41fbf42)

The g_unix_signal_handler then sets the triggers variables,
and calls g_wakeup_signal(that internal wakeup source),
which simply posts and event to the eventfd,
or does a short (1 byte) write to the write end of the pipe.
http://git.gnome.org/browse/glib/tree/glib/gmain.c#n4442
http://git.gnome.org/browse/glib/tree/glib/gwakeup.c#n230

Problem solved, without having to do a full check() everything,
prepare() everything, and poll() again cycle every 500ms.

"back in those days", when this mechanism was not really there,
you could do all that "by hand".
And people did. Very common idiom in glib and other mainloop
applications, also frequently used to "signal" availability of work
or completion of tasks between threads.

static int my_wakeup_fds[2] = { -1, -1 };

Then just pipe2(my_wakeup_fds, O_NONBLOCK), add my_wakeup_fds[0] as
normal read fd source, and add a write(my_wakeup_fds[1], "", 1); to the
signal handlers.

I may even write the patch.
But not before May.

And I did not mention the year ;-)

Cheers,

-- 
: Lars Ellenberg
: LINBIT | Your Way to High Availability
: DRBD/HA support and consulting http://www.linbit.com