[Pacemaker] WARN: G_SIG_dispatch messages

[Jean-Francois Malouin]

* Dejan Muhamedagic <dejanmm at fastmail.fm> [20110224 05:31]:
> On Wed, Feb 23, 2011 at 10:53:23AM -0500, Jean-Francois Malouin wrote:
> > * Dejan Muhamedagic <dejanmm at fastmail.fm> [20110223 09:21]:
> > > Hi,
> > > 
> > > On Mon, Feb 21, 2011 at 01:22:38PM -0500, Jean-Francois Malouin wrote:
> > > > Hi,
> > > > 
> > > > On a cluster that is about to go live I see theses warning popping up
> > > > quite frequently:
> > > > 
> > > > lrmd: [6487]: WARN: G_SIG_dispatch: Dispatch function for SIGCHLD was delayed 240 ms (> 100 ms) before
> > > > +being called (GSource: 0x1542fc0)
> > > > 
> > > > That's on Debian/Squeeze, pacemaker-1.0.9 corosync-1.2.1 and openais-1.1.2
> > > > 
> > > > What do they mean and can I just Forget About It (tm)?
> > > 
> > > Normally, these should indicate that the host can't keep up with
> > > the demand. Did you check the load?
> > 
> > there are not doing much right now, 
> > in terms of load nothing to speak about...
> 
> So, it happens really often? How often? How many resources are
> there? Does it happen on all nodes? You can also open a bugzilla
> with hb_report attached.

It happens only a few times (less than 5) per day. Yet not everyday so it's
very sporadic. I'm just worried that something is lurking in the dark.

It's a 2 nodes cluster running 4 Xen guests:

~# crm_mon -1 -f -n
============
Last updated: Thu Feb 24 10:26:50 2011
Stack: openais
Current DC: helena - partition with quorum
Version: 1.0.9-74392a28b7f31d7ddc86689598bd23114f58978b
2 Nodes configured, 2 expected votes
15 Resources configured.
============

Node puck: online
    resStonitHelena (stonith:external/ipmi) Started 
    resOCFSr1:0 (ocf::heartbeat:Filesystem) Started 
    resPing:0   (ocf::pacemaker:ping) Started 
    resDRBDr1:0 (ocf::linbit:drbd) Master 
    resDRBDr2:0 (ocf::linbit:drbd) Master 
    resOCFSr0:0 (ocf::heartbeat:Filesystem) Started 
    resDRBDr0:0 (ocf::linbit:drbd) Master 
    resDRBDr3:0 (ocf::linbit:drbd) Master 
    resOCFSr2:1 (ocf::heartbeat:Filesystem) Started 
    resOCFSr3:1 (ocf::heartbeat:Filesystem) Started 
Node helena: online
    resDRBDr1:1 (ocf::linbit:drbd) Master 
    resDRBDr2:1 (ocf::linbit:drbd) Master 
    resStonithPuck  (stonith:external/ipmi) Started 
    resXen1 (ocf::heartbeat:Xen) Started 
    resXen2 (ocf::heartbeat:Xen) Started 
    resXen0 (ocf::heartbeat:Xen) Started 
    resDRBDr0:1 (ocf::linbit:drbd) Master 
    resXen3 (ocf::heartbeat:Xen) Started 
    resOCFSr2:0 (ocf::heartbeat:Filesystem) Started 
    resOCFSr0:1 (ocf::heartbeat:Filesystem) Started 
    resOCFSr1:1 (ocf::heartbeat:Filesystem) Started 
    resDRBDr3:1 (ocf::linbit:drbd) Master 
    resOCFSr3:0 (ocf::heartbeat:Filesystem) Started 
    resPing:1   (ocf::pacemaker:ping) Started 

Migration summary:
* Node helena:  pingd=100
* Node puck:  pingd=100

I'll see about submitting a hb_report.
Is there a way to anonymize the report btw?

jf

> 
> Thanks,
> 
> Dejan
> 
> > jf
> > 
> > > 
> > > > A quick google search found a reference to a bug but that's really old
> > > > stuff: http://developerbugs.linuxfoundation.org/show_bug.cgi?id=1684
> > > 
> > > Only for meta-data operations, that should be unrelated.
> > > 
> > > Thanks,
> > > 
> > > Dejan
> > > 
> > > > thanks!
> > > > jf
> > > > 
> > > > _______________________________________________
> > > > Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
> > > > http://oss.clusterlabs.org/mailman/listinfo/pacemaker
> > > > 
> > > > Project Home: http://www.clusterlabs.org
> > > > Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> > > > Bugs: http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker
> > > 
> > > _______________________________________________
> > > Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
> > > http://oss.clusterlabs.org/mailman/listinfo/pacemaker
> > > 
> > > Project Home: http://www.clusterlabs.org
> > > Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> > > Bugs: http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker
> > 
> > 
> > _______________________________________________
> > Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
> > http://oss.clusterlabs.org/mailman/listinfo/pacemaker
> > 
> > Project Home: http://www.clusterlabs.org
> > Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> > Bugs: http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker
> 
> _______________________________________________
> Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
> 
> Project Home: http://www.clusterlabs.org
> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> Bugs: http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker