[Pacemaker] Disconnected from CIB?

[Andrew Beekhof]

On 30/06/2013, at 10:09 PM, Lars Marowsky-Bree <lmb at suse.com> wrote:

> Hi,
> 
> sbd connects to the CIB and watches updates come in to see if pacemaker
> considers the node healthy still, and if the cluster partition is
> quorate according to the CIB. That's all working fine.
> 
> But I've noticed that during start-up of a regular cluster, sbd would
> get disconnected briefly:
> 
> Jun 30 14:02:57 hex-1 sbd: [22662]: info: Node state: online
> Jun 30 14:02:57 hex-1 sbd: [22654]: info: Pacemaker health check: OK
> Jun 30 14:02:57 hex-1 pengine[22687]:   notice: unpack_config: On loss of CCM Quorum: Ignore
> [... pengine, probing, initial resource start-up etc go here ]
> Jun 30 14:03:34 hex-1 cib[22683]:   notice: crm_ipc_prepare: Message exceeds the configured ipc limit (51200 bytes), consider configuring PCMK_ipc_buffer to 106180 or higher to avoid compression overheads
> [...]
> Jun 30 14:03:34 hex-1 crmd[22688]:   notice: process_lrm_event: LRM operation drbd-r0_monitor_31000 (call=142, rc=8, cib-update=1
> 61, confirmed=false) master
> Jun 30 14:03:34 hex-1 crmd[22688]:   notice: process_lrm_event: hex-1-drbd-r0_monitor_31000:142 [ \n ]
> Jun 30 14:03:34 hex-1 sbd: [22662]: WARN: Disconnected from CIB
> [...]
> Jun 30 14:03:36 hex-1 crmd[22688]:   notice: run_graph: Transition 1 (Complete=37, Pending=0, Fired=0, Skipped=20, Incomplete=2, 
> Source=/var/lib/pacemaker/pengine/pe-input-3829.bz2): Stopped
> 
> The disconnect handler is installed via cib->cmds->set_connection_dnotify(cib,
> mon_cib_connection_destroy); sbd is not self-disconnecting here.
> 
> So I wonder if this is a normal occurrence? It doesn't seem to be related to
> the IPC limit; those messages occur at other times and nothing happens. What
> could cause the server side to disconnect from a CIB client?
> 
> Anything I should worry about?

I would say so, because I can't think of a valid reason for it to happen.
You'll probably want to use the blackbox to diagnose this.

Reproducible or random?