[Pacemaker] g_hash_table_lookup: assertion `hash_table != NULL' failed

[Vladislav Bogdanov]

Hi Andrew, hi all,

Upgraded to todays tip and see bunch of assertion messages in logs
(together with some segfaults from pengine).

Here is what I see:

Sep 13 16:53:43 s01-1 pengine: [4120]: ERROR: crm_abort: xpath_search:
Triggered assert at xml.c:2599 : xml_top != NULL
Sep 13 16:56:14 s01-1 pengine: [7075]: ERROR: crm_abort: xpath_search:
Triggered assert at xml.c:2599 : xml_top != NULL
Sep 13 17:00:20 s01-1 pengine: [9281]: ERROR: crm_abort: xpath_search:
Triggered assert at xml.c:2599 : xml_top != NULL
Sep 13 17:01:23 s01-1 crmd: [14368]: ERROR: crm_abort: crm_glib_handler:
Forked child 14536 to record non-fatal assert at utils.c:452 :
g_hash_table_lookup: assertion `hash_table != NULL' failed
Sep 13 17:01:23 s01-1 crmd: [14368]: ERROR: crm_abort: crm_glib_handler:
Forked child 14537 to record non-fatal assert at utils.c:452 :
g_hash_table_lookup: assertion `hash_table != NULL' failed
Sep 13 17:01:23 s01-1 crmd: [14368]: ERROR: crm_abort: crm_glib_handler:
Forked child 14538 to record non-fatal assert at utils.c:452 :
g_hash_table_lookup: assertion `hash_table != NULL' failed
Sep 13 17:01:23 s01-1 crmd: [14368]: ERROR: crm_abort: crm_glib_handler:
Forked child 14540 to record non-fatal assert at utils.c:452 :
g_hash_table_destroy: assertion `hash_table != NULL' failed
Sep 13 17:01:23 s01-1 crmd: [14368]: ERROR: crm_abort: crm_glib_handler:
Forked child 14546 to record non-fatal assert at utils.c:452 :
g_hash_table_lookup: assertion `hash_table != NULL' failed
Sep 13 17:01:23 s01-1 crmd: [14368]: ERROR: crm_abort: crm_glib_handler:
Forked child 14565 to record non-fatal assert at utils.c:452 :
g_hash_table_lookup: assertion `hash_table != NULL' failed
Sep 13 17:01:23 s01-1 crmd: [14368]: ERROR: crm_abort: crm_glib_handler:
Forked child 14581 to record non-fatal assert at utils.c:452 :
g_hash_table_lookup: assertion `hash_table != NULL' failed
Sep 13 17:01:23 s01-1 crmd: [14368]: ERROR: crm_abort: crm_glib_handler:
Forked child 14584 to record non-fatal assert at utils.c:452 :
g_hash_table_destroy: assertion `hash_table != NULL' failed
Sep 13 17:01:24 s01-1 crmd: [14368]: ERROR: crm_abort: crm_glib_handler:
Forked child 14731 to record non-fatal assert at utils.c:452 :
g_hash_table_lookup: assertion `hash_table != NULL' failed
Sep 13 17:01:24 s01-1 crmd: [14368]: ERROR: crm_abort: crm_glib_handler:
Forked child 14732 to record non-fatal assert at utils.c:452 :
g_hash_table_lookup: assertion `hash_table != NULL' failed
Sep 13 17:01:24 s01-1 crmd: [14368]: ERROR: crm_abort: crm_glib_handler:
Forked child 14733 to record non-fatal assert at utils.c:452 :
g_hash_table_lookup: assertion `hash_table != NULL' failed
Sep 13 17:01:24 s01-1 crmd: [14368]: ERROR: crm_abort: crm_glib_handler:
Forked child 14735 to record non-fatal assert at utils.c:452 :
g_hash_table_destroy: assertion `hash_table != NULL' failed
Sep 13 17:01:24 s01-1 crmd: [14368]: ERROR: crm_abort: crm_glib_handler:
Forked child 14771 to record non-fatal assert at utils.c:452 :
g_hash_table_lookup: assertion `hash_table != NULL' failed
Sep 13 17:01:24 s01-1 crmd: [14368]: ERROR: crm_abort: crm_glib_handler:
Forked child 14772 to record non-fatal assert at utils.c:452 :
g_hash_table_lookup: assertion `hash_table != NULL' failed
Sep 13 17:01:24 s01-1 crmd: [14368]: ERROR: crm_abort: crm_glib_handler:
Forked child 14773 to record non-fatal assert at utils.c:452 :
g_hash_table_lookup: assertion `hash_table != NULL' failed
Sep 13 17:01:24 s01-1 crmd: [14368]: ERROR: crm_abort: crm_glib_handler:
Forked child 14775 to record non-fatal assert at utils.c:452 :
g_hash_table_destroy: assertion `hash_table != NULL' failed
Sep 13 17:22:19 s01-1 pengine: [14367]: ERROR: crm_abort: xpath_search:
Triggered assert at xml.c:2599 : xml_top != NULL
Sep 13 17:23:22 s01-1 crmd: [1133]: ERROR: crm_abort: crm_glib_handler:
Forked child 1532 to record non-fatal assert at utils.c:452 :
g_hash_table_lookup: assertion `hash_table != NULL' failed
Sep 13 17:23:22 s01-1 crmd: [1133]: ERROR: crm_abort: crm_glib_handler:
Forked child 1545 to record non-fatal assert at utils.c:452 :
g_hash_table_lookup: assertion `hash_table != NULL' failed
Sep 13 17:23:22 s01-1 crmd: [1133]: ERROR: crm_abort: crm_glib_handler:
Forked child 1560 to record non-fatal assert at utils.c:452 :
g_hash_table_lookup: assertion `hash_table != NULL' failed
Sep 13 17:23:22 s01-1 crmd: [1133]: ERROR: crm_abort: crm_glib_handler:
Forked child 1565 to record non-fatal assert at utils.c:452 :
g_hash_table_destroy: assertion `hash_table != NULL' failed

and
pengine[14367]: segfault at 8 ip 00007f6f310abf89 sp 00007fff5879f1d0
error 4 in libpengine.so.3.0.0[7f6f31099000+5a000]
pengine[1132]: segfault at 8 ip 00007fef1f6a7f89 sp 00007fff0ab135e0
error 4 in libpengine.so.3.0.0[7fef1f695000+5a000]

Ah, these are connected to xml_top assert.

This happens for both nodes and didn't happen before, so perhaps it
connected to latest commits.

Should I provide some more info?

Best,
Vladislav


Should I provide s