[Pacemaker] dc-version and cluster-infrastructure cluster options may be lost when editing

[Vladislav Bogdanov]

20.12.2013 14:15, Vladislav Bogdanov wrote:
> Hi all,
> 
> Just discovered that it is possible to remove options in subject with
> admin action (at least with cibadmin --patch).
> 
> This is notably annoying when updating CIB with 'crm configure load
> update' (which uses crm_diff to create patch) when updating other
> cluster options.
> 
> While most (all?) other options are reset to defaults when they are
> missing in the input, those two in subject are silently removed. That
> is probably a bug. IMO such "internal" options should be preserved
> (filtered?) from external updates.
> 
> I noticed this while debugging problem with one of my resource-agents
> which polls for cluster-infrastructure attribute during start - it
> refused to restart after cluster options update.
> 
> Would be nice to have this fixed in 1.1.11 :)
> 
> Relevant log lines are:
> Dec 20 10:54:21 booter-test-0 cib[3517]:   notice: cib:diff: --         <nvpair id="cib-bootstrap-options-dc-version" name="dc-version" value="1.1.10-3.3.el6-8db9f0b"/>
> Dec 20 10:54:21 booter-test-0 cib[3517]:   notice: cib:diff: --         <nvpair id="cib-bootstrap-options-cluster-infrastructure" name="cluster-infrastructure" value="corosync"/>
> 
> Versions are:
> pacemaker - ClusterLabs/pacemaker/master 0d1ac18 + beekhof/pacemaker/master a269ba8
> crmsh - 4f66cc190185

Actually, although I still shink that pacemaker should not allow
modification of that options, source of my problem is that crmsh doesn't
initialize <cluster_property_set/> node with original values when it is
doing load update.

Example (very long lines are stripped for readability):
# cat u.crm
property $id="cib-bootstrap-options" \
        batch-limit="30"
# crm -d configure load update u.crm
...
DEBUG: clitext: property $id="cib-bootstrap-options" batch-limit="40"
cluster-delay="60s" cluster-recheck-interval="10m"
crmd-transition-delay="0s" dc-deadtime="20s"
default-action-timeout="20s" default-resource-stickiness="100" e...
DEBUG: id_store: saved cib-bootstrap-options-batch-limit
DEBUG: id_store: saved cib-bootstrap-options-cluster-delay
DEBUG: id_store: saved cib-bootstrap-options-cluster-recheck-interval
DEBUG: id_store: saved cib-bootstrap-options-crmd-transition-delay
DEBUG: id_store: saved cib-bootstrap-options-dc-deadtime
DEBUG: id_store: saved cib-bootstrap-options-default-action-timeout
DEBUG: id_store: saved cib-bootstrap-options-default-resource-stickiness
DEBUG: id_store: saved cib-bootstrap-options-election-timeout
DEBUG: id_store: saved cib-bootstrap-options-enable-acl
DEBUG: id_store: saved cib-bootstrap-options-enable-startup-probes
DEBUG: id_store: saved cib-bootstrap-options-enable-container-probes
DEBUG: id_store: saved cib-bootstrap-options-is-managed-default
DEBUG: id_store: saved cib-bootstrap-options-maintenance-mode
DEBUG: id_store: saved cib-bootstrap-options-migration-limit
DEBUG: id_store: saved cib-bootstrap-options-no-quorum-policy
DEBUG: id_store: saved cib-bootstrap-options-node-health-green
DEBUG: id_store: saved cib-bootstrap-options-node-health-red
DEBUG: id_store: saved cib-bootstrap-options-node-health-strategy
DEBUG: id_store: saved cib-bootstrap-options-node-health-yellow
DEBUG: id_store: saved cib-bootstrap-options-pe-error-series-max
DEBUG: id_store: saved cib-bootstrap-options-pe-input-series-max
DEBUG: id_store: saved cib-bootstrap-options-pe-warn-series-max
DEBUG: id_store: saved cib-bootstrap-options-placement-strategy
DEBUG: id_store: saved cib-bootstrap-options-remove-after-stop
DEBUG: id_store: saved cib-bootstrap-options-shutdown-escalation
DEBUG: id_store: saved cib-bootstrap-options-start-failure-is-fatal
DEBUG: id_store: saved cib-bootstrap-options-startup-fencing
DEBUG: id_store: saved cib-bootstrap-options-stonith-action
DEBUG: id_store: saved cib-bootstrap-options-stonith-enabled
DEBUG: id_store: saved cib-bootstrap-options-stonith-timeout
DEBUG: id_store: saved cib-bootstrap-options-stop-all-resources
DEBUG: id_store: saved cib-bootstrap-options-stop-orphan-actions
DEBUG: id_store: saved cib-bootstrap-options-stop-orphan-resources
DEBUG: id_store: saved cib-bootstrap-options-symmetric-cluster
DEBUG: id_store: saved cib-bootstrap-options-dc-version
DEBUG: id_store: saved cib-bootstrap-options-cluster-infrastructure
DEBUG: id_store: saved cib-bootstrap-options-last-lrm-refresh
DEBUG: clitext: rsc_defaults $id="rsc_options" allow-migrate="false"
failure-timeout="10m" migration-threshold="INFINITY"
multiple-active="stop_start" priority="0"
DEBUG: id_store: saved rsc_options-allow-migrate
DEBUG: id_store: saved rsc_options-failure-timeout
DEBUG: id_store: saved rsc_options-migration-threshold
DEBUG: id_store: saved rsc_options-multiple-active
DEBUG: id_store: saved rsc_options-priority
DEBUG: id_store: saved cib-bootstrap-options-batch-limit
DEBUG: update CIB element: property:cib-bootstrap-options
DEBUG: clitext: property $id="cib-bootstrap-options" batch-limit="30"
DEBUG: id_store: saved cib-bootstrap-options-batch-limit
DEBUG: create configuration section rsc_defaults
DEBUG: piping string to crm_verify -V -p
   error: unpack_resources: <-->Resource start-up disabled since no
STONITH resources have been defined
   error: unpack_resources: <-->Either configure some or disable STONITH
with the stonith-enabled option
   error: unpack_resources: <-->NOTE: Clusters with shared data need
STONITH to ensure data integrity
Errors found during check: config not valid
DEBUG: create configuration section rsc_defaults
DEBUG: <?xml version="1.0" ?>
<cib num_updates="5432" dc-uuid="2" update-origin="booter-test-1"
crm_feature_set="3.0.8" validate-with="pacemaker-1.2"
update-client="booter-test-0" epoch="220" admin_epoch="0"
cib-last-written="Mon Dec 23 06:53:00 2013" have-quo
  <configuration>
    <crm_config>
      <cluster_property_set id="cib-bootstrap-options">
        <nvpair name="batch-limit" value="30"
id="cib-bootstrap-options-batch-limit"/>
      </cluster_property_set>
    </crm_config>

...

I would expect that only one option is modified, but crmsh intend to
remove all others.

May be it is possible to fix it by one-line crmsh patch?

Kristoffer, Dejan?

Best,
Vladislav