[ClusterLabs] Upcoming "compact containers handling" feature discussion (Was: Coming in Pacemaker 1.1.17: container bundles)

[Jan Pokorný]

On 18/04/17 15:52 +0200, Jan Pokorný wrote:
> On 17/04/17 09:51 -0500, Ken Gaillot wrote:
>> On 04/13/2017 07:04 AM, Jan Pokorný wrote:
>>> On 03/04/17 09:47 -0500, Ken Gaillot wrote:
>>>> With a group, you could reproduce most of this functionality, though it
>>>> would be more verbose: you'd need to configure ocf:heartbeat:docker,
>>>> ocf:heartbeat:IPaddr2, and ocf:pacemaker:remote resources, plus a
>>>> primitive for your service, as well as constraints that restrict the
>>>> primitive to the guest node and prevent any other resource from running
>>>> on the guest node.
>>>> 
>>>> However, this can do something that a group can't: launch multiple
>>>> instances of a single container image, while associating floating IPs
>>>> and storage mappings specific to each replica. This puts it somewhere
>>>> between a group and a specialized form of clone.
>>> 
>>> In that case, wouldn't it be more systemic to factor any generic
>>> clone/master-like controls (replicas, replicas-per-host, masters) out
>>> of <docker> so it can be reused seemlessly when switching to other
>>> possible containerization backends in the future?
>> 
>> We did consider that. It's not clear how much of the functionality (or
>> terminology) will be applicable to other technologies (currently known
>> and potential future ones). We decided that it would be cleaner to
>> duplicate those fields in other technology tags than to have them not
>> work for certain technologies.
> 
> Then I am confused about this double standard in comparison to
> {port,storage}-mapping sections that are already hoisted out.

Two more thoughts here.

* * *

1. Possibility to lift up container-pacemaker handling generics while
   retaining fully-definedness in respect to schemas (without blind
   spots, i.e., without possibility to define options without effect
   for particlar approach to containers)

I can see that parallel runs are not that easy with some approaches to
containers, especially if they work with exploded root-like file
hierarchy (danger of write conflicts).  I still believe it would be
future-friendly if interface specific options ("image" for "docker")
would not mix with conceptually different pacemaker handling specific
ones like "replicas" that will possibly be shared (with the very same
meaning) as new container interfaces are supported.

RelaxNG is strong enough to cope with that, and each new interface
supported imposes schema changes, anyway.  So a sketch of how such
schema could look like goes as a footnote[*], with its instance
mirroring configuration from the initial post
(http://oss.clusterlabs.org/pipermail/users/2017-April/005380.html):

  <bundle id="httpd-bundle">
 
    <engine replicas="3">
      <docker image="pcmk:httpd" />
    </engine>
 
    <network ip-range-start="192.168.24.112" host-netmask="24">
      <port-mapping id="httpd-port" port="80"/>
    </network>
 
    <storage>
      <!-- [...] -->
    </storage>
 
    <primitive id="httpd-apache"
      class="ocf" provider="heartbeat" type="apache"/>
 
  </bundle>

* * *

2. Terminology clash on "bundle"

Reading runc(8) man page, I'm getting an omnious feeling that we are
late to coin "bundle" as a term in the containers' field as it already
is in use by Open Container Initicative for something rather specific:
https://github.com/opencontainers/runc#creating-an-oci-bundle
https://github.com/opencontainers/runc/blob/v1.0.0-rc3/man/runc.8.md#description

It certainly would not be nice if we made the term ambiguous needlessly.
There were some other suggestions by Ken formerly:
https://github.com/ClusterLabs/pacemaker/pull/1242#discussion_r106726340
(in the same spirit, what about "vagon", one locomotive to pull possibly
"replicated" instances of that?  also would match the suggested "engine"
semi-metaphore).  But original "bucket" would also do.

* * *

[*] sketch of a modified schema -- note that adding a new interface to
    containers is as easy as defining new subelement of "engine", with
    or without reference to "attrs-bundle-replicable" in the respective
    choice group:

  <define name="attrs-bundle-replicable">
    <optional>
      <attribute name="replicas"><data type="integer"/></attribute>
    </optional>
    <optional>
      <attribute name="replicas-per-host"><data type="integer"/></attribute>
    </optional>
    <optional>
      <attribute name="masters"><data type="integer"/></attribute>
    </optional>
  </define>

  <define name="element-bundle">
    <element name="bundle">
      <attribute name="id"><data type="ID"/></attribute>
      <optional>
        <attribute name="description"><text/></attribute>
      </optional>
      <element name="engine">
        <choice>
          <group>
            <ref name="attrs-bundle-replicable"/>
            <element name="docker">
              <attribute name="image"><text/></attribute>
              <optional>
                <attribute name="run-command"> <text/></attribute>
              </optional>
              <optional>
                <attribute name="network"><text/></attribute>
              </optional>
              <optional>
                <attribute name="options"><text/></attribute>
              </optional>
            </element>
          </group>
        </choice>
      </element>
      <optional>
        <element name="network">
          <!-- [...] -->
        </element>
      </optional>
      <optional>
        <element name="storage">
          <!-- [...] -->
        </element>
      </optional>
      <optional>
        <ref name="element-primitive"/>
      </optional>
    </element>
  </define>

-- 
Jan (Poki)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <https://lists.clusterlabs.org/pipermail/users/attachments/20170419/c4b11768/attachment-0003.sig>