[Pacemaker] OCFS2 Error When Adding Node to Cluster

[Darren.Mansell at opengi.co.uk]

Hello everyone.

 

I've just had a bit of a fright trying to join a 3rd node to an existing
live cluster running a controld clone, 02cb clone and an fs clone to
mount a shared volume from a SAN. The existing 2 nodes are running fine
in this configuration.

 

I had the 3rd node running on its own single node cluster, without being
able to see the live 2-node cluster. I put the following config in to
the 3rd node:

 

primitive dlm ocf:pacemaker:controld \

        op monitor interval="120s"

primitive o2cb ocf:ocfs2:o2cb \

        op monitor interval="2m"

primitive fs ocf:heartbeat:Filesystem \

        params device="/dev/dm-0" directory="/opt/SAN/" fstype="ocfs2" \

        op monitor interval="120s"

clone dlm-clone dlm \

        meta globally-unique="false" interleave="true"

clone o2cb-clone o2cb \

        meta globally-unique="false" interleave="true"

clone fs-clone fs \

        meta interleave="true" ordered="true"

colocation o2cb-with-dlm inf: o2cb-clone dlm-clone

colocation fs-with-o2cb inf: fs-clone o2cb-clone

order start-o2cb-after-dlm inf: dlm-clone o2cb-clone

order start-fs-after-o2cb inf: o2cb-clone fs-clone

 

and it mounted the FS and could see the data which the other 2 nodes
were accessing. Then I ensured the 3rd node had all supporting software
installed etc and deleted the CIB using cibadmin -E -force. I then
copied all of /etc/ais from a live node to the 3rd node and restarted
AIS. That's when it all started to go wrong. I eventually managed to get
the cluster back up and running by stopping AIS on the 3rd node and
rebooting both other nodes.