[Pacemaker] Two resource nodes + one quorum node

[Andrew Beekhof]

On 13/06/2013, at 1:57 AM, Digimer <lists at alteeve.ca> wrote:

> On 06/12/2013 03:06 AM, Michael Schwartzkopff wrote:
>> Am Mittwoch, 12. Juni 2013, 09:42:13 schrieb Andrew Beekhof:
>> > On 12/06/2013, at 4:48 AM, Michael Schwartzkopff
>> <misch at clusterbau.com> wrote:
>> > > Am Dienstag, 11. Juni 2013, 22:33:32 schrieb Andrey Groshev:
>> > > > Hi,
>> > > > I want to make Postgres cluster.
>> > > > As far as I understand, for the proper functioning of the cluster
>> must
>> > > > use a quorum (ie, at least three nodes).
>> > > No. Two nodes are enough. See: no-quorum-policy="ignore".
>> > > > But if the databases are large - it is
>> > > > too expensive.
>> > > > One master, two slave, and in addition backup.
>> > > > So I'm trying to make the schema with one master, one slave, and one
>> > > > node
>> > > > only for a quorum.
>> > > One master and one slave should be enough.
>> > But three nodes is better.
>> Definitely yes. But if he has financial limitations (see above) it is
>> better to invest in good fencing than in a third node.
> 
> I build exclusively two-node clusters, and the biggest draw-back is the possibility of a "fence loop". That is, without quorum and with a network error, a node can come up on it's own, fail to contact it's peer and fence it. When the fenced node boots, it comes up, fails to contact it's peer, and fences it. Wash, rinse, repeat.
> 
> To prevent this, I recommend *not* letting your cluster stack start on boot. This does mean that you need to manually start the node(s) on boot, but I have found this to not be an issue. The only times this is needed is when I am cold-booting the cluster or after a node has failed and been fenced.
> 
> In the former case, it is usually at the end of scheduled maintenance so I am there to do it anyway. In the latter case, I don't want a node to rejoin the cluster until I've been able to look into the failure first anyway.
> 
> I also recommend, as a matter of course but doubly so in two-node clusters, to have redundant fence methods. My personal favourite combination is IPMI as the first fence method with switched PDUs as the backup fence method. Run IPMI through your primary fence and the PDUs through your backup switch and you have good fencing, even if a switch or link has failed.

Its certainly possible to build a decent 2-node cluster, but there are several non-obvious steps that are required - preventing fencing loops being one.
For this reason I cannot recommend them for newcomers, because they are also the most likely group to be unaware of the caveats.