[Pacemaker] default gateway for virtual IP

[Max Arnold]

Folks,

I'm stuck trying to get working virtual IP in combination with default gateway. I have two frontend VMs with Nginx and two gateway VMs:

* fe1 - static eth0 ip 1.1.1.1, default gw 1.1.0.1
* fe2 - static eth0 ip 1.1.2.1, default gw 1.1.0.1
* virtual eth0 ip 1.1.3.1 for nginx, shared between fe1 and fe2

* gw1 - no static ip assigned
* gw2 - no static ip assigned
* virtual eth0 ip 1.1.4.1, shared between gw1 and gw2, default gw 1.1.0.1

All four nodes belong to single asymmetric cluster (communication is performed via separate private subnet assigned to eth1). VIP falover works just fine, but gw1 and gw2 are unreachable due to missing default route.

If I add Route primitive with gw=1.1.0.1 to gw1 and gw2, nodes fe1/fe2 immediately lose statically assigned default gateway. Using cloned or duplicate routes for fe1/fe2 doesn't work (I get -1000000 score assigned for one of the nodes). I think pacemaker is confused because default gateway is the same for each VM.

What I want to get is static default gateway for fe1/fe2 (because they always have static IPs) and dynamic default gateway for gw1/gw2 which follows VIP 1.1.4.1.

This is base config without any Route primitives:

node fe1
node fe2
node gw1
node gw2
primitive web_arp ocf:heartbeat:SendArp \
          params ip="1.1.3.1" nic="eth0"
primitive web_ip ocf:heartbeat:IPaddr2 \
          params ip="1.1.3.1" cidr_netmask="16" nic="eth0" \
          op monitor interval="5s"
primitive nginx lsb:nginx \
          meta target-role="Started"
group fe-group web_ip web_arp nginx

primitive gw_arp ocf:heartbeat:SendArp \
          params ip="1.1.4.1" nic="eth0"
primitive gw_ip ocf:heartbeat:IPaddr2 \
          params ip="1.1.4.1" cidr_netmask="16" nic="eth0" \
          op monitor interval="5s"
group gw-group gw_ip gw_arp

location fe-1 fe-group 200: fe1
location fe-2 fe-group 200: fe2
location gw-1 gw-group 200: gw1
location gw-2 gw-group 200: gw2

property $id="cib-bootstrap-options" \
         dc-version="1.0.8-042548a451fce8400660f6031f4da6f0223dd5dd" \
         cluster-infrastructure="openais" \
         expected-quorum-votes="4" \
         stonith-enabled="false" \
         no-quorum-policy="ignore" \
         symmetric-cluster="false" \
         last-lrm-refresh="1312053597"

rsc_defaults $id="rsc-options" \
             resource-stickiness="100"


This is my attempt #1 to add default Route, which leads to dropped static routes on fe1/fe2:

primitive default_route ocf:heartbeat:Route \
          params destination="default" device="eth0" gateway="1.1.0.1"
group gw-group gw_ip gw_arp default_route


This is attempt #2:

primitive default_route ocf:heartbeat:Route \
          params destination="default" device="eth0" gateway="1.1.0.1"
clone default_route_clone default_route
location fe-1-gw default_route_clone 200: fe1
location fe-2-gw default_route_clone 200: fe2

It leads to:
clone_color: default_gw_clone allocation score on fe1: -1000000
clone_color: default_gw_clone allocation score on fe2: -1000000

Plus I can't add it to gw-group (crm complains with some error):
group gw-group gw_ip gw_arp default_route_clone


And I tried to just make separate routes for each VM, but this still doesn't work and is not DRY.

Also I get this error on gw1/gw2 (these machines are not intended to run Nginx, but still attempt to validate its configuration):
ERROR: lsb:nginx: could not parse meta-data: 
ERROR: lsb:nginx: no such resource agent


Software versions:

Ubuntu Lucid
pacemaker 1.0.8+hg15494-2ubuntu2
corosync 1.2.0-0ubuntu1
libheartbeat2 1:3.0.3-1ubuntu1