[Pacemaker] custom resource-agent

Peter Romfeld peter.romfeld.hk at gmail.com
Wed Oct 2 08:13:31 UTC 2013 

In you VPC the change take effect after you got "true", you can check it
with describe-eip or in console :)
right now i just try to add the command with hardcoded variables to IPaddr2
start, just to get it running for now

i put my attempt to create a resource agent on github, maybe someone can
help me find why params are not working?
wget https://raw.github.com/peterromfeldhk/pacemaker/master/AWSFIP

sorry im just started with scripting with nagios_nrpe last project :)
i have 2 problems, 1s2 the OCF_RESKEY_ params dont work as i thought, if i
hardcode the variables the commands are working at least in testrun, the
second big issue i have is the monitoring


On Wed, Oct 2, 2013 at 2:34 PM, David Lang <david at lang.hm> wrote:

> Unless something has changed in the AWS API in the last few months, when
> the aws command exits successfully, that doesn't mean the change has taken
> effect, just that the aws system has accepted the change and it will take
> effect 'soon'
>
>
> David Lang
>
> On Wed, 2 Oct 2013, Peter Romfeld wrote:
>
>  yes i need to use the aws command, i am using a VPC, after issueing the
>> command i get a "true" statement and its done
>>
>> so i only want pacemaker to issue the one-shot command at failover. Here
>> is
>> what i have atm: (i know its still dirty, just lerning pacemaker)
>>
>> primitive drbd_mysql ocf:linbit:drbd \
>>        params drbd_resource="mydata" \
>>        op monitor interval="15s"
>> primitive fs_mysql ocf:heartbeat:Filesystem \
>>        params device="/dev/drbdx" directory="/mountpint" fstype="ext4"
>> options="relatime,barrier=1" \
>>        op start interval="0" timeout="60" \
>>        op stop interval="0" timeout="60" \
>>        op monitor interval="10s" timeout="60s" OCF_CHECK_LEVEL="20" \
>>        meta target-role="started"
>> primitive fvip ocf:heartbeat:AWSFIP \
>>        params fvip="192.168.2.10" region="ap-southeast-1"
>> primitive ip_mysql ocf:heartbeat:IPaddr2 \
>>        params ip="192.168.2.10" cidr_netmask="20" \
>>        op monitor interval="10" \
>>        meta target-role="started"
>> primitive mysqld lsb:mysql
>> group mysql fs_mysql ip_mysql mysqld
>> ms ms_drbd_mysql drbd_mysql \
>>        meta master-max="1" master-node-max="1" clone-max="2"
>> clone-node-max="1" notify="true"
>> colocation mysql_on_drbd inf: fvip mysql ms_drbd_mysql:Master
>> order mysql_after_drbd_and_fvip inf: ms_drbd_mysql:promote fvip:start
>> mysql:start
>>
>> my AWSFIP(adjusted Dummy :%s/dummy/awsfip/g|%s/Dummy/**AWSFIP/g):
>>
>> <parameter name="fvip" unique="1" required="1">
>> <longdesc lang="en">
>> The IPv4 address to be configured in dotted quad notation, for example
>> "192.168.1.1".
>> </longdesc>
>> <shortdesc lang="en">IPv4 address</shortdesc>
>> <content type="string" default="" />
>> </parameter>
>>
>> <parameter name="region" unique="1" required="1">
>> <longdesc lang="en">
>> The name of the AWS region
>> </longdesc>
>> <shortdesc lang="en">AWS region</shortdesc>
>> <content type="string"/>
>> </parameter>
>>
>> awsfip_start() {
>>    awsfip_monitor
>>    Instance_ID=`/usr/bin/curl --silent
>> http://169.254.169.254/latest/**meta-data/instance-id`<http://169.254.169.254/latest/meta-data/instance-id>
>>    ENI_ID=`aws ec2 describe-instances --instance-id $Instance_ID --region
>> $OCF_RESKEY_region | grep NetworkInterfaceId | cut -d '"' -f 4`
>>    if [ $? =  $OCF_SUCCESS ]; then
>>        return $OCF_SUCCESS
>>    fi
>>         aws ec2 assign-private-ip-addresses --network-interface-id $ENI_ID
>> --private-ip-addresses $OCF_RESKEY_fvip --allow-reassignment --region
>> $OCF_RESKEY_region
>>         sleep 4
>>         aws ec2 assign-private-ip-addresses --network-interface-id $ENI_ID
>> --private-ip-addresses $OCF_RESKEY_fvip --allow-reassignment --region
>> $OCF_RESKEY_region
>>         /etc/init.d/networking restart
>>    touch ${OCF_RESKEY_state}
>> }
>>
>> I couldn't get it to work yet, and i don't want to run a external script
>> for it. I can't be so hard to let pacemaker execute an additional one-shot
>> command at failover (in the correct order..)
>>
>> Thanks for your help!
>>
>>
>> On Wednesday, October 02, 2013 07:33 AM, David Lang wrote:
>>
>> the aws command is making the call to inform aws, if you were to bring up
>> the address without making the aws command, would it work? If you are on a
>> Virtual Private Cloud (VPC), it may, but I didn't think it would.
>>
>> If you can make it work without the aws command, then you can just use the
>> standard pacemaker VIP configuration. I know that this doesn't work if you
>> have an external IP that you are moving (you must use an aws call to tell
>> Amazon to move the IP), but it's possible that you don't have to for an
>> internal IP, but I would be surprised.
>>
>> David Lang
>>
>>
>> On Wed, 2 Oct 2013, Peter Romfeld wrote:
>>
>> Hey,
>> when i change the secondary IP per hand or with external script on a
>> Ubuntu
>> Instance I just need:
>> /etc/network/interfaces
>> auto eth0
>> iface eth0 inet dhcp
>>   address 192.168.32.12
>>   netmask 255.255.240.0
>>   gateway 192.168.32.1
>>   up ip addr add 192.168.32.11/20 dev eth0
>>
>> and then run the script which basically just does:
>> #!/bin/sh
>>
>> VIP=172.32.32.11
>> REGION=ap-southeast-1
>>
>> Instance_ID=`/usr/bin/curl --silent
>> http://169.254.169.254/latest/**meta-data/instance-id`<http://169.254.169.254/latest/meta-data/instance-id>
>> ENI_ID=`aws ec2 describe-instances --instance-id $Instance_ID --region
>> $REGION | grep NetworkInterfaceId | cut -d '"' -f 4`
>>
>> aws ec2 assign-private-ip-addresses --network-interface-id $ENI_ID
>> --private-ip-addresses $VIP --allow-reassignment --region $REGION
>>
>>
>> I dont need to inform AWS or restart network, only the correct network
>> config and the one command, when i tested it with pinging from a 3rd
>> instance during IP change i didnt got any interupts. I dont know about
>> monitoring it
>>
>>
>> On Wed, Oct 2, 2013 at 1:38 AM, David Lang <david at lang.hm>
>> <david at lang.hm>wrote:
>>
>> On Tue, 1 Oct 2013, Dejan Muhamedagic wrote:
>>
>> On Tue, Oct 01, 2013 at 10:07:12AM -0700, David Lang wrote:
>>
>>
>> On Tue, 1 Oct 2013, Dejan Muhamedagic wrote:
>>
>> On Tue, Oct 01, 2013 at 07:22:20AM -0700, David Lang wrote:
>>
>>
>> On Tue, 1 Oct 2013, Dejan Muhamedagic wrote:
>>
>> Hi David,
>>
>>
>> On Mon, Sep 30, 2013 at 12:41:23PM -0700, David Lang wrote:
>>
>> On Mon, 30 Sep 2013, David Lang wrote:
>>
>> On Mon, 30 Sep 2013, Michael Schwartzkopff wrote:
>>
>>
>> Am Montag, 30. September 2013, 21:12:56 schrieb Peter Romfeld:
>>
>>
>> I am working in AWS i cant just use a VIP i need to use a floating
>> secondary IP which i reassign through script, i want to let
>> pacemaker
>> handle the reassignment...
>>
>>
>> Please explain the difference of a VIP and a "secondary IP" in
>> your opinion.
>>
>>
>> with AWS you need to inform amazon of the change, not just change
>> the IP on the local box, that requires much more work than a
>> simple local VIP
>>
>>
>> being more detailed, instead of just
>> ifconfig eth0:0 $vip
>> you have to do something like
>>
>> /opt/aws/bin/ec2-assign-****private-ip-addresses -n $ENI_ID
>> --secondary-private-ip-address $VIP --allow-reassignment --region $REGION
>>
>>
>> We may consider adding such an option to IPaddr2. Has anybody
>> ever tried that?
>>
>> pingresult=`ping -c 1 -W 1 $VIP | grep time= | wc -l`
>>
>> if [ "$pingresult" == "0" ]; then
>> echo `date` "-- Restarting network"
>> /sbin/service network restart > /dev/null 2>&1
>>
>>
>> That may break the cluster communication, which may lead to split
>> brain, etc. Is that really the only way?
>>
>>
>> It's not the only way, but you do have the problem that the call to
>> aws management interface is asynchronous, you don't know when it's
>> going to complete, and until it does, the IP doesn't actually work.
>>
>>
>> Wouldn't it be then safer to wait until it starts working, i.e.
>> to monitor in a loop?
>>
>>
>> that's exactly what the snippet of code above is for, to detect when
>> the other box no longer has the address.
>>
>>
>> Hmm, perhaps I'm missing something, but I couldn't notice a loop
>> in that code. What I meant was something like this:
>>
>> while ! ping -c 1 -W 1 $VIP | grep -qs time=; do
>>        :
>> done
>>
>> Then network restart wouldn't be necessary, right? Sorry, I don't
>> know much about aws.
>>
>>
>> I haven't used this exact script before, but I have seen the problem that
>> this script is designed to address. I am not saying that I agree with this
>> script, but it's what Amazon is suggesting, so it's probably a reasonable
>> start.
>>
>>
>> this was a cut-n-paste from the URL provided earlier
>> http://aws.amazon.com/****articles/2127188135977316<http://aws.amazon.com/**articles/2127188135977316>
>> <http://aws.amazon.com/**articles/2127188135977316<http://aws.amazon.com/articles/2127188135977316>
>> ><htt**p://aws.amazon.com/articles/**2127188135977316<http://aws.amazon.com/articles/2127188135977316>
>> >
>>
>>
>>
>> #!/bin/sh
>> # This script will monitor another HA node and take over a Virtual IP
>> (VIP)
>> # if communication with the other node fails
>>
>> # High Availability IP variables
>> # Other node's IP to ping and VIP to swap if other node goes down
>> HA_Node_IP=10.0.0.11
>> VIP=10.0.0.10
>>
>> # Specify the EC2 region that this will be running in
>> REGION=us-west-2
>>
>> # Run aws-apitools-common.sh to set up default environment variables and
>> to
>> # leverage AWS security credentials provided by EC2 roles
>> . /etc/profile.d/aws-apitools-****common.sh
>>
>> # Determine the instance and ENI IDs so we can reassign the VIP to the
>> # correct ENI. Requires EC2 describe-instances and
>> assign-private-ip-address
>> # permissions. The following example EC2 roles policy will authorize these
>> # commands:
>> # {
>> # "Statement": [
>> # {
>> # "Action": [
>> # "ec2:AssignPrivateIpAddresses"****,
>> # "ec2:DescribeInstances"
>> # ],
>> # "Effect": "Allow",
>> # "Resource": "*"
>> # }
>> # ]
>> # }
>>
>> Instance_ID=`/usr/bin/curl --silent http://169.254.169.254/latest/****<http://169.254.169.254/latest/**>
>> meta-data/instance-id`ENI_ID=`****/opt/aws/bin/ec2-describe-****instances
>> <http://169.254.169.254/**latest/meta-data/instance-**
>> idENI_ID=/opt/aws/bin/ec2-**describe-instances<http://169.254.169.254/latest/meta-data/instance-idENI_ID=/opt/aws/bin/ec2-describe-instances>
>> ><http://**169.254.169.254/latest/meta-**data/instance-idENI_ID=/opt/**
>> aws/bin/ec2-describe-instances<http://169.254.169.254/latest/meta-data/instance-idENI_ID=/opt/aws/bin/ec2-describe-instances>
>> **>$Instance_ID
>>
>> --region $REGION | grep eni -m 1 | awk '{print $2;}'`
>>
>> echo `date` "-- Starting HA monitor"
>> while [ . ]; do
>> pingresult=`ping -c 3 -W 1 $HA_Node_IP | grep time= | wc -l`
>>
>> if [ "$pingresult" == "0" ]; then
>> echo `date` "-- HA heartbeat failed, taking over VIP"
>>
>> /opt/aws/bin/ec2-assign-****private-ip-addresses -n $ENI_ID
>> --secondary-private-ip-address $VIP --allow-reassignment --region $REGION
>> pingresult=`ping -c 1 -W 1 $VIP | grep time= | wc -l`
>> if [ "$pingresult" == "0" ]; then
>> echo `date` "-- Restarting network"
>> /sbin/service network restart > /dev/null 2>&1
>> fi
>> sleep 60
>> fi
>> sleep 2
>> done
>>
>>
>>
>> David Lang
>>
>> ______________________________****_________________
>> Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
>> http://oss.clusterlabs.org/****mailman/listinfo/pacemaker<http://oss.clusterlabs.org/**mailman/listinfo/pacemaker>
>> <http://oss.clusterlabs.org/**mailman/listinfo/pacemaker<http://oss.clusterlabs.org/mailman/listinfo/pacemaker>
>> ><ht**tp://oss.clusterlabs.org/**mailman/listinfo/pacemaker<http://oss.clusterlabs.org/mailman/listinfo/pacemaker>
>> >
>>
>>
>> Project Home: http://www.clusterlabs.org
>> Getting started: http://www.clusterlabs.org/****
>> doc/Cluster_from_Scratch.pdf<http://www.clusterlabs.org/**doc/Cluster_from_Scratch.pdf>
>> <http://www.clusterlabs.org/**doc/Cluster_from_Scratch.pdf<http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf>
>> ><**http://www.clusterlabs.org/**doc/Cluster_from_Scratch.pdf<http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf>
>> >
>>
>> Bugs: http://bugs.clusterlabs.org
>>
>>
>>
>>
>> ______________________________**_________________
>>
>> Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
>> http://oss.clusterlabs.org/**mailman/listinfo/pacemaker<http://oss.clusterlabs.org/mailman/listinfo/pacemaker>
>>
>>
>>
>> Project Home: http://www.clusterlabs.org
>>
>> Getting started: http://www.clusterlabs.org/**
>> doc/Cluster_from_Scratch.pdf<http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf>
>>
>> Bugs: http://bugs.clusterlabs.org
>>
>>
>>
>> ______________________________**_________________
>> Pacemaker mailing list:
>> Pacemaker at oss.clusterlabs.**orghttp://oss.clusterlabs.org/**
>> mailman/listinfo/pacemaker<http://oss.clusterlabs.org/mailman/listinfo/pacemaker>
>>
>>
>> Project Home: http://www.clusterlabs.org
>> Getting started: http://www.clusterlabs.org/**
>> doc/Cluster_from_Scratch.pdf<http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf>
>> Bugs: http://bugs.clusterlabs.org
>>
>
> _______________________________________________
>
> Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
>
> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>
>
>
> Project Home: http://www.clusterlabs.org
>
> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>
> Bugs: http://bugs.clusterlabs.org
>
>
> _______________________________________________
> Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>
> Project Home: http://www.clusterlabs.org
> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> Bugs: http://bugs.clusterlabs.org
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.clusterlabs.org/pipermail/pacemaker/attachments/20131002/36b13350/attachment.htm>

More information about the Pacemaker mailing list