[Pacemaker] MySQL HA with pacemaker and DRBD - How to monitor mysql service

emmanuel segura emi2fast at gmail.com
Fri Mar 23 13:48:33 UTC 2012


I'm happy you solved the problem :-)

Il giorno 23 marzo 2012 14:36, coma <coma.inf at gmail.com> ha scritto:

> Yes i missed this line in logs: info: RA output: (mysqld:stop:stderr)
> /usr/lib/ocf/resource.d//heartbeat/mysql: line 45:
> /usr/lib/ocf/lib/heartbeat/ocf-shellfuncs: No such file or directory
>
> some libs are missing on my second node, i don't know why (same install
> with yum on the two nodes), i have copied them from the first node and now
> it's work fine.
> I'm sorry for the inconvenience and thank you very much for your help this
> is much appreciated!
>
>
>
>
> 2012/3/23 emmanuel segura <emi2fast at gmail.com>
>
>> try to look the 45 in your resource agent,i have this in a cluster with
>> mysql
>>
>> =====================================================
>> # Initialization:
>>
>> : ${OCF_FUNCTIONS_DIR=${OCF_ROOT}/resource.d/heartbeat}
>> . ${OCF_FUNCTIONS_DIR}/.ocf-shellfuncs
>> =====================================================
>>
>> Il giorno 23 marzo 2012 14:17, coma <coma.inf at gmail.com> ha scritto:
>>
>> it's done,
>>>
>>> Here is the log from the second node, thank for your help, it very
>>> apreciated!:
>>>
>>> Mar 23 13:54:53 node2 attrd: [3102]: info: find_hash_entry: Creating
>>> hash entry for last-failure-mysqld
>>> Mar 23 13:54:53 node2 attrd: [3102]: info: attrd_perform_update:
>>> Delaying operation last-failure-mysqld=<null>: cib not connected
>>> Mar 23 13:54:53 node2 attrd: [3102]: info: find_hash_entry: Creating
>>> hash entry for fail-count-mysqld
>>> Mar 23 13:54:53 node2 attrd: [3102]: info: attrd_perform_update:
>>> Delaying operation fail-count-mysqld=<null>: cib not connected
>>> Mar 23 13:54:53 node2 lrmd: [3101]: debug: on_msg_add_rsc:client [3104]
>>> adds resource mysqld
>>> Mar 23 13:54:53 node2 crmd: [3104]: info: do_lrm_rsc_op: Performing
>>> key=10:46:7:1f6f7a59-8e04-46d9-8a47-4b1ada0e6ea1 op=mysqld_monitor_0 )
>>> Mar 23 13:54:53 node2 lrmd: [3101]: debug: on_msg_perform_op:2359:
>>> copying parameters for rsc mysqld
>>> Mar 23 13:54:53 node2 lrmd: [3101]: debug: on_msg_perform_op: add an
>>> operation operation monitor[6] on ocf::mysql::mysqld for client 3104, its
>>> parameters: socket=[/var/lib/mysql/mysql.sock]
>>> binary=[/usr/bin/mysqld_safe] group=[mysql] CRM_meta_timeout=[20000]
>>> crm_feature_set=[3.0.5] pid=[/var/run/mysqld/mysqld.pid] user=[mysql]
>>> config=[/etc/my.cnf] datadir=[/data/mysql/databases]  to the operation list.
>>> Mar 23 13:54:53 node2 lrmd: [3101]: info: rsc:mysqld:6: probe
>>> Mar 23 13:54:53 node2 lrmd: [3101]: WARN: Managed mysqld:monitor process
>>> 3223 exited with return code 1.
>>> Mar 23 13:54:53 node2 lrmd: [3101]: info: RA output:
>>> (mysqld:monitor:stderr) /usr/lib/ocf/resource.d//heartbeat/mysql: line 45:
>>> /usr/lib/ocf/lib/heartbeat/ocf-shellfuncs: No such file or directory
>>> Mar 23 13:54:53 node2 crmd: [3104]: debug: create_operation_update:
>>> do_update_resource: Updating resouce mysqld after complete monitor op
>>> (interval=0)
>>> Mar 23 13:54:53 node2 crmd: [3104]: info: process_lrm_event: LRM
>>> operation mysqld_monitor_0 (call=6, rc=1, cib-update=10, confirmed=true)
>>> unknown error
>>> Mar 23 13:54:54 node2 crmd: [3104]: info: do_lrm_rsc_op: Performing
>>> key=3:47:0:1f6f7a59-8e04-46d9-8a47-4b1ada0e6ea1 op=mysqld_stop_0 )
>>> Mar 23 13:54:54 node2 lrmd: [3101]: debug: on_msg_perform_op: add an
>>> operation operation stop[7] on ocf::mysql::mysqld for client 3104, its
>>> parameters: crm_feature_set=[3.0.5]  to the operation list.
>>> Mar 23 13:54:54 node2 lrmd: [3101]: info: rsc:mysqld:7: stop
>>> Mar 23 13:54:54 node2 lrmd: [3101]: info: RA output:
>>> (mysqld:stop:stderr) /usr/lib/ocf/resource.d//heartbeat/mysql: line 45:
>>> /usr/lib/ocf/lib/heartbeat/ocf-shellfuncs: No such file or directory -> I
>>> just saw it, I will look over
>>> Mar 23 13:54:54 node2 lrmd: [3101]: WARN: Managed mysqld:stop process
>>> 3241 exited with return code 1.
>>> Mar 23 13:54:54 node2 crmd: [3104]: debug: create_operation_update:
>>> do_update_resource: Updating resouce mysqld after complete stop op
>>> (interval=0)
>>> Mar 23 13:54:54 node2 crmd: [3104]: info: process_lrm_event: LRM
>>> operation mysqld_stop_0 (call=7, rc=1, cib-update=12, confirmed=true)
>>> unknown error
>>> Mar 23 13:54:54 node2 attrd: [3102]: debug: attrd_local_callback: update
>>> message from node1: fail-count-mysqld=INFINITY
>>> Mar 23 13:54:54 node2 attrd: [3102]: debug: attrd_local_callback: New
>>> value of fail-count-mysqld is INFINITY
>>> Mar 23 13:54:54 node2 attrd: [3102]: info: attrd_trigger_update: Sending
>>> flush op to all hosts for: fail-count-mysqld (INFINITY)
>>> Mar 23 13:54:54 node2 attrd: [3102]: info: attrd_perform_update:
>>> Delaying operation fail-count-mysqld=INFINITY: cib not connected
>>> Mar 23 13:54:54 node2 attrd: [3102]: debug: attrd_local_callback: update
>>> message from node1: last-failure-mysqld=1332507294
>>> Mar 23 13:54:54 node2 attrd: [3102]: debug: attrd_local_callback: New
>>> value of last-failure-mysqld is 1332507294
>>> Mar 23 13:54:54 node2 attrd: [3102]: info: attrd_trigger_update: Sending
>>> flush op to all hosts for: last-failure-mysqld (1332507294)
>>> Mar 23 13:54:54 node2 attrd: [3102]: info: attrd_perform_update:
>>> Delaying operation last-failure-mysqld=1332507294: cib not connected
>>> Mar 23 13:54:54 node2 attrd: [3102]: info: attrd_trigger_update: Sending
>>> flush op to all hosts for: last-failure-mysqld (1332507294)
>>> Mar 23 13:54:54 node2 cib: [3100]: debug: cib_process_xpath: cib_query:
>>> //cib/status//node_state[@id='node2']//transient_attributes//nvpair[@name='last-failure-mysqld']
>>> does not exist
>>> Mar 23 13:54:54 node2 attrd: [3102]: info: attrd_perform_update: Sent
>>> update 4: last-failure-mysqld=1332507294
>>> Mar 23 13:54:54 node2 attrd: [3102]: info: attrd_trigger_update: Sending
>>> flush op to all hosts for: fail-count-mysqld (INFINITY)
>>> Mar 23 13:54:54 node2 cib: [3100]: debug: cib_process_xpath: cib_query:
>>> //cib/status//node_state[@id='node2']//transient_attributes//nvpair[@name='fail-count-mysqld']
>>> does not exist
>>> Mar 23 13:54:54 node2 attrd: [3102]: info: attrd_perform_update: Sent
>>> update 11: fail-count-mysqld=INFINITY
>>> Mar 23 13:54:54 node2 attrd: [3102]: debug: attrd_cib_callback: Update 4
>>> for last-failure-mysqld=1332507294 passed
>>> Mar 23 13:54:54 node2 attrd: [3102]: debug: attrd_cib_callback: Update
>>> 11 for fail-count-mysqld=INFINITY passed
>>>
>>>
>>>
>>>
>>> 2012/3/23 emmanuel segura <emi2fast at gmail.com>
>>>
>>>> The first thing you can do it's eliminate this "location
>>>> master-prefer-node-1 Cluster-VIP 25: node1"
>>>>
>>>> because you have your virtual in group and i would like to see the log
>>>> from the second node
>>>>
>>>> Thanks :-)
>>>>
>>>> Il giorno 23 marzo 2012 13:42, coma <coma.inf at gmail.com> ha scritto:
>>>>
>>>> Thank you for your responses,
>>>>>
>>>>> i have fixed my migration-treshold problem with lsb:mysqld ressource
>>>>> (i can see migration-threshold=2 with crm_mon failcounts so its ok) bu
>>>>> failover still doesn't work when mysql fail (but work fine when node fail
>>>>> or standby).
>>>>> So i've tried with the ocf ressource agent, it's work fine on my first
>>>>> node but fail on my second node with unknown error.
>>>>>
>>>>> crm_mon --failcount:
>>>>>
>>>>> Failed actions:
>>>>>     mysqld_monitor_0 (node=node2, call=6, rc=1, status=complete):
>>>>> unknown error
>>>>>     mysqld_stop_0 (node=node2, call=7, rc=1, status=complete): unknown
>>>>> error
>>>>>
>>>>>
>>>>> I have exactly the same mysql packages version and configuration on my
>>>>> two nodes (with proper permissions), also corosync/heartbeat and pacemaker
>>>>> are in the same version too:
>>>>>
>>>>> corosynclib-1.2.7-1.1.el5
>>>>> corosync-1.2.7-1.1.el5
>>>>> pacemaker-libs-1.1.5-1.1.el5
>>>>> pacemaker-1.1.5-1.1.el5
>>>>> heartbeat-3.0.3-2.3.el5
>>>>> heartbeat-libs-3.0.3-2.3.el5
>>>>> heartbeat-debuginfo-3.0.2-2.el5
>>>>>
>>>>> So i don't andersand wy it's works on one node but not on the second?
>>>>>
>>>>>
>>>>> ressource config:
>>>>>
>>>>> primitive mysqld ocf:heartbeat:mysql \
>>>>>         params binary="/usr/bin/mysqld_safe" config="/etc/my.cnf" \
>>>>>         user="mysql" group="mysql" pid="/var/run/mysqld/mysqld.pid" \
>>>>>         datadir="/data/mysql/databases"
>>>>> socket="/var/lib/mysql/mysql.sock" \
>>>>>         op start interval="0" timeout="120" \
>>>>>         op stop interval="0" timeout="120" \
>>>>>         op monitor interval="30" timeout="30" depth="0" \
>>>>>         target-role="Started"
>>>>>
>>>>>
>>>>> And same with (i have created test database / table + grant test user
>>>>> on it):
>>>>>
>>>>> primitive mysqld ocf:heartbeat:mysql \
>>>>>         params binary="/usr/bin/mysqld_safe" config="/etc/my.cnf" \
>>>>>         datadir="/data/mysql/databases" user="mysql" \
>>>>>         pid="/var/run/mysqld/mysqld.pid"
>>>>> socket="/var/lib/mysql/mysql.sock" \
>>>>>         test_passwd="test" test_table="Cluster.dbcheck"
>>>>> test_user="test" \
>>>>>         op start interval="0" timeout="120" \
>>>>>         op stop interval="0" timeout="120" \
>>>>>         op monitor interval="30s" timeout="30s" OCF_CHECK_LEVEL="1" \
>>>>>         meta migration-threshold="3" target-role="Started"
>>>>>
>>>>>
>>>>> Full config:
>>>>>
>>>>> node node2 \
>>>>>         attributes standby="off"
>>>>> node node1 \
>>>>>         attributes standby="off"
>>>>> primitive Cluster-VIP ocf:heartbeat:IPaddr2 \
>>>>>         params ip="x.x.x.x" broadcast="x.x.x.x" nic="eth0"
>>>>> cidr_netmask="21" iflabel="VIP1" \
>>>>>         op monitor interval="10s" timeout="20s" \
>>>>>         meta is-managed="true"
>>>>> primitive datavg ocf:heartbeat:LVM \
>>>>>         params volgrpname="datavg" exclusive="true" \
>>>>>         op start interval="0" timeout="30" \
>>>>>         op stop interval="0" timeout="30"
>>>>> primitive drbd_mysql ocf:linbit:drbd \
>>>>>         params drbd_resource="drbd-mysql" \
>>>>>         op monitor interval="15s"
>>>>> primitive fs_mysql ocf:heartbeat:Filesystem \
>>>>>         params device="/dev/datavg/data" directory="/data"
>>>>> fstype="ext3"
>>>>> primitive mysqld ocf:heartbeat:mysql \
>>>>>         params binary="/usr/bin/mysqld_safe" config="/etc/my.cnf"
>>>>> user="mysql" group="mysql" pid="/var/run/mysqld/mysqld.pid"
>>>>> datadir="/data/mysql/databases" socket="/var/lib/mysql/mysql.sock" \
>>>>>         op start interval="0" timeout="120" \
>>>>>         op stop interval="0" timeout="120" \
>>>>>         op monitor interval="30" timeout="30" depth="0"
>>>>> group mysql datavg fs_mysql Cluster-VIP mysqld
>>>>> ms ms_drbd_mysql drbd_mysql \
>>>>>         meta master-max="1" master-node-max="1" clone-max="2"
>>>>> clone-node-max="1" notify="true"
>>>>> location master-prefer-node-1 Cluster-VIP 25: node1
>>>>> colocation mysql_on_drbd inf: mysql ms_drbd_mysql:Master
>>>>> order mysql_after_drbd inf: ms_drbd_mysql:promote mysql:start
>>>>> property $id="cib-bootstrap-options" \
>>>>>
>>>>> dc-version="1.1.5-1.1.el5-01e86afaaa6d4a8c4836f68df80ababd6ca3902f" \
>>>>>         cluster-infrastructure="openais" \
>>>>>         expected-quorum-votes="2" \
>>>>>         stonith-enabled="false" \
>>>>>         no-quorum-policy="ignore" \
>>>>>         last-lrm-refresh="1332504626"
>>>>> rsc_defaults $id="rsc-options" \
>>>>>         resource-stickiness="100"
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> 2012/3/22 Andreas Kurz <andreas at hastexo.com>
>>>>>
>>>>>>  On 03/22/2012 03:23 PM, coma wrote:
>>>>>> > Thank you for your responses,
>>>>>> >
>>>>>> > I have added the migration-treshold on my mysqld ressource, when i
>>>>>> kill
>>>>>> > or manually stop mysql on one node, there is not failover on the
>>>>>> second
>>>>>> > node.
>>>>>> > Also, when i look crm_mon --failcounts, i can see "mysqld:
>>>>>> > migration-threshold=1000000 fail-count=1000000", so i don"t
>>>>>> anderstand
>>>>>> > why migration-threshold not equal 2?
>>>>>> >
>>>>>> > Migration summary:
>>>>>> > * Node node1:
>>>>>> >    mysqld: migration-threshold=1000000 fail-count=1000000
>>>>>> > * Node node2:
>>>>>> >
>>>>>> > Failed actions:
>>>>>> >     mysqld_monitor_10000 (node=node1, call=90, rc=7,
>>>>>> status=complete):
>>>>>> > not running
>>>>>> >     mysqld_stop_0 (node=node1, call=93, rc=1, status=complete):
>>>>>> unknown
>>>>>> > error
>>>>>>
>>>>>> The lsb init script you are using seems to be not LSB compliant ...
>>>>>> looks like it returns an error on stopping an already stopped mysql.
>>>>>>
>>>>>>
>>>>>> http://clusterlabs.org/doc/en-US/Pacemaker/1.1/html-single/Pacemaker_Explained/index.html#ap-lsb
>>>>>>
>>>>>> Fix the script ... or better use the ocf resource agent.
>>>>>>
>>>>>> Regards,
>>>>>> Andreas
>>>>>>
>>>>>> --
>>>>>> Need help with Pacemaker?
>>>>>> http://www.hastexo.com/now
>>>>>>
>>>>>> >
>>>>>> >
>>>>>> >
>>>>>> > configuration:
>>>>>> >
>>>>>> > node node1 \
>>>>>> >         attributes standby="off"
>>>>>> > node node2 \
>>>>>> >         attributes standby="off"
>>>>>> > primitive Cluster-VIP ocf:heartbeat:IPaddr2 \
>>>>>> >         params ip="x.x.x.x" broadcast="x.x.x.x" nic="eth0"
>>>>>> > cidr_netmask="21" iflabel="VIP1" \
>>>>>> >         op monitor interval="10s" timeout="20s" \
>>>>>> >         meta is-managed="true"
>>>>>> > primitive datavg ocf:heartbeat:LVM \
>>>>>> >         params volgrpname="datavg" exclusive="true" \
>>>>>> >         op start interval="0" timeout="30" \
>>>>>> >         op stop interval="0" timeout="30"
>>>>>> > primitive drbd_mysql ocf:linbit:drbd \
>>>>>> >         params drbd_resource="drbd-mysql" \
>>>>>> >         op monitor interval="15s"
>>>>>> > primitive fs_mysql ocf:heartbeat:Filesystem \
>>>>>> >         params device="/dev/datavg/data" directory="/data"
>>>>>> fstype="ext3"
>>>>>> > primitive mysqld lsb:mysqld \
>>>>>> >         op monitor interval="10s" timeout="30s" \
>>>>>> >         op start interval="0" timeout="120" \
>>>>>> >         op stop interval="0" timeout="120" \
>>>>>> >         meta target-role="Started" migration-threshold="2"
>>>>>> > failure-timeout="20s"
>>>>>> > group mysql datavg fs_mysql Cluster-VIP mysqld
>>>>>> > ms ms_drbd_mysql drbd_mysql \
>>>>>> >         meta master-max="1" master-node-max="1" clone-max="2"
>>>>>> > clone-node-max="1" notify="true"
>>>>>> > location master-prefer-node-1 Cluster-VIP 25: node1
>>>>>> > colocation mysql_on_drbd inf: mysql ms_drbd_mysql:Master
>>>>>> > order mysql_after_drbd inf: ms_drbd_mysql:promote mysql:start
>>>>>> > property $id="cib-bootstrap-options" \
>>>>>> >
>>>>>> > dc-version="1.1.5-1.1.el5-01e86afaaa6d4a8c4836f68df80ababd6ca3902f"
>>>>>> \
>>>>>> >         cluster-infrastructure="openais" \
>>>>>> >         expected-quorum-votes="2" \
>>>>>> >         stonith-enabled="false" \
>>>>>> >         no-quorum-policy="ignore" \
>>>>>> >         last-lrm-refresh="1332425337"
>>>>>> > rsc_defaults $id="rsc-options" \
>>>>>> >         resource-stickiness="100"
>>>>>> >
>>>>>> >
>>>>>> >
>>>>>> > 2012/3/22 Andreas Kurz <andreas at hastexo.com <mailto:
>>>>>> andreas at hastexo.com>>
>>>>>> >
>>>>>> >     On 03/22/2012 01:51 PM, coma wrote:
>>>>>> >     > Ah yes thank you, the service status mysql is now monitored,
>>>>>> but the
>>>>>> >     > failover is not performed?
>>>>>> >
>>>>>> >     As long as local restarts are successful there is no need for a
>>>>>> failover
>>>>>> >     ... there is migration-treshold to limit local restart tries.
>>>>>> >
>>>>>> >     Regards,
>>>>>> >     Andreas
>>>>>> >
>>>>>> >     --
>>>>>> >     Need help with Pacemaker?
>>>>>> >     http://www.hastexo.com/now
>>>>>> >
>>>>>> >     >
>>>>>> >     >
>>>>>> >     >
>>>>>> >     > 2012/3/22 emmanuel segura <emi2fast at gmail.com
>>>>>> >     <mailto:emi2fast at gmail.com> <mailto:emi2fast at gmail.com
>>>>>> >     <mailto:emi2fast at gmail.com>>>
>>>>>> >     >
>>>>>> >     >     sorry
>>>>>> >     >     I think you missed the op monitor operetion in your
>>>>>> primitive
>>>>>> >     definition
>>>>>> >     >
>>>>>> >     >
>>>>>> >     >
>>>>>> >     >     Il giorno 22 marzo 2012 11:52, emmanuel segura
>>>>>> >     <emi2fast at gmail.com <mailto:emi2fast at gmail.com>
>>>>>> >     >     <mailto:emi2fast at gmail.com <mailto:emi2fast at gmail.com>>>
>>>>>> ha
>>>>>> >     scritto:
>>>>>> >     >
>>>>>> >     >         I think you missed the op monitor operetion you
>>>>>> primitive
>>>>>> >     definition
>>>>>> >     >
>>>>>> >     >         Il giorno 22 marzo 2012 11:33, coma <
>>>>>> coma.inf at gmail.com
>>>>>> >     <mailto:coma.inf at gmail.com>
>>>>>> >     >         <mailto:coma.inf at gmail.com <mailto:coma.inf at gmail.com
>>>>>> >>>
>>>>>> >     ha scritto:
>>>>>> >     >
>>>>>> >     >             Hello,
>>>>>> >     >
>>>>>> >     >             I have a question about mysql service monitoring
>>>>>> into a
>>>>>> >     >             MySQL HA cluster with pacemaker and DRBD,
>>>>>> >     >             I have set up a configuration to allow a failover
>>>>>> between
>>>>>> >     >             two nodes, it work fine when a node is offline (or
>>>>>> >     standby),
>>>>>> >     >             but i want to know if it is possible to monitor
>>>>>> the mysql
>>>>>> >     >             service to perform a failover if mysql is stopped
>>>>>> or
>>>>>> >     >             unavailable?
>>>>>> >     >
>>>>>> >     >             Thank you in advance for any response.
>>>>>> >     >
>>>>>> >     >             My crm configuration:
>>>>>> >     >
>>>>>> >     >             node node1 \
>>>>>> >     >                     attributes standby="off"
>>>>>> >     >             node node2 \
>>>>>> >     >                     attributes standby="off"
>>>>>> >     >             primitive Cluster-VIP ocf:heartbeat:IPaddr2 \
>>>>>> >     >                     params ip="x.x.x.x" broadcast="x.x.x.x"
>>>>>> nic="eth0"
>>>>>> >     >             cidr_netmask="21" iflabel="VIP1" \
>>>>>> >     >                     op monitor interval="10s" timeout="20s" \
>>>>>> >     >                     meta is-managed="true"
>>>>>> >     >             primitive datavg ocf:heartbeat:LVM \
>>>>>> >     >                     params volgrpname="datavg"
>>>>>> exclusive="true" \
>>>>>> >     >                     op start interval="0" timeout="30" \
>>>>>> >     >                     op stop interval="0" timeout="30"
>>>>>> >     >             primitive drbd_mysql ocf:linbit:drbd \
>>>>>> >     >                     params drbd_resource="drbd-mysql" \
>>>>>> >     >                     op monitor interval="15s"
>>>>>> >     >             primitive fs_mysql ocf:heartbeat:Filesystem \
>>>>>> >     >                     params device="/dev/datavg/data"
>>>>>> directory="/data"
>>>>>> >     >             fstype="ext3"
>>>>>> >     >             primitive mysqld lsb:mysqld
>>>>>> >     >             group mysql datavg fs_mysql Cluster-VIP mysqld
>>>>>> >     >             ms ms_drbd_mysql drbd_mysql \
>>>>>> >     >                     meta master-max="1" master-node-max="1"
>>>>>> >     >             clone-max="2" clone-node-max="1" notify="true"
>>>>>> >     >             location master-prefer-node-1 Cluster-VIP 25:
>>>>>> node1
>>>>>> >     >             colocation mysql_on_drbd inf: mysql
>>>>>> ms_drbd_mysql:Master
>>>>>> >     >             order mysql_after_drbd inf: ms_drbd_mysql:promote
>>>>>> >     mysql:start
>>>>>> >     >             property $id="cib-bootstrap-options" \
>>>>>> >     >
>>>>>> >     >
>>>>>> >
>>>>>> dc-version="1.1.5-1.1.el5-01e86afaaa6d4a8c4836f68df80ababd6ca3902f"
>>>>>> >     >             \
>>>>>> >     >                     cluster-infrastructure="openais" \
>>>>>> >     >                     expected-quorum-votes="2" \
>>>>>> >     >                     stonith-enabled="false" \
>>>>>> >     >                     no-quorum-policy="ignore" \
>>>>>> >     >                     last-lrm-refresh="1332254494"
>>>>>> >     >             rsc_defaults $id="rsc-options" \
>>>>>> >     >                     resource-stickiness="100"
>>>>>> >     >
>>>>>> >     >
>>>>>> >     >             _______________________________________________
>>>>>> >     >             Pacemaker mailing list:
>>>>>> Pacemaker at oss.clusterlabs.org
>>>>>> >     <mailto:Pacemaker at oss.clusterlabs.org>
>>>>>> >     >             <mailto:Pacemaker at oss.clusterlabs.org
>>>>>> >     <mailto:Pacemaker at oss.clusterlabs.org>>
>>>>>> >     >
>>>>>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>>>>> >     >
>>>>>> >     >             Project Home: http://www.clusterlabs.org
>>>>>> >     >             Getting started:
>>>>>> >     >
>>>>>> http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>>>>>> >     >             Bugs: http://bugs.clusterlabs.org
>>>>>> >     >
>>>>>> >     >
>>>>>> >     >
>>>>>> >     >
>>>>>> >     >         --
>>>>>> >     >         esta es mi vida e me la vivo hasta que dios quiera
>>>>>> >     >
>>>>>> >     >
>>>>>> >     >
>>>>>> >     >
>>>>>> >     >     --
>>>>>> >     >     esta es mi vida e me la vivo hasta que dios quiera
>>>>>> >     >
>>>>>> >     >     _______________________________________________
>>>>>> >     >     Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
>>>>>> >     <mailto:Pacemaker at oss.clusterlabs.org>
>>>>>> >     >     <mailto:Pacemaker at oss.clusterlabs.org
>>>>>> >     <mailto:Pacemaker at oss.clusterlabs.org>>
>>>>>> >     >     http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>>>>> >     >
>>>>>> >     >     Project Home: http://www.clusterlabs.org
>>>>>> >     >     Getting started:
>>>>>> >     http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>>>>>> >     >     Bugs: http://bugs.clusterlabs.org
>>>>>> >     >
>>>>>> >     >
>>>>>> >     >
>>>>>> >     >
>>>>>> >     > _______________________________________________
>>>>>> >     > Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
>>>>>> >     <mailto:Pacemaker at oss.clusterlabs.org>
>>>>>> >     > http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>>>>> >     >
>>>>>> >     > Project Home: http://www.clusterlabs.org
>>>>>> >     > Getting started:
>>>>>> >     http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>>>>>> >     > Bugs: http://bugs.clusterlabs.org
>>>>>> >
>>>>>> >
>>>>>> >
>>>>>> >     _______________________________________________
>>>>>> >     Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
>>>>>> >     <mailto:Pacemaker at oss.clusterlabs.org>
>>>>>> >     http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>>>>> >
>>>>>> >     Project Home: http://www.clusterlabs.org
>>>>>> >     Getting started:
>>>>>> http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>>>>>> >     Bugs: http://bugs.clusterlabs.org
>>>>>> >
>>>>>> >
>>>>>> >
>>>>>> >
>>>>>> > _______________________________________________
>>>>>> > Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
>>>>>> > http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>>>>> >
>>>>>> > Project Home: http://www.clusterlabs.org
>>>>>> > Getting started:
>>>>>> http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>>>>>> > Bugs: http://bugs.clusterlabs.org
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
>>>>>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>>>>>
>>>>>> Project Home: http://www.clusterlabs.org
>>>>>> Getting started:
>>>>>> http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>>>>>> Bugs: http://bugs.clusterlabs.org
>>>>>>
>>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
>>>>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>>>>
>>>>> Project Home: http://www.clusterlabs.org
>>>>> Getting started:
>>>>> http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>>>>> Bugs: http://bugs.clusterlabs.org
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> esta es mi vida e me la vivo hasta que dios quiera
>>>>
>>>> _______________________________________________
>>>> Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
>>>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>>>
>>>> Project Home: http://www.clusterlabs.org
>>>> Getting started:
>>>> http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>>>> Bugs: http://bugs.clusterlabs.org
>>>>
>>>>
>>>
>>> _______________________________________________
>>> Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
>>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>>
>>> Project Home: http://www.clusterlabs.org
>>> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>>> Bugs: http://bugs.clusterlabs.org
>>>
>>>
>>
>>
>> --
>> esta es mi vida e me la vivo hasta que dios quiera
>>
>> _______________________________________________
>> Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>
>> Project Home: http://www.clusterlabs.org
>> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>> Bugs: http://bugs.clusterlabs.org
>>
>>
>
> _______________________________________________
> Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>
> Project Home: http://www.clusterlabs.org
> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> Bugs: http://bugs.clusterlabs.org
>
>


-- 
esta es mi vida e me la vivo hasta que dios quiera
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.clusterlabs.org/pipermail/pacemaker/attachments/20120323/d7537120/attachment.htm>


More information about the Pacemaker mailing list