[ClusterLabs] pacemaker remote configuration on ubuntu 14.04

Fri Mar 11 21:11:11 CET 2016

On 03/10/2016 11:36 PM, Сергей Филатов wrote:
> This one is the right log

Something in the cluster configuration and state (for example, an
unsatisfied constraint) is preventing the cluster from starting the
resource:

Mar 10 04:00:53 [11785] controller-1.domain.com    pengine:     info:
native_print:     compute-1       (ocf::pacemaker:remote):        Stopped
Mar 10 04:00:53 [11785] controller-1.domain.com    pengine:     info:
native_color:     Resource compute-1 cannot run anywhere


> 
> 
> 
>> On 10 Mar 2016, at 08:17, Сергей Филатов <filatecs at gmail.com 
>> <mailto:filatecs at gmail.com>> wrote:
>>
>> pcs resource show compute-1
>>
>>  Resource: compute-1 (class=ocf provider=pacemaker type=remote)
>>  Operations: monitor interval=60s (compute-1-monitor-interval-60s)
>>
>> Can’t find _start_0 template in pacemaker logs
>> I don’t have ipv6 address for remote node, but I guess it should be listening 
>> on both
>>
>> attached pacemaker.log for cluster node
>> <pacemaker.log.tar.gz>
>>
>>
>>> On 09 Mar 2016, at 10:23, Ken Gaillot <kgaillot at redhat.com 
>>> <mailto:kgaillot at redhat.com>> wrote:
>>>
>>> On 03/08/2016 11:38 PM, Сергей Филатов wrote:
>>>> ssh -p 3121 compute-1
>>>> ssh_exchange_identification: read: Connection reset by peer
>>>>
>>>> That’s what I get in /var/log/pacemaker.log after restarting pacemaker_remote:
>>>> Mar 09 05:30:27 [28031] compute-1.domain.com <http://compute-1.domain.com/> 
>>>>       lrmd:     info: crm_signal_dispatch:  Invoking handler for signal 15: 
>>>> Terminated
>>>> Mar 09 05:30:27 [28031] compute-1.domain.com <http://compute-1.domain.com/> 
>>>>       lrmd:     info: lrmd_shutdown:        Terminating with  0 clients
>>>> Mar 09 05:30:27 [28031] compute-1.domain.com <http://compute-1.domain.com/> 
>>>>       lrmd:     info: qb_ipcs_us_withdraw:  withdrawing server sockets
>>>> Mar 09 05:30:27 [28031] compute-1.domain.com <http://compute-1.domain.com/> 
>>>>       lrmd:     info: crm_xml_cleanup:      Cleaning up memory from libxml2
>>>> Mar 09 05:30:27 [28193] compute-1.domain.com <http://compute-1.domain.com/> 
>>>>       lrmd:     info: crm_log_init:         Changed active directory to 
>>>> /var/lib/heartbeat/cores/root
>>>> Mar 09 05:30:27 [28193] compute-1.domain.com <http://compute-1.domain.com/> 
>>>>       lrmd:     info: qb_ipcs_us_publish:   server name: lrmd
>>>> Mar 09 05:30:27 [28193] compute-1.domain.com <http://compute-1.domain.com/> 
>>>>       lrmd:   notice: lrmd_init_remote_tls_server:  Starting a tls listener 
>>>> on port 3121.
>>>> Mar 09 05:30:28 [28193] compute-1.domain.com <http://compute-1.domain.com/> 
>>>>       lrmd:   notice: bind_and_listen:      Listening on address ::
>>>> Mar 09 05:30:28 [28193] compute-1.domain.com <http://compute-1.domain.com/> 
>>>>       lrmd:     info: qb_ipcs_us_publish:   server name: cib_ro
>>>> Mar 09 05:30:28 [28193] compute-1.domain.com <http://compute-1.domain.com/> 
>>>>       lrmd:     info: qb_ipcs_us_publish:   server name: cib_rw
>>>> Mar 09 05:30:28 [28193] compute-1.domain.com <http://compute-1.domain.com/> 
>>>>       lrmd:     info: qb_ipcs_us_publish:   server name: cib_shm
>>>> Mar 09 05:30:28 [28193] compute-1.domain.com <http://compute-1.domain.com/> 
>>>>       lrmd:     info: qb_ipcs_us_publish:   server name: attrd
>>>> Mar 09 05:30:28 [28193] compute-1.domain.com <http://compute-1.domain.com/> 
>>>>       lrmd:     info: qb_ipcs_us_publish:   server name: stonith-ng
>>>> Mar 09 05:30:28 [28193] compute-1.domain.com <http://compute-1.domain.com/> 
>>>>       lrmd:     info: qb_ipcs_us_publish:   server name: crmd
>>>> Mar 09 05:30:28 [28193] compute-1.domain.com <http://compute-1.domain.com/> 
>>>>       lrmd:     info: main:         Starting
>>>
>>> It looks like the cluster is not even trying to connect to the remote
>>> node. pacemaker_remote here is binding only to IPv6, so the cluster will
>>> need to contact it on that address.
>>>
>>> What is your ocf:pacemaker:remote resource configuration?
>>>
>>> Check your cluster node logs for the start action -- if your resource is
>>> named R, the start action will be R_start_0. There will be two nodes of
>>> interest: the node assigned the remote node resource, and the DC.
>>>
>>>> I got only pacemaker-remote resource-agents pcs installed, so no 
>>>> /etc/default/pacemaker file on remote node
>>>> selinux is disabled and I specifically opened firewall on 2224, 3121 and 
>>>> 21064 tcp and 5405 udp
>>>>
>>>>> On 08 Mar 2016, at 08:51, Ken Gaillot <kgaillot at redhat.com 
>>>>> <mailto:kgaillot at redhat.com>> wrote:
>>>>>
>>>>> On 03/07/2016 09:10 PM, Сергей Филатов wrote:
>>>>>> Thanks for an answer. Turned out the problem was not in ipv6.
>>>>>> Remote node is listening on 3121 port and it’s name is resolving fine.
>>>>>> Got authkey file at /etc/pacemaker on both remote and cluster nodes.
>>>>>> What can I check in addition? Is there any walkthrough for ubuntu?
>>>>>
>>>>> Nothing specific to ubuntu, but there's not much distro-specific to it.
>>>>>
>>>>> If you "ssh -p 3121" to the remote node from a cluster node, what do you
>>>>> get?
>>>>>
>>>>> pacemaker_remote will use the usual log settings for pacemaker (probably
>>>>> /var/log/pacemaker.log, probably configured in /etc/default/pacemaker on
>>>>> ubuntu). You should see "New remote connection" in the remote node's log
>>>>> when the cluster tries to connect, and "LRMD client connection
>>>>> established" if it's successful.
>>>>>
>>>>> As always, check for firewall and SELinux issues.
>>>>>
>>>>>>
>>>>>>> On 07 Mar 2016, at 09:40, Ken Gaillot <kgaillot at redhat.com 
>>>>>>> <mailto:kgaillot at redhat.com>> wrote:
>>>>>>>
>>>>>>> On 03/06/2016 07:43 PM, Сергей Филатов wrote:
>>>>>>>> Hi,
>>>>>>>> I’m trying to set up pacemaker_remote resource on ubuntu 14.04
>>>>>>>> I followed "remote node walkthrough” guide 
>>>>>>>> (http://clusterlabs.org/doc/en-US/Pacemaker/1.1/html-single/Pacemaker_Remote/#idm140473081667280 
>>>>>>>> <http://clusterlabs.org/doc/en-US/Pacemaker/1.1/html-single/Pacemaker_Remote/#idm140473081667280>)
>>>>>>>> After creating ocf:pacemaker:remote resource on cluster node, remote 
>>>>>>>> node doesn’t show up as online.
>>>>>>>> I guess I need to configure remote agent to listen on ipv4, where can I 
>>>>>>>> configure it?
>>>>>>>> Or is there any other steps to set up remote node besides the ones 
>>>>>>>> mentioned in guide?
>>>>>>>> tcp6       0      0 :::3121                 :::* 
>>>>>>>>                    LISTEN      21620/pacemaker_rem off (0.00/0/0)
>>>>>>>>
>>>>>>>> pacemaker and pacemaker_remote are 1.12 version
>>>>>>>
>>>>>>>
>>>>>>> pacemaker_remote will try to bind to IPv6 addresses first, and only if
>>>>>>> that fails, will it bind to IPv4. There is no way to configure this
>>>>>>> behavior currently, though it obviously would be nice to have.
>>>>>>>
>>>>>>> The only workarounds I can think of are to make IPv6 connections work
>>>>>>> between the cluster and the remote node, or disable IPv6 on the remote
>>>>>>> node. Using IPv6, there could be an issue if your name resolution
>>>>>>> returns both IPv4 and IPv6 addresses for the remote host; you could
>>>>>>> potentially work around that by adding an IPv6-only name for it, and
>>>>>>> using that as the server option to the remote resource.
>>
>