[ClusterLabs] Antw: Re: Antw: Re: [Question] About movement of pacemaker_remote.
renayama19661014 at ybb.ne.jp
renayama19661014 at ybb.ne.jp
Tue Aug 4 09:40:13 UTC 2015
Hi Andrew,
> Do you know if this behaviour still exists?
> A LOT of work went into the remote node logic in the last couple of months, its
> possible this was fixed as a side-effect.
It is the latest and does not confirm it.
I confirm it.
Many Thanks!
Hideo Yamauchi.
----- Original Message -----
> From: Andrew Beekhof <andrew at beekhof.net>
> To: renayama19661014 at ybb.ne.jp; Cluster Labs - All topics related to open-source clustering welcomed <users at clusterlabs.org>
> Cc:
> Date: 2015/8/4, Tue 13:16
> Subject: Re: [ClusterLabs] Antw: Re: Antw: Re: [Question] About movement of pacemaker_remote.
>
>
>> On 12 May 2015, at 12:12 pm, renayama19661014 at ybb.ne.jp wrote:
>>
>> Hi All,
>>
>> The problem is like a buffer becoming NULL after crm_resouce -C practice
> somehow or other after having rebooted remote node.
>>
>> I incorporated log in a source code and confirmed it.
>>
>> ------------------------------------------------
>> crm_remote_recv_once(crm_remote_t * remote)
>> {
>> (snip)
>> /* automatically grow the buffer when needed */
>> if(remote->buffer_size < read_len) {
>> remote->buffer_size = 2 * read_len;
>> crm_trace("Expanding buffer to %u bytes",
> remote->buffer_size);
>>
>> remote->buffer = realloc_safe(remote->buffer,
> remote->buffer_size + 1);
>> CRM_ASSERT(remote->buffer != NULL);
>> }
>>
>> #ifdef HAVE_GNUTLS_GNUTLS_H
>> if (remote->tls_session) {
>> if (remote->buffer == NULL) {
>> crm_info("### YAMAUCHI buffer is NULL [buffer_zie[%d]
> readlen[%d]", remote->buffer_size, read_len);
>> }
>> rc = gnutls_record_recv(*(remote->tls_session),
>> remote->buffer +
> remote->buffer_offset,
>> remote->buffer_size -
> remote->buffer_offset);
>> (snip)
>> ------------------------------------------------
>>
>> May 12 10:54:01 sl7-01 crmd[30447]: info: crm_remote_recv_once: ###
> YAMAUCHI buffer is NULL [buffer_zie[1326] readlen[40]
>> May 12 10:54:02 sl7-01 crmd[30447]: info: crm_remote_recv_once: ###
> YAMAUCHI buffer is NULL [buffer_zie[1326] readlen[40]
>> May 12 10:54:04 sl7-01 crmd[30447]: info: crm_remote_recv_once: ###
> YAMAUCHI buffer is NULL [buffer_zie[1326] readlen[40]
>
> Do you know if this behaviour still exists?
> A LOT of work went into the remote node logic in the last couple of months, its
> possible this was fixed as a side-effect.
>
>>
>> ------------------------------------------------
>>
>> gnutls_record_recv processes an empty buffer and becomes the error.
>>
>> ------------------------------------------------
>> (snip)
>> ssize_t
>> _gnutls_recv_int(gnutls_session_t session, content_type_t type,
>> gnutls_handshake_description_t htype,
>> gnutls_packet_t *packet,
>> uint8_t * data, size_t data_size, void *seq,
>> unsigned int ms)
>> {
>> int ret;
>>
>> if (packet == NULL && (type != GNUTLS_ALERT && type !=
> GNUTLS_HEARTBEAT)
>> && (data_size == 0 || data == NULL))
>> return gnutls_assert_val(GNUTLS_E_INVALID_REQUEST);
>>
>> (sip)
>> ssize_t
>> gnutls_record_recv(gnutls_session_t session, void *data, size_t data_size)
>> {
>> return _gnutls_recv_int(session, GNUTLS_APPLICATION_DATA, -1, NULL,
>> data, data_size, NULL,
>> session->internals.record_timeout_ms);
>> }
>> (snip)
>> ------------------------------------------------
>>
>> Best Regards,
>> Hideo Yamauchi.
>>
>>
>>
>> ----- Original Message -----
>>> From: "renayama19661014 at ybb.ne.jp"
> <renayama19661014 at ybb.ne.jp>
>>> To: "users at clusterlabs.org" <users at clusterlabs.org>
>>> Cc:
>>> Date: 2015/5/11, Mon 16:45
>>> Subject: Re: [ClusterLabs] Antw: Re: Antw: Re: [Question] About
> movement of pacemaker_remote.
>>>
>>> Hi Ulrich,
>>>
>>> Thank you for comments.
>>>
>>>> So your host and you resource are both named "snmp1"? I
> also
>>> don't
>>>> have much experience with cleaning up resources for a node that is
> offline.
>>> What
>>>> change should it make (while the node is offline)?
>>>
>>>
>>> The name of the remote resource and the name of the remote node make
> same
>>> "snmp1".
>>>
>>>
>>> (snip)
>>> primitive snmp1 ocf:pacemaker:remote \
>>> params \
>>> server="snmp1" \
>>> op start interval="0s" timeout="60s"
>>> on-fail="ignore" \
>>> op monitor interval="3s" timeout="15s"
> \
>>> op stop interval="0s" timeout="60s"
>>> on-fail="ignore"
>>>
>>> primitive Host-rsc1 ocf:heartbeat:Dummy \
>>> op start interval="0s" timeout="60s"
>>> on-fail="restart" \
>>> op monitor interval="10s" timeout="60s"
>>> on-fail="restart" \
>>> op stop interval="0s" timeout="60s"
>>> on-fail="ignore"
>>>
>>> primitive Remote-rsc1 ocf:heartbeat:Dummy \
>>> op start interval="0s" timeout="60s"
>>> on-fail="restart" \
>>> op monitor interval="10s" timeout="60s"
>>> on-fail="restart" \
>>> op stop interval="0s" timeout="60s"
>>> on-fail="ignore"
>>>
>>> location loc1 Remote-rsc1 \
>>> rule 200: #uname eq snmp1
>>> location loc3 Host-rsc1 \
>>> rule 200: #uname eq bl460g8n1
>>> (snip)
>>>
>>> The pacemaker_remoted of the snmp1 node stops in SIGTERM.
>>> I reboot pacemaker_remoted of the snmp1 node afterwards.
>>> And I execute crm_resource command, but the snmp1 node remains
> off-line.
>>>
>>> After having executed crm_resource command, the remote node thinks that
> it is
>>> right movement to become the snmp1 online.
>>>
>>>
>>>
>>> Best Regards,
>>> Hideo Yamauchi.
>>>
>>>
>>>
>>>
>>>
>>> ----- Original Message -----
>>>> From: Ulrich Windl <Ulrich.Windl at rz.uni-regensburg.de>
>>>> To: users at clusterlabs.org; renayama19661014 at ybb.ne.jp
>>>> Cc:
>>>> Date: 2015/5/11, Mon 15:39
>>>> Subject: Antw: Re: [ClusterLabs] Antw: Re: [Question] About
> movement of
>>> pacemaker_remote.
>>>>
>>>>>>> <renayama19661014 at ybb.ne.jp> schrieb am
> 11.05.2015 um
>>> 06:22
>>>> in Nachricht
>>>> <361916.15877.qm at web200006.mail.kks.yahoo.co.jp>:
>>>>> Hi All,
>>>>>
>>>>> I matched the OS version of the remote node with a host once
> again and
>>>
>>>>> confirmed it in Pacemaker1.1.13-rc2.
>>>>>
>>>>> It was the same even if I made a host RHEL7.1.(bl460g8n1)
>>>>> I made the remote host RHEL7.1.(snmp1)
>>>>>
>>>>> The first crm_resource -C fails.
>>>>> --------------------------------
>>>>> [root at bl460g8n1 ~]# crm_resource -C -r snmp1
>>>>> Cleaning up snmp1 on bl460g8n1
>>>>> Waiting for 1 replies from the CRMd. OK
>>>>>
>>>>> [root at bl460g8n1 ~]# crm_mon -1 -Af
>>>>> Last updated: Mon May 11 12:44:31 2015
>>>>> Last change: Mon May 11 12:43:30 2015
>>>>> Stack: corosync
>>>>> Current DC: bl460g8n1 - partition WITHOUT quorum
>>>>> Version: 1.1.12-7a2e3ae
>>>>> 2 Nodes configured
>>>>> 3 Resources configured
>>>>>
>>>>>
>>>>> Online: [ bl460g8n1 ]
>>>>> RemoteOFFLINE: [ snmp1 ]
>>>>
>>>> So your host and you resource are both named "snmp1"? I
> also
>>> don't
>>>> have much experience with cleaning up resources for a node that is
> offline.
>>> What
>>>> change should it make (while the node is offline)?
>>>>
>>>>>
>>>>> Host-rsc1 (ocf::heartbeat:Dummy): Started bl460g8n1
>>>>> Remote-rsc1 (ocf::heartbeat:Dummy): Started bl460g8n1
> (failure
>>> ignored)
>>>>>
>>>>> Node Attributes:
>>>>> * Node bl460g8n1:
>>>>> + ringnumber_0 : 192.168.101.21 is UP
>>>>> + ringnumber_1 : 192.168.102.21 is UP
>>>>>
>>>>> Migration summary:
>>>>> * Node bl460g8n1:
>>>>> snmp1: migration-threshold=1 fail-count=1000000
>>> last-failure='Mon
>>>> May 11
>>>>> 12:44:28 2015'
>>>>>
>>>>> Failed actions:
>>>>> snmp1_start_0 on bl460g8n1 'unknown error' (1):
> call=5,
>>>> status=Timed
>>>>> Out, exit-reason='none', last-rc-change='Mon May
> 11
>>> 12:43:31
>>>> 2015', queued=0ms,
>>>>> exec=0ms
>>>>> --------------------------------
>>>>>
>>>>>
>>>>> The second crm_resource -C succeeded and was connected to the
> remote
>>> host.
>>>>
>>>> Then the node was online it seems.
>>>>
>>>> Regards,
>>>> Ulrich
>>>>
>>>>> --------------------------------
>>>>> [root at bl460g8n1 ~]# crm_mon -1 -Af
>>>>> Last updated: Mon May 11 12:44:54 2015
>>>>> Last change: Mon May 11 12:44:48 2015
>>>>> Stack: corosync
>>>>> Current DC: bl460g8n1 - partition WITHOUT quorum
>>>>> Version: 1.1.12-7a2e3ae
>>>>> 2 Nodes configured
>>>>> 3 Resources configured
>>>>>
>>>>>
>>>>> Online: [ bl460g8n1 ]
>>>>> RemoteOnline: [ snmp1 ]
>>>>>
>>>>> Host-rsc1 (ocf::heartbeat:Dummy): Started bl460g8n1
>>>>> Remote-rsc1 (ocf::heartbeat:Dummy): Started snmp1
>>>>> snmp1 (ocf::pacemaker:remote): Started bl460g8n1
>>>>>
>>>>> Node Attributes:
>>>>> * Node bl460g8n1:
>>>>> + ringnumber_0 : 192.168.101.21 is UP
>>>>> + ringnumber_1 : 192.168.102.21 is UP
>>>>> * Node snmp1:
>>>>>
>>>>> Migration summary:
>>>>> * Node bl460g8n1:
>>>>> * Node snmp1:
>>>>> --------------------------------
>>>>>
>>>>> The gnutls of a host and the remote node was the next
> version.
>>>>>
>>>>> gnutls-devel-3.3.8-12.el7.x86_64
>>>>> gnutls-dane-3.3.8-12.el7.x86_64
>>>>> gnutls-c++-3.3.8-12.el7.x86_64
>>>>> gnutls-3.3.8-12.el7.x86_64
>>>>> gnutls-utils-3.3.8-12.el7.x86_64
>>>>>
>>>>>
>>>>> Best Regards,
>>>>> Hideo Yamauchi.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> ----- Original Message -----
>>>>>> From: "renayama19661014 at ybb.ne.jp"
>>>> <renayama19661014 at ybb.ne.jp>
>>>>>> To: Cluster Labs - All topics related to open-source
> clustering
>>>> welcomed
>>>>> <users at clusterlabs.org>
>>>>>> Cc:
>>>>>> Date: 2015/4/28, Tue 14:06
>>>>>> Subject: Re: [ClusterLabs] Antw: Re: [Question] About
> movement of
>>>>> pacemaker_remote.
>>>>>>
>>>>>> Hi David,
>>>>>>
>>>>>> Even if the result changed the remote node to RHEL7.1, it
> was the
>>> same.
>>>>>>
>>>>>>
>>>>>> I try it with a host node of pacemaker as RHEL7.1 this
> time.
>>>>>>
>>>>>>
>>>>>> I noticed an interesting phenomenon.
>>>>>> The remote node fails in a reconnection in the first
> crm_resource.
>>>>>> However, the remote node succeeds in a reconnection in
> the second
>>>>> crm_resource.
>>>>>>
>>>>>> I think that I have some problem with the point where I
> cut the
>>>> connection
>>>>> with
>>>>>> the remote node first.
>>>>>>
>>>>>> Best Regards,
>>>>>> Hideo Yamauchi.
>>>>>>
>>>>>>
>>>>>> ----- Original Message -----
>>>>>>> From: "renayama19661014 at ybb.ne.jp"
>>>>>> <renayama19661014 at ybb.ne.jp>
>>>>>>> To: Cluster Labs - All topics related to open-source
>>> clustering
>>>> welcomed
>>>>>> <users at clusterlabs.org>
>>>>>>> Cc:
>>>>>>> Date: 2015/4/28, Tue 11:52
>>>>>>> Subject: Re: [ClusterLabs] Antw: Re: [Question] About
>
>>> movement of
>>>>>> pacemaker_remote.
>>>>>>>
>>>>>>> Hi David,
>>>>>>> Thank you for comments.
>>>>>>>> At first glance this looks gnutls related.
> GNUTLS is
>>>> returning -50
>>>>>> during
>>>>>>> receive
>>>>>>>
>>>>>>>> on the client side (pacemaker's side). -50
> maps to
>>>> 'invalid
>>>>>>> request'. >debug: crm_remote_recv_once:
> TLS
>>> receive
>>>> failed: The
>>>>>>> request is invalid. >We treat this error as fatal
> and
>>> destroy
>>>> the
>>>>>> connection.
>>>>>>> I've never encountered
>>>>>>>> this error and I don't know what causes it.
> It's
>>>> possible
>>>>>>> there's a bug in
>>>>>>>> our gnutls usage... it's also possible
> there's a
>>> bug
>>>> in the
>>>>>> version
>>>>>>> of gnutls
>>>>>>>> that is in use as well.
>>>>>>> We built the remote node in RHEL6.5.
>>>>>>> Because it may be a problem of gnutls, I confirm it
> in
>>> RHEL7.1.
>>>>>>>
>>>>>>> Best Regards,
>>>>>>> Hideo Yamauchi.
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Users mailing list: Users at clusterlabs.org
>>>>>>> http://clusterlabs.org/mailman/listinfo/users
>>>>>>>
>>>>>>> Project Home: http://www.clusterlabs.org
>>>>>>> Getting started:
>>>> http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>>>>>>> Bugs: http://bugs.clusterlabs.org
>>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Users mailing list: Users at clusterlabs.org
>>>>>> http://clusterlabs.org/mailman/listinfo/users
>>>>>>
>>>>>> Project Home: http://www.clusterlabs.org
>>>>>> Getting started:
>>>> http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>>>>>> Bugs: http://bugs.clusterlabs.org
>>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Users mailing list: Users at clusterlabs.org
>>>>> http://clusterlabs.org/mailman/listinfo/users
>>>>>
>>>>> Project Home: http://www.clusterlabs.org
>>>>> Getting started:
>>> http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>>>>> Bugs: http://bugs.clusterlabs.org
>>>>
>>>
>>> _______________________________________________
>>> Users mailing list: Users at clusterlabs.org
>>> http://clusterlabs.org/mailman/listinfo/users
>>>
>>> Project Home: http://www.clusterlabs.org
>>> Getting started:
> http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>>> Bugs: http://bugs.clusterlabs.org
>>>
>>
>> _______________________________________________
>> Users mailing list: Users at clusterlabs.org
>> http://clusterlabs.org/mailman/listinfo/users
>>
>> Project Home: http://www.clusterlabs.org
>> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>> Bugs: http://bugs.clusterlabs.org
>
More information about the Users
mailing list