[Pacemaker] Corosync using unicast instead of multicast

Mon Nov 8 16:07:40 UTC 2010

On 11/08/2010 05:50 AM, Dan Frincu wrote:
> Hi,
>
> Steven Dake wrote:
>> On 11/05/2010 01:30 AM, Dan Frincu wrote:
>>> Hi,
>>>
>>> Alan Jones wrote:
>>>> This question should be on the openais list, however, I happen to know
>>>> the answer.
>>>> To get up and running quickly you can configure broadcast with the
>>>> version you have.
>>>>
>>> I've done that already, however I was a little concerned as to what
>>> Steven Dake said on the openais mailing list about using broadcast
>>> "Broadcast and redundant ring probably don't work to well together.".
>>>
>>> I've also done some testing and saw that the broadcast address used is
>>> 255.255.255.255, regardless of what the bindnetaddr network address is,
>>> and quite frankly, I was hoping to see a directed broadcast address.
>>> This wasn't the case, therefore I wonder whether this was the issue that
>>> Steven was referring to, because by using the 255.255.255.255 as a
>>> broadcast address, there is the slight chance that some application
>>> running in the same network might send a broadcast packet using the same
>>
>> This can happen with multicast or unicast modes as well. If a third
>> party application communicates on the multicast/port combo or unicast
>> port of a cluster node, there is conflict.
>>
>> With encryption, corosync encrypts and authenticates all packets,
>> ignoring packets without a proper signature. The signatures are
>> difficult to spoof. Without encryption, bad things happen in this
>> condition.
>>
>> For more details, read "SECURITY" file in our source distribution.
>>
> OK, I read the SECURITY file, a lot of overhead is added, I understand
> the reasons why it does it this way, not going to go into the details
> right now. Basically enabling encryption ensures that any traffic going
> between the nodes is both encrypted and authenticated, so rogue messages
> that happen to reach the exact network socket will be discarded. I'll
> come back to this a little bit later.
>
> Then again, I have this sentence in my head that I can't seem to get rid
> of "Broadcast and redundant ring probably don't work to well together,
> broadcast and redundant ring probably don't work to well together...."
> and also I read "OpenAIS now provides broadcast network communication in
> addition to multicast. This functionality is considered Technology
> Preview for standalone usage of OpenAIS", therefore I'm a little bit
> more concerned.
>
> Can you shed some light on this please? Two questions:
>
> 1) What do you mean by "Broadcast and redundant ring probably don't work
> to well together"?
>

broadcast requires a specific port to run on.  As a result, the ports 
should be different for each interface.  I have not done any specific 
testing on broadcast with redundant ring - you would probably be the first.

> 2) Is using Corosync's broadcast feature instead of multicast stable
> enough to be used in production systems?
>

Personally I'd wait for 2.0 for this feature and use bonding for the moment.

> Thank you in advance.
>
> Best regards,
>
> Dan
>>> port as configured on the cluster. How would the cluster react to that,
>>> would it ignore the packet, would it wreak havoc?
>>>
>>> Regards,
>>>
>>> Dan
>>>
>>> That's my main concern right now.
>>>> Corosync can distinguish separate clusters with the multicast address
>>>> and port that become payload to the messages.
>>>> The patch you referred to can be applied to the top of tree for
>>>> corosync or you can wait for a new release 1.3.0 planned for the end
>>>> of November.
>>>> Alan
>>>>
>>>> On Thu, Nov 4, 2010 at 1:02 AM, Dan Frincu<dfrincu at streamwide.ro>
>>>> wrote:
>>>>
>>>>> Hi all,
>>>>>
>>>>> I'm having an issue with a setup using the following:
>>>>> cluster-glue-1.0.6-1.6.el5.x86_64.rpm
>>>>> cluster-glue-libs-1.0.6-1.6.el5.x86_64.rpm
>>>>> corosync-1.2.7-1.1.el5.x86_64.rpm
>>>>> corosynclib-1.2.7-1.1.el5.x86_64.rpm
>>>>> drbd83-8.3.2-6.el5_3.x86_64.rpm
>>>>> kmod-drbd83-8.3.2-6.el5_3.x86_64.rpm
>>>>> openais-1.1.3-1.6.el5.x86_64.rpm
>>>>> openaislib-1.1.3-1.6.el5.x86_64.rpm
>>>>> pacemaker-1.0.9.1-1.el5.x86_64.rpm
>>>>> pacemaker-libs-1.0.9.1-1.el5.x86_64.rpm
>>>>> resource-agents-1.0.3-2.el5.x86_64.rpm
>>>>>
>>>>> This is a two-node HA cluster, with the nodes interconnected via
>>>>> bonded
>>>>> interfaces through the switch. The issue is that I have no control
>>>>> of the
>>>>> switch itself, can't do anything about that, and from what I
>>>>> understand the
>>>>> environment doesn't allow enabling multicast on the switch. In this
>>>>> situation, how can I have the setup functional (with redundant rings,
>>>>> rrp_mode: active) without using multicast.
>>>>>
>>>>> I've seen that individual network sockets are formed between nodes,
>>>>> unicast
>>>>> sockets, as well as the multicast sockets. I'm interested in
>>>>> knowing how
>>>>> will the lack of multicast affect the redundant rings, connectivity,
>>>>> failover, etc.
>>>>>
>>>>> I've also seen this page
>>>>> https://lists.linux-foundation.org/pipermail/openais/2010-October/015271.html
>>>>>
>>>>> And here it states using UDPU transport mode avoids using multicast or
>>>>> broadcast, but it's a patch, is this integrated in any of the newer
>>>>> versions
>>>>> of corosync?
>>>>>
>>>>> Thank you in advance.
>>>>>
>>>>> Regards,
>>>>>
>>>>> Dan
>>>>>
>>>>> --
>>>>> Dan FRINCU
>>>>> Systems Engineer
>>>>> CCNA, RHCE
>>>>> Streamwide Romania
>>>>>
>>>>> _______________________________________________
>>>>> Pacemaker mailing list:Pacemaker at oss.clusterlabs.org
>>>>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>>>>
>>>>> Project Home:http://www.clusterlabs.org
>>>>> Getting
>>>>> started:http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>>>>> Bugs:
>>>>> http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>> _______________________________________________
>>>> Pacemaker mailing list:Pacemaker at oss.clusterlabs.org
>>>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>>>
>>>> Project Home:http://www.clusterlabs.org
>>>> Getting started:http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>>>> Bugs:http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker
>>>>
>>>>
>>>
>>> --
>>> Dan FRINCU
>>> Systems Engineer
>>> CCNA, RHCE
>>> Streamwide Romania
>>>
>>>
>>>
>>> _______________________________________________
>>> Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
>>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>>
>>> Project Home: http://www.clusterlabs.org
>>> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>>> Bugs:
>>> http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker
>>>
>>
>