[Pacemaker] stonith - using multiple fencing devices for one node to fence device with redundant power sources

Fri Oct 4 08:43:56 UTC 2013

Hi Guys,

thanks a lot for the tip, fencing_topology seems to be exactly what I
need! However, there seems to be the problem, I'm not sure whether
it's me, pacemaker or stonith agent..

I've set 4 stonith primitives, as per document:

primitive stonith-vbox3-1-off stonith:fence_netio \
	  params ipaddr="10.76.6.13" login="admin" passwd="admin" port="1" pcmk_host_list="vbox4" verbose="1" debug="/tmp/stonith-1-off.log" power_wait="1" action="off"
primitive stonith-vbox3-1-on stonith:fence_netio \
	  params ipaddr="10.76.6.13" login="admin" passwd="admin" port="1" pcmk_host_list="vbox4" verbose="1" debug="/tmp/stonith-1-on.log" power_wait="1" action="on"
primitive stonith-vbox3-2-off stonith:fence_netio \
	  params ipaddr="10.76.6.12" login="admin" passwd="admin" port="2" pcmk_host_list="vbox4" verbose="1" debug="/tmp/stonith-2-off.log" power_wait="1" action="off"
primitive stonith-vbox3-2-on stonith:fence_netio \
	  params ipaddr="10.76.6.12" login="admin" passwd="admin" port="2" pcmk_host_list="vbox4" verbose="1" debug="/tmp/stonith-2-on.log" power_wait="1" action="on"

one ON and one OFF for each PDU.

then I set fencing topology as follows:

fencing_topology vbox4: stonith-vbox3-1-off,stonith-vbox3-2-off,stonith-vbox3-1-on,stonith-vbox3-2-on

(btw my crmsh complaints about syntax here, but it seems to put in into CIB anyways,
so more on this maybe later)

then when I kill the node, thus firing up the node fencing operation, following happens:

(log snippet)

Oct  4 10:11:53 vbox3 stonith-ng[4170]:   notice: initiate_remote_stonith_op: Initiating remote operation reboot for vbox4: 0c54accc-6c1c-49dc-be0f-7bbe00fdb917 (0)
Oct  4 10:11:56 vbox3 stonith-ng[4170]:   notice: log_operation: Operation 'reboot' [18469] (call 0 from crmd.4174) for host 'vbox4' with device 'stonith-vbox3-1-off' returned: 0 (OK)
Oct  4 10:11:56 vbox3 stonith-ng[4170]:   notice: process_remote_stonith_exec: Call to stonith-vbox3-1-off for vbox4 on behalf of crmd.4174 at vbox3: passed (0)
Oct  4 10:11:58 vbox3 stonith-ng[4170]:   notice: log_operation: Operation 'reboot' [18527] (call 0 from crmd.4174) for host 'vbox4' with device 'stonith-vbox3-2-off' returned: 0 (OK)
Oct  4 10:11:58 vbox3 stonith-ng[4170]:   notice: process_remote_stonith_exec: Call to stonith-vbox3-2-off for vbox4 on behalf of crmd.4174 at vbox3: passed (0)
Oct  4 10:12:01 vbox3 stonith-ng[4170]:   notice: log_operation: Operation 'reboot' [18533] (call 0 from crmd.4174) for host 'vbox4' with device 'stonith-vbox3-1-on' returned: 0 (OK)
Oct  4 10:12:01 vbox3 stonith-ng[4170]:   notice: process_remote_stonith_exec: Call to stonith-vbox3-1-on for vbox4 on behalf of crmd.4174 at vbox3: passed (0)
Oct  4 10:12:04 vbox3 stonith-ng[4170]:   notice: log_operation: Operation 'reboot' [18539] (call 0 from crmd.4174) for host 'vbox4' with device 'stonith-vbox3-2-on' returned: 0 (OK)
Oct  4 10:12:04 vbox3 stonith-ng[4170]:   notice: process_remote_stonith_exec: Call to stonith-vbox3-2-on for vbox4 on behalf of crmd.4174 at vbox3: passed (0)

the order looks good, but according to stonith agent debug logs, reboot operation is always executed
(exactly P1: off,on P2: off,on, P1: off,on P2: off,on)
 instead of P1: off P2: off P1: on P2: on!

before I try digging deeper into this, does somebody of You idea on where the problem could be?

Does "Operation 'reboot' [18539] (call 0 from crmd.4174) for host 'vbox4' with device 'stonith-vbox3-2-on'"
mean that reboot action is always executed? Than would certainly be the problem, but why is that,
if I have different actions defined? Is this a bug, or some my fault?

thanks a lot in advance!

with best regards

nik

On Fri, Oct 04, 2013 at 10:18:56AM +0200, Lars Marowsky-Bree wrote:
> On 2013-10-03T23:50:15, Digimer <lists at alteeve.ca> wrote:
> 
> > > digimer's hack works, but it makes my eyes bleed. ;-)
> > meanie!
> 
> That's not because of what you diligently debugged and described,
> though, but because it's necessary. In my opinion, 90%+ of all setups
> that actually need to use more than one device per level will need your
> document, and that is quite complex to force on users.
> 
> (Just imagine that for multiple nodes that share power switches!)
> 
> 
> Regards,
>     Lars
> 
> -- 
> Architect Storage/HA
> SUSE LINUX Products GmbH, GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer, HRB 21284 (AG Nürnberg)
> "Experience is the name everyone gives to their mistakes." -- Oscar Wilde
> 
> 
> _______________________________________________
> Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
> 
> Project Home: http://www.clusterlabs.org
> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> Bugs: http://bugs.clusterlabs.org
> 

-- 
-------------------------------------
Ing. Nikola CIPRICH
LinuxBox.cz, s.r.o.
28.rijna 168, 709 00 Ostrava

tel.:   +420 591 166 214
fax:    +420 596 621 273
mobil:  +420 777 093 799
www.linuxbox.cz

mobil servis: +420 737 238 656
email servis: servis at linuxbox.cz
-------------------------------------
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: <https://lists.clusterlabs.org/pipermail/pacemaker/attachments/20131004/a677d3ef/attachment-0004.sig>