<br><br><div class="gmail_quote">2011/6/8 Dejan Muhamedagic <span dir="ltr"><<a href="mailto:dejanmm@fastmail.fm">dejanmm@fastmail.fm</a>></span><br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<div><div></div><div class="h5">On Tue, Jun 07, 2011 at 11:19:25AM -0600, Serge Dubrouski wrote:<br>
> On Tue, Jun 7, 2011 at 9:55 AM, Dejan Muhamedagic <<a href="mailto:dejanmm@fastmail.fm">dejanmm@fastmail.fm</a>>wrote:<br>
><br>
> > On Tue, Jun 07, 2011 at 09:47:17AM -0600, Serge Dubrouski wrote:<br>
> > > On Tue, Jun 7, 2011 at 9:39 AM, Dejan Muhamedagic <<a href="mailto:dejanmm@fastmail.fm">dejanmm@fastmail.fm</a><br>
> > >wrote:<br>
> > ><br>
> > > > Hi,<br>
> > > ><br>
> > > > On Tue, Jun 07, 2011 at 08:45:07AM -0600, Serge Dubrouski wrote:<br>
> > > > > No, RA acts like it should. It can't find necessary software and<br>
> > returns<br>
> > > > > OCF_NOT_CONFIGURED, all RAs act this way. You have to install all<br>
> > > > software<br>
> > > > > used in you cluster on all nodes even if you are not actually<br>
> > planning to<br>
> > > > > run that software on some of them.<br>
> > > ><br>
> > > > This is not so. A resource agents should be able to figure out if<br>
> > > > there's no software installed and then return NOT_INSTALLED or<br>
> > > > NOT_RUNNING.<br>
> > > ><br>
> > ><br>
> > > That changes exit code but doesn't change the requirements to have that<br>
> > > software installed and able to report that it's down.<br>
> ><br>
> > If it's not installed, then it cannot run. The difference between<br>
> > NOT_INSTALLED and NOT_CONFIGURED is that in the former case<br>
> > pacemaker won't try to start the resource on that node whereas in<br>
> > the latter it will give up on the resource completely.<br>
> ><br>
><br>
> Thanks for clarifications. Now it better explains Ruslan's case. DRBD isn't<br>
> installed so it returns NOT_INSTALLED and that's treated as definitely DOWN<br>
> by Pacemaker when it runs status/monitor operations. iSCSI in it's turn is<br>
> installed but not configured and that's treated as state is UNKNOWN. Am I<br>
> correct?<br>
<br>
</div></div>No idea, he didn't provide enough information (logs) :)<br>
<br>
Thanks,<br>
<font color="#888888"><br>
Dejan<br>
</font><div><div></div><div class="h5"><br>
> ><br>
> > Cheers,<br>
> ><br>
> > Dejan<br>
> ><br>
> > > > Thanks,<br>
> > > ><br>
> > > > Dejan<br>
> > > ><br>
> > > > > On Tue, Jun 7, 2011 at 8:02 AM, ruslan usifov <<br>
> > <a href="mailto:ruslan.usifov@gmail.com">ruslan.usifov@gmail.com</a><br>
> > > > >wrote:<br>
> > > > ><br>
> > > > > > Thanks fo replay i undestend already this moment. Now i think that<br>
> > this<br>
> > > > is<br>
> > > > > > problem of ocf:heartbeat:iSCSITarget RA, which return unproperly<br>
> > return<br>
> > > > > > code, when no any iscssi target software installed<br>
> > > > > ><br>
> > > > > > 2011/6/7 Serge Dubrouski <<a href="mailto:sergeyfd@gmail.com">sergeyfd@gmail.com</a>><br>
> > > > > ><br>
> > > > > >> This questions pops up over and over again. Pacemaker has to make<br>
> > sure<br>
> > > > > >> that your resources aren't up anywhere in the cluster before start<br>
> > > > them up<br>
> > > > > >> on designated nodes. That means that it has to be able to run<br>
> > > > status/monitor<br>
> > > > > >> operations for all configured resources on all configured nodes.<br>
> > You<br>
> > > > can't<br>
> > > > > >> just add a 3rd quorum node into cluster you have to make sure that<br>
> > all<br>
> > > > RAs<br>
> > > > > >> that you use can run on that 3rd node properly.<br>
> > > > > >><br>
> > > > > >> On Tue, Jun 7, 2011 at 1:58 AM, ruslan usifov <<br>
> > > > <a href="mailto:ruslan.usifov@gmail.com">ruslan.usifov@gmail.com</a>>wrote:<br>
> > > > > >><br>
> > > > > >>> Hello<br>
> > > > > >>><br>
> > > > > >>> I have 3 node cluster (in future we add another one node) with<br>
> > follow<br>
> > > > > >>> configuration:<br>
> > > > > >>><br>
> > > > > >>> crm(live)configure# show<br>
> > > > > >>> node drbd1<br>
> > > > > >>> node drbd2<br>
> > > > > >>> node drbd3<br>
> > > > > >>> primitive drbd_web ocf:linbit:drbd \<br>
> > > > > >>> params drbd_resource="web" \<br>
> > > > > >>> op monitor interval="10s" timeout="60s"<br>
> > > > > >>> primitive drbd_web-U ocf:linbit:drbd \<br>
> > > > > >>> params drbd_resource="web-U" \<br>
> > > > > >>> op monitor interval="10s" timeout="60s"<br>
> > > > > >>> primitive iscsi_ip_web ocf:heartbeat:IPaddr2 \<br>
> > > > > >>> params ip="192.168.19.91" nic="eth1:1" cidr_netmask="24"<br>
> > > > > >>> primitive iscsi_web_target ocf:heartbeat:iSCSITarget \<br>
> > > > > >>> params iqn="iqn.2010-06.playrix.local:san.web" \<br>
> > > > > >>> op monitor interval="10s" timeout="30s"<br>
> > > > > >>> primitive iscsi_web_target_lun0 ocf:heartbeat:iSCSILogicalUnit \<br>
> > > > > >>> params lun="0" path="/dev/drbd10"<br>
> > > > > >>> target_iqn="iqn.2010-06.playrix.local:san.web"<br>
> > > > > >>> group iscsi_web iscsi_ip_web iscsi_web_target<br>
> > iscsi_web_target_lun0<br>
> > > > > >>> ms ms_drbd_web drbd_web \<br>
> > > > > >>> meta master-max="1" master-node-max="1" clone-max="2"<br>
> > > > > >>> clone-node-max="1" notify="true" globally-unique="false"<br>
> > > > > >>> target-role="Started" is-managed="true"<br>
> > > > > >>> ms ms_drbd_web-U drbd_web-U \<br>
> > > > > >>> meta master-max="1" master-node-max="1" clone-max="1"<br>
> > > > > >>> clone-node-max="1" notify="true" is-managed="true"<br>
> > > > globally-unique="false"<br>
> > > > > >>> location ms_drbd_web-U_on_drbd1_or_drbd2 ms_drbd_web-U \<br>
> > > > > >>> rule $id="ms_drbd_web-U_on_drbd1_or_drbd2-rule" -inf:<br>
> > #uname<br>
> > > > ne<br>
> > > > > >>> drbd1 and #uname ne drbd2<br>
> > > > > >>> location ms_drbd_web_on_drbd1_or_drbd2 ms_drbd_web \<br>
> > > > > >>> rule $id="ms_drbd_web_on_drbd1_or_drbd2-rule" -inf:<br>
> > #uname ne<br>
> > > > > >>> drbd1 and #uname ne drbd2<br>
> > > > > >>> colocation drbd_web-U_on_drbd_web inf: ms_drbd_web-U:Master<br>
> > > > > >>> ms_drbd_web:Master<br>
> > > > > >>> colocation iscsi_ip_web_on_drbd_web inf: iscsi_ip_web<br>
> > > > ms_drbd_web:Master<br>
> > > > > >>> colocation iscsi_web_on_drbd_web-U inf: iscsi_web<br>
> > > > ms_drbd_web-U:Master<br>
> > > > > >>> order iscsi_web_after_ms_drbd_web-U inf: ms_drbd_web-U:start<br>
> > > > iscsi_web<br>
> > > > > >>> order ms_drbd_web-U_after_iscsi_ip_web inf: iscsi_ip_web:start<br>
> > > > > >>> ms_drbd_web-U:start<br>
> > > > > >>> order ms_drbd_web-U_before_ms_drbd_web inf: ms_drbd_web:promote<br>
> > > > > >>> iscsi_ip_web:start<br>
> > > > > >>> property $id="cib-bootstrap-options" \<br>
> > > > > >>><br>
> > dc-version="1.0.11-db98485d06ed3fe0fe236509f023e1bd4a5566f1"<br>
> > > > \<br>
> > > > > >>> cluster-infrastructure="openais" \<br>
> > > > > >>> expected-quorum-votes="3" \<br>
> > > > > >>> stonith-enabled="false" \<br>
> > > > > >>> last-lrm-refresh="1307432239" \<br>
> > > > > >>> symmetric-cluster="true"<br>
> > > > > >>><br>
> > > > > >>><br>
> > > > > >>><br>
> > > > > >>> In this configuration i want that all resources ran only on drbd1<br>
> > and<br>
> > > > > >>> drb2 nodes. And as i understand wit location constraint i should<br>
> > > > reach this<br>
> > > > > >>> objective. And all resource mast run on drbd1 and drbb2 nodes.<br>
> > But i<br>
> > > > got<br>
> > > > > >>> follow error:<br>
> > > > > >>><br>
> > > > > >>> Failed actions:<br>
> > > > > >>> iscsi_web_target_monitor_0 (node=drbd3, call=5, rc=6,<br>
> > > > > >>> status=complete): not configured<br>
> > > > > >>> iscsi_web_target_lun0_monitor_0 (node=drbd3, call=6, rc=6,<br>
> > > > > >>> status=complete): not configured<br>
> > > > > >>><br>
> > > > > >>><br>
> > > > > >>><br>
> > > > > >>> And i confused why drbd3? There is nothing must run o monitored.<br>
> > > > Please<br>
> > > > > >>> if it is possible explain this behavior<br>
> > > > > >>><br>
> > > > > >>><br>
> > > > > >>><br>
> > > > > >>> _______________________________________________<br>
> > > > > >>> Pacemaker mailing list: <a href="mailto:Pacemaker@oss.clusterlabs.org">Pacemaker@oss.clusterlabs.org</a><br>
> > > > > >>> <a href="http://oss.clusterlabs.org/mailman/listinfo/pacemaker" target="_blank">http://oss.clusterlabs.org/mailman/listinfo/pacemaker</a><br>
> > > > > >>><br>
> > > > > >>> Project Home: <a href="http://www.clusterlabs.org" target="_blank">http://www.clusterlabs.org</a><br>
> > > > > >>> Getting started:<br>
> > > > <a href="http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf" target="_blank">http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf</a><br>
> > > > > >>> Bugs:<br>
> > > > > >>><br>
> > > ><br>
> > <a href="http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker" target="_blank">http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker</a><br>
> > > > > >>><br>
> > > > > >>><br>
> > > > > >><br>
> > > > > >><br>
> > > > > >> --<br>
> > > > > >> Serge Dubrouski.<br>
> > > > > >><br>
> > > > > >> _______________________________________________<br>
> > > > > >> Pacemaker mailing list: <a href="mailto:Pacemaker@oss.clusterlabs.org">Pacemaker@oss.clusterlabs.org</a><br>
> > > > > >> <a href="http://oss.clusterlabs.org/mailman/listinfo/pacemaker" target="_blank">http://oss.clusterlabs.org/mailman/listinfo/pacemaker</a><br>
> > > > > >><br>
> > > > > >> Project Home: <a href="http://www.clusterlabs.org" target="_blank">http://www.clusterlabs.org</a><br>
> > > > > >> Getting started:<br>
> > > > <a href="http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf" target="_blank">http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf</a><br>
> > > > > >> Bugs:<br>
> > > > > >><br>
> > > ><br>
> > <a href="http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker" target="_blank">http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker</a><br>
> > > > > >><br>
> > > > > >><br>
> > > > > ><br>
> > > > > > _______________________________________________<br>
> > > > > > Pacemaker mailing list: <a href="mailto:Pacemaker@oss.clusterlabs.org">Pacemaker@oss.clusterlabs.org</a><br>
> > > > > > <a href="http://oss.clusterlabs.org/mailman/listinfo/pacemaker" target="_blank">http://oss.clusterlabs.org/mailman/listinfo/pacemaker</a><br>
> > > > > ><br>
> > > > > > Project Home: <a href="http://www.clusterlabs.org" target="_blank">http://www.clusterlabs.org</a><br>
> > > > > > Getting started:<br>
> > > > <a href="http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf" target="_blank">http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf</a><br>
> > > > > > Bugs:<br>
> > > > > ><br>
> > > ><br>
> > <a href="http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker" target="_blank">http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker</a><br>
> > > > > ><br>
> > > > > ><br>
> > > > ><br>
> > > > ><br>
> > > > > --<br>
> > > > > Serge Dubrouski.<br>
> > > ><br>
> > > > > _______________________________________________<br>
> > > > > Pacemaker mailing list: <a href="mailto:Pacemaker@oss.clusterlabs.org">Pacemaker@oss.clusterlabs.org</a><br>
> > > > > <a href="http://oss.clusterlabs.org/mailman/listinfo/pacemaker" target="_blank">http://oss.clusterlabs.org/mailman/listinfo/pacemaker</a><br>
> > > > ><br>
> > > > > Project Home: <a href="http://www.clusterlabs.org" target="_blank">http://www.clusterlabs.org</a><br>
> > > > > Getting started:<br>
> > <a href="http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf" target="_blank">http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf</a><br>
> > > > > Bugs:<br>
> > > ><br>
> > <a href="http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker" target="_blank">http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker</a><br>
> > > ><br>
> > > ><br>
> > > > _______________________________________________<br>
> > > > Pacemaker mailing list: <a href="mailto:Pacemaker@oss.clusterlabs.org">Pacemaker@oss.clusterlabs.org</a><br>
> > > > <a href="http://oss.clusterlabs.org/mailman/listinfo/pacemaker" target="_blank">http://oss.clusterlabs.org/mailman/listinfo/pacemaker</a><br>
> > > ><br>
> > > > Project Home: <a href="http://www.clusterlabs.org" target="_blank">http://www.clusterlabs.org</a><br>
> > > > Getting started:<br>
> > <a href="http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf" target="_blank">http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf</a><br>
> > > > Bugs:<br>
> > > ><br>
> > <a href="http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker" target="_blank">http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker</a><br>
> > > ><br>
> > ><br>
> > ><br>
> > ><br>
> > > --<br>
> > > Serge Dubrouski.<br>
> ><br>
> > > _______________________________________________<br>
> > > Pacemaker mailing list: <a href="mailto:Pacemaker@oss.clusterlabs.org">Pacemaker@oss.clusterlabs.org</a><br>
> > > <a href="http://oss.clusterlabs.org/mailman/listinfo/pacemaker" target="_blank">http://oss.clusterlabs.org/mailman/listinfo/pacemaker</a><br>
> > ><br>
> > > Project Home: <a href="http://www.clusterlabs.org" target="_blank">http://www.clusterlabs.org</a><br>
> > > Getting started: <a href="http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf" target="_blank">http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf</a><br>
> > > Bugs:<br>
> > <a href="http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker" target="_blank">http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker</a><br>
> ><br>
> ><br>
> > _______________________________________________<br>
> > Pacemaker mailing list: <a href="mailto:Pacemaker@oss.clusterlabs.org">Pacemaker@oss.clusterlabs.org</a><br>
> > <a href="http://oss.clusterlabs.org/mailman/listinfo/pacemaker" target="_blank">http://oss.clusterlabs.org/mailman/listinfo/pacemaker</a><br>
> ><br>
> > Project Home: <a href="http://www.clusterlabs.org" target="_blank">http://www.clusterlabs.org</a><br>
> > Getting started: <a href="http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf" target="_blank">http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf</a><br>
> > Bugs:<br>
> > <a href="http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker" target="_blank">http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker</a><br>
> ><br>
><br>
><br>
><br>
> --<br>
> Serge Dubrouski.<br>
<br>
> _______________________________________________<br>
> Pacemaker mailing list: <a href="mailto:Pacemaker@oss.clusterlabs.org">Pacemaker@oss.clusterlabs.org</a><br>
> <a href="http://oss.clusterlabs.org/mailman/listinfo/pacemaker" target="_blank">http://oss.clusterlabs.org/mailman/listinfo/pacemaker</a><br>
><br>
> Project Home: <a href="http://www.clusterlabs.org" target="_blank">http://www.clusterlabs.org</a><br>
> Getting started: <a href="http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf" target="_blank">http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf</a><br>
> Bugs: <a href="http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker" target="_blank">http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker</a><br>
<br>
<br>
_______________________________________________<br>
Pacemaker mailing list: <a href="mailto:Pacemaker@oss.clusterlabs.org">Pacemaker@oss.clusterlabs.org</a><br>
<a href="http://oss.clusterlabs.org/mailman/listinfo/pacemaker" target="_blank">http://oss.clusterlabs.org/mailman/listinfo/pacemaker</a><br>
<br>
Project Home: <a href="http://www.clusterlabs.org" target="_blank">http://www.clusterlabs.org</a><br>
Getting started: <a href="http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf" target="_blank">http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf</a><br>
Bugs: <a href="http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker" target="_blank">http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker</a><br>
</div></div></blockquote></div><br><br>Here was in my logs on drbd3(where iscsitarget doesn't installed)<br><br><br>Jun 07 11:33:10 drbd3 lrmd: [2391]: info: RA output: (iscsi_web_target:probe:stderr) 2011/06/07_11:33:10 ERROR: Unsupported iSCSI target implementation ""!<br>
Jun 07 11:33:11 drbd3 lrmd: [2391]: info: RA output: (iscsi_web_target_lun0:probe:stderr) 2011/06/07_11:33:11 ERROR: Missing resource parameter "implementation"!<br><br>