[Pacemaker] resources not migrating when some are not runnable on one node, maybe because of groups or master/slave clones?

[Andrew Beekhof]

On Mon, Jul 30, 2012 at 11:52 PM, Phil Frost <phil at macprofessionals.com> wrote:
> On 07/29/2012 11:15 PM, Andrew Beekhof wrote:
>>
>> If I run:
>>
>> tools/crm_simulate -x ~/Dropbox/phil.xml -Ss | grep "promotion score"
>>
>> I see:
>>
>> drbd_exports:1 promotion score on storage02: 110
>> drbd_exports:0 promotion score on storage01: 6
>>
>> The 100 coming from one of your rules which says:
>>
>>             <!--# storage02 is a much more capable machine, so prefer
>> that.-->
>>
>> So I'm not really understanding why you think we'd migrate everything
>> to storage01.
>
>
> Because of the colocation rules:
>
> (ip_storage vg_exports) -> drbd_exports_ms
> nfs_kernel_server_clone -> nfs_common_clone -> portmap_clone
> export_test -> fs_test -> vg_exports
> export_test -> nfs_kernel_server_clone

Ok, I see what you're saying now.

The problem is, we've already promoted drbd_exports_ms before it comes
time to allocate export_test to a node.
The allocation of drbd_exports_ms uses the colocation chain to get
hints as to where things that depend on it can/cant run, but at that
point export_test doesn't know enough to say "only storage01 because
thats the only possible location for nfs_kernel_server_clone!".

Sometimes what is obvious to us is not at all obvious to the PE,
because it doesn't work iteratively (and if it did, you might be
waiting minutes/hours for a result like other A* search algorithms).
Every design has its trade-offs, unfortunately you just hit one of ours.

Perhaps try a more linear colocation chain.
Or setting on-fail=standby for some of the clone operations.


>
> export_test must be on nfs_kernel_server_clone, and also the DRBD master
> (indirectly, through fs_test -> vg_exports -> drbd_exports_ms). But,
> storage02 can't run nfs_kernel_server_clone (location foo).
>
> I know the rules are complicated, but I couldn't find a simpler
> configuration that still showed the problem. They make sense if you consider
> the services they represent though:
>
> - to make an export, you need an NFS server and a filesystem mounted
> - to mount a filesystem, you need the volume group, and therefore need to be
> the DRBD master
>
>
>
> _______________________________________________
> Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>
> Project Home: http://www.clusterlabs.org
> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> Bugs: http://bugs.clusterlabs.org