[Pacemaker] Resource Scheduler Parameters

[btinsley]

On Tue, Apr 7, 2009 at 6:49 AM, btinsley <btinsley at gmail.com> wrote:

> On Tue, Apr 7, 2009 at 6:13 AM, btinsley <btinsley at gmail.com> wrote:
>
>> On Tue, Apr 7, 2009 at 4:44 AM, Andrew Beekhof <beekhof at gmail.com> wrote:
>>
>>> On Tue, Apr 7, 2009 at 11:39, Dejan Muhamedagic <dejanmm at fastmail.fm>
>>> wrote:
>>> > On Tue, Apr 07, 2009 at 08:07:27AM +0200, Andrew Beekhof wrote:
>>> >> Nothing in pacemaker - but the lrmd could be doing something that may
>>> >> be involved
>>> >
>>> > AFAIK, it doesn't.
>>>
>>> Ok, I was just thinking that its the one that ultimately spawns the
>>> process... so anything it did would also be inherited by the client.
>>>
>>>  btinsley: Perhaps look at /proc/<pid>/sched for the lrmd and aisexec
>>> processes.  That may give you an idea of where the settings are coming
>>> from.
>>>
>>> >
>>> >>
>>> >> On Tue, Apr 7, 2009 at 02:21, btinsley <btinsley at gmail.com> wrote:
>>> >> > This is a little early in investigation on my end, but is there
>>> anything
>>> >> > that Pacemaker... or potentially OpenAIS, does that would restrict a
>>> cluster
>>> >> > resource from setting the scheduler type and/or priority? I have
>>> been
>>> >> > tinkering with KVM instances and there is a 100% repeatable
>>> difference
>>> >> > between how it behaves when Pacemaker starts the resource and when I
>>> start
>>> >> > it via the same OCF script on the command line (as the root user).
>>> When it
>>> >> > is started via Pacemaker, it seems to be unable to change its
>>> scheduler and
>>> >> > priority and eventually the whole system is brought to its knees and
>>> all
>>> >> > monitors get stuck in the "waiting on I/O" (D) state, which freaks
>>> Pacemaker
>>> >> > out...rightfully so ;-)
>>> >> >
>>> >> > Looking at /proc/<pid>/sched shows:
>>> >> >
>>> >> > ...
>>> >> > policy???????????????????????????? :??????????????????? 0
>>> >> > prio?????????????????????????????? :??????????????????? 0
>>> >> > ...
>>> >> >
>>> >> > And invoking the resource from the command line shows:
>>> >> >
>>> >> > ...
>>> >> > policy???????????????????????????? :??????????????????? 2
>>> >> > prio?????????????????????????????? :??????????????????? 120
>>> >> > ...
>>> >> >
>>> >> > Invoking ulimit with the -e and -r parameters with a value of
>>> "unlimited" in
>>> >> > the OCF script does no good. Thoughts?
>>> >> >
>>> >> >
>>>
>>
>>
>> Thanks! I will do that this morning and go from there.
>>
>>
>>
> OK, aisexec and all Pacemaker processes have policy 2 (SCHED_RR) and
> priority 0. All cluster resources have the same values (I think I inverted
> the policy lines in the paste above). I also looked at other non-clustered
> processes on the system (syslog-ng, sshd, etc) and they all have policy 0
> (SCHED_OTHER) and priority 120, which is what the KVM process appears to
> need to function properly. I will post to the AIS list and see what they say
> since it looks like tweaking the scheduler parameters begins there.
>
>

AIS guys said to upgrade to the latest Whitetank :-)  I did and the behavior
is the same, but it's not necessarily incorrect. The aisexec process sets
itself to the realtime scheduling class, which does the same for all of the
Pacemaker processes when they are spawned. This is probably how you want the
cluster daemons to run. However, when lrmd spawns resource scripts
*everything* the script does also inherits the realtime scheduling class.
I'm not sure this is how you want all your clustered applications running
(or all the other stuff a resource script may do). Thoughts here?

As a workaround, I added calls to the chrt program in each resource script
to "downgrade" the scheduler to SCHED_OTHER and set the priority to zero,
which is the system default.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.clusterlabs.org/pipermail/pacemaker/attachments/20090408/0f7960c6/attachment-0002.htm>