[ClusterLabs] [ClusterLabs Developers] Resource Agent language discussion

[Jehan-Guillaume de Rorthais]

On Tue, 11 Aug 2015 06:42:37 +0200
"Fabio M. Di Nitto" <fabbione at fabbione.net> wrote: 
>On 8/7/2015 5:14 PM, Jehan-Guillaume de Rorthais wrote:
>> Hi Jan,
>> 
>> On Fri, 7 Aug 2015 15:36:57 +0200
>> Jan Pokorný <jpokorny at redhat.com> wrote:
>> 
>>> On 07/08/15 12:09 +0200, Jehan-Guillaume de Rorthais wrote:
>>>> Now, I would like to discuss about the language used to write a RA in
>>>> Pacemaker. I never seen discussion or page about this so far.
>>>
>>> it wasn't in such a "heretic :)" tone, but I tried to show that it
>>> is extremely hard (if not impossible in some instances) to write
>>> bullet-proof code in bash (or POSIX shell, for that matter) because
>>> it's so cumbersome to move from "whitespace-delimited words as
>>> a single argument" and "words as standalone arguments" back and forth,
>>> connected with quotation-desired/-counterproductive madness
>>> (what if one wants to indeed pass quotation marks as legitimate
>>> characters within the passed value, etc.) few months back:
>>>
>>> http://clusterlabs.org/pipermail/users/2015-May/000403.html
>>> (even on developers list, but with fewer replies and broken threading:
>>> http://clusterlabs.org/pipermail/developers/2015-May/000023.html).
>> 
>> Thanks for the links and history. You add some more argument to my points :)
>> 
>>>> HINT: I don't want to discuss (neither troll about) what is the best
>>>> language. I would like to know why **ALL** the RA are written in
>>>> bash
>>>
>>> I would expect the original influence were the init scripts (as RAs
>>> are mostly just enriched variants to support more flexible
>>> configuration and better diagnostics back to the cluster stack),
>>> which in turn were born having simplicity and ease of debugging
>>> (maintainability) in mind.
>> 
>> That sounds legitimate. And bash is still appropriate for some simple RA.
>> 
>> But for the same ease of code debugging and maintainability arguments (and
>> many others), complexe RA shouldn't use shell as language.
>
> so beside the language you can/want to use, from a development
> perspective you guys are probably right that in some cases, more complex
> languages could be a better fit.
> 
> But you forgot to position yourself as end user and the reason why we
> currently use bash/shell.
> 
> first of all, our end user is not necessarily a developer. Most of them
> are in fact sysadmins and one common that sysadmins have is that they
> know bash/shell.

I understand that. However, most sysadmins know the basics of bash, maybe some
advanced bash, to interact with the system. But how many of them are actually
doing proper **development** in bash? Use arrays, escape parameters, protects
against unwanted globing, use substitutions instead of
cut/sed/awk/grep/whatever, check their return code, declare "typed" variable,
mess with their scope, ...?

Sysadmins are happy with bash for simple tasks/scripts or just doing sysadmin.
When it comes to development, they turn to perl or python nowadays. If they
want to open and debug a RA, this is not some sysadmin anymore, this is
development. At least, I would expect published RA to be well tested and
"production" ready, not having trivial bug easy to fix.

> If needs arise to debug a RA, shell is pretty much the only common
> denominator with our user base.

But as a sysadmin who tries to write robust bash script and as a PostgreSQL DBA,
if I open the pgsql RA agent, I become a bit nervous. Try to run shellcheck on
the pgsql and mysql RA.

Proper and robust bash development often require to use syntaxes, features
and cautions a lot of people will not be familiar with. Worst, sometime you
must stick outside of good practices: consider some optional args to a
command you must not quote as instance... It just becomes quickly a headache.

So all-in-all, as a sysadmin, watching at some RA code (not all of them) doesn't
give me much more confidence about them. And reading the discussions pointed by
Jan earlier in this thread confirm this feeling.

> The other problem i see in using other languages is how they operate
> under extreme conditions (memory pressure, disk I/O etc).
> 
> Just for the fun of it, 
[...]

> Granted, it´s an incredibly small test et all, but all I am trying to
> say is that Cluster is supposed to be as reliable as possible under
> extreme conditions.

Ok, I understand this argument. Bash is reliable and lighter, so better fit for
extreme condition. But then, I would expect the project to provide a C
library as well. I'm not kidding, I understand bash is a good compromise. But
as far as I can read the code and debug if needed, whatever the language.
Again, this is development tasks.

If I pick the PostgreSQL project, the engine can be extended using SQL,
PL/PgSQL, python, perl, C, and many other languages (even shell), to add
operators, types, functions/procedures, background worker (C only), ... People
extending or using external modules are responsible to test and validate them.
Sometime, when such a module is really useful, they are swallowed in the
PostgreSQL core officialy (xml, autovacuum, FTS(tsearch), ...). But at least,
PostgreSQL provides API and libraries for these languages.

Considering robustness and extreme condition, database and clustering solutions
share the same needs and attention. 

Note that 3 RA are using perl from bash for some more complex (though really
simple) tasks: apache-conf.sh, nginx and pgsql. As long as perl has been loaded,
why not writing the whole RA in perl? ;-)

> In most systems, all commands required to execute a RA in shell are
> already cached in ram and requirements to re-run them are minimal (and
> could save a system).
>
> with Perl, there was no caching that I could see (even executing the
> command several times), with lots of I/O to load modules from disks.

If hitting 10MB on memory or disk is a problem on your server, your RA is
probably not your main problem by this time.

> So given that, is it worth rewriting the RA in another language (and
> what defines a "simple" vs "complex" ras from above)? or wouldn´t it
> better to just fix the current ones for stuff like escapes and handling
> of spaces in the options?