[ClusterLabs] [ClusterLabs Developers] Resource Agent language discussion

[Fabio M. Di Nitto]

On 8/11/2015 11:01 AM, Jehan-Guillaume de Rorthais wrote:
> On Tue, 11 Aug 2015 06:42:37 +0200
> "Fabio M. Di Nitto" <fabbione at fabbione.net> wrote: 
>> On 8/7/2015 5:14 PM, Jehan-Guillaume de Rorthais wrote:
>>> Hi Jan,
>>>
>>> On Fri, 7 Aug 2015 15:36:57 +0200
>>> Jan Pokorný <jpokorny at redhat.com> wrote:
>>>
>>>> On 07/08/15 12:09 +0200, Jehan-Guillaume de Rorthais wrote:
>>>>> Now, I would like to discuss about the language used to write a RA in
>>>>> Pacemaker. I never seen discussion or page about this so far.
>>>>
>>>> it wasn't in such a "heretic :)" tone, but I tried to show that it
>>>> is extremely hard (if not impossible in some instances) to write
>>>> bullet-proof code in bash (or POSIX shell, for that matter) because
>>>> it's so cumbersome to move from "whitespace-delimited words as
>>>> a single argument" and "words as standalone arguments" back and forth,
>>>> connected with quotation-desired/-counterproductive madness
>>>> (what if one wants to indeed pass quotation marks as legitimate
>>>> characters within the passed value, etc.) few months back:
>>>>
>>>> http://clusterlabs.org/pipermail/users/2015-May/000403.html
>>>> (even on developers list, but with fewer replies and broken threading:
>>>> http://clusterlabs.org/pipermail/developers/2015-May/000023.html).
>>>
>>> Thanks for the links and history. You add some more argument to my points :)
>>>
>>>>> HINT: I don't want to discuss (neither troll about) what is the best
>>>>> language. I would like to know why **ALL** the RA are written in
>>>>> bash
>>>>
>>>> I would expect the original influence were the init scripts (as RAs
>>>> are mostly just enriched variants to support more flexible
>>>> configuration and better diagnostics back to the cluster stack),
>>>> which in turn were born having simplicity and ease of debugging
>>>> (maintainability) in mind.
>>>
>>> That sounds legitimate. And bash is still appropriate for some simple RA.
>>>
>>> But for the same ease of code debugging and maintainability arguments (and
>>> many others), complexe RA shouldn't use shell as language.
>>
>> so beside the language you can/want to use, from a development
>> perspective you guys are probably right that in some cases, more complex
>> languages could be a better fit.
>>
>> But you forgot to position yourself as end user and the reason why we
>> currently use bash/shell.
>>
>> first of all, our end user is not necessarily a developer. Most of them
>> are in fact sysadmins and one common that sysadmins have is that they
>> know bash/shell.
> 
> I understand that. However, most sysadmins know the basics of bash, maybe some
> advanced bash, to interact with the system. But how many of them are actually
> doing proper **development** in bash? Use arrays, escape parameters, protects
> against unwanted globing, use substitutions instead of
> cut/sed/awk/grep/whatever, check their return code, declare "typed" variable,
> mess with their scope, ...?
> 
> Sysadmins are happy with bash for simple tasks/scripts or just doing sysadmin.
> When it comes to development, they turn to perl or python nowadays. If they
> want to open and debug a RA, this is not some sysadmin anymore, this is
> development. At least, I would expect published RA to be well tested and
> "production" ready, not having trivial bug easy to fix.
> 
>> If needs arise to debug a RA, shell is pretty much the only common
>> denominator with our user base.
> 
> But as a sysadmin who tries to write robust bash script and as a PostgreSQL DBA,
> if I open the pgsql RA agent, I become a bit nervous. Try to run shellcheck on
> the pgsql and mysql RA.
> 
> Proper and robust bash development often require to use syntaxes, features
> and cautions a lot of people will not be familiar with. Worst, sometime you
> must stick outside of good practices: consider some optional args to a
> command you must not quote as instance... It just becomes quickly a headache.
> 
> So all-in-all, as a sysadmin, watching at some RA code (not all of them) doesn't
> give me much more confidence about them. And reading the discussions pointed by
> Jan earlier in this thread confirm this feeling.
> 
>> The other problem i see in using other languages is how they operate
>> under extreme conditions (memory pressure, disk I/O etc).
>>
>> Just for the fun of it, 
> [...]
> 
>> Granted, it´s an incredibly small test et all, but all I am trying to
>> say is that Cluster is supposed to be as reliable as possible under
>> extreme conditions.
> 
> Ok, I understand this argument. Bash is reliable and lighter, so better fit for
> extreme condition. But then, I would expect the project to provide a C
> library as well. I'm not kidding, I understand bash is a good compromise. But
> as far as I can read the code and debug if needed, whatever the language.
> Again, this is development tasks.
> 
> If I pick the PostgreSQL project, the engine can be extended using SQL,
> PL/PgSQL, python, perl, C, and many other languages (even shell), to add
> operators, types, functions/procedures, background worker (C only), ... People
> extending or using external modules are responsible to test and validate them.
> Sometime, when such a module is really useful, they are swallowed in the
> PostgreSQL core officialy (xml, autovacuum, FTS(tsearch), ...). But at least,
> PostgreSQL provides API and libraries for these languages.
> 
> Considering robustness and extreme condition, database and clustering solutions
> share the same needs and attention. 
> 
> Note that 3 RA are using perl from bash for some more complex (though really
> simple) tasks: apache-conf.sh, nginx and pgsql. As long as perl has been loaded,
> why not writing the whole RA in perl? ;-)
> 
>> In most systems, all commands required to execute a RA in shell are
>> already cached in ram and requirements to re-run them are minimal (and
>> could save a system).
>>
>> with Perl, there was no caching that I could see (even executing the
>> command several times), with lots of I/O to load modules from disks.
> 
> If hitting 10MB on memory or disk is a problem on your server, your RA is
> probably not your main problem by this time.

That is a bad assumption that shouldn´t be done either directions.

We could argue the other way around :)

If loading 10MB of $fancy_language_script fails to stop a service that
is driving the server bad, vs loading 500k of shell that saves the node
to be fenced, then the language become a problem.

Fabio

> 
>> So given that, is it worth rewriting the RA in another language (and
>> what defines a "simple" vs "complex" ras from above)? or wouldn´t it
>> better to just fix the current ones for stuff like escapes and handling
>> of spaces in the options?
>