[ClusterLabs] [ClusterLabs Developers] Resource Agent language discussion

Tue Aug 11 09:40:55 UTC 2015

I'm butting in here by saying that I don't think that the performance
discussion is very useful. Performance can always be improved, and by that
logic everything should be hand-written in assembly. Correctness and
maintainability should be of much higher importance. Premature
optimization.... Besides, Bash isn't exactly the fastest choice out there.

Best regards

Alexander

On Tue, Aug 11, 2015 at 11:15 AM, Fabio M. Di Nitto <fabbione at fabbione.net>
wrote:

>
>
> On 8/11/2015 11:01 AM, Jehan-Guillaume de Rorthais wrote:
> > On Tue, 11 Aug 2015 06:42:37 +0200
> > "Fabio M. Di Nitto" <fabbione at fabbione.net> wrote:
> >> On 8/7/2015 5:14 PM, Jehan-Guillaume de Rorthais wrote:
> >>> Hi Jan,
> >>>
> >>> On Fri, 7 Aug 2015 15:36:57 +0200
> >>> Jan Pokorný <jpokorny at redhat.com> wrote:
> >>>
> >>>> On 07/08/15 12:09 +0200, Jehan-Guillaume de Rorthais wrote:
> >>>>> Now, I would like to discuss about the language used to write a RA in
> >>>>> Pacemaker. I never seen discussion or page about this so far.
> >>>>
> >>>> it wasn't in such a "heretic :)" tone, but I tried to show that it
> >>>> is extremely hard (if not impossible in some instances) to write
> >>>> bullet-proof code in bash (or POSIX shell, for that matter) because
> >>>> it's so cumbersome to move from "whitespace-delimited words as
> >>>> a single argument" and "words as standalone arguments" back and forth,
> >>>> connected with quotation-desired/-counterproductive madness
> >>>> (what if one wants to indeed pass quotation marks as legitimate
> >>>> characters within the passed value, etc.) few months back:
> >>>>
> >>>> http://clusterlabs.org/pipermail/users/2015-May/000403.html
> >>>> (even on developers list, but with fewer replies and broken threading:
> >>>> http://clusterlabs.org/pipermail/developers/2015-May/000023.html).
> >>>
> >>> Thanks for the links and history. You add some more argument to my
> points :)
> >>>
> >>>>> HINT: I don't want to discuss (neither troll about) what is the best
> >>>>> language. I would like to know why **ALL** the RA are written in
> >>>>> bash
> >>>>
> >>>> I would expect the original influence were the init scripts (as RAs
> >>>> are mostly just enriched variants to support more flexible
> >>>> configuration and better diagnostics back to the cluster stack),
> >>>> which in turn were born having simplicity and ease of debugging
> >>>> (maintainability) in mind.
> >>>
> >>> That sounds legitimate. And bash is still appropriate for some simple
> RA.
> >>>
> >>> But for the same ease of code debugging and maintainability arguments
> (and
> >>> many others), complexe RA shouldn't use shell as language.
> >>
> >> so beside the language you can/want to use, from a development
> >> perspective you guys are probably right that in some cases, more complex
> >> languages could be a better fit.
> >>
> >> But you forgot to position yourself as end user and the reason why we
> >> currently use bash/shell.
> >>
> >> first of all, our end user is not necessarily a developer. Most of them
> >> are in fact sysadmins and one common that sysadmins have is that they
> >> know bash/shell.
> >
> > I understand that. However, most sysadmins know the basics of bash,
> maybe some
> > advanced bash, to interact with the system. But how many of them are
> actually
> > doing proper **development** in bash? Use arrays, escape parameters,
> protects
> > against unwanted globing, use substitutions instead of
> > cut/sed/awk/grep/whatever, check their return code, declare "typed"
> variable,
> > mess with their scope, ...?
> >
> > Sysadmins are happy with bash for simple tasks/scripts or just doing
> sysadmin.
> > When it comes to development, they turn to perl or python nowadays. If
> they
> > want to open and debug a RA, this is not some sysadmin anymore, this is
> > development. At least, I would expect published RA to be well tested and
> > "production" ready, not having trivial bug easy to fix.
> >
> >> If needs arise to debug a RA, shell is pretty much the only common
> >> denominator with our user base.
> >
> > But as a sysadmin who tries to write robust bash script and as a
> PostgreSQL DBA,
> > if I open the pgsql RA agent, I become a bit nervous. Try to run
> shellcheck on
> > the pgsql and mysql RA.
> >
> > Proper and robust bash development often require to use syntaxes,
> features
> > and cautions a lot of people will not be familiar with. Worst, sometime
> you
> > must stick outside of good practices: consider some optional args to a
> > command you must not quote as instance... It just becomes quickly a
> headache.
> >
> > So all-in-all, as a sysadmin, watching at some RA code (not all of them)
> doesn't
> > give me much more confidence about them. And reading the discussions
> pointed by
> > Jan earlier in this thread confirm this feeling.
> >
> >> The other problem i see in using other languages is how they operate
> >> under extreme conditions (memory pressure, disk I/O etc).
> >>
> >> Just for the fun of it,
> > [...]
> >
> >> Granted, it´s an incredibly small test et all, but all I am trying to
> >> say is that Cluster is supposed to be as reliable as possible under
> >> extreme conditions.
> >
> > Ok, I understand this argument. Bash is reliable and lighter, so better
> fit for
> > extreme condition. But then, I would expect the project to provide a C
> > library as well. I'm not kidding, I understand bash is a good
> compromise. But
> > as far as I can read the code and debug if needed, whatever the language.
> > Again, this is development tasks.
> >
> > If I pick the PostgreSQL project, the engine can be extended using SQL,
> > PL/PgSQL, python, perl, C, and many other languages (even shell), to add
> > operators, types, functions/procedures, background worker (C only), ...
> People
> > extending or using external modules are responsible to test and validate
> them.
> > Sometime, when such a module is really useful, they are swallowed in the
> > PostgreSQL core officialy (xml, autovacuum, FTS(tsearch), ...). But at
> least,
> > PostgreSQL provides API and libraries for these languages.
> >
> > Considering robustness and extreme condition, database and clustering
> solutions
> > share the same needs and attention.
> >
> > Note that 3 RA are using perl from bash for some more complex (though
> really
> > simple) tasks: apache-conf.sh, nginx and pgsql. As long as perl has been
> loaded,
> > why not writing the whole RA in perl? ;-)
> >
> >> In most systems, all commands required to execute a RA in shell are
> >> already cached in ram and requirements to re-run them are minimal (and
> >> could save a system).
> >>
> >> with Perl, there was no caching that I could see (even executing the
> >> command several times), with lots of I/O to load modules from disks.
> >
> > If hitting 10MB on memory or disk is a problem on your server, your RA is
> > probably not your main problem by this time.
>
> That is a bad assumption that shouldn´t be done either directions.
>
> We could argue the other way around :)
>
> If loading 10MB of $fancy_language_script fails to stop a service that
> is driving the server bad, vs loading 500k of shell that saves the node
> to be fenced, then the language become a problem.
>
> Fabio
>
> >
> >> So given that, is it worth rewriting the RA in another language (and
> >> what defines a "simple" vs "complex" ras from above)? or wouldn´t it
> >> better to just fix the current ones for stuff like escapes and handling
> >> of spaces in the options?
> >
>
> _______________________________________________
> Users mailing list: Users at clusterlabs.org
> http://clusterlabs.org/mailman/listinfo/users
>
> Project Home: http://www.clusterlabs.org
> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> Bugs: http://bugs.clusterlabs.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.clusterlabs.org/pipermail/users/attachments/20150811/0607d679/attachment.htm>