[ClusterLabs] Antw: Re: [ClusterLabs Developers] Resource Agent language discussion

Tue Aug 11 08:56:11 UTC 2015

Hello!

Just adding my opinion. I think that the current interface to RAs with
environment variables is very good. It's similar to CGI which was hugely
successful due to it's flexibility.

As an RA implementation language, however, most classic shells and Bash in
particular certainly has its drawbacks. Lack of complex data structures,
difficulties performing more advanced arithmetics and handling errors
robustly are some of the problems which spring to mind.

I do understand Bash being the default choice, since it is a common
denominator for many POSIX-like systems. That said, I think it could be
useful to add some wrappers and examples for more traditional languages,
just like how CGI does it. Python springs to mind as a suitable language
for medium-sized RA:s: it's very popular, integrates well with it's
environment and has extensive tooling for testing.

I don't think performance really is an issue with RA:s. The scripts are
called very infrequently and aren't generally doing any heavy lifting.
Startup latency is the only thing which I can envision being a problem with
certain language choices.

Best regards,

Alexander

On Tue, Aug 11, 2015 at 8:30 AM, Ulrich Windl <
Ulrich.Windl at rz.uni-regensburg.de> wrote:

> >>> "Fabio M. Di Nitto" <fabbione at fabbione.net> schrieb am 11.08.2015 um
> 06:42
> in
> Nachricht <55C97D3D.5040601 at fabbione.net>:
>
> [...]
> > first of all, our end user is not necessarily a developer. Most of them
> > are in fact sysadmins and one common that sysadmins have is that they
> > know bash/shell.
>
> But actually you can make much more errors in a shell script than in a Perl
> (just  one language example) script. With "errors" I mean errors that
> aren't
> discovered until the script runs, and errors that are discovered during
> run (by
> the interpreter, not by the programmer).
>
> >
> > If needs arise to debug a RA, shell is pretty much the only common
> > denominator with our user base.
>
> Why? Debugging Perl is much easier than debugging a shell script. Or do you
> talk about "-x" to trace the commands?
>
> One of the biggest problems in shell is the "text substitution model" when
> using variables. Things depend very much what is in a variable (like
> backslashes quotes, spaces, etc.).
>
> >
> > The other problem i see in using other languages is how they operate
> > under extreme conditions (memory pressure, disk I/O etc).
>
> I agree that the old Bourne shell was quite robust under extreme
> conditions,
> but the shell is only the basic part. Every command invoked by the shell
> has to
> be robust also. And most important: The programmer has to take care about
> exceptional conditions. In my experience especially that is hardly ever
> done in
> shell scripts.
>
> >
> > Just for the fun of it, I did some basic profiling of "hello world" in
> > bash and perl. Please dońt take this as an attempt to start a
> > "language" flame war here. I just want to explain differences on why
> > shell vs others.
> >
> > Perl is at least 3 times slower than bash
> > Perl uses at least 4/5 times more memory to execute the same command
>
> Did you also comapre it with "echo 'hello world'"? ;-) I mean which RA
> consists of exaclty one line? If you read a directory with 500 entries,
> detecting all that are files, I guess Perl should be already faster than
> the
> comparable shell script, because Perl does not need to run external
> processes
> for most of the commands being used.
>
>
> >
> > Granted, it́s an incredibly small test et all, but all I am trying to
> > say is that Cluster is supposed to be as reliable as possible under
> > extreme conditions.
>
> So program RAs in C or anoter language that has almost no runtime that
> could
> be unreliable? Oh well, unreliable... I'd better stop here...
>
> >
> > In most systems, all commands required to execute a RA in shell are
> > already cached in ram and requirements to re-run them are minimal (and
> > could save a system).
>
> But still with shared libraries there is overhead to get the started as a
> process. (Having statically linked binaries would be different). Oh well,
> for
> your Perl example: Did you run it more than once?
>
> You'll also have to compare loading time vs. execution time. While simple
> scripts seem faster in shell, you'll have real trouble implementing any
> complex
> algorithm in a shell script.
>
> >
> > with Perl, there was no caching that I could see (even executing the
> > command several times), with lots of I/O to load modules from disks.
>
> That's not fair: You say the files for external commands and the shell is
> cached, but the files for Perl are not? Why should that be?
>
> I did a test myself (SLES11SP3 x86_64 in a Xen-PVM) (you did not post your
> test):
> ---
> v04:/tmp> echo "echo 'hello world'" >hi.sh
> v04:/tmp> chmod +x hi.sh
> v04:/tmp> echo 'print "hello world\n";' >hi.pl
> v04:/tmp> chmod +x hi.pl
> v04:/tmp> time sh hi.sh
> hello world
>
> real    0m0.011s
> user    0m0.000s
> sys     0m0.004s
> v04:/tmp> time perl hi.pl
> hello world
>
> real    0m0.009s
> user    0m0.004s
> sys     0m0.004s
> v04:/tmp> #again
> v04:/tmp> time sh hi.sh
> hello world
>
> real    0m0.007s
> user    0m0.000s
> sys     0m0.004s
> v04:/tmp> time perl hi.pl
> hello world
>
> real    0m0.007s
> user    0m0.004s
> sys     0m0.000s
> ---
> So Perl is NOT any slower than Shell in this simple case!
>
> >
> > So given that, is it worth rewriting the RA in another language (and
> > what defines a "simple" vs "complex" ras from above)? or wouldńt it
> > better to just fix the current ones for stuff like escapes and handling
> > of spaces in the options?
>
> I could not verify most of your arguments, sorry!
>
> >
> > Just 2c
> > Fabio
> [...]
>
>
> _______________________________________________
> Users mailing list: Users at clusterlabs.org
> http://clusterlabs.org/mailman/listinfo/users
>
> Project Home: http://www.clusterlabs.org
> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> Bugs: http://bugs.clusterlabs.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.clusterlabs.org/pipermail/users/attachments/20150811/f7976786/attachment.htm>