Programming languages startup times - 2013 roundup

I just revised the study I did a year ago about programming languages startup times. It all started because I was writing some small script that would be frequently invoked and I wanted to know how did the startup times of Bash and Perl compare against each other. The results were not at all what I expected and I extended the investigation to other languages. The main conclusion for me was that Bash and Perl had very similar startup times, which let me stick with Perl, much to my delight.

That post received some attention this week due to my refering to it in another blog, which made me want to repeat it to see if anything has changed in the meantime and to do it a little bit more properly. Also, I got some feedback and suggestions to extend it even further. So, in order to make it easier for me to repeat it and, perhaps, to incent people to replicate it in other platforms and with other languages, I've written a simple script called startup-times to automate the benchmark process.

The script is written in Perl (you guessed it!) and uses the Benchmark module to calculate the timings. This time I investigated 12 programming languages, two compiled (C and Java) and 10 interpreted. Running it on my laptop, which is still the same I used a year ago, a Dell Latitude E6410, now running Lubuntu 13.10, I got this:

$ ./startup-times
Bash: GNU bash, versão 4.2.45(1)-release (x86_64-pc-linux-gnu)
  timethis for 1: 10.1873 wallclock secs ( 0.08 usr +  0.92 sys =  1.00 CPU) @ 3840.00/s (n=3840)

C: gcc (Ubuntu/Linaro 4.7.3-1ubuntu1) 4.7.3
  timethis for 1: 5.09504 wallclock secs ( 0.09 usr +  1.04 sys =  1.13 CPU) @ 3964.60/s (n=4480)

Java: javac 1.7.0_25
  timethis for 1: 304.648 wallclock secs ( 0.14 usr  1.05 sys + 246.21 cusr 52.60 csys = 300.00 CPU) @ 12.80/s (n=3840)

JavaSun: javac 1.7.0_25
  timethis for 1: 208.54 wallclock secs ( 0.11 usr  0.96 sys + 159.18 cusr 44.00 csys = 204.25 CPU) @ 17.55/s (n=3584)

Ksh:   version         sh (AT&T Research) 93u+ 2012-08-01
  timethis for 1: 9.63142 wallclock secs ( 0.07 usr +  1.02 sys =  1.09 CPU) @ 3793.58/s (n=4135)

Lua: Lua 5.2
  timethis for 1: 7.12142 wallclock secs ( 0.12 usr +  0.98 sys =  1.10 CPU) @ 3258.18/s (n=3584)

PHP: PHP 5.4.9-4ubuntu2.2 (cli) (built: Jul 15 2013 18:23:35)
  timethis for 1: 44.1422 wallclock secs ( 0.03 usr  1.07 sys + 23.97 cusr 13.64 csys = 38.71 CPU) @ 86.80/s (n=3360)

Perl: This is perl 5, version 14, subversion 2 (v5.14.2) built for x86_64-linux-gnu-thread-multi
  timethis for 1: 11.7166 wallclock secs ( 0.09 usr +  1.05 sys =  1.14 CPU) @ 3627.19/s (n=4135)

Python: Python 2.7.4
  timethis for 1: 55.0902 wallclock secs ( 0.12 usr  1.01 sys + 31.30 cusr 15.82 csys = 48.25 CPU) @ 69.64/s (n=3360)

Ruby: ruby 1.9.3p194 (2012-04-20 revision 35410) [x86_64-linux]
  timethis for 1: 68.0358 wallclock secs ( 0.02 usr  1.08 sys + 45.19 cusr 13.79 csys = 60.08 CPU) @ 63.91/s (n=3840)

TCL: TCL 8.5
  timethis for 1: 18.4099 wallclock secs ( 0.17 usr  0.88 sys +  5.37 cusr  6.38 csys = 12.80 CPU) @ 233.28/s (n=2986)

Tcsh: tcsh 6.18.01 (Astron) 2012-02-14 (x86_64-unknown-linux) options wide,nls,dl,al,kan,rh,nd,color,filec
  timethis for 1: 26.4094 wallclock secs ( 0.11 usr  0.91 sys +  6.70 cusr  6.76 csys = 14.48 CPU) @ 231.98/s (n=3359)

Zsh: zsh 5.0.0 (x86_64-unknown-linux-gnu)
  timethis for 1: 15.1896 wallclock secs ( 0.11 usr  0.99 sys +  0.50 cusr  0.82 csys =  2.42 CPU) @ 1586.78/s (n=3840)


LANGUAGE   CALLS/s  NULL(ms)     SCORE
       C   879.286     1.137     1.000
     Lua   503.271     1.987     1.747
     Ksh   429.324     2.329     2.048
    Bash   376.941     2.653     2.333
    Perl   352.917     2.834     2.491
     Zsh   252.805     3.956     3.478
     TCL   162.196     6.165     5.421
    Tcsh   127.190     7.862     6.913
     PHP    76.118    13.138    11.552
  Python    60.991    16.396    14.417
    Ruby    56.441    17.718    15.579
 JavaSun    17.186    58.186    51.163
    Java    12.605    79.335    69.759

I think a graph makes some things clearer.



There are a few things to notice. The first one is that Lua beat all other interpreted languages. Rob Hoelz urged me to include it, already predicting this. I'm embarrassed to confess that I don't know much about Lua, even though it's a language with roots in Brazil.

All shells (ksh, bash, zsh, and tcsh) have good and comparable startup times. Among the heavier scripting languages just Lua, Perl, and TCL are in the same ballpark. I've left Tcsh out of the green group because it's the slowest and nobody should program in csh, anyway.

I've put PHP, Python, and Ruby in the yellow group. Their median startup time is six times higher than the green group median. So, for instance, in terms of performance alone for small and frequently used scripts this means that you can get six times more bang for buck with Perl than with Python or Ruby. :-)

Java is another story. I even tried two different JDKs: the OpenJDK that comes with Ubuntu and the SunOracle JDK to see how much they differ. Not much. Both crawl in comparison with all other languages. There seems to be a fair amount of discussion about this "problem". Even in academia. But I couldn't find a solution. It seems that Java simply isn't cut for this particular niche of programming.