« July 16, 2014 | Main | July 27, 2014 »

Thursday, July 24, 2014

Floating Point Benchmark: Lua Language Added

I have posted an update to my trigonometry-intense floating point benchmark which adds Lua to the list of languages in which the benchmark is implemented. A new release of the benchmark collection including Lua is now available for downloading.

Lua was developed with the intention of being a small-footprint scripting language which could be easily embedded in applications. Despite this design goal, which it has achieved superbly, being widely adopted as the means of extensibility for numerous games and applications, it is a remarkably sophisticated language, with support for floating point, complex data structures, object oriented programming, and functional programming. It is a modern realisation of what I attempted to achieve with Atlast in 1990, but with a syntax which most programmers will find familiar and a completely memory-safe architecture (unless compromised by user extensions). If I were developing an application for which I needed scripting or user extensibility, Lua would be my tool of choice, and in porting the benchmark to the language I encountered no problems whatsoever—indeed, it worked the first time.

The relative performance of the various language implementations (with C taken as 1) is as follows. All language implementations of the benchmark listed below produced identical results to the last (11th) decimal place.

Language Relative
Time		 Details
C 1 GCC 3.2.3 -O3, Linux
Visual Basic .NET 0.866 All optimisations, Windows XP
FORTRAN 1.008 GNU Fortran (g77) 3.2.3 -O3, Linux
Pascal 1.027
1.077		 Free Pascal 2.2.0 -O3, Linux
GNU Pascal 2.1 (GCC 2.95.2) -O3, Linux
Java 1.121 Sun JDK 1.5.0_04-b05, Linux
Visual Basic 6 1.132 All optimisations, Windows XP
Haskell 1.223 GHC 7.4.1-O2 -funbox-strict-fields, Linux
Ada 1.401 GNAT/GCC 3.4.4 -O3, Linux
Go 1.481 Go version go1.1.1 linux/amd64, Linux
Simula 2.099 GNU Cim 5.1, GCC 4.8.1 -O2, Linux
Lua 2.515
22.7		 LuaJIT 2.0.3, Linux
Lua 5.2.3, Linux
Python 2.633
30.0		 PyPy 2.2.1 (Python 2.7.3), Linux
Python 2.7.6, Linux
Erlang 3.663
9.335		 Erlang/OTP 17, emulator 6.0, HiPE [native, {hipe, [o3]}]
Byte code (BEAM), Linux
ALGOL 60 3.951 MARST 2.7, GCC 4.8.1 -O3, Linux
Lisp 7.41
19.8		 GNU Common Lisp 2.6.7, Compiled, Linux
GNU Common Lisp 2.6.7, Interpreted
Smalltalk 7.59 GNU Smalltalk 2.3.5, Linux
Forth 9.92 Gforth 0.7.0, Linux
COBOL 12.5
46.3		 Micro Focus Visual COBOL 2010, Windows 7
Fixed decimal instead of computational-2
Algol 68 15.2 Algol 68 Genie 2.4.1 -O3, Linux
Perl 23.6 Perl v5.8.0, Linux
Ruby 26.1 Ruby 1.8.3, Linux
JavaScript 27.6
39.1
46.9		 Opera 8.0, Linux
Internet Explorer 6.0.2900, Windows XP
Mozilla Firefox 1.0.6, Linux
QBasic 148.3 MS-DOS QBasic 1.1, Windows XP Console

The performance of the reference implementation of Lua is comparable to other scripting languages which compile to and execute byte-codes, such as Perl, Python, and Ruby. Raw CPU performance is rarely important in a scripting language, as it is mostly used as “glue” to invoke facilities of the host application which run at native code speed.

Update: I have added results in the above table for the benchmark run under the LuaJIT just-in-time compiler for Lua, generating code for the x86_64 architecture. This runs almost ten times faster than the standard implementation of Lua and is comparable with other compiled languages. The benchmark ran without any modifications on LuaJIT.

I have also added results from running the Python benchmark under the PyPy just-in-time compiler for Python. Again, there was a dramatic speed increase, vaulting Python into the ranks of compiled languages. Since the Python benchmark was last run with the standard implementation of Python in 2006, I re-ran it on Python 2.7.6 and found, compared to C on an x86_64 architecture, to be substantially slower. I do not know whether this is due to better performance in C code, worse performance in Python, or due to changes in machine architecture compared to the 32-bit system on which the benchmark was run in 2006. (2014-07-26 22:25 UTC)

Posted at 22:32 Permalink