« November 21, 2017 | Main | November 30, 2017 »

Friday, November 24, 2017

Floating Point Benchmark: Modula-2 Language Added, Python 3.x Added

I have posted a new edition of the floating point benchmark collection which adds the Modula-2 language to those in which the benchmark has been implemented. In addition, this release adds a version of the benchmark compatible with Python version 3.x (a Python 2.x version of the benchmark has been included in the collection since November 2006).

Modula-2 was developed by Niklaus Wirth between 1977 and 1985. He viewed the language as the successor to Pascal and Modula, and used it for all system and application programming for his Lilith workstation. Modula-2 extended the earlier languages by supporting separate compilation and encapsulation with a module mechanism, coroutines for concurrent processes, and facilities for access to low-level data structures for system programming. Several versions of Modula-2 were released between 1983 and 1988, refining the language specification. In 1996 and 1998, ISO issued three standards under ISO/IEC 10514 for the base language and extensions for object-oriented programming and generics.

Modula-2 is a conventional imperative programming language. Programmers familiar with Pascal will have little difficulty mastering Modula-2. Modula-2 has been used for a number of projects in industry including embedded systems including General Motors' first engine control computer.

Philippe Guiochon developed a Modula-2 version of the benchmark, starting from the FreeBASIC version of the program. He contributed two versions of the program for different compilers: J.P.I. TopSpeed v3.1 for MS-DOS and Excelsior XDS v2.60 beta for Windows 32. Due to small differences in the libraries supported by these compilers, two different versions of the benchmark source code are supplied as fbench_JPI.mod and fbench_XDS.mod. M. Guiochon ran timing tests on a variety of computers, which are included in the source code for the benchmarks.

Since I do not have a machine which can run these programs (Fourmilab is Microsoft free), I was unable to run an archival benchmark and comparison against C for the language comparison table. Since there is a GNU Modula-2 compiler, built atop the GCC (GNU Compiler Collection) infrastructure, that supports the ISO dialect of the language, I decided to see if I could get the code working with it.

I used fbench_JPI.mod as the point of departure to develop a version compatible with GNU Modula-2 in ISO mode with its ISO libraries. I restructured the program, removed the INTRIG code (I'm interested in comparing languages, not their library mathematical functions), and tried to make it more in keeping with the style of Modula-2 than its Basic ancestor. This program, fbench.mod, compiled with GNU Modula-2 version gm2-1.6.4, which is built on gcc-6.4.0 with:

gm2 -O3 -fiso -flibs=iso,pim -Wpedantic fbench.mod -o fbench
is the basis for the timing tests.

After a preliminary run to estimate an iteration count, I ran five runs of 175,495,723 iterations on an idle machine with a mean run time of 294.85 seconds, or 1.6801 microseconds per iteration. This compares with the reference C implementation, which on GCC 5.4.0 ran 166,051,660 iterations in a mean time of 296.536, or 1.7858 microseconds per iteration.

Thus the execution time is 0.941 times that of C (Modula2 is about 6% faster than C). Note that this is almost precisely the speed ratio of the C++ version of the benchmark (0.939) compiled with GCC compared to the C implementation compiled with the same compiler. This makes sense since the back-end of the compiler is the same for both the G++ and GM2 compilers, and the structure of the programs they're compiling is similar.

Philippe Guiochon then re-tested the fbench.mod program with the Excelsior XDS Modula-2 compiler on Windows 32, and encountered both compile errors and accuracy problems. The XDS compiler defines the REAL type to be 32-bit single-precision floating point, which is insufficient for the accuracy needs of this algorithm, and requires variables to be declared as LONGREAL to use 64-bit double precision (which then requires importing the trigonometric functions from the LongMath module). In addition, this compiler does not permit enumeration types to be used as array indices or bounds in FOR statements and requires them to be explicitly converted to CARDINAL with the ORD() function. Using the ORD() function is ugly but compatible with GNU Modula-2, but declaring variables as LONGREAL causes that compiler to use the Intel 80-bit floating point type (GCC's “long double”) instead of the standard 64-bit data type it uses for REAL. Since this may affect the timing (and still other compilers may define REAL and LONGREAL in yet other ways), I have kept this version separate as fbench_iso_XDS.mod and used the original fbench.mod for the timing tests with GNU Modula-2.

Source code for all of these programs is included in the modula2 directory of the distribution.

Completely unrelated to Modula-2, this release of the benchmark collection adds an implementation of the benchmark compatible with version 3 of the Python language. The original version of the Python benchmark, developed in November 2006, was based upon version 2 of Python. Python version 3 made some incompatible changes to the syntax and semantics of the language, requiring changes in many programs. The impact of the changes on the benchmark were relatively minor and required only the following modifications:

With these changes, the benchmark ran with no errors or warnings on Python 3.5.2 and PyPy 5.9.0-beta0, which is based upon Python 3.5.3. The timings were comparable to those on Python 2.7.6 and PyPy 2.2.1, so I did not bother re-running archival timing tests.

I have restructured the python directory in the distribution to contain two subdirectories, python2 and python3, which contain fbench.py files for their respective versions of the language.

Download floating point benchmark collection

Language Relative
C 1 GCC 3.2.3 -O3, Linux
JavaScript 0.372
Mozilla Firefox 55.0.2, Linux
Safari 11.0, MacOS X
Brave 0.18.36, Linux
Google Chrome 61.0.3163.91, Linux
Chromium 60.0.3112.113, Linux
Node.js v6.11.3, Linux
Chapel 0.528
Chapel 1.16.0, -fast, Linux
Parallel, 64 threads
Visual Basic .NET 0.866 All optimisations, Windows XP
C++ 0.939
G++ 5.4.0, -O3, Linux, double
long double (80 bit)
__float128 (128 bit)
MPFR (128 bit)
MPFR (512 bit)
Modula-2 0.941 GNU Modula-2 gm2-1.6.4 -O3, Linux
FORTRAN 1.008 GNU Fortran (g77) 3.2.3 -O3, Linux
Pascal 1.027
Free Pascal 2.2.0 -O3, Linux
GNU Pascal 2.1 (GCC 2.95.2) -O3, Linux
Swift 1.054 Swift 3.0.1, -O, Linux
Rust 1.077 Rust 0.13.0, --release, Linux
Java 1.121 Sun JDK 1.5.0_04-b05, Linux
Visual Basic 6 1.132 All optimisations, Windows XP
Haskell 1.223 GHC 7.4.1-O2 -funbox-strict-fields, Linux
Scala 1.263 Scala 2.12.3, OpenJDK 9, Linux
FreeBASIC 1.306 FreeBASIC 1.05.0, Linux
Ada 1.401 GNAT/GCC 3.4.4 -O3, Linux
Go 1.481 Go version go1.1.1 linux/amd64, Linux
Julia 1.501 Julia version 0.6.1 64-bit -O2 --check-bounds=no, Linux
Simula 2.099 GNU Cim 5.1, GCC 4.8.1 -O2, Linux
Lua 2.515
LuaJIT 2.0.3, Linux
Lua 5.2.3, Linux
Python 2.633
PyPy 2.2.1 (Python 2.7.3), Linux
Python 2.7.6, Linux
Erlang 3.663
Erlang/OTP 17, emulator 6.0, HiPE [native, {hipe, [o3]}]
Byte code (BEAM), Linux
ALGOL 60 3.951 MARST 2.7, GCC 4.8.1 -O3, Linux
PL/I 5.667 Iron Spring PL/I 0.9.9b beta, Linux
Lisp 7.41
GNU Common Lisp 2.6.7, Compiled, Linux
GNU Common Lisp 2.6.7, Interpreted
Smalltalk 7.59 GNU Smalltalk 2.3.5, Linux
Ruby 7.832 Ruby 2.4.2p198, Linux
Forth 9.92 Gforth 0.7.0, Linux
Prolog 11.72
SWI-Prolog 7.6.0-rc2, Linux
GNU Prolog 1.4.4, Linux, (limited iterations)
COBOL 12.5
Micro Focus Visual COBOL 2010, Windows 7
Fixed decimal instead of computational-2
Algol 68 15.2 Algol 68 Genie 2.4.1 -O3, Linux
Perl 23.6 Perl v5.8.0, Linux
BASICA/GW-BASIC 53.42 Bas 2.4, Linux
QBasic 148.3 MS-DOS QBasic 1.1, Windows XP Console
Mathematica 391.6 Mathematica, Raspberry Pi 3, Raspbian

Posted at 22:25 Permalink