« Univac Document Archive: 1107 SLEUTH II and PROCS Manuals Added | Main | Floating Point Benchmark: Modula-2 Language Added, Python 3.x Added »

Tuesday, November 21, 2017

Floating Point Benchmark: Julia Language Added

I have posted a new edition of the floating point benchmark collection which adds the Julia language.

Julia is a language intended for numerical computation in science and engineering. It combines aspects of object orientation, functional programming, and conventional imperative languages. It has a dynamic type system, automatic storage management with garbage collection, macros, and support for parallel processing in both the single instruction multiple data (SIMD) and symmetrical multiprocessing paradigms. An extensive mathematical function library is included, and support for complex numbers, multiple precision integers and floating point, and vector and matrix algebra are built in.

An interactive evaluator is provided, and programs are compiled into machine code with a just in time (JIT) compiler based upon the LLVM compiler infrastructure.

Implementation of the benchmark was straightforward, and based upon the C++ version. Some restructuring was required because Julia does not provide the class and method mechanism of C++, nor does it allow passing pointers as function arguments.

As with all ports of the benchmark, I tried to do things in the style of the language. This means I adopted a style much like that of the Haskell and Scala benchmarks, using a trace context to manage the state as a ray is traced through the surfaces of the design. The transitSurface() function, which accounts for the bulk of the benchmark's run time, takes a TraceContext as an argument and returns a TraceContext updated to reflect the propagation of the ray through the current surface. This means on each call to the function a new TraceContext is allocated and one discarded to be cleaned up by garbage collection. You might think this overhead could be reduced by replacing the dynamically allocated TraceContext with global mutable variables, but that makes things dramatically worse. The program fbench_static.jl uses static variables as the original C and Fortran versions do, and runs almost four and a half times slower than fbench.jl.

I did some preliminary timing tests to try different compiler options, none of which seemed to make much difference in run time. I ran the timing tests with code compiled with the “-O2 --check-bounds=no” options. The latter suppresses array bound checking. After initial runs to determine an iteration count resulting in a run time of around five minutes, I ran five timing tests. Julia's @time macro was used to time the inner loop. For an iteration count of 110,327,622, the mean time of of five runs on an idle system was 295.681 seconds, or 2.68 microseconds per iteration.

My runs of the reference C benchmark on GCC 5.4.0 had a mean time of 296.536 seconds for 166,051,660 iterations, or 1.7858 microseconds per iteration. Thus, for this benchmark, the run time for Julia was 1.501 times longer than that of C or, equivalently, Julia was 50% slower than C.

Download floating point benchmark collection

Language Relative
C 1 GCC 3.2.3 -O3, Linux
JavaScript 0.372
Mozilla Firefox 55.0.2, Linux
Safari 11.0, MacOS X
Brave 0.18.36, Linux
Google Chrome 61.0.3163.91, Linux
Chromium 60.0.3112.113, Linux
Node.js v6.11.3, Linux
Chapel 0.528
Chapel 1.16.0, -fast, Linux
Parallel, 64 threads
Visual Basic .NET 0.866 All optimisations, Windows XP
C++ 0.939
G++ 5.4.0, -O3, Linux, double
long double (80 bit)
__float128 (128 bit)
MPFR (128 bit)
MPFR (512 bit)
FORTRAN 1.008 GNU Fortran (g77) 3.2.3 -O3, Linux
Pascal 1.027
Free Pascal 2.2.0 -O3, Linux
GNU Pascal 2.1 (GCC 2.95.2) -O3, Linux
Swift 1.054 Swift 3.0.1, -O, Linux
Rust 1.077 Rust 0.13.0, --release, Linux
Java 1.121 Sun JDK 1.5.0_04-b05, Linux
Visual Basic 6 1.132 All optimisations, Windows XP
Haskell 1.223 GHC 7.4.1-O2 -funbox-strict-fields, Linux
Scala 1.263 Scala 2.12.3, OpenJDK 9, Linux
FreeBASIC 1.306 FreeBASIC 1.05.0, Linux
Ada 1.401 GNAT/GCC 3.4.4 -O3, Linux
Go 1.481 Go version go1.1.1 linux/amd64, Linux
Julia 1.501 Julia version 0.6.1 64-bit -O2 --check-bounds=no, Linux
Simula 2.099 GNU Cim 5.1, GCC 4.8.1 -O2, Linux
Lua 2.515
LuaJIT 2.0.3, Linux
Lua 5.2.3, Linux
Python 2.633
PyPy 2.2.1 (Python 2.7.3), Linux
Python 2.7.6, Linux
Erlang 3.663
Erlang/OTP 17, emulator 6.0, HiPE [native, {hipe, [o3]}]
Byte code (BEAM), Linux
ALGOL 60 3.951 MARST 2.7, GCC 4.8.1 -O3, Linux
PL/I 5.667 Iron Spring PL/I 0.9.9b beta, Linux
Lisp 7.41
GNU Common Lisp 2.6.7, Compiled, Linux
GNU Common Lisp 2.6.7, Interpreted
Smalltalk 7.59 GNU Smalltalk 2.3.5, Linux
Ruby 7.832 Ruby 2.4.2p198, Linux
Forth 9.92 Gforth 0.7.0, Linux
Prolog 11.72
SWI-Prolog 7.6.0-rc2, Linux
GNU Prolog 1.4.4, Linux, (limited iterations)
COBOL 12.5
Micro Focus Visual COBOL 2010, Windows 7
Fixed decimal instead of computational-2
Algol 68 15.2 Algol 68 Genie 2.4.1 -O3, Linux
Perl 23.6 Perl v5.8.0, Linux
BASICA/GW-BASIC 53.42 Bas 2.4, Linux
QBasic 148.3 MS-DOS QBasic 1.1, Windows XP Console
Mathematica 391.6 Mathematica, Raspberry Pi 3, Raspbian

Posted at November 21, 2017 22:35