« Reversing Myopia | Main | Reading List: Down with Power »

Tuesday, October 9, 2012

Floating Point Benchmark: Forth Language Added

I have posted an update to my trigonometry-intense floating point benchmark which adds Forth to the list of languages in which the benchmark is implemented. A new release of the benchmark collection including Forth is now available for downloading.

The Forth benchmark was developed using Gforth 0.7.0. Care was taken to use only ANS Forth in the implementation; the following word sets are used in the program:

Editing of floating point numbers is done with a stripped-down version of the fpout17 library linked on the Gforth Links to Forth programs page. The author of this program is identified only as “Ed”, and the source code indicates it is in the public domain. I have stripped out all functionality not required by the benchmark and changed some names to avoid redefinition warnings in some Forth systems. Ed, if I knew who you were, I'd buy you a beer—this code saved me a great deal of tedious work.

The relative performance of the various language implementations (with C taken as 1) is as follows. All language implementations of the benchmark listed below produced identical results to the last (11th) decimal place.

Language Relative
Time		 Details
C 1 GCC 3.2.3 -O3, Linux
Visual Basic .NET 0.866 All optimisations, Windows XP
FORTRAN 1.008 GNU Fortran (g77) 3.2.3 -O3, Linux
Pascal 1.027
1.077		 Free Pascal 2.2.0 -O3, Linux
GNU Pascal 2.1 (GCC 2.95.2) -O3, Linux
Java 1.121 Sun JDK 1.5.0_04-b05, Linux
Visual Basic 6 1.132 All optimisations, Windows XP
Haskell 1.223 GHC 7.4.1-O2 -funbox-strict-fields, Linux
Ada 1.401 GNAT/GCC 3.4.4 -O3, Linux
Lisp 7.41
19.8		 GNU Common Lisp 2.6.7, Compiled, Linux
GNU Common Lisp 2.6.7, Interpreted
Smalltalk 7.59 GNU Smalltalk 2.3.5, Linux
Forth 9.92 Gforth 0.7.0, Linux
COBOL 12.5
46.3		 Micro Focus Visual COBOL 2010, Windows 7
Fixed decimal instead of computational-2
Python 17.6 Python 2.3.3 -OO, Linux
Perl 23.6 Perl v5.8.0, Linux
Ruby 26.1 Ruby 1.8.3, Linux
JavaScript 27.6
39.1
46.9		 Opera 8.0, Linux
Internet Explorer 6.0.2900, Windows XP
Mozilla Firefox 1.0.6, Linux
QBasic 148.3 MS-DOS QBasic 1.1, Windows XP Console

The performance compared to native C code is consistent with my experience with ATLAST more than two decades ago. There, I saw a speed penalty of a Forth-like language of between a factor of 5 and 8 compared to C code, whereas here it's around a factor of 10. But much has changed since then: optimisation of C code is much better, and we've moved from 16 bit segmented architectures to flat address spaces where native code has additional advantages compared to threaded code.

I also ran the benchmark under an evaluation version of Forth, Inc.'s SwiftForth system. I do not report results in the table above because the output differed in the last decimal place due to apparent problems in rounding floating point numbers for display. (I did not investigate this—the problem may be an incompatibility in the floating point output editing code I used.) If you wish to repeat this experiment, you'll need to un-comment the indicated statements at the top of fbench.fs and adjust the path for including the floating point library to where it is installed on your system. SwiftForth ran the benchmark around 14% slower than Gforth.

Please bear in mind that very few people do this kind of scientific computation in Forth—it is mostly used for device control and embedded software applications. Indeed, floating point came late to Forth, with fixed-point being the rule for most of the language's history. Consequently, this benchmark isn't remotely representative of the performance of most of the things people do in Forth. It is what it is—a benchmark more than thirty years old which was uncanny in its ability to determine how fast AutoCAD would run on a machine back in the day. Thinking about re-implementing AutoCAD in Forth? 

    ?crazy IF
        honk
    THEN

Posted at October 9, 2012 21:52