fbench

Introduction

Fbench is a complete optical design raytracing algorithm, shorn of its user interface and recast into portable C. It not only determines execution speed on an extremely floating point (including trigonometric functions) intensive real-world application, it checks accuracy on an algorithm that is exquisitely sensitive to errors. The performance of this program is typically far more sensitive to changes in the efficiency of the trigonometric library routines than the average floating point program.

The benchmark may be compiled in two modes. If the symbol INTRIG is defined, built-in trigonometric and square root routines will be used for all calculations. Timings made with INTRIG defined reflect the machine's basic floating point performance for the arithmetic operators. If INTRIG is not defined, the system library <math.h> functions are used. Results with INTRIG not defined reflect the system's library performance and/or floating point hardware support for trig functions and square root. Results with INTRIG defined are a good guide to general floating point performance, while results with INTRIG undefined indicate the performance of an application which is math function intensive.

Special note regarding errors in accuracy: this program has generated numbers identical to the last digit it formats and checks on the following machines, floating point architectures, and languages:

Inaccuracies reported by this program should be taken very seriously indeed, as the program has been demonstrated to be invariant under changes in floating point format, as long as the format is a recognised double precision format. If you encounter errors, please remember that they are just as likely to be in the floating point editing library or the trigonometric libraries (unless compiled with INTRIG defined) as in the low level operator code. Now that virtually all computers use IEEE floating point format, differences in results are almost certainly indicative of errors in code generation, optimisation, or library routines.

The benchmark assumes that results are basically reliable, and only tests the last result computed against the reference. If you're running on a suspect system you can compile this program with ACCURACY defined. This will generate a version which executes as an infinite loop, performing the ray trace and checking the results on every pass. All incorrect results will be reported.

Benchmark Results for Various Systems

Representative timings are given below. All have been normalised as if run for 1000 iterations.

All brand and product names are trademarks or registered trademarks of their respective companies. Results of this benchmark may or may not be representative of the performance of listed systems for other programs and workloads. Lawyers burn spontaneously in an atmosphere of fluorine.

Comparing Languages

This benchmark was created primarily to compare the performance of different computers and implementations of the C language. Over the years, however, the benchmark has been ported to a variety of different programming languages. The following table compares the relative performance of these languages taking the run time of the C version as 1. For example, a language with a “Relative Time ” of 5 will take five times as long to complete the benchmark as the reference C implementation. Language benchmarks were compared to the run time of the C implementation on the same machine, with the iteration count of both adjusted to run about five minutes. Relative performance was then calculated as the ratio of time per iteration. None of the primary results for language implementations tested exploit the thread-level parallelism implemented in modern processors. All of these runs produced precisely the expected results.

These results should not be interpreted as representative of the overall performance of the various languages for a broad variety of tasks. Each language port is a straightforward translation of the reference C algorithm, and does not exploit additional features (such as vector and matrix operations) which may be present in the target language. However, the nature of the algorithm does not lend itself to such optimisations.

Special thanks to Jim White (“mathimagics”)—approach with extreme caution, known to be in possession of weapons of math instruction—who ported the benchmark to Visual Basic 6, Java, and Scilab; and to John Nagle, who ported the benchmark to Go.

Expected Numerical Results

The C language version of this benchmark contains code which automatically verifies the results of the computation with those expected. Implementations in some other languages may simply print the results and leave it up to you to check that they are correct. The following is the output from a correct execution of fbench in the other languages. There may be slight differences in the annotations and formatting, but the numbers should be absolutely identical.

                       Focal Length          Angle to Axis
Marginal ray         47.09479120920          0.04178472683
Paraxial ray         47.08372160249          0.04177864821

Longitudinal spherical aberration:        -0.01106960671
  (Maximum permissible):                   0.05306749907
                                             Acceptable

Offense against sine condition (coma):     0.00008954761
    (Maximum permissible):                 0.00250000000
                                             Acceptable

Axial chromatic aberration:                0.00448229032
    (Maximum permissible):                 0.05306749907
                                             Acceptable

Don't worry about the terminology—unless you're an optical designer it'll probably make no sense whatsoever. What's important is that the numbers agree to the last decimal place; if they don't, it's a sign something is amiss. If you've compiled the benchmark with aggressive optimisation, you might try more conservative settings to see if that corrects the results.

FFBENCH: Fast Fourier Transform Benchmark

Floating Point Benchmarks

Fourmilab Home Page