HotBits Pseudorandom Generator
HotBits Pseudorandom GeneratorWhile it's essential that the theory behind a pseudorandom number generator be well understood, and that its software realisation be carefully verified to implement the theoretical design, there is no substitute for detailed statistical testing of the actual output of the generator against the expectation for genuinely random data.
A large data set produced by the HotBits pseudorandom generator has been subjected to the scrutiny of three different randomness test suites, whose results are presented below. You can download this large data set and subject it to your own analyses, if you are so inclined.
There are many different ways to test for randomness, but all of them, in essence, boil down to computing a mathematical metric from the data stream being tested and comparing the result with the expectation value for an infinite sequence of genuinely random data. For a truly random sequence, any value is equally probable. The sequence of bits “0000000000000000” is just as likely to occur in a random data stream as “1100100100001111”, and is no less “random”. (The latter sequence is, in fact, the first sixteen bits of the mathematical constant π in binary, whose algorithmic complexity is only modestly greater than the all zero sequence!)
Randomness can be defined only statistically over a long sequence, which is why it is essential to test a large data set. Data can fail to be random in many ways. For example, one of the most obvious tests one can apply to a sequence of binary data is to count the number of ones and zeroes: as the length of the sequence increases, the difference in these values can be used to calculate the probability the sequence is random. But this test, used in isolation, would consider a sequence of alternating zero and one bits (“0101010101010101…”) perfectly random. Hence, it must be used as part of a test battery, including other measures which are sensitive to repeating patterns, improbably long runs of zeroes and ones, and other, more subtle, deviations from randomness.
Speaking as a programmer and not a mathematician or statistician, the two widely-used randomness test batteries: Diehard and the NIST SP 800-22 Statistical Test Suite, whose results are reported below, are quite messy and fragile programs. When using them, it is wise to use data sets of the same size as those employed in the examples supplied with the programs, and to select test parameters identical to those of the examples. In my experience, deviating from the domain in which the programs are known to have been tested may yield surprising and dismaying results. Also, before testing your own data with one of these test batteries, be sure to re-run the examples in the documentation and verify that you're able to reproduce the published results; changes in compilers and libraries, file formats, and operating system compatibility issues may have to be resolved before you can obtain reliable results from these tests.
The Fourmilab ENT program is a public domain utility which tests binary data sequences, either as a series of 8 bit bytes, or as a bit stream, with five standard tests for randomness which are described in the document linked to above. These are all straightforward mathematical metrics, and while they identify major departures from randomness, may miss subtle forms of bias identified by the more comprehensive test suites. The following are the results of an ENT test of the same 11,468,800 data set used for the Diehard test battery in the next section.
Entropy = 7.999984 bits per byte. Optimum compression would reduce the size of this 11468800 byte file by 0 percent. Chi square distribution for 11468800 samples is 262.19, and randomly would exceed this value 36.51 percent of the times. Arithmetic mean value of data bytes is 127.5088 (127.5 = random). Monte Carlo value for Pi is 3.140586335 (error 0.03 percent). Serial correlation coefficient is -0.000152 (totally uncorrelated = 0.0).
Professor George Marsaglia of Florida State University published the “Diehard Battery of Tests of Randomness” in 1995, as part of the Marsaglia Random Number CDROM. The Diehard tests are rather “quirky” measurements of randomness compared to the mathematical properties tested by ENT. Diehard tests include items such as a spacings between birthdays in a random population, monkeys pounding on keyboards, and games of craps. These tests, however, can be exquisitely sensitive to subtle departures from randomness, and their results can all be expressed as the probability the results obtained would be observed in a genuinely random sequence. Probability values close to zero or one indicate potential problems, while probabilities in the middle of the range are expected for random sequences. Please read the “NOTE” at the top of the results presented below about interpreting the reported probability values: with hundreds of probability values computed, some may be expected, purely by chance, to be close to one or zero.
The Diehard test suite was run on a 11,468,800 byte data set extracted from the beginning of the 16,779,776 HotBits pseudorandom test data set. I limited the data set length to that used by other Diehard examples to avoid possible problems in the code dependent upon the size of the data set.
NOTE
Most of the tests in DIEHARD return a p-value, which
should be uniform on [0,1) if the input file contains truly
independent random bits. Those p-values are obtained by
p=1-F(X), where F is the assumed distribution of the sample
random variable X---often normal. But that assumed F is often just
an asymptotic approximation, for which the fit will be worst
in the tails. Thus you should not be surprised with occasion-
al p-values near 0 or 1, such as .0012 or .9983. When a bit
stream really FAILS BIG, you will get p`s of 0 or 1 to six
or more places. By all means, do not, as a Statistician
might, think that a p < .025 or p> .975 means that the RNG
has "failed the test at the .05 level". Such p`s happen
among the hundreds that DIEHARD produces, even with good RNGs.
So keep in mind that "p happens"
Enter the name of the file to be tested.
This must be a form="unformatted",access="direct" binary
file of about 10-12 million bytes. Enter file name:
../FourmilabHotBits_Pseudo_Diehard.dat
HERE ARE YOUR CHOICES:
1 Birthday Spacings
2 Overlapping Permutations
3 Ranks of 31x31 and 32x32 matrices
4 Ranks of 6x8 Matrices
5 Monkey Tests on 20-bit Words
6 Monkey Tests OPSO,OQSO,DNA
7 Count the 1`s in a Stream of Bytes
8 Count the 1`s in Specific Bytes
9 Parking Lot Test
10 Minimum Distance Test
11 Random Spheres Test
12 The Sqeeze Test
13 Overlapping Sums Test
14 Runs Test
15 The Craps Test
16 All of the above
To choose any particular tests, enter corresponding numbers.
Enter 16 for all tests. If you want to perform all but a few
tests, enter corresponding numbers preceded by "-" sign.
Tests are executed in the order they are entered.
Enter your choices.
16
|-------------------------------------------------------------|
| This is the BIRTHDAY SPACINGS TEST |
|Choose m birthdays in a "year" of n days. List the spacings |
|between the birthdays. Let j be the number of values that |
|occur more than once in that list, then j is asymptotically |
|Poisson distributed with mean m^3/(4n). Experience shows n |
|must be quite large, say n>=2^18, for comparing the results |
|to the Poisson distribution with that mean. This test uses |
|n=2^24 and m=2^10, so that the underlying distribution for j |
|is taken to be Poisson with lambda=2^30/(2^26)=16. A sample |
|of 200 j''s is taken, and a chi-square goodness of fit test |
|provides a p value. The first test uses bits 1-24 (counting |
|from the left) from integers in the specified file. Then the|
|file is closed and reopened, then bits 2-25 of the same inte-|
|gers are used to provide birthdays, and so on to bits 9-32. |
|Each set of bits provides a p-value, and the nine p-values |
|provide a sample for a KSTEST. |
|------------------------------------------------------------ |
RESULTS OF BIRTHDAY SPACINGS TEST FOR ../FourmilabHotBits_Pseudo_Diehard.dat
(no_bdays=1024, no_days/yr=2^24, lambda=16.00, sample size=500)
Bits used mean chisqr p-value
1 to 24 15.55 19.9922 0.274631
2 to 25 15.68 8.4203 0.956715
3 to 26 15.93 21.9942 0.184944
4 to 27 16.01 23.8379 0.123908
5 to 28 15.66 14.6740 0.618950
6 to 29 15.70 13.2294 0.720702
7 to 30 15.56 19.3852 0.306874
8 to 31 15.89 22.9957 0.149392
9 to 32 15.85 4.2297 0.999248
degree of freedoms is: 17
---------------------------------------------------------------
p-value for KStest on those 9 p-values: 0.355503
|-------------------------------------------------------------|
| THE OVERLAPPING 5-PERMUTATION TEST |
|This is the OPERM5 test. It looks at a sequence of one mill-|
|ion 32-bit random integers. Each set of five consecutive |
|integers can be in one of 120 states, for the 5! possible or-|
|derings of five numbers. Thus the 5th, 6th, 7th,...numbers |
|each provide a state. As many thousands of state transitions |
|are observed, cumulative counts are made of the number of |
|occurences of each state. Then the quadratic form in the |
|weak inverse of the 120x120 covariance matrix yields a test |
|equivalent to the likelihood ratio test that the 120 cell |
|counts came from the specified (asymptotically) normal dis- |
|tribution with the specified 120x120 covariance matrix (with |
|rank 99). This version uses 1,000,000 integers, twice. |
|-------------------------------------------------------------|
OPERM5 test for file
(For samples of 1,000,000 consecutive 5-tuples)
sample 1
chisquare=93.926036 with df=99; p-value= 0.625264
_______________________________________________________________
sample 2
chisquare=103.403568 with df=99; p-value= 0.361050
_______________________________________________________________
|-------------------------------------------------------------|
|This is the BINARY RANK TEST for 31x31 matrices. The leftmost|
|31 bits of 31 random integers from the test sequence are used|
|to form a 31x31 binary matrix over the field {0,1}. The rank |
|is determined. That rank can be from 0 to 31, but ranks< 28 |
|are rare, and their counts are pooled with those for rank 28.|
|Ranks are found for 40,000 such random matrices and a chisqu-|
|are test is performed on counts for ranks 31,30,28 and <=28. |
|-------------------------------------------------------------|
Rank test for binary matrices (31x31) from ../FourmilabHotBits_Pseudo_Diehard.dat
RANK OBSERVED EXPECTED (O-E)^2/E SUM
r<=28 212 211.4 0.002 0.002
r=29 5119 5134.0 0.044 0.045
r=30 22993 23103.0 0.524 0.570
r=31 11676 11551.5 1.341 1.911
chi-square = 1.911 with df = 3; p-value = 0.591
--------------------------------------------------------------
|-------------------------------------------------------------|
|This is the BINARY RANK TEST for 32x32 matrices. A random 32x|
|32 binary matrix is formed, each row a 32-bit random integer.|
|The rank is determined. That rank can be from 0 to 32, ranks |
|less than 29 are rare, and their counts are pooled with those|
|for rank 29. Ranks are found for 40,000 such random matrices|
|and a chisquare test is performed on counts for ranks 32,31,|
|30 and <=29. |
|-------------------------------------------------------------|
Rank test for binary matrices (32x32) from ../FourmilabHotBits_Pseudo_Diehard.dat
RANK OBSERVED EXPECTED (O-E)^2/E SUM
r<=29 199 211.4 0.729 0.729
r=30 5129 5134.0 0.005 0.734
r=31 23237 23103.0 0.777 1.511
r=32 11435 11551.5 1.175 2.686
chi-square = 2.686 with df = 3; p-value = 0.443
--------------------------------------------------------------
|-------------------------------------------------------------|
|This is the BINARY RANK TEST for 6x8 matrices. From each of |
|six random 32-bit integers from the generator under test, a |
|specified byte is chosen, and the resulting six bytes form a |
|6x8 binary matrix whose rank is determined. That rank can be|
|from 0 to 6, but ranks 0,1,2,3 are rare; their counts are |
|pooled with those for rank 4. Ranks are found for 100,000 |
|random matrices, and a chi-square test is performed on |
|counts for ranks 6,5 and (0,...,4) (pooled together). |
|-------------------------------------------------------------|
Rank test for binary matrices (6x8) from ../FourmilabHotBits_Pseudo_Diehard.dat
bits 1 to 8
RANK OBSERVED EXPECTED (O-E)^2/E SUM
r<=4 851 944.3 9.218 9.218
r=5 21665 21743.9 0.286 9.505
r=6 77484 77311.8 0.384 9.888
chi-square = 9.888 with df = 2; p-value = 0.007
--------------------------------------------------------------
bits 2 to 9
RANK OBSERVED EXPECTED (O-E)^2/E SUM
r<=4 876 944.3 4.940 4.940
r=5 21828 21743.9 0.325 5.265
r=6 77296 77311.8 0.003 5.269
chi-square = 5.269 with df = 2; p-value = 0.072
--------------------------------------------------------------
bits 3 to 10
RANK OBSERVED EXPECTED (O-E)^2/E SUM
r<=4 927 944.3 0.317 0.317
r=5 21750 21743.9 0.002 0.319
r=6 77323 77311.8 0.002 0.320
chi-square = 0.320 with df = 2; p-value = 0.852
--------------------------------------------------------------
bits 4 to 11
RANK OBSERVED EXPECTED (O-E)^2/E SUM
r<=4 922 944.3 0.527 0.527
r=5 21703 21743.9 0.077 0.604
r=6 77375 77311.8 0.052 0.655
chi-square = 0.655 with df = 2; p-value = 0.721
--------------------------------------------------------------
bits 5 to 12
RANK OBSERVED EXPECTED (O-E)^2/E SUM
r<=4 990 944.3 2.212 2.212
r=5 21667 21743.9 0.272 2.484
r=6 77343 77311.8 0.013 2.496
chi-square = 2.496 with df = 2; p-value = 0.287
--------------------------------------------------------------
bits 6 to 13
RANK OBSERVED EXPECTED (O-E)^2/E SUM
r<=4 953 944.3 0.080 0.080
r=5 21843 21743.9 0.452 0.532
r=6 77204 77311.8 0.150 0.682
chi-square = 0.682 with df = 2; p-value = 0.711
--------------------------------------------------------------
bits 7 to 14
RANK OBSERVED EXPECTED (O-E)^2/E SUM
r<=4 958 944.3 0.199 0.199
r=5 21535 21743.9 2.007 2.206
r=6 77507 77311.8 0.493 2.699
chi-square = 2.699 with df = 2; p-value = 0.259
--------------------------------------------------------------
bits 8 to 15
RANK OBSERVED EXPECTED (O-E)^2/E SUM
r<=4 931 944.3 0.187 0.187
r=5 21409 21743.9 5.158 5.345
r=6 77660 77311.8 1.568 6.914
chi-square = 6.914 with df = 2; p-value = 0.032
--------------------------------------------------------------
bits 9 to 16
RANK OBSERVED EXPECTED (O-E)^2/E SUM
r<=4 937 944.3 0.056 0.056
r=5 21562 21743.9 1.522 1.578
r=6 77501 77311.8 0.463 2.041
chi-square = 2.041 with df = 2; p-value = 0.360
--------------------------------------------------------------
bits 10 to 17
RANK OBSERVED EXPECTED (O-E)^2/E SUM
r<=4 915 944.3 0.909 0.909
r=5 21675 21743.9 0.218 1.127
r=6 77410 77311.8 0.125 1.252
chi-square = 1.252 with df = 2; p-value = 0.535
--------------------------------------------------------------
bits 11 to 18
RANK OBSERVED EXPECTED (O-E)^2/E SUM
r<=4 927 944.3 0.317 0.317
r=5 21679 21743.9 0.194 0.511
r=6 77394 77311.8 0.087 0.598
chi-square = 0.598 with df = 2; p-value = 0.742
--------------------------------------------------------------
bits 12 to 19
RANK OBSERVED EXPECTED (O-E)^2/E SUM
r<=4 971 944.3 0.755 0.755
r=5 21769 21743.9 0.029 0.784
r=6 77260 77311.8 0.035 0.819
chi-square = 0.819 with df = 2; p-value = 0.664
--------------------------------------------------------------
bits 13 to 20
RANK OBSERVED EXPECTED (O-E)^2/E SUM
r<=4 957 944.3 0.171 0.171
r=5 21744 21743.9 0.000 0.171
r=6 77299 77311.8 0.002 0.173
chi-square = 0.173 with df = 2; p-value = 0.917
--------------------------------------------------------------
bits 14 to 21
RANK OBSERVED EXPECTED (O-E)^2/E SUM
r<=4 1003 944.3 3.649 3.649
r=5 21564 21743.9 1.488 5.137
r=6 77433 77311.8 0.190 5.327
chi-square = 5.327 with df = 2; p-value = 0.070
--------------------------------------------------------------
bits 15 to 22
RANK OBSERVED EXPECTED (O-E)^2/E SUM
r<=4 937 944.3 0.056 0.056
r=5 21672 21743.9 0.238 0.294
r=6 77391 77311.8 0.081 0.375
chi-square = 0.375 with df = 2; p-value = 0.829
--------------------------------------------------------------
bits 16 to 23
RANK OBSERVED EXPECTED (O-E)^2/E SUM
r<=4 960 944.3 0.261 0.261
r=5 21771 21743.9 0.034 0.295
r=6 77269 77311.8 0.024 0.318
chi-square = 0.318 with df = 2; p-value = 0.853
--------------------------------------------------------------
bits 17 to 24
RANK OBSERVED EXPECTED (O-E)^2/E SUM
r<=4 907 944.3 1.473 1.473
r=5 21894 21743.9 1.036 2.510
r=6 77199 77311.8 0.165 2.674
chi-square = 2.674 with df = 2; p-value = 0.263
--------------------------------------------------------------
bits 18 to 25
RANK OBSERVED EXPECTED (O-E)^2/E SUM
r<=4 891 944.3 3.008 3.008
r=5 21865 21743.9 0.674 3.683
r=6 77244 77311.8 0.059 3.742
chi-square = 3.742 with df = 2; p-value = 0.154
--------------------------------------------------------------
bits 19 to 26
RANK OBSERVED EXPECTED (O-E)^2/E SUM
r<=4 927 944.3 0.317 0.317
r=5 21798 21743.9 0.135 0.452
r=6 77275 77311.8 0.018 0.469
chi-square = 0.469 with df = 2; p-value = 0.791
--------------------------------------------------------------
bits 20 to 27
RANK OBSERVED EXPECTED (O-E)^2/E SUM
r<=4 960 944.3 0.261 0.261
r=5 21727 21743.9 0.013 0.274
r=6 77313 77311.8 0.000 0.274
chi-square = 0.274 with df = 2; p-value = 0.872
--------------------------------------------------------------
bits 21 to 28
RANK OBSERVED EXPECTED (O-E)^2/E SUM
r<=4 899 944.3 2.173 2.173
r=5 21729 21743.9 0.010 2.183
r=6 77372 77311.8 0.047 2.230
chi-square = 2.230 with df = 2; p-value = 0.328
--------------------------------------------------------------
bits 22 to 29
RANK OBSERVED EXPECTED (O-E)^2/E SUM
r<=4 917 944.3 0.789 0.789
r=5 21808 21743.9 0.189 0.978
r=6 77275 77311.8 0.018 0.996
chi-square = 0.996 with df = 2; p-value = 0.608
--------------------------------------------------------------
bits 23 to 30
RANK OBSERVED EXPECTED (O-E)^2/E SUM
r<=4 870 944.3 5.846 5.846
r=5 21730 21743.9 0.009 5.855
r=6 77400 77311.8 0.101 5.956
chi-square = 5.956 with df = 2; p-value = 0.051
--------------------------------------------------------------
bits 24 to 31
RANK OBSERVED EXPECTED (O-E)^2/E SUM
r<=4 981 944.3 1.426 1.426
r=5 21712 21743.9 0.047 1.473
r=6 77307 77311.8 0.000 1.473
chi-square = 1.473 with df = 2; p-value = 0.479
--------------------------------------------------------------
bits 25 to 32
RANK OBSERVED EXPECTED (O-E)^2/E SUM
r<=4 952 944.3 0.063 0.063
r=5 21693 21743.9 0.119 0.182
r=6 77355 77311.8 0.024 0.206
chi-square = 0.206 with df = 2; p-value = 0.902
--------------------------------------------------------------
TEST SUMMARY, 25 tests on 100,000 random 6x8 matrices
These should be 25 uniform [0,1] random variates:
0.007125 0.071771 0.852026 0.720644 0.287044
0.711015 0.259425 0.031529 0.360388 0.534677
0.741541 0.664108 0.917171 0.069691 0.828897
0.852784 0.262621 0.153941 0.790941 0.871890
0.327879 0.607826 0.050904 0.478683 0.902092
The KS test for those 25 supposed UNI's yields
KS p-value = 0.613828
|-------------------------------------------------------------|
| THE BITSTREAM TEST |
|The file under test is viewed as a stream of bits. Call them |
|b1,b2,... . Consider an alphabet with two "letters", 0 and 1|
|and think of the stream of bits as a succession of 20-letter |
|"words", overlapping. Thus the first word is b1b2...b20, the|
|second is b2b3...b21, and so on. The bitstream test counts |
|the number of missing 20-letter (20-bit) words in a string of|
|2^21 overlapping 20-letter words. There are 2^20 possible 20|
|letter words. For a truly random string of 2^21+19 bits, the|
|number of missing words j should be (very close to) normally |
|distributed with mean 141,909 and sigma 428. Thus |
| (j-141909)/428 should be a standard normal variate (z score)|
|that leads to a uniform [0,1) p value. The test is repeated |
|twenty times. |
|-------------------------------------------------------------|
THE OVERLAPPING 20-TUPLES BITSTREAM TEST for ../FourmilabHotBits_Pseudo_Diehard.dat
(20 bits/word, 2097152 words 20 bitstreams. No. missing words
should average 141909.33 with sigma=428.00.)
----------------------------------------------------------------
BITSTREAM test results for ../FourmilabHotBits_Pseudo_Diehard.dat.
Bitstream No. missing words z-score p-value
1 142655 1.74 0.040735
2 141574 -0.78 0.783328
3 141930 0.05 0.480741
4 142988 2.52 0.005863
5 141824 -0.20 0.579013
6 141602 -0.72 0.763640
7 140819 -2.55 0.994575
8 141909 -0.00 0.500308
9 142310 0.94 0.174599
10 142222 0.73 0.232531
11 141587 -0.75 0.774307
12 142571 1.55 0.061057
13 142357 1.05 0.147790
14 141227 -1.59 0.944558
15 142613 1.64 0.050079
16 142403 1.15 0.124366
17 141939 0.07 0.472366
18 141434 -1.11 0.866626
19 142002 0.22 0.414292
20 142392 1.13 0.129716
----------------------------------------------------------------
|-------------------------------------------------------------|
| OPSO means Overlapping-Pairs-Sparse-Occupancy |
|The OPSO test considers 2-letter words from an alphabet of |
|1024 letters. Each letter is determined by a specified ten |
|bits from a 32-bit integer in the sequence to be tested. OPSO|
|generates 2^21 (overlapping) 2-letter words (from 2^21+1 |
|"keystrokes") and counts the number of missing words---that |
|is 2-letter words which do not appear in the entire sequence.|
|That count should be very close to normally distributed with |
|mean 141,909, sigma 290. Thus (missingwrds-141909)/290 should|
|be a standard normal variable. The OPSO test takes 32 bits at|
|a time from the test file and uses a designated set of ten |
|consecutive bits. It then restarts the file for the next de- |
|signated 10 bits, and so on. |
|------------------------------------------------------------ |
OPSO test for file ../FourmilabHotBits_Pseudo_Diehard.dat
Bits used No. missing words z-score p-value
23 to 32 141923 0.0471 0.481202
22 to 31 142068 0.5471 0.292142
21 to 30 141946 0.1264 0.449688
20 to 29 141587 -1.1115 0.866820
19 to 28 142251 1.1782 0.119364
18 to 27 141900 -0.0322 0.512833
17 to 26 141834 -0.2598 0.602475
16 to 25 142323 1.4264 0.076869
15 to 24 141512 -1.3701 0.914673
14 to 23 141806 -0.3563 0.639196
13 to 22 142153 0.8402 0.200386
12 to 21 142083 0.5989 0.274632
11 to 20 141937 0.0954 0.461993
10 to 19 141692 -0.7494 0.773196
9 to 18 141655 -0.8770 0.809757
8 to 17 141759 -0.5184 0.697903
7 to 16 142185 0.9506 0.170907
6 to 15 141886 -0.0804 0.532060
5 to 14 141652 -0.8873 0.812553
4 to 13 141329 -2.0011 0.977311
3 to 12 141484 -1.4667 0.928765
2 to 11 141395 -1.7736 0.961931
1 to 10 141994 0.2920 0.385156
-----------------------------------------------------------------
|------------------------------------------------------------ |
| OQSO means Overlapping-Quadruples-Sparse-Occupancy |
| The test OQSO is similar, except that it considers 4-letter|
|words from an alphabet of 32 letters, each letter determined |
|by a designated string of 5 consecutive bits from the test |
|file, elements of which are assumed 32-bit random integers. |
|The mean number of missing words in a sequence of 2^21 four- |
|letter words, (2^21+3 "keystrokes"), is again 141909, with |
|sigma = 295. The mean is based on theory; sigma comes from |
|extensive simulation. |
|------------------------------------------------------------ |
OQSO test for file ../FourmilabHotBits_Pseudo_Diehard.dat
Bits used No. missing words z-score p-value
28 to 32 142037 0.4328 0.332587
27 to 31 141937 0.0938 0.462635
26 to 30 141503 -1.3774 0.915804
25 to 29 142043 0.4531 0.325232
24 to 28 141911 0.0057 0.497742
23 to 27 142038 0.4362 0.331357
22 to 26 142266 1.2091 0.113322
21 to 25 142168 0.8768 0.190285
20 to 24 142170 0.8836 0.188449
19 to 23 141659 -0.8486 0.801941
18 to 22 141909 -0.0011 0.500446
17 to 21 141961 0.1752 0.430480
16 to 20 142025 0.3921 0.347491
15 to 19 142410 1.6972 0.044831
14 to 18 141714 -0.6621 0.746058
13 to 17 141648 -0.8859 0.812155
12 to 16 141666 -0.8248 0.795271
11 to 15 141388 -1.7672 0.961404
10 to 14 141360 -1.8621 0.968708
9 to 13 141773 -0.4621 0.678008
8 to 12 142046 0.4633 0.321579
7 to 11 142115 0.6972 0.242843
6 to 10 141701 -0.7062 0.759969
5 to 9 141985 0.2565 0.398779
4 to 8 142396 1.6497 0.049499
3 to 7 141850 -0.2011 0.579697
2 to 6 142271 1.2260 0.110099
1 to 5 142229 1.0836 0.139265
-----------------------------------------------------------------
|------------------------------------------------------------ |
| The DNA test considers an alphabet of 4 letters: C,G,A,T,|
|determined by two designated bits in the sequence of random |
|integers being tested. It considers 10-letter words, so that|
|as in OPSO and OQSO, there are 2^20 possible words, and the |
|mean number of missing words from a string of 2^21 (over- |
|lapping) 10-letter words (2^21+9 "keystrokes") is 141909. |
|The standard deviation sigma=339 was determined as for OQSO |
|by simulation. (Sigma for OPSO, 290, is the true value (to |
|three places), not determined by simulation. |
|------------------------------------------------------------ |
DNA test for file ../FourmilabHotBits_Pseudo_Diehard.dat
Bits used No. missing words z-score p-value
31 to 32 141737 -0.5083 0.694395
30 to 31 142108 0.5860 0.278922
29 to 30 142253 1.0138 0.155345
28 to 29 141943 0.0993 0.460441
27 to 28 142137 0.6716 0.250921
26 to 27 141716 -0.5703 0.715761
25 to 26 141939 0.0875 0.465128
24 to 25 141786 -0.3638 0.641998
23 to 24 142411 1.4799 0.069456
22 to 23 142122 0.6273 0.265216
21 to 22 141494 -1.2252 0.889743
20 to 21 141361 -1.6175 0.947114
19 to 20 141654 -0.7532 0.774331
18 to 19 141416 -1.4553 0.927200
17 to 18 141870 -0.1160 0.546181
16 to 17 141989 0.2350 0.407099
15 to 16 141348 -1.6558 0.951123
14 to 15 141921 0.0344 0.486269
13 to 14 142018 0.3206 0.374272
12 to 13 141749 -0.4729 0.681875
11 to 12 141741 -0.4965 0.690246
10 to 11 142120 0.6214 0.267153
9 to 10 142473 1.6627 0.048182
8 to 9 141873 -0.1072 0.542672
7 to 8 142040 0.3855 0.349949
6 to 7 142105 0.5772 0.281903
5 to 6 142287 1.1141 0.132624
4 to 5 141561 -1.0275 0.847913
3 to 4 141821 -0.2606 0.602784
2 to 3 141849 -0.1780 0.570625
1 to 2 142051 0.4179 0.338008
-----------------------------------------------------------------
|-------------------------------------------------------------|
| This is the COUNT-THE-1''s TEST on a stream of bytes. |
|Consider the file under test as a stream of bytes (four per |
|32 bit integer). Each byte can contain from 0 to 8 1''s, |
|with probabilities 1,8,28,56,70,56,28,8,1 over 256. Now let |
|the stream of bytes provide a string of overlapping 5-letter|
|words, each "letter" taking values A,B,C,D,E. The letters are|
|determined by the number of 1''s in a byte: 0,1,or 2 yield A,|
|3 yields B, 4 yields C, 5 yields D and 6,7 or 8 yield E. Thus|
|we have a monkey at a typewriter hitting five keys with vari-|
|ous probabilities (37,56,70,56,37 over 256). There are 5^5 |
|possible 5-letter words, and from a string of 256,000 (over- |
|lapping) 5-letter words, counts are made on the frequencies |
|for each word. The quadratic form in the weak inverse of |
|the covariance matrix of the cell counts provides a chisquare|
|test: Q5-Q4, the difference of the naive Pearson sums of |
|(OBS-EXP)^2/EXP on counts for 5- and 4-letter cell counts. |
|-------------------------------------------------------------|
Test result for the byte stream from ../FourmilabHotBits_Pseudo_Diehard.dat
(Degrees of freedom: 5^4-5^3=2500; sample size: 2560000)
chisquare z-score p-value
2455.34 -0.632 0.736165
|-------------------------------------------------------------|
| This is the COUNT-THE-1''s TEST for specific bytes. |
|Consider the file under test as a stream of 32-bit integers. |
|From each integer, a specific byte is chosen , say the left- |
|most: bits 1 to 8. Each byte can contain from 0 to 8 1''s, |
|with probabilitie 1,8,28,56,70,56,28,8,1 over 256. Now let |
|the specified bytes from successive integers provide a string|
|of (overlapping) 5-letter words, each "letter" taking values |
|A,B,C,D,E. The letters are determined by the number of 1''s,|
|in that byte: 0,1,or 2 ---> A, 3 ---> B, 4 ---> C, 5 ---> D, |
|and 6,7 or 8 ---> E. Thus we have a monkey at a typewriter |
|hitting five keys with with various probabilities: 37,56,70, |
|56,37 over 256. There are 5^5 possible 5-letter words, and |
|from a string of 256,000 (overlapping) 5-letter words, counts|
|are made on the frequencies for each word. The quadratic form|
|in the weak inverse of the covariance matrix of the cell |
|counts provides a chisquare test: Q5-Q4, the difference of |
|the naive Pearson sums of (OBS-EXP)^2/EXP on counts for 5- |
|and 4-letter cell counts. |
|-------------------------------------------------------------|
Test results for specific bytes from ../FourmilabHotBits_Pseudo_Diehard.dat
(Degrees of freedom: 5^4-5^3=2500; sample size: 256000)
bits used chisquare z-score p-value
1 to 8 2516.72 0.236 0.406559
2 to 9 2535.88 0.507 0.305920
3 to 10 2533.92 0.480 0.315718
4 to 11 2701.94 2.856 0.002146
5 to 12 2572.94 1.032 0.151148
6 to 13 2416.50 -1.181 0.881171
7 to 14 2513.33 0.189 0.425238
8 to 15 2540.32 0.570 0.284262
9 to 16 2548.39 0.684 0.246879
10 to 17 2557.56 0.814 0.207835
11 to 18 2533.87 0.479 0.315968
12 to 19 2476.96 -0.326 0.627699
13 to 20 2625.44 1.774 0.038031
14 to 21 2499.60 -0.006 0.502267
15 to 22 2554.00 0.764 0.222529
16 to 23 2461.60 -0.543 0.706434
17 to 24 2434.98 -0.920 0.821089
18 to 25 2420.05 -1.131 0.870889
19 to 26 2381.78 -1.672 0.952731
20 to 27 2421.64 -1.108 0.866115
21 to 28 2522.10 0.313 0.377308
22 to 29 2613.88 1.611 0.053638
23 to 30 2497.12 -0.041 0.516251
24 to 31 2507.88 0.111 0.455644
25 to 32 2484.32 -0.222 0.587723
|-------------------------------------------------------------|
| THIS IS A PARKING LOT TEST |
|In a square of side 100, randomly "park" a car---a circle of |
|radius 1. Then try to park a 2nd, a 3rd, and so on, each |
|time parking "by ear". That is, if an attempt to park a car |
|causes a crash with one already parked, try again at a new |
|random location. (To avoid path problems, consider parking |
|helicopters rather than cars.) Each attempt leads to either|
|a crash or a success, the latter followed by an increment to |
|the list of cars already parked. If we plot n: the number of |
|attempts, versus k: the number successfully parked, we get a |
|curve that should be similar to those provided by a perfect |
|random number generator. Theory for the behavior of such a |
|random curve seems beyond reach, and as graphics displays are|
|not available for this battery of tests, a simple characteriz|
|ation of the random experiment is used: k, the number of cars|
|successfully parked after n=12,000 attempts. Simulation shows|
|that k should average 3523 with sigma 21.9 and is very close |
|to normally distributed. Thus (k-3523)/21.9 should be a st- |
|andard normal variable, which, converted to a uniform varia- |
|ble, provides input to a KSTEST based on a sample of 10. |
|-------------------------------------------------------------|
CDPARK: result of 10 tests on file ../FourmilabHotBits_Pseudo_Diehard.dat
(Of 12000 tries, the average no. of successes should be
3523.0 with sigma=21.9)
No. succeses z-score p-value
3514 -0.4110 0.659449
3526 0.1370 0.445521
3538 0.6849 0.246694
3503 -0.9132 0.819442
3536 0.5936 0.276387
3490 -1.5068 0.934075
3509 -0.6393 0.738676
3527 0.1826 0.427537
3565 1.9178 0.027568
3543 0.9132 0.180558
Square side=100, avg. no. parked=3525.10 sample std.=20.75
p-value of the KSTEST for those 10 p-values: 1.000000
|-------------------------------------------------------------|
| THE MINIMUM DISTANCE TEST |
|It does this 100 times: choose n=8000 random points in a |
|square of side 10000. Find d, the minimum distance between |
|the (n^2-n)/2 pairs of points. If the points are truly inde-|
|pendent uniform, then d^2, the square of the minimum distance|
|should be (very close to) exponentially distributed with mean|
|.995 . Thus 1-exp(-d^2/.995) should be uniform on [0,1) and |
|a KSTEST on the resulting 100 values serves as a test of uni-|
|formity for random points in the square. Test numbers=0 mod 5|
|are printed but the KSTEST is based on the full set of 100 |
|random choices of 8000 points in the 10000x10000 square. |
|-------------------------------------------------------------|
This is the MINIMUM DISTANCE test for file ../FourmilabHotBits_Pseudo_Diehard.dat
Sample no. d^2 mean equiv uni
5 0.8382 0.7229 0.569323
10 0.1134 0.5972 0.107731
15 0.1682 0.5648 0.155528
20 0.4103 0.7428 0.337937
25 0.2766 0.7865 0.242664
30 0.4146 0.8082 0.340795
35 1.1169 0.8560 0.674531
40 2.5136 0.8642 0.920043
45 5.8444 0.9585 0.997188
50 0.1718 1.0642 0.158565
55 0.7041 1.0677 0.507202
60 0.8090 1.0447 0.556493
65 1.5655 1.0754 0.792664
70 0.1773 1.0495 0.163256
75 0.5912 1.0249 0.447968
80 0.2378 1.0036 0.212573
85 0.0188 1.0016 0.018765
90 1.1563 0.9884 0.687175
95 4.4738 0.9979 0.988850
100 0.8557 1.0258 0.576822
--------------------------------------------------------------
Result of KS test on 100 transformed mindist^2's: p-value=0.982511
|-------------------------------------------------------------|
| THE 3DSPHERES TEST |
|Choose 4000 random points in a cube of edge 1000. At each |
|point, center a sphere large enough to reach the next closest|
|point. Then the volume of the smallest such sphere is (very |
|close to) exponentially distributed with mean 120pi/3. Thus |
|the radius cubed is exponential with mean 30. (The mean is |
|obtained by extensive simulation). The 3DSPHERES test gener-|
|ates 4000 such spheres 20 times. Each min radius cubed leads|
|to a uniform variable by means of 1-exp(-r^3/30.), then a |
| KSTEST is done on the 20 p-values. |
|-------------------------------------------------------------|
The 3DSPHERES test for file ../FourmilabHotBits_Pseudo_Diehard.dat
sample no r^3 equiv. uni.
1 61.663 0.871962
2 27.417 0.599037
3 25.799 0.576821
4 78.359 0.926609
5 4.653 0.143667
6 98.950 0.963055
7 11.933 0.328171
8 24.072 0.551743
9 51.971 0.823137
10 13.204 0.356049
11 12.261 0.335496
12 6.585 0.197069
13 2.023 0.065201
14 20.853 0.500979
15 28.613 0.614718
16 24.303 0.555193
17 10.829 0.302995
18 32.518 0.661734
19 142.132 0.991241
20 1.623 0.052664
--------------------------------------------------------------
p-value for KS test on those 20 p-values: 0.921166
|-------------------------------------------------------------|
| This is the SQUEEZE test |
| Random integers are floated to get uniforms on [0,1). Start-|
| ing with k=2^31=2147483647, the test finds j, the number of |
| iterations necessary to reduce k to 1, using the reduction |
| k=ceiling(k*U), with U provided by floating integers from |
| the file being tested. Such j''s are found 100,000 times, |
| then counts for the number of times j was <=6,7,...,47,>=48 |
| are used to provide a chi-square test for cell frequencies. |
|-------------------------------------------------------------|
RESULTS OF SQUEEZE TEST FOR ../FourmilabHotBits_Pseudo_Diehard.dat
Table of standardized frequency counts
(obs-exp)^2/exp for j=(1,..,6), 7,...,47,(48,...)
0.6 0.1 1.1 -0.5 -0.9 2.0
-1.3 0.6 0.8 0.5 2.2 0.3
1.2 -0.2 -1.8 -1.2 1.7 0.3
0.8 -1.1 -0.6 -0.6 2.0 -2.3
-0.6 0.3 -0.5 -0.0 -0.3 -1.4
-0.8 0.9 -0.5 0.8 0.8 0.6
-1.4 -1.7 1.3 0.4 0.1 0.0
1.8
Chi-square with 42 degrees of freedom:51.069196
z-score=0.989530, p-value=0.159208
_____________________________________________________________
|-------------------------------------------------------------|
| The OVERLAPPING SUMS test |
|Integers are floated to get a sequence U(1),U(2),... of uni- |
|form [0,1) variables. Then overlapping sums, |
| S(1)=U(1)+...+U(100), S2=U(2)+...+U(101),... are formed. |
|The S''s are virtually normal with a certain covariance mat- |
|rix. A linear transformation of the S''s converts them to a |
|sequence of independent standard normals, which are converted|
|to uniform variables for a KSTEST. |
|-------------------------------------------------------------|
Results of the OSUM test for ../FourmilabHotBits_Pseudo_Diehard.dat
Test no p-value
1 0.967041
2 0.235812
3 0.073502
4 0.448751
5 0.349435
6 0.603653
7 0.096515
8 0.313113
9 0.291257
10 0.103890
_____________________________________________________________
p-value for 10 kstests on 100 kstests:0.126408
|-------------------------------------------------------------|
| This is the RUNS test. It counts runs up, and runs down,|
|in a sequence of uniform [0,1) variables, obtained by float- |
|ing the 32-bit integers in the specified file. This example |
|shows how runs are counted: .123,.357,.789,.425,.224,.416,.95|
|contains an up-run of length 3, a down-run of length 2 and an|
|up-run of (at least) 2, depending on the next values. The |
|covariance matrices for the runs-up and runs-down are well |
|known, leading to chisquare tests for quadratic forms in the |
|weak inverses of the covariance matrices. Runs are counted |
|for sequences of length 10,000. This is done ten times. Then|
|another three sets of ten. |
|-------------------------------------------------------------|
The RUNS test for file ../FourmilabHotBits_Pseudo_Diehard.dat
(Up and down runs in a sequence of 10000 numbers)
Set 1
runs up; ks test for 10 p's: 0.729472
runs down; ks test for 10 p's: 0.504524
Set 2
runs up; ks test for 10 p's: 0.513458
runs down; ks test for 10 p's: 0.591819
|-------------------------------------------------------------|
|This the CRAPS TEST. It plays 200,000 games of craps, counts|
|the number of wins and the number of throws necessary to end |
|each game. The number of wins should be (very close to) a |
|normal with mean 200000p and variance 200000p(1-p), and |
|p=244/495. Throws necessary to complete the game can vary |
|from 1 to infinity, but counts for all>21 are lumped with 21.|
|A chi-square test is made on the no.-of-throws cell counts. |
|Each 32-bit integer from the test file provides the value for|
|the throw of a die, by floating to [0,1), multiplying by 6 |
|and taking 1 plus the integer part of the result. |
|-------------------------------------------------------------|
RESULTS OF CRAPS TEST FOR ../FourmilabHotBits_Pseudo_Diehard.dat
No. of wins: Observed Expected
98276 98585.858586
z-score=-1.386, pvalue=0.91711
Analysis of Throws-per-Game:
Throws Observed Expected Chisq Sum of (O-E)^2/E
1 66425 66666.7 0.876 0.876
2 37400 37654.3 1.718 2.594
3 27254 26954.7 3.323 5.916
4 19072 19313.5 3.019 8.935
5 14023 13851.4 2.125 11.061
6 10077 9943.5 1.791 12.852
7 7199 7145.0 0.408 13.260
8 5160 5139.1 0.085 13.345
9 3692 3699.9 0.017 13.361
10 2740 2666.3 2.037 15.399
11 1934 1923.3 0.059 15.458
12 1385 1388.7 0.010 15.468
13 1016 1003.7 0.150 15.618
14 741 726.1 0.304 15.922
15 511 525.8 0.419 16.341
16 365 381.2 0.684 17.025
17 270 276.5 0.155 17.180
18 192 200.8 0.388 17.568
19 153 146.0 0.337 17.905
20 102 106.2 0.167 18.073
21 289 287.1 0.012 18.085
Chisq= 18.09 for 20 degrees of freedom, p= 0.58180
SUMMARY of craptest on ../FourmilabHotBits_Pseudo_Diehard.dat
p-value for no. of wins: 0.917106
p-value for throws/game: 0.581801
_____________________________________________________________
The following results were produced by testing a sequence of 16,779,776 bytes from the HotBits generator with version 2.1.2 of the U.S. National Institute of Standards and Technology Statistical Test Suite for Random and Pseudorandom Number Generators for Cryptographic Applications as described in NIST Special Publication 800-22rev1a [PDF]. This test suite is supplied as C source code, which I built with GCC 5.4.0 on a Linux x86_64 machine.
The data set used in the following tests is identical in the first 11,468,800 bytes to that used in the ENT and Diehard tests above. This is the complete data set I generated for testing; due to the sensitivity of the Diehard tests to the data set size, I limited the data I tested to be the same as that used in the Diehard examples and tested that data set with ENT. The NIST tests seem to have no problems with large data sets, so I used the complete data set in these tests. Here is the command and parameters I used to run the test.
$ ./assess 1048576
G E N E R A T O R S E L E C T I O N
______________________________________
[0] Input File [1] Linear Congruential
[2] Quadratic Congruential I [3] Quadratic Congruential II
[4] Cubic Congruential [5] XOR
[6] Modular Exponentiation [7] Blum-Blum-Shub
[8] Micali-Schnorr [9] G Using SHA-1
Enter Choice: 0
User Prescribed Input File: FourmilabHotBits_Pseudo.dat
S T A T I S T I C A L T E S T S
_________________________________
[01] Frequency [02] Block Frequency
[03] Cumulative Sums [04] Runs
[05] Longest Run of Ones [06] Rank
[07] Discrete Fourier Transform [08] Nonperiodic Template Matchings
[09] Overlapping Template Matchings [10] Universal Statistical
[11] Approximate Entropy [12] Random Excursions
[13] Random Excursions Variant [14] Serial
[15] Linear Complexity
INSTRUCTIONS
Enter 0 if you DO NOT want to apply all of the
statistical tests to each sequence and 1 if you DO.
Enter Choice: 1
P a r a m e t e r A d j u s t m e n t s
-----------------------------------------
[1] Block Frequency Test - block length(M): 128
[2] NonOverlapping Template Test - block length(m): 9
[3] Overlapping Template Test - block length(m): 9
[4] Approximate Entropy Test - block length(m): 10
[5] Serial Test - block length(m): 16
[6] Linear Complexity Test - block length(M): 500
Select Test (0 to continue): 0
How many bitstreams? 128
Input File Format:
[0] ASCII - A sequence of ASCII 0's and 1's
[1] Binary - Each byte in data file contains 8 bits of data
Select input mode: 1
Statistical Testing In Progress.........
Statistical Testing Complete!!!!!!!!!!!!
I enabled all tests and used the parameters from the examples in the paper documenting the tests. Unless you really understand what you're doing, have studied the math underlying the test, and have read the code that implements it, it's a bad idea to get too creative changing these parameters.
After all the tests have completed, a summary report is written, which is reproduced below. The key values to look at for each test are the “P-value”: the probability the results obtained were due to chance, and the “Proportion” of sequences generated which were deemed to pass the test. Extremal P-values (very close to zero or one), and Proportions below those indicated in the comment at the end of the table are indicative of potential problems.
------------------------------------------------------------------------------ RESULTS FOR THE UNIFORMITY OF P-VALUES AND THE PROPORTION OF PASSING SEQUENCES ------------------------------------------------------------------------------ generator is <FourmilabHotBits_Pseudo.dat> ------------------------------------------------------------------------------ C1 C2 C3 C4 C5 C6 C7 C8 C9 C10 P-VALUE PROPORTION STATISTICAL TEST ------------------------------------------------------------------------------ 11 17 13 10 15 16 11 11 10 14 0.788728 126/128 Frequency 9 16 7 8 8 16 12 14 18 20 0.051391 128/128 BlockFrequency 13 15 15 18 9 10 8 16 12 12 0.500934 126/128 CumulativeSums 14 10 17 11 14 16 13 11 11 11 0.848588 123/128 CumulativeSums 19 12 14 16 10 12 15 10 8 12 0.484646 125/128 Runs 10 7 8 11 15 16 12 20 8 21 0.022503 128/128 LongestRun 14 10 8 16 12 10 15 6 17 20 0.095617 127/128 Rank 10 10 21 16 14 11 10 14 11 11 0.350485 127/128 FFT 13 12 12 13 16 11 9 13 16 13 0.922036 125/128 NonOverlappingTemplate 13 7 12 19 10 16 10 13 15 13 0.422034 127/128 NonOverlappingTemplate 9 12 14 15 12 10 10 8 16 22 0.141256 127/128 NonOverlappingTemplate 15 9 14 14 16 4 17 18 7 14 0.060239 127/128 NonOverlappingTemplate 8 11 15 18 11 17 15 12 14 7 0.311542 128/128 NonOverlappingTemplate 13 15 16 9 14 14 10 11 13 13 0.900104 126/128 NonOverlappingTemplate 17 11 9 10 10 16 15 14 16 10 0.568055 125/128 NonOverlappingTemplate 15 9 16 13 11 13 16 13 13 9 0.804337 128/128 NonOverlappingTemplate 14 9 12 13 10 9 18 19 13 11 0.392456 127/128 NonOverlappingTemplate 15 11 13 14 9 13 13 18 10 12 0.788728 126/128 NonOverlappingTemplate 12 10 15 13 18 10 14 13 10 13 0.804337 126/128 NonOverlappingTemplate 14 13 9 13 15 10 9 13 15 17 0.739918 126/128 NonOverlappingTemplate 18 12 13 18 9 9 14 11 15 9 0.392456 125/128 NonOverlappingTemplate 8 14 6 19 11 14 17 10 13 16 0.162606 128/128 NonOverlappingTemplate 15 13 12 11 17 12 12 9 12 15 0.875539 126/128 NonOverlappingTemplate 15 13 11 10 11 16 18 12 10 12 0.739918 128/128 NonOverlappingTemplate 7 15 15 14 15 15 21 9 6 11 0.066882 127/128 NonOverlappingTemplate 12 14 17 10 9 12 18 12 12 12 0.689019 127/128 NonOverlappingTemplate 11 16 13 12 5 16 13 14 11 17 0.392456 128/128 NonOverlappingTemplate 12 10 20 15 20 4 20 12 9 6 0.001919 126/128 NonOverlappingTemplate 12 12 16 10 19 11 12 17 10 9 0.437274 126/128 NonOverlappingTemplate 14 9 12 7 20 14 9 13 13 17 0.222869 128/128 NonOverlappingTemplate 12 10 7 20 12 12 12 12 17 14 0.337162 127/128 NonOverlappingTemplate 14 10 16 13 18 15 15 8 12 7 0.350485 125/128 NonOverlappingTemplate 19 11 8 9 12 17 13 14 10 15 0.364146 128/128 NonOverlappingTemplate 10 22 18 8 13 12 8 12 10 15 0.078086 127/128 NonOverlappingTemplate 13 14 7 11 13 13 16 14 12 15 0.819544 127/128 NonOverlappingTemplate 13 17 15 11 14 10 8 11 16 13 0.689019 128/128 NonOverlappingTemplate 8 10 14 13 20 15 18 11 10 9 0.195163 126/128 NonOverlappingTemplate 9 18 11 7 14 12 17 14 13 13 0.452799 127/128 NonOverlappingTemplate 14 10 9 9 15 18 13 12 10 18 0.407091 124/128 NonOverlappingTemplate 16 20 10 5 14 9 15 13 13 13 0.155209 127/128 NonOverlappingTemplate 15 11 13 10 13 13 15 13 13 12 0.985035 128/128 NonOverlappingTemplate 8 9 12 13 14 17 18 12 17 8 0.275709 126/128 NonOverlappingTemplate 13 7 16 16 6 11 18 18 13 10 0.110952 127/128 NonOverlappingTemplate 12 12 13 13 14 10 13 12 12 17 0.964295 127/128 NonOverlappingTemplate 14 16 11 10 10 13 15 8 18 13 0.568055 127/128 NonOverlappingTemplate 14 12 10 14 13 14 15 16 8 12 0.848588 127/128 NonOverlappingTemplate 12 14 13 10 7 10 11 25 16 10 0.025193 127/128 NonOverlappingTemplate 9 9 13 18 12 11 19 18 5 14 0.063482 128/128 NonOverlappingTemplate 15 17 12 13 14 10 13 14 11 9 0.848588 127/128 NonOverlappingTemplate 14 7 15 5 12 11 16 16 14 18 0.148094 125/128 NonOverlappingTemplate 10 14 7 13 12 14 18 13 15 12 0.637119 127/128 NonOverlappingTemplate 10 15 14 14 19 7 17 14 10 8 0.213309 127/128 NonOverlappingTemplate 10 19 7 13 14 22 8 6 14 15 0.014216 127/128 NonOverlappingTemplate 14 12 5 16 20 10 12 16 11 12 0.170294 127/128 NonOverlappingTemplate 12 12 13 15 10 18 13 10 17 8 0.534146 126/128 NonOverlappingTemplate 18 14 11 15 12 10 15 12 11 10 0.772760 125/128 NonOverlappingTemplate 15 8 14 11 8 8 15 19 16 14 0.232760 127/128 NonOverlappingTemplate 13 15 13 17 14 7 13 15 11 10 0.671779 128/128 NonOverlappingTemplate 8 11 17 9 14 12 11 17 12 17 0.452799 125/128 NonOverlappingTemplate 14 13 13 16 11 12 11 11 18 9 0.756476 126/128 NonOverlappingTemplate 16 13 6 8 15 12 17 14 10 17 0.253551 127/128 NonOverlappingTemplate 16 15 10 15 12 14 10 9 14 13 0.834308 125/128 NonOverlappingTemplate 8 16 21 15 19 10 13 7 8 11 0.033288 127/128 NonOverlappingTemplate 16 15 15 9 14 11 12 12 10 14 0.862344 128/128 NonOverlappingTemplate 16 9 20 9 16 14 16 11 10 7 0.134686 128/128 NonOverlappingTemplate 8 13 12 15 9 8 15 14 25 9 0.016911 126/128 NonOverlappingTemplate 10 13 10 17 13 9 12 13 14 17 0.723129 127/128 NonOverlappingTemplate 12 14 12 19 15 7 15 7 14 13 0.311542 128/128 NonOverlappingTemplate 18 11 20 14 10 12 8 14 7 14 0.155209 128/128 NonOverlappingTemplate 11 10 10 19 12 14 13 12 14 13 0.772760 127/128 NonOverlappingTemplate 17 11 9 14 12 15 9 13 20 8 0.242986 126/128 NonOverlappingTemplate 18 9 10 11 12 17 10 14 12 15 0.568055 127/128 NonOverlappingTemplate 17 7 17 14 5 18 12 14 12 12 0.122325 126/128 NonOverlappingTemplate 12 14 5 13 15 20 10 11 14 14 0.232760 127/128 NonOverlappingTemplate 11 13 14 13 8 12 14 18 17 8 0.468595 127/128 NonOverlappingTemplate 17 12 12 16 14 6 11 14 17 9 0.350485 126/128 NonOverlappingTemplate 14 17 11 14 10 10 18 12 9 13 0.602458 128/128 NonOverlappingTemplate 18 10 16 13 14 10 13 15 9 10 0.602458 126/128 NonOverlappingTemplate 16 13 14 13 13 12 10 8 18 11 0.671779 127/128 NonOverlappingTemplate 12 14 12 14 16 16 7 10 17 10 0.517442 126/128 NonOverlappingTemplate 13 15 13 19 14 3 12 18 8 13 0.057146 127/128 NonOverlappingTemplate 16 14 11 13 13 9 13 14 12 13 0.957319 124/128 NonOverlappingTemplate 13 12 12 13 16 11 9 13 16 13 0.922036 125/128 NonOverlappingTemplate 8 10 14 16 16 6 14 14 17 13 0.311542 128/128 NonOverlappingTemplate 19 9 18 10 10 8 12 12 17 13 0.213309 125/128 NonOverlappingTemplate 11 10 16 17 12 7 13 18 15 9 0.311542 127/128 NonOverlappingTemplate 16 12 14 15 12 14 11 12 10 12 0.957319 127/128 NonOverlappingTemplate 16 14 13 11 12 14 3 15 18 12 0.178278 126/128 NonOverlappingTemplate 18 10 16 7 19 7 14 12 12 13 0.148094 128/128 NonOverlappingTemplate 16 14 10 11 9 13 17 13 11 14 0.788728 127/128 NonOverlappingTemplate 15 15 12 11 13 12 12 9 14 15 0.941144 126/128 NonOverlappingTemplate 16 13 16 9 11 10 13 15 14 11 0.819544 128/128 NonOverlappingTemplate 13 12 11 15 8 20 16 8 11 14 0.299251 127/128 NonOverlappingTemplate 20 12 10 8 14 10 16 18 10 10 0.178278 124/128 NonOverlappingTemplate 15 15 9 19 12 15 7 15 8 13 0.253551 126/128 NonOverlappingTemplate 10 17 10 20 16 12 11 13 9 10 0.299251 125/128 NonOverlappingTemplate 12 15 12 9 17 10 9 20 15 9 0.242986 126/128 NonOverlappingTemplate 10 8 20 10 17 16 19 9 8 11 0.048716 127/128 NonOverlappingTemplate 13 13 3 16 13 11 16 16 15 12 0.222869 127/128 NonOverlappingTemplate 7 13 12 12 16 7 21 13 9 18 0.063482 127/128 NonOverlappingTemplate 9 9 12 15 17 17 12 7 15 15 0.350485 128/128 NonOverlappingTemplate 11 8 14 11 18 12 12 15 10 17 0.534146 128/128 NonOverlappingTemplate 10 10 11 16 15 10 16 11 11 18 0.568055 127/128 NonOverlappingTemplate 10 20 15 12 8 9 8 14 21 11 0.048716 126/128 NonOverlappingTemplate 4 8 12 13 20 11 15 16 12 17 0.060239 128/128 NonOverlappingTemplate 19 8 16 12 11 13 15 9 10 15 0.392456 125/128 NonOverlappingTemplate 16 15 19 9 13 8 6 15 14 13 0.186566 124/128 NonOverlappingTemplate 13 6 12 14 10 10 19 18 11 15 0.213309 126/128 NonOverlappingTemplate 9 14 11 13 11 12 13 18 12 15 0.819544 127/128 NonOverlappingTemplate 11 9 8 15 16 15 16 16 10 12 0.534146 126/128 NonOverlappingTemplate 13 12 14 11 19 16 11 11 11 10 0.689019 126/128 NonOverlappingTemplate 16 11 13 17 11 12 8 16 13 11 0.689019 126/128 NonOverlappingTemplate 13 15 13 12 9 12 17 13 14 10 0.875539 127/128 NonOverlappingTemplate 10 9 15 14 13 13 10 12 13 19 0.654467 128/128 NonOverlappingTemplate 18 11 14 10 13 11 10 18 12 11 0.602458 127/128 NonOverlappingTemplate 14 7 8 12 14 16 15 11 17 14 0.468595 125/128 NonOverlappingTemplate 20 13 9 15 11 19 11 9 13 8 0.148094 124/128 NonOverlappingTemplate 11 12 15 16 11 15 9 12 18 9 0.585209 128/128 NonOverlappingTemplate 6 15 10 20 7 16 9 16 16 13 0.060239 127/128 NonOverlappingTemplate 17 10 13 15 12 17 10 13 9 12 0.689019 125/128 NonOverlappingTemplate 15 12 12 11 13 13 10 13 17 12 0.941144 126/128 NonOverlappingTemplate 13 12 11 15 12 11 11 13 18 12 0.900104 127/128 NonOverlappingTemplate 15 14 11 8 10 11 12 14 16 17 0.671779 126/128 NonOverlappingTemplate 10 19 12 16 15 9 8 10 17 12 0.275709 126/128 NonOverlappingTemplate 12 16 14 9 12 25 8 9 12 11 0.028181 128/128 NonOverlappingTemplate 12 15 16 15 10 16 7 12 11 14 0.637119 127/128 NonOverlappingTemplate 12 9 12 13 11 16 14 15 13 13 0.941144 127/128 NonOverlappingTemplate 15 12 10 13 12 18 16 8 11 13 0.637119 127/128 NonOverlappingTemplate 15 8 13 15 8 11 19 13 16 10 0.337162 126/128 NonOverlappingTemplate 11 13 12 11 10 15 9 12 22 13 0.311542 128/128 NonOverlappingTemplate 10 10 11 14 15 10 16 14 15 13 0.862344 127/128 NonOverlappingTemplate 14 13 14 8 13 19 9 14 15 9 0.452799 126/128 NonOverlappingTemplate 14 16 10 9 13 14 12 17 10 13 0.772760 128/128 NonOverlappingTemplate 12 12 9 15 12 17 14 10 12 15 0.834308 126/128 NonOverlappingTemplate 20 11 16 14 12 14 11 11 6 13 0.299251 123/128 NonOverlappingTemplate 20 10 15 11 11 13 15 10 12 11 0.551026 127/128 NonOverlappingTemplate 14 13 12 12 11 15 16 7 13 15 0.788728 126/128 NonOverlappingTemplate 11 13 17 10 6 14 14 15 11 17 0.422034 127/128 NonOverlappingTemplate 11 9 11 9 16 13 14 16 14 15 0.756476 128/128 NonOverlappingTemplate 11 15 17 16 10 9 14 12 12 12 0.772760 126/128 NonOverlappingTemplate 16 21 10 15 8 17 8 10 11 12 0.110952 126/128 NonOverlappingTemplate 11 12 15 16 14 11 16 13 10 10 0.862344 128/128 NonOverlappingTemplate 14 11 6 11 14 17 14 19 11 11 0.311542 125/128 NonOverlappingTemplate 11 15 11 14 20 9 9 11 15 13 0.437274 126/128 NonOverlappingTemplate 12 12 14 13 9 16 22 9 11 10 0.213309 128/128 NonOverlappingTemplate 14 14 10 11 20 9 13 10 10 17 0.350485 128/128 NonOverlappingTemplate 14 10 11 12 12 11 13 15 18 12 0.862344 127/128 NonOverlappingTemplate 14 15 13 10 10 9 20 12 10 15 0.437274 125/128 NonOverlappingTemplate 15 11 6 17 14 14 11 8 15 17 0.287306 126/128 NonOverlappingTemplate 18 15 10 5 10 12 17 12 16 13 0.213309 125/128 NonOverlappingTemplate 16 15 17 12 14 13 10 10 7 14 0.568055 128/128 NonOverlappingTemplate 10 10 17 11 11 10 16 14 16 13 0.706149 128/128 NonOverlappingTemplate 16 15 16 14 5 12 13 11 9 17 0.287306 126/128 NonOverlappingTemplate 17 14 13 10 19 8 12 12 11 12 0.500934 128/128 NonOverlappingTemplate 18 8 12 9 13 18 11 14 16 9 0.299251 125/128 NonOverlappingTemplate 16 14 11 14 12 10 12 14 12 13 0.970538 124/128 NonOverlappingTemplate 11 9 15 12 16 11 12 13 17 12 0.819544 128/128 OverlappingTemplate 12 16 11 13 18 16 10 13 12 7 0.500934 126/128 Universal 8 14 13 12 7 18 14 13 12 17 0.407091 125/128 ApproximateEntropy 11 6 6 9 8 5 7 3 11 10 0.374107 75/76 RandomExcursions 3 8 8 8 10 9 7 10 8 5 0.681642 76/76 RandomExcursions 12 11 2 5 10 6 6 7 7 10 0.169178 75/76 RandomExcursions 7 6 11 11 9 9 5 9 8 1 0.197677 76/76 RandomExcursions 7 8 6 10 11 8 6 8 5 7 0.846579 76/76 RandomExcursions 10 7 7 10 8 9 9 9 4 3 0.534146 72/76 RandomExcursions 8 4 13 10 4 6 8 5 9 9 0.266044 75/76 RandomExcursions 11 6 7 9 9 9 7 6 4 8 0.768138 75/76 RandomExcursions 6 9 10 10 10 4 8 3 11 5 0.266044 74/76 RandomExcursionsVariant 6 9 10 9 8 10 3 8 2 11 0.197677 75/76 RandomExcursionsVariant 7 10 8 7 8 9 8 3 7 9 0.821681 75/76 RandomExcursionsVariant 7 11 5 7 12 8 6 6 7 7 0.651990 75/76 RandomExcursionsVariant 8 6 4 14 6 11 6 13 4 4 0.026435 75/76 RandomExcursionsVariant 7 5 11 6 8 11 9 3 9 7 0.450564 76/76 RandomExcursionsVariant 5 9 5 11 7 7 8 11 7 6 0.681642 75/76 RandomExcursionsVariant 4 6 10 6 8 12 5 5 13 7 0.169178 75/76 RandomExcursionsVariant 5 7 4 11 8 9 8 10 5 9 0.592591 75/76 RandomExcursionsVariant 6 11 6 5 10 9 6 7 6 10 0.681642 75/76 RandomExcursionsVariant 12 8 7 10 4 6 6 7 7 9 0.622249 75/76 RandomExcursionsVariant 11 6 15 5 11 6 6 6 6 4 0.061150 76/76 RandomExcursionsVariant 8 10 9 11 3 4 13 4 5 9 0.079817 76/76 RandomExcursionsVariant 4 14 6 10 5 9 4 10 4 10 0.066882 76/76 RandomExcursionsVariant 6 10 9 8 7 5 8 6 9 8 0.929192 76/76 RandomExcursionsVariant 7 12 3 6 7 10 7 11 10 3 0.156248 76/76 RandomExcursionsVariant 8 9 5 6 6 14 6 12 6 4 0.132858 75/76 RandomExcursionsVariant 6 6 11 9 9 4 10 6 2 13 0.087086 75/76 RandomExcursionsVariant 13 10 20 13 13 13 17 10 10 9 0.392456 126/128 Serial 14 12 13 12 21 8 14 7 9 18 0.100508 126/128 Serial 14 16 17 12 10 13 15 17 10 4 0.178278 126/128 LinearComplexity - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - The minimum pass rate for each statistical test with the exception of the random excursion (variant) test is approximately = 123 for a sample size = 128 binary sequences. The minimum pass rate for the random excursion (variant) test is approximately = 72 for a sample size = 76 binary sequences. For further guidelines construct a probability table using the MAPLE program provided in the addendum section of the documentation. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
In addition to the test summary, detailed results from the various tests are written to a number of data files. You can download a ZIPped archive containing all of these results, and the HotBits pseudorandom data stream which was tested to produce them from the links below.
|