« Reading List: A Time for War | Main | Reading List: Bargaining Position »

Sunday, March 31, 2013

Reading List: Colossus

Copeland, B. Jack, ed. Colossus. Oxford: Oxford University Press, 2006. ISBN 978-0-19-953680-1.
During World War II the British codebreakers at Bletchley Park provided intelligence to senior political officials and military commanders which was vital in winning the Battle of the Atlantic and discerning German strategic intentions in the build-up to the invasion of France and the subsequent campaign in Europe. Breaking the German codes was just barely on the edge of possibility with the technology of the time, and required recruiting a cadre of exceptionally talented and often highly eccentric individuals and creating tools which laid the foundations for modern computer technology.

At the end of the war, all of the work of the codebreakers remained under the seal of secrecy: in Winston Churchill's history of the war it was never mentioned. Part of this was due to the inertia of the state to relinquish its control over information, but also because the Soviets, emerging as the new adversary, might adopt some of the same cryptographic techniques used by the Germans and concealing that they had been compromised might yield valuable information from intercepts of Soviet communications.

As early as the 1960s, publications in the United States began to describe the exploits of the codebreakers, and gave the mistaken impression that U.S. codebreakers were in the vanguard simply because they were the only ones allowed to talk about their wartime work. The heavy hand of the Official Secrets Act suppressed free discussion of the work at Bletchley Park until June 2000, when the key report, written in 1945, was allowed to be published.

Now it can be told. Fortunately, many of the participants in the work at Bletchley were young and still around when finally permitted to discuss their exploits. This volume is largely a collection of their recollections, many in great technical detail. You will finally understand precisely which vulnerabilities of the German cryptosystems permitted them to be broken (as is often the case, it was all-too-clever innovations by the designers intended to make the encryption “unbreakable” which provided the door into it for the codebreakers) and how sloppy key discipline among users facilitated decryption. For example, it was common to discover two or more messages encrypted with the same key. Since encryption was done by a binary exclusive or (XOR) of the bits of the Baudot teleprinter code, with that of the key (generated mechanically from a specified starting position of the code machine's wheels), if you have two messages encrypted with the same key, you can XOR them together, taking out the key and leaving you with the XOR of the plaintext of the two messages. This, of course, will be gibberish, but you can then take common words and phrases which occur in messages and “slide” them along the text, XORing as you go, to see if the result makes sense. If it does, you've recovered part of the other message, and by XORing with either message, that part of the key. This is something one could do in microseconds today with the simplest of computer programs, but in the day was done in kiloseconds by clerks looking up the XOR of Baudot codes in tables one by one (at least until they memorised them, which the better ones did).

The chapters are written by people with expertise in the topic discussed, many of whom were there. The people at Bletchley had to make up the terminology for the unprecedented things they were doing as they did it. Due to the veil of secrecy dropped over their work, many of their terms were orphaned. What we call “bits” they called “pulses”, “binary addition” XOR, and ones and zeroes of binary notation crosses and dots. It is all very quaint and delightful, and used in most of these documents.

After reading this book you will understand precisely how the German codes were broken, what Colossus did, how it was built and what challenges were overcome in constructing it, and how it was integrated into a system incorporating large numbers of intuitive humans able to deliver near-real-time intelligence to decision makers. The level of detail may be intimidating to some, but for the first time it's all there. I have never before read any description of the key flaw in the Lorenz cipher which Colossus exploited and how it processed messages punched on loops of paper tape to break into them and recover the key.

The aftermath of Bletchley was interesting. All of the participants were sworn to secrecy and all of their publications kept under high security. But the know-how they had developed in electronic computation was their own, and many of them went to Manchester to develop the pioneering digital computers developed there. The developers of much of this technology could not speak of whence it came, and until recent years the history of computing has been disconnected from its roots.

As a collection of essays, this book is uneven and occasionally repetitive. But it is authentic, and an essential document for anybody interested in how codebreaking was done in World War II and how electronic computation came to be.

Posted at March 31, 2013 23:46