« Reading List: They Thought They Were Free | Main | Reading List: Little Wars »

Tuesday, September 26, 2006

Reading List: The Trouble with Physics

Smolin, Lee. The Trouble with Physics. New York: Houghton Mifflin, 2006. ISBN 0-618-55105-0.
The first forty years of the twentieth century saw a revolution in fundamental physics: special and general relativity changed our perception of space, time, matter, energy, and gravitation; quantum theory explained all of chemistry while wiping away the clockwork determinism of classical mechanics and replacing it with a deeply mysterious theory which yields fantastically precise predictions yet nobody really understands at its deepest levels; and the structure of the atom was elucidated, along with important clues to the mysteries of the nucleus. In the large, the universe was found to be enormously larger than expected and expanding—a dynamic arena which some suspected might have an origin and a future vastly different than its present state.

The next forty years worked out the structure and interactions of the particles and forces which constitute matter and govern its interactions, resulting in a standard model of particle physics with precisely defined theories which predicted all of the myriad phenomena observed in particle accelerators and in the highest energy events in the heavens. The universe was found to have originated in a big bang no more distant than three times the age of the Earth, and the birth cry of the universe had been detected by radio telescopes.

And then? Unexpected by almost all practitioners of high energy particle physics, which had become an enterprise larger by far than all of science at the start of the century, progress stopped. Since the wrapping up of the standard model around 1975, experiments have simply confirmed its predictions (with the exception of the discovery of neutrino oscillations and consequent mass, but that can be accommodated within the standard model without changing its structure), and no theoretical prediction of phenomena beyond the standard model has been confirmed experimentally.

What went wrong? Well, we certainly haven't reached the End of Science or even the End of Physics, because the theories which govern phenomena in the very small and very large—quantum mechanics and general relativity—are fundamentally incompatible with one another and produce nonsensical or infinite results when you attempt to perform calculations in the domain—known to exist from astronomical observations—where both must apply. Even a calculation as seemingly straightforward as estimating the energy of empty space yields a result which is 120 orders of magnitude greater than experiment shows it to be: perhaps the most embarrassing prediction in the history of science.

In the first chapter of this tour de force, physicist Lee Smolin poses “The Five Great Problems in Theoretical Physics”, all of which are just as mysterious today as they were thirty-five years ago. Subsequent chapters explore the origin and nature of these problems, and how it came to be, despite unprecedented levels of funding for theoretical and experimental physics, that we seem to be getting nowhere in resolving any of these fundamental enigmas.

This prolonged dry spell in high energy physics has seen the emergence of string theory (or superstring theory, or M-theory, or whatever they're calling it this year) as the dominant research program in fundamental physics. At the outset, there were a number of excellent reasons to believe that string theory pointed the way to a grand unification of all of the forces and particles of physics, and might answer many, if not all, of the Great Problems. This motivated many very bright people, including the author (who, although most identified with loop quantum gravity research, has published in string theory as well) to pursue this direction. What is difficult for an outsider to comprehend, however, is how a theoretical program which, after thirty-five years of intensive effort, has yet to make a single prediction testable by a plausible experiment; has failed to predict any of the major scientific surprises that have occurred over those years such as the accelerating expansion of the universe and the apparent variation in the fine structure constant; that does not even now exist in a well-defined mathematical form; and has not been rigorously proved to be a finite theory; has established itself as a virtual intellectual monopoly in the academy, forcing aspiring young theorists to work in string theory if they are to have any hope of finding a job, receiving grants, or obtaining tenure.

It is this phenomenon, not string theory itself, which, in the author's opinion, is the real “Trouble with Physics”. He considers string theory as quite possibly providing clues (though not the complete solution) to the great problems, and finds much to admire in many practitioners of this research. But monoculture is as damaging in academia as in agriculture, and when it becomes deeply entrenched in research institutions, squeezes out other approaches of equal or greater merit. He draws the distinction between “craftspeople”, who are good at performing calculations, filling in blanks, and extending an existing framework, and “seers”, who make the great intellectual leaps which create entirely new frameworks. After thirty-five years with no testable result, there are plenty of reasons to suspect a new framework is needed, yet our institutions select out those most likely to discover them, or force them to spend their most intellectually creative years doing tedious string theory calculations at the behest of their elders.

In the final chapters, Smolin looks at how academic science actually works today: how hiring and tenure decisions are made, how grant applications are evaluated, and the difficult career choices young physicists must make to work within this system. When reading this, the word “Gosplan” (Госпла́н) kept flashing through my mind, for the process he describes resembles nothing so much as central planning in a command economy: a small group of senior people, distant from the facts on the ground and the cutting edge of intellectual progress, trying to direct a grand effort in the interest of “efficiency”. But the lesson of more than a century of failed socialist experiments is that, in the timeless words of Rocket J. Squirrel, “that trick never works”—the decisions inevitably come down on the side of risk aversion, and are often influenced by cronyism and toadying to figures in authority. The concept of managing risk and reward by building a diversified portfolio of low and high risk placements which is second nature to managers of venture capital funds and industrial research and development laboratories appears to be totally absent in academic science, which is supposed to be working on the most difficult and fundamental questions. Central planning works abysmally for cement and steel manufacturing; how likely is it to spark the next scientific revolution?

There is much more to ponder: why string theory, as presently defined, cannot possibly be a complete theory which subsumes general relativity; hints from experiments which point to new physics beyond string theory; stories of other mathematically beautiful theories (such as SU(5) grand unification) which experiment showed to be dead wrong; and a candid view of the troubling groupthink, appeal to authority, and intellectual arrogance of some members of the string theory community. As with all of Smolin's writing, this is a joy to read, and you get the sense that he's telling you the straight story, as honestly as he can, not trying to sell you something. If you're interested in these issues, you'll probably also want to read Leonard Susskind's pro-string The Cosmic Landscape and Peter Woit's sceptical Not Even Wrong.

Posted at September 26, 2006 23:52