« Astronomical Numbers | Main | Reading List: A.I. Apocalypse »

Saturday, April 18, 2015

Reading List: Einstein's Unification

van Dongen, Jeroen. Einstein's Unification. Cambridge: Cambridge University Press, 2010. ISBN 978-0-521-88346-7.
In 1905 Albert Einstein published four papers which transformed the understanding of space, time, mass, and energy; provided physical evidence for the quantisation of energy; and observational confirmation of the existence of atoms. These publications are collectively called the Annus Mirabilis papers, and vaulted the largely unknown Einstein to the top rank of theoretical physicists. When Einstein was awarded the Nobel Prize in Physics in 1921, it was for one of these 1905 papers which explained the photoelectric effect. Einstein's 1905 papers are masterpieces of intuitive reasoning and clear exposition, and demonstrated Einstein's technique of constructing thought experiments based upon physical observations, then deriving testable mathematical models from them. Unlike so many present-day scientific publications, Einstein's papers on special relativity and the equivalence of mass and energy were accessible to anybody with a college-level understanding of mechanics and electrodynamics and used no special jargon or advanced mathematics. Being based on well-understood concepts, neither cited any other scientific paper.

While special relativity revolutionised our understanding of space and time, and has withstood every experimental test to which it has been subjected in the more than a century since it was formulated, it was known from inception that the theory was incomplete. It's called special relativity because it only describes the behaviour of bodies under the special case of uniform unaccelerated motion in the absence of gravity. To handle acceleration and gravitation would require extending the special theory into a general theory of relativity, and it is upon this quest that Einstein next embarked.

As before, Einstein began with a simple thought experiment. Just as in special relativity, where there is no experiment which can be done in a laboratory without the ability to observe the outside world that can determine its speed or direction of uniform (unaccelerated) motion, Einstein argued that there should be no experiment an observer could perform in a sufficiently small closed laboratory which could distinguish uniform acceleration from the effect of gravity. If one observed objects to fall with an acceleration equal to that on the surface of the Earth, the laboratory might be stationary on the Earth or in a space ship accelerating with a constant acceleration of one gravity, and no experiment could distinguish the two situations. (The reason for the “sufficiently small” qualification is that since gravity is produced by massive objects, the direction a test particle will fall depends upon its position with respect to the centre of gravity of the body. In a very large laboratory, objects dropped far apart would fall in different directions. This is what causes tides.)

Einstein called this observation the “equivalence principle”: that the effects of acceleration and gravity are indistinguishable, and that hence a theory which extended special relativity to incorporate accelerated motion would necessarily also be a theory of gravity. Einstein had originally hoped it would be straightforward to reconcile special relativity with acceleration and gravity, but the deeper he got into the problem, the more he appreciated how difficult a task he had undertaken. Thanks to the Einstein Papers Project, which is curating and publishing all of Einstein's extant work, including notebooks, letters, and other documents, the author (a participant in the project) has been able to reconstruct Einstein's ten-year search for a viable theory of general relativity.

Einstein pursued a two-track approach. The bottom up path started with Newtonian gravity and attempted to generalise it to make it compatible with special relativity. In this attempt, Einstein was guided by the correspondence principle, which requires that any new theory which explains behaviour under previously untested conditions must reproduce the tested results of existing theory under known conditions. For example, the equations of motion in special relativity reduce to those of Newtonian mechanics when velocities are small compared to the speed of light. Similarly, for gravity, any candidate theory must yield results identical to Newtonian gravitation when field strength is weak and velocities are low.

From the top down, Einstein concluded that any theory compatible with the principle of equivalence between acceleration and gravity must exhibit general covariance, which can be thought of as being equally valid regardless of the choice of co-ordinates (as long as they are varied without discontinuities). There are very few mathematical structures which have this property, and Einstein was drawn to Riemann's tensor geometry. Over years of work, Einstein pursued both paths, producing a bottom-up theory which was not generally covariant which he eventually rejected as in conflict with experiment. By November 1915 he had returned to the top-down mathematical approach and in four papers expounded a generally covariant theory which agreed with experiment. General relativity had arrived.

Einstein's 1915 theory correctly predicted the anomalous perihelion precession of Mercury and also predicted that starlight passing near the limb of the Sun would be deflected by twice the angle expected based on Newtonian gravitation. This was confirmed (within a rather large margin of error) in an eclipse expedition in 1919, which made Einstein's general relativity front page news around the world. Since then precision tests of general relativity have tested a variety of predictions of the theory with ever-increasing precision, with no experiment to date yielding results inconsistent with the theory.

Thus, by 1915, Einstein had produced theories of mechanics, electrodynamics, the equivalence of mass and energy, and the mechanics of bodies under acceleration and the influence of gravitational fields, and changed space and time from a fixed background in which physics occurs to a dynamical arena: “Matter and energy tell spacetime how to curve. Spacetime tells matter how to move.” What do you do, at age 36, having figured out, largely on your own, how a large part of the universe works?

Much of Einstein's work so far had consisted of unification. Special relativity unified space and time, matter and energy. General relativity unified acceleration and gravitation, gravitation and geometry. But much remained to be unified. In general relativity and classical electrodynamics there were two field theories, both defined on the continuum, both with unlimited range and an inverse square law, both exhibiting static and dynamic effects (although the details of gravitomagnetism would not be worked out until later). And yet the theories seemed entirely distinct: gravity was always attractive and worked by the bending of spacetime by matter-energy, while electromagnetism could be either attractive or repulsive, and seemed to be propagated by fields emitted by point charges—how messy.

Further, quantum theory, which Einstein's 1905 paper on the photoelectric effect had helped launch, seemed to point in a very different direction than the classical field theories in which Einstein had worked. Quantum mechanics, especially as elaborated in the “new” quantum theory of the 1920s, seemed to indicate that aspects of the universe such as electric charge were discrete, not continuous, and that physics could, even in principle, only predict the probability of the outcome of experiments, not calculate them definitively from known initial conditions. Einstein never disputed the successes of quantum theory in explaining experimental results, but suspected it was a theory based upon phenomena which did not explain what was going on at a deeper level. (For example, the physical theory of elasticity explains experimental results and makes predictions within its domain of applicability, but it is not fundamental. All of the effects of elasticity are ultimately due to electromagnetic forces between atoms in materials. But that doesn't mean that the theory of elasticity isn't useful to engineers, or that they should do their spring calculations at the molecular level.)

Einstein undertook the search for a unified field theory, which would unify gravity and electromagnetism, just as Maxwell had unified electrostatics and magnetism into a single theory. In addition, Einstein believed that a unified field theory would be antecedent to quantum theory, and that the probabilistic results of quantum theory could be deduced from the more fundamental theory, which would remain entirely deterministic. From 1915 until his death in 1955 Einstein's work concentrated mostly on the quest for a unified field theory. He was aided by numerous talented assistants, many of whom went on to do important work in their own right. He explored a variety of paths to such a theory, but ultimately rejected each one, in turn, as either inconsistent with experiment or unable to explain phenomena such as point particles or quantisation of charge.

As the author documents, Einstein's approach to doing physics changed in the years after 1915. While before he was guided both by physics and mathematics, in retrospect he recalled and described his search of the field equations of general relativity as having followed the path of discovering the simplest and most elegant mathematical structure which could explain the observed phenomena. He thus came, like Dirac, to argue that mathematical beauty was the best guide to correct physical theories.

In the last forty years of his life, Einstein made no progress whatsoever toward a unified field theory, apart from discarding numerous paths which did not work. He explored a variety of approaches: “semivectors” (which turned out just to be a reformulation of spinors), five-dimensional models including a cylindrically compactified dimension based on Kaluza-Klein theory, and attempts to deduce the properties of particles and their quantum behaviour from nonlinear continuum field theories.

In seeking to unify electromagnetism and gravity, he ignored the strong and weak nuclear forces which had been discovered over the years and merited being included in any grand scheme of unification. In the years after World War II, many physicists ceased to worry about the meaning of quantum mechanics and the seemingly inherent randomness in its predictions which so distressed Einstein, and adopted a “shut up and calculate” approach as their computations were confirmed to ever greater precision by experiments.

So great was the respect for Einstein's achievements that only rarely was a disparaging word said about his work on unified field theories, but toward the end of his life it was outside the mainstream of theoretical physics, which had moved on to elaboration of quantum theory and making quantum theory compatible with special relativity. It would be a decade after Einstein's death before astronomical discoveries would make general relativity once again a frontier in physics.

What can we learn from the latter half of Einstein's life and his pursuit of unification? The frontier of physics today remains unification among the forces and particles we have discovered. Now we have three forces to unify (counting electromagnetism and the weak nuclear force as already unified in the electroweak force), plus two seemingly incompatible kinds of particles: bosons (carriers of force) and fermions (what stuff is made of). Six decades (to the day) after the death of Einstein, unification of gravity and the other forces remains as elusive as when he first attempted it.

It is a noble task to try to unify disparate facts and theories into a common whole. Much of our progress in the age of science has come from such unification. Einstein unified space and time; matter and energy; acceleration and gravity; geometry and motion. We all benefit every day from technologies dependent upon these fundamental discoveries. He spent the last forty years of his life seeking the next grand unification. He never found it. For this effort we should applaud him.

I must remark upon how absurd the price of this book is. At Amazon as of this writing, the hardcover is US$ 102.91 and the Kindle edition is US$ 88. Eighty-eight Yankee dollars for a 224 page book which is ranked #739,058 in the Kindle store?

Posted at April 18, 2015 15:09