« August 11, 2010 | Main | August 25, 2010 »

Thursday, August 19, 2010

Reading List: Plastic Fantastic

Reich, Eugenie Samuel. Plastic Fantastic. New York: St. Martin's Press, 2009. ISBN 978-0-230-62384-2.
Boosters of Big Science, and the politicians who rely upon its pronouncements to justify their policy prescriptions often cite the self-correcting nature of the scientific process: peer review subjects the work of researchers to independent and dispassionate scrutiny before results are published, and should an incorrect result make it into print, the failure of independent researchers to replicate it will inevitably call it into question and eventually cause it to be refuted.

Well, that's how it works in theory. Theory is very big in contemporary Big Science. This book is about how things work in fact, in the real world, and it's quite a bit different. At the turn of the century, there was no hotter property in condensed matter physics than Hendrik Schön, a junior researcher at Bell Labs who, in rapid succession reported breakthroughs in electronic devices fabricated from organic molecules including:

  • Organic field effect transistors
  • Field-induced superconductivity in organic crystals
  • Fractional quantum Hall effect in organic materials
  • Organic crystal laser
  • Light emitting organic transistor
  • Organic Josephson junction
  • High temperature superconductivity in C60
  • Single electron organic transistors

In the year 2001, Schön published a paper in a peer reviewed journal at a rate of one every eight days, with many reaching the empyrean heights of Nature, Science, and Physical Review. Other labs were in awe of his results, and puzzled because every attempt they made to replicate his experiments failed, often in ways which seemed to indicate the descriptions of experiments he published were insufficient for others to replicate them. Theorists also raised their eyebrows at Schön's results, because he claimed breakdown properties of sputtered aluminium oxide insulating layers far beyond measured experimental results, and behaviour of charge transport in his organic substrates which didn't make any sense according to the known properties of such materials.

The experimenters were in a tizzy, trying to figure out why they couldn't replicate Schön's results, while the theorists were filling blackboards trying to understand how his incongruous results could possibly make sense. His superiors were basking in the reflected glory of his ascendence into the élite of experimental physicists and the reflection of his glory upon their laboratory.

In April 2002, while waiting in the patent attorney's office at Bell Labs, researchers Julia Hsu and Lynn Loo were thumbing through copies of Schön's papers they'd printed out as background documentation for the patent application they were preparing, when Loo noticed that two graphs of inverter outputs, one in a Nature paper describing a device made of a layer of thousands of organic molecules, and another in a Science paper describing an inverter made of just one or two active molecules were identical, right down to the instrumental noise. When this was brought to the attention of Schön's manager and word of possible irregularities in Schön's publications began to make its way through the condensed matter physics grapevine, his work was subjected to intense scrutiny both within Bell Labs and by outside researchers, and additional instances of identical graphs re-labelled for entirely different experiments came to hand. Bell Labs launched a formal investigation in May 2002, which concluded, in a report issued the following September, that Schön had committed at least 16 instances of scientific misconduct, fabricating the experimental data he reported from mathematical functions, with no evidence whatsoever that he had ever built the devices he claimed to have, or performed the experiments described in his papers. A total of twenty-one papers authored by Schön in Science, Nature, and Physical Review were withdrawn, as well as a number in less prestigious venues.

What is fascinating in this saga of flat-out fraud and ultimate exposure and disgrace is how completely the much-vaunted system of checks and balances of industrial scale Big Science and peer review in the most prestigious journals completely fell on its face at the hands of a fraudster in a junior position with little or no scientific track record who was willing to make up data to confirm the published expectations of the theorists, and figured out how to game the peer review system, using criticisms of his papers as a guide to make up additional data to satisfy the objections of the referees. As a former manager of a group of ambitious and rambunctious technologists, what strikes me is how utterly Schön's colleagues and managers at Bell Labs failed in overseeing his work and vetting his results. “Extraordinary claims require extraordinary evidence”, and Schön was making and publishing extraordinary claims at the rate of almost one a week in 2001, and yet not once did anybody at Bell Labs insist on observing him perform one of the experiments he claimed to be performing, even after other meticulous experimenters in laboratories around the world reported that they were unable to replicate his results. Think about it—if a junior software developer in your company claimed to have developed a miraculous application, wouldn't you want to see a demo before issuing a press release about it and filing a patent application? And yet nobody at Bell Labs thought to do so with Schön's work.

The lessons from this episode are profound, and I see little evidence that they have been internalised by the science establishment. A great deal of experimental science is now guided by the expectations of theorists; it is difficult to obtain funding for an experimental program which looks for effects not anticipated by theory. In such an environment, an unscrupulous scientist willing to make up data that conforms to the prejudices of the theorists may be able to publish in prestigious journals and be considered a rising star of science based on an entirely fraudulent corpus of work. Because scientists, especially in the Anglo-Saxon culture, are loath to make accusations of fraud (as the author notes, in the golden age of British science such an allegation might well result in a duel being fought), failure to replicate experimental results is often assumed to be a failure by the replicator to precisely reproduce the circumstances of the original investigator, not to call into question the veracity of the reported work. Schön's work consisted of desktop experiments involving straightforward measurements of electrical properties of materials, which were about as simple as anything in contemporary science to evaluate and independently replicate. Now think of how vulnerable research on far less clear cut topics such as global climate, effects of diet on public health, and other topics would be to fraudulent, agenda-driven “research”. Also, Schön got caught only because he became sloppy in his frenzy of publication, duplicating graphs and data sets from one paper to another. How long could a more careful charlatan get away with it?

Quite aside from the fascinating story and its implications for the integrity of the contemporary scientific enterprise, this is a superbly written narrative which reads more like a thriller than an account of a regrettable episode in science. But it is entirely factual, and documented with extensive end notes citing original sources.

Posted at 22:15 Permalink