'How Bold Predictions Hurt Science'

UPDATE: Read also Richard Gallagher’s “Authors of our own misfortune

How many expert assurances or warnings must turn out to be conspicuously wrong for the authority of science and scientists to be diminished?“: that’s the ominous conclusion of a beautifully no-holds-barred article today:

Promises, Promises – Ill-judged predictions and projections can be embarrassing at best and, at worst, damaging to the authority of science and science policy. by Stuart Blackman – The Scientist, Vol 23, Issue 11, Page 28

The article is full of interesting quotes. Excerpts:

  • It doesn’t take anything so extreme as scientific fraud to scupper what may have seemed, at the time, to be a well-grounded scientific prediction. At its most enthusiastic, science has always been prone to promise rather more, and sooner, than it has managed to deliver
  • Scientists have a strong incentive to make bold predictions—namely, to obtain funding, influence, and high-profile publications. But [...] unfulfilled predictions [...] can be a blow for patients, policy makers, and for the reputation of science itself
  • [The 1995 Varmus NIH expert panel concluded that] ‘overzealous representation of clinical gene therapy has [led to] misrepresentation [that] threatens confidence in the field and will inevitably lead to disappointment in both medical and lay communities
  • says Brian Wynne, professor of science studies at Lancaster University, UK. ‘Every research proposal these days [...] has got to include an [impact]  statement [...] basically requiring scientists to make promises, and to exaggerate those promises.
  • As British fertility expert Robert Winston told the BBC in 2005: ‘We tend often to really have rather too much overconfidence. We may exaggerate, simply because [...] we need support [...] We can go about persuading people a bit too vigorously sometimes.
  • Predictions can also create a sense of haste and urgency that can impede cool, calm reflection on how to proceed at the policy level. [Nik Brown, co-director of the Science and Technology Studies Unit, University of York, UK] says it can create a pressure to legislate before experts properly understand a new research path and its potential.
  • Research [by Joan Haran, Cesagen Research Fellow at Cardiff University, UK shows that] ‘Because of the high esteem in which scientists are held, it becomes very hard to mount a critique of their promises,‘ [...] Scientists defending their corner is understandable, says Haran, but it should be recognized that it can be at the expense of healthy skepticism.
  • Predictions can also create a sense of haste and urgency that can impede cool, calm reflection on how to proceed at the policy level. [Brown] says it can create a pressure to legislate before experts properly understand a new research path and its potential. [Sociologist Christine Hauskeller, Senior Research Fellow at the ESRC Centre for Genomics in Society, University of Exeter, UK adds that] this is not only a waste of financial and legal resources [...] but it serves to narrow social and scientific possibilities
  • Hilary Rose [professor emerita of the sociology of science at the University of Bradford, UK and Gresham College London] believes that an overemphasis on certain research trajectories, and overoptimistic expectations of what they can deliver, can obscure political and social solutions to problems

Parts of the article are specific to climate science.

The last line in a “Some famous (and infamous) predictions” table classifies as “Right or Wrong? PENDING” this 2007 “prediction

The Intergovernmental Panel on Climate Change’s 4th Assessment Report projects that global surface air temperatures will increase by between 1.1 and 6.4°C over preindustrial levels by the end of the century

A speech at the Copenhagen Climate Conference of February 2009 by the then Danish Prime Minister is mentioned as example of “politicians [trying to] ‘fob off responsibility to scientists’

[Don’t] provide us with too many moving targets, because it is already a very, very complicated process,‘ he said. ‘I need fixed targets and certain figures, and not too many considerations on uncertainty and risk and things like that.‘ Such demands, says [Dan Sarewitz, director of the Consortium for Science, Policy, and Outcomes at Arizona State University], can tempt scientists into providing simplistic and unqualified extrapolations from the current state of knowledge to possible future scenarios.

Is it time to design guidelines to “predict responsibly” then? These are Blackman’s suggestions:

  1. Avoid simple timelines: “try to communicate the complexities of the process rather than make a specific prediction”
  2. Learn from history: “heed the lessons of past predictions and promises”
  3. State the caveats: “inform the public also of the current limitations”
  4. Remember what you don’t know: “scientists know a lot less about technology and innovation and political context”

The Great Haddock Revival…

Too much of a good news, one imagines, so it won’t make mainstream headlines any time soon: “The Great Haddock Revival – In the near-empty seas, one species has surged back to life. Can the others follow?

today, despite the odds, haddock stocks are soaring

Alas, nobody has got much clue about why haddocks are back, or more precisely, why there are even more of them than ever recorded.

The Great Haddock Revival

The Great Haddock Revival

Suggestions on the reasons abound, let’s see what researchers will come up with and more importantly, if any other previously-overfished species becomes a success story as well.