Financial Crisis to Climate Negotiators: DO NOT OVER-RELY ON MODELS!!!

The ongoing financial crisis can and will teach all of us many lessons, also in terms of climate and AGW. And no, I do not mean the rather naive made-up litany visible at ClimateProgress.

I refer to something much more profound, especially since there is now evidence that even the guys at RealClimate do not fully understand what they are dealing with, when they deal with climate models (even after loving models to death).

Consensus is in fact emerging about three advices that went missing during the build-up of the financial troubles:

  • DO NOT OVER-RELY ON MODELS
  • DO NOT JUST PUSH TOWARD EVER MORE COMPLEX MODELS
  • BE SENSIBLE WHEN MANAGING RISK

====

I have prepared a couple of quick lists from three recent articles on the business pages of the International Herald Tribune (full attributed quotes at the bottom of the blog; and yes, do keep in mind those are from analysts that are experts in their field indeed).

First, what went wrong. It’s evident that plenty of it directly related to the climate debate:

  • Models gave a false sense of precision. They can now be seen as educated guess calculated to many decimal places. At the time, they appeared precise, and yet proceeded to ultimately demonstrate themselves as totally off base
  • Until the crisis, the field (of financial risk modeling) enjoyed a halo of academic credibility
  • In general, there was too much focus on quantitative issues and data and models. People did not know what to do with things that cannot generally modeled as a quantifiable risk
  • Risk managers were also too busy with models and bringing up data that could not be absorbed by senior management
  • Better modeling, more wisely applied, would have helped, but so would have common sense in senior management

Obviously, the danger lies in the fact that to confuse the model with the world is to embrace a future disaster, as humans (or the climate) do not just obey mathematical rules that can be modeled.

What should be done? I wish the negotiators in Poznan had the following list in mind:

  • Understand that risk is a function of behavior more than of models
  • Consider that risk management is about making big-picture choices, not just trying to prevent losses
  • Acknowledge that risk may mean different things, like hazard, threat, gamble, chance, possibility, or opportunity
  • Accept that models are useful as points of information. They shouldn’t drive risk tolerance and shouldn’t be used to tell anybody how to manage firms (or nations)

Models getting translated to the real world of company or national policy suffer indeed from a “chinese whispers syndrome”, with the original caveat-full expert statements awfully simplified and distorted for the benefit of the business directors (or national politicians).

At the end of the day, the problem is not the models. The models are tools, perhaps the devil’s but still just tools. The problem is putting all eggs in the models basket, in financial just as in AGW terms.

====

(original quotes)

(1) From In fallout from crisis, rethinking risk and human judgment, by Lynnley Browning; IHT, Wednesday, November 19, 2008

[…] to cope with uncertainty and “slippery slopes” […] “With this crisis, everybody is re-evaluating the concept of risk management,” said Richard Phillips, a professor of risk management and insurance at Georgia State University […]

The scrutiny goes beyond a dissection of the complex mathematical models created by financial engineering [and focuses] “on the overreliance on models,” said Carol Fox of the Risk and Insurance Management Society […]

Because nearly all risk-management models failed to predict or protect against the crisis, Fox said, insurers will increasingly view risk “more as a function of behavior than of models.”

Going forward, she said, insurers will use models “as a point of information, but it won’t drive risk tolerance” […].

“People have been managing the wrong risk […] ” said Peter Bernstein, a historian and the author of “Against the Odds: The Remarkable Story of Risk.” ”Risk management is about making choices, not preventing losses. […]

the financial crisis has made clear is that risk, and how one deals with it, can mean wildly different things to different companies, from gamble, hazard or chance to threat, possibility or opportunity. It can be a bucket of nasty things to be avoided, or a daring play. […]

It didn’t help matters that until the crisis, the field enjoyed a halo of academic credibility. “All these rocket scientists with Ph.D.s provided reassurance to decision makers and buyers,” said Paul Bracken, a professor of political science at Yale University.

[According to] Robert Merton, the Harvard Business School professor who received the Nobel in economic science in 1997 […] “A lot of it is straightforward things, like judgments made to accept ratings. We’ve got to get these financial engineers and quant types out of the banks and get sensible types in.” […]

“Our definition of risk became confused with obeying the law,” said Bill Sharon, chief executive of Sorms, a risk-management consulting firm. […]

Now, insurers are increasingly looking at risk management as a process applying […] to big-picture questions […].

After all, said Martin Grace, associate director of the Center for Risk Management and Insurance Research at Georgia State University, “you can have math models, but that doesn’t tell you how to manage the firm.”

(2) From When crisis hit, a global framework for limiting risk proved ineffective by Conrad de Aenlle, IHT, Wednesday, November 19, 2008

[…] Even if [The Basel II international accord on banking supervision] had been put into practice immediately, it might not have averted the crisis. Critics contend that the various models, formulas and equations used to determine asset quality provide a false sense of precision, leaving bankers and regulators with no clear idea of where they stand. The numbers that are derived amount to an educated guess calculated to umpteen decimal places.

“There has been too much focus on quantitative issues and data and models and a lack of understanding of what the main risks are in the business model,” said Peter Neu, a principal in Frankfurt for the Boston Consulting Group. “Risk managers are too busy with models and bringing up data that can’t be absorbed by senior management.”

A shortcoming of some models is that their risk projections come with a caveat that they are assumed to be accurate during normal market conditions. […]

(3) From Wall Street’s extreme sport: Financial engineering by Steve Lohr, IHT, November 5, 2008

“Complexity, transparency, liquidity and leverage have all played a huge role in this crisis,” said Leslie Rahl, president of Capital Market Risk Advisors, a risk-management consulting firm. “And these are things that are not generally modeled as a quantifiable risk.”

The miss by Wall Street analysts shows how models can be precise out to several decimal places, and yet be totally off base

The quantitative models typically have their origins in academia and often the physical sciences. In academia, the focus is on problems that can be solved, proved and published — not messy, intractable challenges. In science, the models derive from particle flows in a liquid or a gas, which conform to the neat, crisp laws of physics.

“To confuse the model with the world is to embrace a future disaster driven by the belief that humans obey mathematical rules.”

Better modeling, more wisely applied, would have helped, Lindsey said, but so would have common sense in senior management