Expert accountability

The ruling last week by an Italian court that seven scientists were guilty of manslaughter for failing to warn citizens about the threat of a major earthquake in the city of l'Aquila, raises some difficult questions about accountability and ethics in relation to the provision of 'expert' opinion and scientific advice to governments and the wider public. The scientific community and global media have reacted with outrage, comparing the verdict and threat of lengthy jail sentences for the scientists involved with the persecution of Galileo and the burning of witches.

I agree in principle and in spirit with this sense of outrage. The ruling appears to betray a lack of understanding of the inherent uncertainty in any scientific endeavour, which is particularly pronounced when it comes to forecasting future events. Furthermore, putting science on trial in this way threatens to curtail the progress of scientific inquiry generally, and more specifically--in the nearer term--to make the position of those charged with civil protection and disaster prevention, in Italy at least, almost untenable. However, there remains an important point relating to accountability and ethics that appears to have been overlooked amidst all the righteous indignation.

In an excellent article in Nature, Stephen S. Hall outlines the sequence of events that led up to the tragedy. In an extraordinary meeting of the National Commission for the Forecast and Prevention of Major Risks, the seven scientists--who were all members of the Commission--reached the conclusion that an earthquake was "unlikely" (if not impossible). This in turn was interpreted by a government official, speaking at a press conference, as meaning the situation in l'Aquila was "certainly normal" and posed "no danger". The same official further added that the sequence of minor quakes and tremors that had been occurring in the region was in fact "favourable ... because of the continuous discharge of energy".

This interpretation is apparently contrary to the scientific evidence. The same article, by Hall, quotes Thomas Jordan, director of the Southern California Earthquake Center at the University of Southern California in Los Angeles, and chair of the International Commission on Earthquake Forecasting (ICEF), as suggesting that in the aftermath of a medium-sized shock in a seismic swarm (a sequence of tremors), the risk of a major quake can increase anywhere from 100-fold to nearly 1,000-fold in the short term, although the overall probability of a major quake remains relatively low--at around 2%, according to a study of other earthquake-prone zones in Italy (G. Grandori et al. Bull. Seismol. Soc. Am. 78, 1538–1549; 1988), also quoted by Hall.

A 1,000- (or even 100-) fold increase in the probability of a major quake, would have been a very different message for public consumption than the "anaesthetizing" reassurances given to the media at the press conference. Clearly, the most egregious error committed here was in the over-, or indeed mis-interpretation, of the scientific evidence by the government official who spoke to the press. The scientists were--apparently--correct in their assessment that a major quake remained "unlikely", although not impossible. But, was this the best way of characterizing the risks for the general public?

One of the people affected by the L'Aquila earthquake, quoted in Hall's article, admits that he feels "betrayed by science"--"Either they didn't know certain things, which is a problem, or they didn't know how to communicate what they did know, which is also a problem."

It may be that the error here was not in mis-stating the risk, but in not being specific enough about it. There is an understandable reluctance to use statistics, probabilities and scientific terminology in the public communication of scientific evidence. But at times we take this too far. The public is not stupid, and--problems with common misunderstandings in relation to probability notwithstanding--would be better served by the scientific community and public officials avoiding condescending reassurance in favour of the clear presentation of facts.

Is woolly language, such as "unlikely" really any more useful or informative than saying simply "we don't know"? Might the committee have been better to present the available statistical evidence--including details about the changes in probabilities--while acknowledging that the precise timing and location of a major quake is essentially unpredictable?

The advice could have stopped short of ordering a full-scale evacuation--which, on the basis of the best available evidence, would have been unnecessary 98% of the time--and instead, simply presented that evidence, enabling people to make their own informed decisions about what level of risk they were willing to accept.

Returning to the issue of accountability and ethics, to what extent should scientific or other experts be held accountable for the advice they give to governments or to the wider public? In the case of the l'Aquila tragedy, a more relevant question might be; should researchers be held accountable for the way in which their findings are interpreted by policy-makers and, in turn, by the media? Should researchers generally consider how their findings are likely to be interpreted before making them public? This almost certainly places an unreasonable burden on researchers.

And yet--while researchers can't be expected to control how others interpret their findings, a greater effort needs to be made in communicating the science--its achievements and its limitations--directly to the public. With the recent proliferation of sources for news, opinion and analysis, the authority of traditional media outlets and the role of journalists and editors as the gatekeepers of public information, is increasingly being challenged. This presents both a challenge and an opportunity for scientific engagement with a wider audience. It is increasingly difficult for members of the public to distinguish the signal of scientific or expert analysis from (a) noise and (b) intentionally biased or deceitful opinion emanating from thinly disguised lobbyists, portraying themselves as independent 'experts'. On the other hand, the internet enables researchers to disseminate their findings, methods and data without intermediation by journalists or politicians.

The 'science on trial' headlines may sound melodramatic, but scientists from both the social and hard-sciences are right to feel they are being challenged to justify their art as at no other time in living memory. Public confidence in "science"--in its broadest sense--has been undermined by episodes such as the 'climategate' controversy. The discipline of economics, similarly, has been widely criticised for not predicting the financial crisis--and more fundamentally, for persisting with models that appear unable to explain 'real world' phenomena. This critique certainly has some merit, and economics as a discipline is evolving to take account of the lessons from related disciplines, notably psychology, biology and epidemiology. However, it has to be recognised--both by researchers and those who would criticise their efforts--that models, by their very nature, are imperfect simplifications of the world they are trying to explain. One clear responsibility of any researcher, is to think carefully about the domain of validity of the models that they use [PDF], to define their limitations, and to communicate this in an unambiguous and honest way, along with any findings from the research.

Reflecting on the trial of the seismologists, the former president of Italy's National Institute of Geophysics and Volcanology, concludes that "scientists have to shut up". On the contrary, the lesson for scientists from this tragedy and the subsequent trial, is to be more proactive in our engagement with the public.

A good starting point might be the establishment of a voluntary code of ethics for researchers. This would include, for example, a commitment to publish annually a list of all sources of funding for one's research. Furthermore, the code might also contain a commitment to make public not only our research findings but also the data and methodology used (including relevant context, limitations and assumptions). Signing up to this code could be a prerequisite for any government advisers, and could similarly become a useful tool for the media in screening 'expert' commentators.

More generally this code would be based on three fundamental guiding principles; honesty, transparency and humility. Going back to Hall's Nature article, he quotes a man who lost his wife and daughter in the earthquake, lamenting the fact that "the science, on this occasion, was dramatically superficial, and it betrayed the culture of prudence and good sense that our parents taught us on the basis of experience and of the wisdom of the previous generations." Perhaps the greatest lesson from this tragedy is the need for a greater degree of humility when it comes to the predictive powers of even the most sophisticated scientific models. 

Letter to FT on Lomborg's false dichotomy

The following letter was published in the Financial Times today (20 December, 2010), in response to this article by Bjorn Lomborg (from the FT, 9 December, 2010).

Sir,

In the article “Time for Europe to show real vision on climate” (FT, 10^th December), Bjorn Lomborg argues that "We will never succeed in making fossil fuels so expensive that no one wants them [...] Instead, we should make green energy so cheap that everyone wants it". There is an inherent inconsistency in this line of reasoning, as the decision to switch from burning fossil fuels to some alternative depends on the relative prices of each.

The figures quoted in the article seem to suggest that emissions reduction is very expensive, while developing green tech alternatives would be relatively cheap. These cannot be simultaneously true. If cheap alternative energy sources are developed then emissions reduction becomes cheap as a consequence. Incidentally, it is notable that no reference is made to the assumptions and uncertainties inherent in the types of estimates to which the author refers.

As regards his policy recommendations, Lomborg advocates supply side solutions to climate change without specifying how the necessary investment might be stimulated. Does he envisage governments giving large-scale subsidies to the green energy sector? Such schemes are unlikely to be cheap.

Private sector investment in green energy alternatives is constrained partly by uncertainty, which pushes up the risk premium required by investors. One way of reducing this uncertainty - and thus stimulating greater investment - is to formulate more coherent climate policy, something which Lomborg seems to think is no longer worthwhile.

The simple, but false, dichotomous interpretation of the available policy options presented in this article may be a good way of getting media attention, but it is not a good way of thinking about complex socio-economic or scientific policy issues.

Yours etc.

Tom McDermott & Stefano F. Verde, Trinity College Dublin

Tim Laing, LSE

and Aurelie Mejean, CIRED, Paris

Update: Lomborg's response to our letter is here.

An open letter on Ireland's proposed Climate Change Bill

A chara,

Climate change is a real and significant threat to human welfare, particularly in the poorest parts of the world. While an effective 'solution' to this threat will require global cooperation over a sustained period of time, this is no excuse for not acting now to begin the process of reducing our dependence on carbon-intensive activities.

Ireland has a great record of leading the world in initiatives aimed at reducing global poverty. This work should be complemented and reinforced by action on climate change. Ireland could take the lead in demonstrating to other rich countries (and the rapidly developing 'emerging economies') that reducing carbon emissions can be achieved without jeopardising economic or social welfare. In fact, these goals can be enhanced by such initiatives. This is not only the morally right thing to do, it is also in our interests. Such leadership would help to restore Ireland's image internationally, which has been so tarnished by the excesses, greed and corruption of our recent economic boom and bust. At the same time, intelligent climate legislation could provide an additional source of revenue for government, while potentially improving our competitiveness over the long-term.

Unfortunately, the proposed Climate Change Bill produced by The Oireachtas Joint Committee on Climate Change and Energy Security (and due to be debated in the Dail today, Thursday), will not achieve any of these worthy goals. The proposed bill would legislate for ambitious emissions reduction targets, with the Taoiseach responsible for ensuring that these targets are achieved. The Taoiseach would also indicate what levels of emissions he/she expects each year. How is the Taoiseach to predict annual emissions levels or to enforce any such medium to long-term targets? This is equivalent to imposing legislation that requires the Taoiseach to predict levels of economic growth each year, or somehow to enforce medium to long-term economic targets.

Unless this legislation envisages an entirely new, centrally-planned economic system in this country, I do not see how its objectives can be achieved.

Legislation of this nature will do two things:

1) It provides a convenient sound-bite for politicians to hide behind. It is relatively easy to say "we have proposed/introduced legislation that will force emissions to fall by x% by 2050" etc. without actually specifying how such targets will be achieved (i.e. without having to stand up to various interest groups who may stand to lose from specific climate-related legislation).

2) Such a law obliges the presiding government to make various interventions to attempt to reduce GHG emissions. Crucially, however, it leaves the choice of specific interventions as a purely political decision. How will the government of the day decide how and where to reduce emissions? On what basis? We surely should be sufficiently well chastened in this country by recent experience of political interventions in the property sector (in the form of tax breaks etc.) to understand what a dangerous scenario this type of legislation will create.

We should not allow politicians the convenience of meaningless targets to hide behind, or the opportunity to use climate change legislation as a means of making themselves and their friends better off in the next round of crony-political-economy.

Setting ambitious long-term targets might sound good, but in reality this does not provide any greater certainty to businesses, investors, or consumers, simply because such targets are purely aspirational and are not credible without specific measures to achieve them.

The optimal climate change policy from both an equity and an efficiency perspective is to place a tax on carbon emissions, and allow people to choose the best way for them of reducing carbon dependency. This would obviously have revenue raising potential - revenue that is so desperately needed right now - while any potential threat to vulnerable people could be mitigated by using part of the revenue raised to provide reimbursements to those on low-incomes. Taxes are never popular, but the people of Ireland are acutely aware right now that taxes must rise. In every crisis there lies opportunity. If only we had the courage to embrace this one.

Is mise,

Tom McDermott

Update: A version of this letter was published on the irisheconomy.ie blog on 11 November 2010.

Winning the battle with climate change skeptics?

The following post came out of a debate I was having with my brother about the role of "skeptical" views on climate change in the climate policy debate ...

The likely (and worst) effects of climate change in poor countries are almost indistinguishable from the effects of being poor.* The idea that (in the short term at least, and based on current expectations) money would be better spent on pursuing development goals rather than mitigation efforts (from poor countries' perspective), has often been made in the academic literature on climate change and development (see for example this article by Richard Tol).

For example, malaria represents a "winnable battle" (although "winning" is not as straightforward as some authors have suggested, see this article by Jeffrey Sachs & Pia Malaney), and deserves more attention and money. However, there is a very important and subtle distinction that some of the climate "skeptic" commentary seems to have missed (or intentionally omitted). That is, climate change will make these so-called winnable battles much harder to win. Furthermore, unmitigated climate change will have - almost certainly - disastrous consequences (especially for poor countries and poor people everywhere).

Business as usual is not a good option (as argued in this RealClimate post and this Economist article). The projections now being made about what unmitigated climate change will look like are startling, and scary (see for example this article by Andrei Sokolov and colleagues, and the report from the Hadley Centre in the UK, as reported here in The SundayTimes). The debate needs to be about how much mitigation is optimal, when and how to pursue it, and how to balance resources devoted to mitigation with those devoted to adaptation (from a poor country's perspective this essentially means development).

This is a very worthwhile debate, and should not be stifled by those who use the ugly term "climate denier" and make personal attacks on anyone 'silly enough' to be skeptical about climate change - or the proposed policy responses to it. (Ironically, this is exactly the sort of small-minded condescension that environmentalists and other 'social crusaders' have had to put up with for years).

One thing that is dangerous about the skeptics' approach - and the headlines they generate in some sections of the popular media - is that people are very willing to believe a voice that tells them "everything is fine, everything is going to be ok". We fundamentally prefer the easy option (psychologists term this "cognitive fluency"). We find it very difficult, on the other hand, to take a threat seriously if we have difficulty imagining it happening (people are a lot more frightened by the idea of being attacked by a shark than killed by falling debris from a plane, but apparently that is irrational - the latter being 30 times more likely! This is the so-called 'availability bias' - see this article by Oliver Burkeman, writing in the Guardian).

Climate change, and its potential consequences definitely fall into this category of being difficult to imagine.

As a footnote to this discussion, I'm not sure what is meant by saying that climate change is an 'unwinnable battle' (as I think has been suggested by some of the skeptics who, although not explicitly doubting the existence of climate change - and the contribution human activities are making to it - have challenged the rationale for attempting to prevent it through emissions reductions). There is a strong degree of irreversibility in climate change. However, the level of warming/change is still up for grabs - the irreversibility aspect makes precautionary emissions reduction now, although expensive, all the more justifiable (as the Economist also argues, in the article cited above). If the 'unwinnable' aspect is the contention that we will never change people's behaviour, then I think that is very naive. People will respond to the right incentives (a tax on carbon being the best way to create those incentives), and will change behaviour very quickly when the alternative becomes economically more rewarding. Not easy to get this right, but hardly 'unwinnable'.

*Note: That is, excluding the potential for inundation leading to the destruction - or at least forced relocation - of entire communities or even nations. In fact, this has already happened to the Charteret Islanders of Papua New Guinea.