The impact of grey literature on climate projections

Most food crop cultivation in Africa is rain-fed, but climate change is affecting vital rainfall patterns and pushing up temperatures, diminishing yields that could halve in some countries by 2020. This warning has been widely quoted since it first appeared in a synthesis report for policy-makers in 2007 by the authoritative UN Intergovernmental Panel on Climate Change (IPCC).

Clouds of doubt gathered over the statement after it emerged that the IPCC report had based the projection on a non-peer reviewed research paper -otherwise known as "grey literature".

The claim was published in Sunday Times newspaper in the UK on 7 February, in a report headlined "Africagate: top British scientist says UN panel is losing credibility".

A flood of allegations from all quarters then began to question the credibility of the 2007 assessment report. Governments and policy-makers use the IPCC assessment reports to formulate plans and strategies for coping with climate change.

So, is the projection incorrect? We asked the IPCC and other scientists, but we will have to wait until the end of August 2010 to find out.

On 10 March the UN announced that a Netherlands-based group of 15 national academies of science would review how the IPCC does its work. The Panel publishes periodic assessments by the three committees that deal with the causes of climate change, its impact, and mitigation options. The review committee will also consider whether the IPCC should use non-peer reviewed papers.

What other scientists say

The projection that crop yields could be reduced by 50 percent in some African countries, contained in the synthesis report, was based on a paper cited in the Panel's report on impacts.

''In fact, we think Africa faces some of the toughest impacts on agriculture in the world, just not as extreme as the IPCC statement suggested''

Written by Ali Agoumi, a Moroccan climate expert, the paper "is a summary of technical studies and research conducted" in three countries - Morocco, Algeria and Tunisia - submitted to the UN Framework Convention on Climate Change, "and is a perfectly legitimate IPCC reference", wrote a group of scientists involved in the panel's reports on the popular blog RealClimate, run by them on 14 February.

"The IPCC cites 18,000 references in the AR4 [Fourth assessment report]; the vast majority of these are peer-reviewed scientific journal papers ... it is indispensable to use gray sources, since many valuable data are published in them," the scientists said in defence of grey literature.

"Reports by government statistics offices, the International Energy Agency, World bank ... This is particularly true when it comes to regional impacts in the least developed countries, where knowledgeable local experts exist who have little chance, or impetus, to publish in international science journals," the scientists commented.

Yet it is not the use of grey literature that seems to be the issue. David Lobell, of Stanford University's Program on Food Security and the Environment, who has worked extensively on projecting the impact of climate change on crop yields in Africa, called the IPCC statement in the synthesis report "ill-advised".

"The original syntax was technically correct (i.e., worst years are 50 percent yields drops, and these could become more common), but it was easily misinterpreted as a statement about average yields," he said.

IPCC officials often quoted the statement to illustrate the possible impact of climate change on food production in Africa, when the document on which it was based only referred to three North African countries.

"Part of the problem was that the scientific literature on some of these issues was quite lacking at the time [when the report was being compiled – the report takes three years to be written]," Lobell acknowledged.

"One risk now is that people could interpret the IPCC statement as being wrong, saying that Africa doesn't really face a threat, or that other IPCC statements are also in doubt. In fact, we think Africa faces some of the toughest impacts on agriculture in the world, just not as extreme as the IPCC statement suggested."

The scandal

The Sunday Times report in the UK was among several that followed a controversy dubbed "Climategate", which broke in November 2009, ahead of the UN climate change conference in Copenhagen in December. The row started when hacked emails by the Climatic Research Unit at the University in East Anglia, one of the centres that supplied data to the IPCC, were published on a website for climate change sceptics.

Since then, newspapers across the world have published allegations by climate change sceptics, organizations, government officials, and other publications that the IPCC manipulated climate change data or used non-peer reviewed papers as the basis for some of its projections.

The UN panel defended its work until it was forced to admit in January that it had erroneously projected that 80 percent of the Himalayan glacier area would very likely be gone by 2035.

The panel had sourced the projection from a document produced by the World Wide Fund for Nature (WWF), which had used an interview by a scientist with the magazine, New Scientist, in 1999 as the source of the projection, and not a scientific publication.