From buying vaccines to leasing helicopters, the UN agreed more than 30,000 contracts worth a total of $17.7 billion last year. But if you want to know more – which pharmaceutical companies sell most to which UN agency, for example – you have some work to do. That’s because, to the annoyance of open data advocates, the information is public but not machine-readable: The tables in the UN’s annual procurement report run over 387 pages of a 739-page PDF.
The Annual Statistical Report on United Nations Procurement (ASR), compiled by the UN Office for Project Services (UNOPS) on behalf of a network of UN procurement officials, gathers data from multiple incompatible systems and classification schemes to arrive at an overview of spending. In an email, UNOPS told IRIN that the challenge includes pulling together the report from various agencies with differing “data availability and time constraints”.
In this uneven feat of transparency, 39 UN entities disclose their spending on external suppliers of goods and services. The UN has decided not to make the data available in a fully flexible open format. UNOPS told IRIN it was “assessing the risks and opportunities of the different options, any changes to reporting will require sign off by the UN agencies.” In comparison, the European Commission provides downloadable files of contract awards – more than 500,000 records are listed for 2015.
Analysts contacted by IRIN nevertheless welcomed the UN’s openness with its data, which is better than many other institutions in the humanitarian and development sectors. “There are a whole load of organisations who don’t publish details of their procurement spend at all,” said Rupert Simons, chief executive of Publish What You Fund, an aid transparency watchdog.
But Simons and others questioned the rationale for not releasing the data in spreadsheet form. It’s a missed opportunity, they say, and although public, it falls far short of one of the characteristics of open data, including this definition from the UN: “machine-readable structured data, in non-proprietary format”.
PDF files are readable to the naked eye, but stripping the formatting and layout to get to the words and numbers on the page often results in messy data. Aidan McGuire, of SensibleCode, which runs an online service to convert large PDF tables, says that for open data, PDF is “not an ideal format and should be avoided”. (IRIN has used the service on this and other PDF documents).
PDFs may attract scorn from open data advocates – they’re known sometimes as “data graveyards”, but choosing a format and type of report depends on who the audience is. The ASR report may be trying to be too many things to too many people, according to Simons. National government officials in aid-recipient countries may be well-served with narrative reports in PDF format, but NGO and civil society users often want the ability to sift and analyse data, he explained. UNOPS confirmed the report had a “wide audience with different needs and demands”.
Highlights from 2016 report
Companies from three countries (Belgium, India, and the United States) won 20 percent of the purchase orders. Overall, developing and “transition” countries landed more than half of the awards. UNICEF is the biggest spender ($3.5 billion). The largest categories of expenditure were: medical ($2.8 b), transportation ($2.4 b), food ($1.9 b) and administration services ($1.7 b). The largest deal was a $566 million UNICEF engagement with GlaxoSmithKline. The report also makes reference to environmental and other sustainability factors, as well as efficiency gains in joint procurement across UN entities.
According to the report, UN contract spending is up by 0.8 percent. But the figures are not exactly on a like-for-like basis: 2016 saw the inclusion of the International Organization for Migration for the first time ($522 million of spending), and the removal of a $1.1 billion United Nations Development Programme scheme by which it procured goods for member states.
Loopholes and noise
Key players in the humanitarian community have renewed commitments to transparency but have a long way to go: A 12 June report by Development Initiatives for the Grand Bargain group of humanitarian donors and agencies found transparency across a range of indicators to be “very poor” for more than half the signatory organisations. The ASR report includes the UN’s big humanitarian agencies, along with peacekeeping and a range of smaller bodies.
Even among the minority of Grand Bargain signatories that supply datasets formatted in the International Aid Transparency Initiative (IATI) standard, few contain details on contractors. Only about half of the organisations surveyed by Simons’ group for its annual aid transparency ranking publish any data on contracts, he said.
IRIN will again, as in previous years, be trawling the ASR data for insights and revelations. Previous findings include the UN paying a sanctioned entity in Central African Republic, and some eye-watering spending at hotels in Damascus.
As a first stage in the process, we ran the PDF through a SensibleCode’s extraction service and found that about $14 billion of contract records may automatically be converted. That leaves about 20 percent (in dollar terms) to clean up manually.
In any dataset of this scale and varied provenance, there will be some “noise” in the data in addition to the conversion headaches. For example, the same companies may appear with different spellings and punctuation, making it hard to calculate total sales by vendor. Types of goods and services may not be categorised the same by various agencies.
One contract we’re curious about is for “spacecraft”, which may (or may not) be a typo. Stay tuned.
UNOPS said its aim was to “continue to improve the data quality and detail”.
Another loophole is that vendors can be anonymised at the discretion of the UN agency – more than $1.7 billion of contracts have an “unspecified” vendor. Tens of millions also have the vendor withheld for security reasons – a legitimate practice in high-risk countries.
UN managers may not relish opening their books so that civil society and journalists can pick holes in their spending. But the exercise serves an important purpose.
“The humanitarian community goes to its big funders every year and says, ‘please give us more money’, and that request for funding is never even close to being fulfilled,” Simons said. “I think it’s entirely fair to enquire into the costing.”
bp/ag
