The first output of a web‐based survey shows that the European Cohesion Policy is only halfway to accomplishing a paradigm shift to open data, with differences in performance both between and ‐ in some cases ‐ within European Countries.
Low scores are attributed to the formats the authorities are choosing when publishing their data on the web, while other indicators such as the level of granularity are positively influenced by the requirements of current regulations.

Availability of Open Data on projects and beneficiaries of the European Cohesion Policy (or Regional Policy) – which is the second-biggest EU policy after agriculture with a budget of EUR 347 billion for the period 2007-13 – can surely help foster transparency in the use of public money in Europe.

The European Union lacks common initiatives such as the US Recovery.gov or USA spending.gov to track government spending and improve transparency of public policies. In particular, in the case of Structural Funds, there is no single point of access to the data, since each single EU Region and National agency acting as Managing Authority of the Funds is responsible for publishing data on the beneficiaries and the amount of public funding received.  This implies that hundreds of Managing Authorities are following different paths and implementing different information strategies when opening up their data. Many databases (often simple PDF lists) are now uploaded to regional or national institutional websites, showing huge variation not only in the way they can be accessed (formats, search masks, data visualization etc.) but also in content and quality of data provided (detail level, granularity, description, etc.).

Last summer, after a first analysis on the prevailing formats, I started to design an independent web-based survey on the overall quality of data published by each Managing Authority responsible for the 434 Operational Programmes approved in July 2009. Data was collected in October 2010 by me and Chiara Assunta Ricci, a brilliant PhD student in Economics at La Sapienza University of Rome. We were inspired by what people at the fantastic project Farmsubsidy.org had been able to do with the data from the other big EU policy, the Common Agricultural Policy. While their greatest achievement is having gathered the PDF documents every State has to publish on line in one real database, they also provide an evaluation of the lists of projects they have used by a transparency composite indicator. The same exercise could be applied to Structural Funds.

The first output of the survey was published a few days ago in the European Journal of ePractice. Here you can download the paper and here the full issue “The Openness of Government”. The paper is based on David Osimo’s seminal proposal for a “Benchmarking 2.0” and represents a pilot of a measurement framework for comparing governments’ efforts to make data available.

This exercise could represent a first step for improving current ‘traditional’ EU e-Government benchmarking. In fact, the new edition “Digitizing Public Services in Europe: Putting ambition into action – 9th Benchmark Measurement” (page 19 and 137) confirms the importance of updating and expanding the scope of the analysis by including new metrics on “Transparent and Open Government” .
The evaluation scheme is based on the Eight principles of Open Government Data, which are considered as a key reference and a worldwide de facto standard. This scheme is meant to be flexible and could be applied to other kinds of Government Data. Gianfranco Andriola, one of the promoters of the Italian Open Data Licence, helped me define the methodological approach for the principles “format” and “licence”. I must also thank the “man behind the curtain” of Spaghetti Open Data initiative, Matteo Brunati aka Dagoneye, for his suggestions about open formats, and Sergio Scicchitano for his many advices and support.

Results can be summarized as follows:

1. The European Cohesion Policy is only halfway to accomplishing a paradigm shift to open data, with differences in performance both between and – in some cases – within European countries. Best performing countries such as the Czech Republic and Finland obtain a score of 71%, while the worst performing Member State is Latvia with 25%. Countries from the eastern Europe often appear in the first half of the chart.
2. Very low scores are attributed to the formats the authorities are choosing when publishing their data on the web, while other indicators such as the level of granularity are positively influenced by the requirements of current regulations.
3. A considerable difference in performance is shown when comparing datasets that are shared and centralized at national level with those which are managed by a single regional authority. This variation is also statistically significant with regard to all the indicators examined, and is probably due to the fact that a centrally managed programme has the advantage that information flows are easier to manage and local actions are more easily coordinated.
4. The use of open, machine-processable and linked-data formats have unexpected advantages in terms of transparency and re-use of the data by the public and private sector. The application of these technical principles does not need extra budget or major changes in government organization and information management; nor does it require the update of existing software and infrastructures. What is needed today is the promotion among national and local authorities of the culture of transparency and the raising of awareness of the benefits that could derive from opening up existing data and information in a re-usable way.