According to the current 2007-13 regulation, all regional and national agencies responsible for managing one of the 434 Operational Programmes funded by the 2007-13 Structural Funds must publish on the web a list of businesses or public authorities that have received public funding and the amount of funding received. But the way they do this varies greatly across Europe.

The second output of the evaluation activity of the availability and quality of open data on European Structural Funds is now being published. It’s a benchmarking report (in Italian only, at least for now) that I prepared with the help of my colleague Chiara Ricci for the DG Regional Policy of the Italian Ministry of Economic Development.
I had the chance to present it at the Annual Meeting between the European Commission and the Italian Managing Authorities of the European Regional Development Fund (ERDF), held on October 27-28, 2011 in Rome. It was an extraordinary opportunity to talk about the benefits of open government data in front of a number of high-level representatives of all regional and central institutions involved in the implementation of Regional Policy in Italy (here is my presentation).

The report features brand new data on detail, accessibility, formats, and other characteristics of the datasets on the recipients and the projects funded by European Regional Policy (“lists of beneficiaries”). It’s a new wave of data collected in October 2011, exactly one year after the first web-based survey. A total of 32 characteristics are taken into account in the evaluation process, including the presence of search masks and visualization systems.

The map of Structural Funds transparency reported below shows a core component of this research, that is the format of the data published by each region. The map shows the average score of all the regional and multi-regional programmes that have an impact of that specific territory. A very low score is attributed to PDFs and to HTML reports that split the data into multiple tables or pages (regions in red or orange) . Higher scores are assigned to the XLS format, which is machine-processable (in yellow). The highest scores are attributed to the few regions in Europe that publish data in an open format such as CSV (in green), since no data is currently published in XML or JSON or RDF (see the report for the details about the construction of the index). You can also find the link to the datasets by clicking on each region. The link to the Regional Programme is displayed where there is more than one dataset available.

One year after the first survey, the level of openness has not improved. About two-thirds of EU Operational Programmes still publish their data in PDF, while only 2% use open formats. A radical change is necessary to meet the requirements of new 2014-2020 regulation, as proposed a few weeks ago by the European Commission, which include the use of CSV or XML format.

This is very good news for the open data movement. Earlier today Commissioners Hahn and Andor presented the European Commission proposals for the new Structural Funds regulations for the period 2014-2020. The new general regulation includes an article that force EU countries and regions to open up their data on projects and beneficiaries of Regional Policy. EU Regional Policy (or Cohesion Policy) is worth € 376 billion, more than a third of the entire budget of the Union.
In the last few months, technical and policy recommendations on how to improve the rules of this policy concerning transparency were provided by two studies (one commissioned by the European Parliament and the other by the DG Regional Policy [pdf]) and one independent web-based survey. Plus, organizations such as Transparency International advocated better rules and practices, such as the creation of a centralized website that contains all EU funds beneficiaries and that publishes the data respecting the 8 principles of Open Government Data.

The good news is that most of these recommendations have been incorporated in the drafts of the new regulations. In particular, Art. 105 (Chapter II, Information and Communication) states that EU countries “shall in order to ensure transparency in the support of the Funds maintain a list of operations by operational programme and by Fund in CSV or XML format which shall be accessible through the single website or the single website portal providing a list and summary of all operational programmes in that Member State”. It has been demostrated that the presence of a single website covering all data from the local institutions will likely improve the performance of the country in terms of transparency.

The minimum set of information to be provided – currently limited to three items – has been extended to cover new interesting data such as postcodes of beneficiaries. The data fields that must be included are listed in Annex V:

• Beneficiary name (only legal entities; no natural persons shall be named);
• Operation name;
• Operation summary;
• Operation start date;
• Operation end date (expected date for physical completion or full implementation of the operation);
• Total eligible expenditure allocated to the operation;
• EU co-financing rate (as per priority axis);
• Operation postcode;
• Country;
• Name of category of intervention for the operation;
• Date of last update of the list of operations.
• The headings of the data fields and the names of the operations shall be also provided in at least one other official language of the European Union.

In my opinion, this proposal is probably a good compromise between the need to introduce new, more transparent ways to publish data and the current level of technical and administrative capacity of EU regions. However, a few important features characterizing real open data are still missing. For example the data should be released in a linked-data formats such as the RDF. Plus, a clear indication of the license under which the data are released should be provided. Introducing these features now – even though the RDF format seems now pretty advanced – is particularly important seeing that it is rather difficult to modify a multi-annual regulation once it is approved.

Obviously it will be crucial to monitor the actual implementation of these rules across the European Union. Luckily, official regulations have demonstrated to be a powerful tool, far more persuasive than other initiatives, such as the European Transparency Initiative, undertaken by the Commission after the approval of the official regulations. As already demonstrated, the level of compliance with regulations among EU agencies is extremely high, given that the Commission has the power to stop the flow of money from the EU to the Regions if these rules are broken.