Benchmarking open data efforts through indices and rankings: Assessing development and contexts of use

Abstract

This paper aims to provide a broad perspective on the development of benchmarking open data efforts through indices and rankings over the years, both at the level of countries and allowing for a cross-country comparison. The methodology follows a systematic search for the relevant resources, their classification and identification of six open data benchmarks to be further analyzed, the identification of their key components through decomposition, their description, and identifying the similarities and differences. Three major groups of indices and four periods that characterize the efforts to benchmark and measure the development of open data are identified, where the first measure the openness of the selected categories of data, the second focuses on different aspects of the open data ecosystem, using a large number of variables, and the third is a combination of both approaches. Recommendations as well as trends that can form the benchmarking frameworks in the future are also discussed. The findings are of a high importance for individual countries, which allow for correct and accurate interpretation of the results changes in the scope of a given index or rank, i.e., whether the difference in results is the result of national efforts or the subject of changes in the specific index, as well as how to combine and interpret the results of a number of indices for correct decision-making and for the definition of the future actions where the results vary significantly. In addition, the findings are also important for international organizations publishing benchmarking reports.