The Sustainable Development Goals (SDGs) require a huge amount of data to measure progress and achieve the goals. In many countries around the world this is an acute challenge due to financial, institutional, and capacity constraints.

This is prompting the development of innovative ways for governments, and specifically National Statistical Offices (NSOs), to generate or curate additional data. Many are looking to the private sector as potential partners to share a wide range of environmental, commercial, and social data that can help monitor attainment of the SDGs. Government policymakers and official statisticians have expressed concern, however, about the feasibility of using third-party data. Worries include data existing in different formats and therefore being irreconcilable, and the feasibility and security of sharing data across public and private institutions.

The Challenge

In 2016, the Colombian National Administrative Department of Statistics (known as DANE) conducted a data gap analysis of the data currently available to monitor Colombia’s progress on the SDGs. It found that within the Colombian government, there was information available for 54 percent of the global SDG indicators, partial information for 30 percent, and no data (and / or no methodological clarity on how to measure the indicator) for 16 percent (Administrative Department National Statistics – DANE, nd).

The solution

In response to this analysis, the Sustainable Development Solutions Network (SDSN) and Centro de Pensamiento Estratégico Internacional (Cepei) decided to collaborate to test new potential methods for collecting missing SDG data from the private sector. The aim was to transmit private sector data addressing a deficiency defined by the data gap analysis to DANE, the national institution in Colombia responsible for measuring SDG indicators. The project explored the best governance arrangements to facilitate data exchange, as well as the technical challenges associated with data reconciliation. The group hypothesized that a technological data reconciliation platform would support the data sharing process. The Bogotá Chamber of Commerce (BCoC) was selected as a partner for the project based on its willingness to share a wide range of highly useful datasets and strong management-level commitment to the SDGs.

Lessons Learned

Building trust between partners 

✔ Dialogue between the private sector and the NSO was an important element before initiating the technical aspects of the work.

✔ There are protocols within institutions that are not easily calculable in programmatic planning; e.g. what was perceived of as a simple request for a sample of a given database in fact required analysis by and approval from BCoC’s legal unit.

✔ Raising awareness about the SDGs as a starting point for discussions with stakeholders is a crucial element.

✔ Although support from senior management inside the institutions is necessary for speed and practicality, it does not guarantee the smooth running of the project. Senior management is not necessarily aware of the constraints that staff are confronted with when it comes to data sharing.

Harmonization 

✔ Databases need to have a dictionary of variables or metadata prior to using a data reconciliation tool.

✔ Partners must define clear definitions of concepts to avoid misinterpretations in the process. Trade-offs between data access and confidentiality.

✔ Partners need to make legal units aware of the statistical purposes of data without compromising confidentiality of the data.

✔ Administrative and legal units require more time than knowledge management or data analysis units to run analyses and feasibility studies regarding the data sharing process.

Trade-offs between data access and confidentiality

✔ Partners need to make legal units aware of the statistical purposes of data without compromising confidentiality of the data.

✔ Administrative and legal units require more time than knowledge management or data analysis units to run analyses and feasibility studies regarding the data sharing process.

Technology tools are not always the answer

✔ To ensure high-quality, accurate information, BCoc needs to improve its collection, validation, and quality processes regarding the Mercantile Register prior to sending the data to DANE.