By Diana Harper
As the international development community has increasingly come to understand the role of complexity in sustainable social change, programs seeking to achieve systems change seem to have accelerated in recent years. Of particular interest are innovation initiatives, which seek to identify, pilot, and scale different approaches to solve “wicked” problems that haven’t responded to previous strategies.
With such intervention approaches in place, the next challenge is how to evaluate their effects and learn and adapt accordingly. Historical approaches to evaluation in development have assumed a tightly-controlled implementation environment and a linear theory of change—neither of which are up to the task of the complexity of the real world and of innovation initiatives.
LINC recently completed primary research on innovations to strengthen data ecosystems and local networks in Tanzania as part of a wider engagement under SPACES. A summary of findings is listed below; read the full report for more details.
Background and Approach
The $21.8 million Data Collaborative for Local Impact (DCLI) program is the result of a partnership between the Millennium Challenge Corporation (MCC) and the President’s Emergency Plan for AIDS Relief (PEPFAR). DCLI includes a series of innovative activities to promote data use for decision-making; it is implemented by MCC and funded by PEPFAR. To assess the effectiveness of the program in Tanzania, MCC engaged SPACES to conduct an independent evaluation after the majority of the programs had been implemented.
Using a systems perspective, we examined the results of the program to understand the actual and unanticipated effects of the program, as well as the pathways through which change occurred. In order to do this, we adapted an outcomes-harvesting methodology in order to identify meaningful changes in the local context related to the outcomes of interest, and then worked backwards to identify whether and how the project contributed to these changes. This helped to explore why certain innovations were sustained, as well as how the project activities affected and were affected by outside factors. In total, we conducted 45 key informant interviews (KIIs) from September to November 2019.
Alongside this data collection, we also conducted secondary research to review project reports and communications materials, academic and grey literature, and news and media sources.
Results and Recommendations
Overall, the research reflected that the DCLI program has had a positive effect on several key outcomes of interest. Respondents demonstrated that the program has increased data skills of individuals and organizations, improved data use for service delivery and resource allocation at the local level and fostered an improved data culture and attitudes toward data.
Of note, the vast majority of those interviewed were no longer engaged or receiving direct benefits from the DCLI program. Yet the research showed that activities initiated under DCLI are continuing beyond the life of the program in a variety of ways, and that these efforts supported long-term change in the local context. For example:
- Entrepreneurs supported by DCLI are continuing to build and scale their data innovations as well as their businesses.
- Those who have been trained and mentored under DCLI are using their skills to improve the quality of their organization’s service delivery and advocacy efforts.
- New local leaders in communities where DCLI focused are using findings from DCLI-supported community dialogues to understand their constituents’ priorities.
In this way, it appeared that, beyond the program implementation period, DCLI has begun to spur a wider set of changes in the data ecosystem. The research identified a set of recommendations for data ecosystem initiatives in Tanzania and in other settings. In particular, the research reflected that when local actors lead change, programs are more effective and sustainable. Additionally, the research showed that flexible and adaptable program structures are needed to lay the groundwork for innovations to emerge. By using a systems approach to evaluation, we were able to uncover wider effects of this innovation initiative and generate learning for how to best influence change in data ecosystems.