Search

Despite an emerging body of scholarship on applying generative AI (GenAI) to qualitative data analysis, this area remains underdeveloped. This article evaluates how GenAI can support thematic analysis using a publicly available interview dataset from Lumivero. It introduces Guided AI Thematic Analysis (GAITA), an adaptation of King et al.’s (2018) Template Analysis. This framework positions researchers as a reflexive instrument and intellectual leader while thoroughly guiding GPT-4 in four...

The CGIAR 2030 Research and Innovation Strategy commits organizational change with seven ways of working, including “Making the digital revolution central to our way of working”. In that context, Artificial Intelligence (AI), introduces both opportunities and risks to evaluation practice. Guided by the CGIAR-wide Evaluation Framework, integrating AI tools requires a governance approach to balance innovation with ethical responsibility, ensuring transparency, fairness, accountability, and...

One of the main questions evaluators have when using AI is: "How can I know if it performs well?" To answer this question, we have developed a guidance note aimed at capturing what we have learned so far about integrating AI into evaluation and to offer a framework for further exploration. Since we began our experiments with Large Language Models (LLMs) in the Spring of 2023, we have made significant progress in using Generative Artificial Intelligence (GenAI) for processing and analyzing...

What it takes to create smart velocity

This article presents an artificial intelligence-assisted causal mapping pipeline for gathering and analysing stakeholder perspectives at scale. Evidence relevant to constructing a programme theory, as well as evidence for the causal influences flowing through it, are both collected at the same time, without the evaluator needing to possess a prior theory. The method uses an artificial intelligence interviewer to conduct interviews, automated coding to identify causal claims in the...

Generative AI (GenAI) is evolving rapidly, and its use in violence against women research remains new and largely experimental. While researchers are beginning to explore GenAI’s potential, uncertainty around its risks, ethical challenges, and real-world application persists. Recognising the need for clear guidance, the Sexual Violence Research Initiative (SVRI) partnered with The MERL Tech Initiative (MTI) to develop this resource. This guide aims to provide background on GenAI, strengthen...

Relational state capacity (RSC) integrates the quality of citizen-state relationships into the concept of state capacity, emphasizing mutual recognition between citizens and state agents as foundational for societal problem-solving. Traditional state capacity definitions overemphasize technical and institutional elements, overlooking relational dynamics. RSC is a latent societal resource activated through mutual recognition in citizen-state exchanges with potential to improve outcomes across...

The post outlines the background and development of a new approach to assessing value for money (VfM) in public policies and social programs. King identifies a key problem: while there's increasing pressure to demonstrate VfM in public programs, traditional assessment methods are incomplete, with economics and evaluation operating as separate rather than integrated disciplines. Key points about the research development: 1. King developed a conceptual model for VfM evaluation, published in...

A curated collection of resources on Adaptive Management and MEL in International Development

Essential how-to guidance for government product managers and digital teams.

This paper describes, synthesises and analyses the work of 25 Action Research Groups that were the heartbeat of the CLARISSA Systemic Action Research programme. The CLARISSA programme worked to create participatory and systemic solutions to the worst forms of child labour in the adult entertainment sector in Nepal, and the leather sector in Bangladesh. The paper explores how the action research groups were set up, and how they worked through key stages of trust and relationship building,...

This article provides a comprehensive overview of rubric development in evaluation practice, focusing on stakeholder engagement and practical implementation. King explains that rubrics are essential tools for implementing evaluative reasoning, serving as frameworks that combine evidence, criteria, and standards to make transparent value judgments. The piece emphasizes that effective rubric development is inherently participatory, requiring meaningful stakeholder engagement for both ethical...

In an evolving international development landscape, the push for localization has gained urgency. Global South Intermediary Organizations (GSIs) are emerging as key players, offering a promising pathway to achieving localization programming. To harness their potential, it is crucial to develop a deeper understanding of their functions, structures, and strategies. However, they should complement—not replace—direct funding relationships with grassroots organizations. The Trust,...

Government offices are filled with siloed, sector-specific digital systems that strain their capacity to make decisions, use data effectively, and achieve ambitious sustainable development goals. Public investments in digital development, transformation, and infrastructure can only meet citizen needs if data and systems are consolidated and interoperable. While interoperability is a sensible approach to building digital public infrastructure, transforming existing systems is easier said...

This five-step framework, developed and tested by a foundation, embeds learning in emergent systems change strategies. It prioritizes the testing of hypotheses and assumptions, uses learning questions, and calls for examining both confirming and disconfirming evidence. --- A framework for embedding learning in systems change strategies and for testing strategic uncertainties. Learning and evaluation approaches that accompany systems change efforts need to fit with and support the emergent...

The Reduced Access Analytical Methods (RAAM) toolkit is a practical resource designed to help humanitarian practitioners overcome monitoring challenges in reduced access environments. Reduced access can be caused by natural disaster, conflict, political instability, or other factors, and typically makes it difficult to conduct normal monitoring of program implementation. The RAAM toolkit offers technical and managerial tools for a menu of analytical methods that can fill information gaps...

Over the course of this paper we lay out the basis of the Carve-Out method, an approach that leverages a behavioural framework to allow organisations (even those burdened with layers of bureaucracy and entrenched ideas) to create intentionally designed environments that give people the space, resources and support they need to explore and test new ideas. Ideas that may go on to transform the core of their organisation. The Carve-Out method is based on a simple insight: That if we want to...

The guide begins with a section on what we at TAF have learned about practicing Strategy Testing, highlighting things that we didn’t know when we started out. Some of these lessons may seem obvious, but others may strike you as less so. In any case, we believe that, when taken together, they provide a useful reminder of some of the pitfalls that arise in efforts to practice Adaptive Management. Moreover, they help to remind us that there are multiple layers to doing Strategy Testing...

With growing evidence of what works in education, governments and their partners strive towards scale. While implementation research to adapt solutions holds promise for testing what works, where and for whom, its use in education is limited. Sixty-three international education stakeholders from government, academia, think tanks, local and international non-governmental organizations, multi- and bi-laterals and foundations offer insight into three modes of implementation research that differ...