Your search

Realist evaluation and experimental designs are both well-established approaches to evaluation. Over the past 10 years, realist trials—evaluations purposefully combining realist evaluation and experimental designs—have emerged. Informed by a comprehensive review of published realist trials, this article examines to what extent and how realist trials align with quality standards for realist evaluations and randomized controlled trials and to what extent and how the realist and trial aspects...

Evaluators are interested in capturing how things causally influence one another. They are also interested in capturing how stakeholders think things causally i...

Participatory action research (PAR) is an approach to research that prioritizes the value of experiential knowledge for tackling problems caused by unequal and harmful social systems, and for envisioning and implementing alternatives. PAR involves the participation and leadership of those people experiencing issues, who take action to produce emancipatory social change, through conducting systematic research to generate new knowledge. This Primer sets out key considerations for the design of...

Achieving impact through research for development programmes (R4D) requires engagement with diverse stakeholders across the research, development and policy divides. Understanding how such programmes support the emergence of outcomes, therefore, requires a focus on the relational aspects of engagement and collaboration. Increasingly, evaluation of large research collaborations is employing social network analysis (SNA), making use of its relational view of causation. In this paper, we use...

The United Kingdom Research and Innovation (UKRI) Global Challenges Research Fund (GCRF) aimed to address global challenges to achieve the United Nations (UN) Sustainable Development Goals through 12 interdisciplinary research hubs. This research documents key lessons learned around working with Theory of Change (ToC) to guide Monitoring, Evaluation and Learning (MEL) within these complex research for development hubs. Interviews and document reviews were conducted in ten of the research...

Research for development (R4D) funding is increasingly expected to demonstrate value for money (VfM). However, the dominance of positivist approaches to evaluating VfM, such as cost-benefit analysis, do not fully account for the complexity of R4D funds and risk undermining efforts to contribute to transformational development. This paper posits an alternative approach to evaluating VfM, using the UK’s Global Challenges Research Fund and the Newton Fund as case studies. Based on a...

The complexity of issues addressed by research for development (R4D) requires collaborations between partners from a range of disciplines and cultural contexts. Power asymmetries within such partnerships may obstruct the fair distribution of resources, responsibilities and benefits across all partners. This paper presents a cross-case analysis of five R4D partnership evaluations, their methods and how they unearthed and addressed power asymmetries. It contributes to the field of R4D...

“Hacking by the prompt”—writing simple yet creative conversational instructions in ChatGPT's message window—revealed many valuable additions to the evaluator's toolbox for all stages of the evaluation process. This includes the production of terms of reference and proposals for the dissemination of final reports. ChatGPT does not come with an instruction book, so evaluators must experiment creatively to understand its potential. The surprising performance of ChatGPT leads to the question:...

Large language models (LLMs) are a type of generative artificial intelligence (AI) designed to produce text-based content. LLMs use deep learning techniques and massively large data sets to understand, summarize, generate, and predict new text. LLMs caught the public eye in early 2023 when ChatGPT (the first consumer facing LLM) was released. LLM technologies are driven by recent advances in deep-learning AI techniques, where language models are trained on extremely large text data from the...

While many knowledge workers may fear that the rise of artificial intelligence (AI) will threaten their jobs, this article argues that small evaluation businesses should embrace AI tools to increase their value in the marketplace and remain relevant. In this article, consultants from a research, evaluation, and strategy firm, Intention 2 Impact, Inc., make a case for using AI tools to disrupt business as usual in evaluation from theoretical and practical perspectives. Theoretically, AI may...

Introduction:The Most Significant Change (MSC) technique is a complex-aware monitoring and evaluation tool, widely recognized for various adaptive management purposes. The documentation of practical examples using the MSC technique for an ongoing monitoring purpose is limited. We aim to fill the current gap by documenting and sharing the experience and lessons learned of The Challenge Initiative (TCI), which is scaling up evidence-based family planning (FP) and adolescent and youth sexual...

This article explores the challenges of monitoring and evaluating politically informed and adaptive programmes in the international development field. We assess the strengths and weaknesses of some specific evaluation methodologies which have been suggested as particularly appropriate for these kinds of programmes based on scholarly literature and the practical experience of the authors in using them. We suggest that those methods which assume generative causality are particularly well...

Issues of power are not new to program evaluation. What is new is a consideration of how programming uses insights into incentives that shape and adapt implementation. How should one evaluate in a way that explicitly assesses the ways in which a program considers power? One of the innovative topics deriving from the democracy and governance space is the approach of thinking and working politically (TWP) which is seeing increased use in development programming. TWP suggests different mental...

This chapter focuses on facilitation given by the Innovation Caucus to provide expert academic critique to inform Innovate UK's strategy for UK business innovation. As Innovate UK's strategy developed, it became evident that several policy domains needed critical insights and evidence. An academic critical friend provided the latest academic insights, evidence and wider perspective of the actors in the UK Innovation system, insights of which policy makers often lack. The chapter gave case...

Evaluators who take a complexity-aware approach must consider tradeoffs related to theoretical parsimony, falsifiability, and measurement validity. These tradeoffs may be particularly pronounced with ex-post evaluation designs in which program theory development and monitoring frameworks are often completed before the evaluator is engaged. In this chapter, we argue that theory-based evaluation (TBE) approaches can address unique ex-post evaluation challenges that complexity-aware evaluation...

This chapter examines good practices in implementing effective Monitoring, Evaluation, and Learning (MEL) systems within complex international development Democracy, Human Rights, and Governance (DRG) programs, which are characterized by challenges of non-linearity, limited evidence of theories of change, and contextual and politically contingent nature of outcomes. The chapter presents three cases of MEL systems in complex projects implemented by Pact across distinct and diverse operating...

Background: Addressing today’s sustainability challenges requires adopting a systemic approach where social and ecological systems are treated as integrated social-ecological systems. Such systems are complex, and the international development sector increasingly recognises the need to account for the complexity of the systems that they seek to transform. Purpose: This paper sketches out the elements of a complexity-aware monitoring and evaluation (M&E) system for international...

This article explores the potential of using participatory action research as an adaptive programming modality to drive learning and innovation to tackle the drivers of (and seek to eliminate) the Worst Forms of Child Labour. We draw on our experience from early phases of implementation of a large-scale action research programme, which despite the constraints covid-19 posed in moving to full implementation and participatory engagement with children and other stakeholders on the ground, is...

Meeting the United Nations Sustainable Development Goals (SDGs) will require adapting or redirecting a variety of very complex global and local human systems. It is essential that development scholars and practitioners have tools to understand the dynamics of these systems and the key drivers of their behavior, such as barriers to progress and leverage points for driving sustainable change. System dynamics tools are well suited to address this challenge, but they must first be adapted for...