Understanding stakeholders and political networks - Practical Guidance Note - Piron, Laure-Hélène; Mwamba, Wilfred (FCDO and TWP CoP, 2026)
In 2023, the UK Foreign, Commonwealth & Development Office and the Thinking and Working Politically Community of Practice published Understanding political economy analysis and thinking and working politically. This complementary note sets out more detailed guidance on how to undertake a dynamic Stakeholder Analysis and Political Network Mapping, which are core elements of political economy analysis and other forms of politically informed context analysis. This technical note is for use alongside the UK Foreign, Commonwealth & Development Office (FCDO) guide to understanding political economy analysis (PEA) and thinking and working politically and is part of a series of guides on the use of PEA in practice. It sets out more detailed guidance on how to undertake a dynamic stakeholder analysis and political network mapping, both of which can be used to support international cooperation and development partnerships. Stakeholder analysis considers the key actors relevant to an area of work, political network mapping goes further, analysing the relationships between actors. Both tools help practitioners move from a general understanding towards specific, evidence-based decisions. These approaches can contribute to the design and implementation of a strategy, policy dialogue or development programme, and to the management of risks This guide has been produced jointly with the Thinking and Working Politically Community of Practice (TWP CoP). The TWP CoP is a global network of practitioners and researchers in international cooperation committed to helping practitioners understand how change happens and why. The TWP CoP Secretariat has been hosted by the University of Birmingham with support from FCDO.

Critical thinking, creative thinking, systems thinking and many more: A comparative bibliometric analysis of prevalence and distribution - Crilly, Nathan (Thinking Skills and Creativity, 2026)
To compare discourses related to different ways of thinking, a bibliometric analysis was performed on the Dimensions database of research documents. Terms related to 78 ways of thinking were ranked according to the number of documents containing them over the last five years (2020 to 2024). Twenty ways of thinking were subjected to further analyses, examining (i) those that are most prevalent (with the top five being critical thinking, design thinking, creative thinking, systems thinking then computational thinking); (ii) the different suffixes appended to each term (with critical thinking mostly referred to as a skill, systems thinking mostly referred to as an approach and design thinking almost equally referred to as a method, approach and process); (iii) the proportion of documents focusing on each of them, rather than simply referring to them (with computational thinking having the highest such proportion); (iv) their rise to prevalence since 1975 (with critical thinking and systems thinking slowly gaining prevalence compared to the more recent rise of design thinking and computational thinking); (v) the frequency with which they are referred to together (with critical thinking and creative thinking co-occurring most often); (vi) their distribution across various academic disciplines (with futures thinking being quite evenly distributed, but computational thinking being highly concentrated); (vii) their distribution across the Sustainable Development Goals (with critical thinking, systems thinking and design thinking being most prevalent in this respect). The findings and visualisations provide a useful basis for identifying, comparing, selecting and combining different ways of thinking.
Keywords: Bibliometric analysis; Interdisciplinary; Sustainable development goals; Ways of thinking

Regenerative Evaluation - Influences, Pathways & Practices - Boone, Katie; Collins, Kirsten Bording; Kaiser, Jannik; Martins, Laila; Reppart, Mateah Spencer; Strasser, Tim; Van den Daele, Geetha (Uniity Effect, 2026)
Regenerative Evaluation: Evaluation in Service of Life To foster change today means navigating a gap between two realities: the reality of reporting (demanding accountability and proof of impact) and the reality of the planet (calling for regeneration, complexity, and interconnectedness). For too long, evaluation has forced a choice, functioning as an extractive exercise—mining data to count the "fruits" of a project while ignoring the health of the "soil" that sustains them. Regenerative Evaluation bridges this gap. Moving beyond extractive measurement, it is an evolving practice that embraces rigor while actively nourishing the systems it assesses. This document outlines its core components: ● The 9 Guiding Principles: The internal values and mindsets that orient our judgment and align our work with living systems. ● The 7 Core Practices: A practical, three-phased cycle of specific methods and habits that translate these principles into tangible evidence and action. ● The 3 Essential Evaluation Strands: The evaluation lineages (systemic, power-conscious, and ecological) that this practice weaves together. ● Applications and Future Pathways: Entry points for bringing this work to life across projects, organizational vitality, funding, and policy.

In-between Spaces - Intermediaries Rethinking Funding Practices. A learning pap er on Funding and Entrepreneurship Supp ort for Adaptation & Resilience - CAIL (Climate Adaptation Innovation Learning and SILP, 2026)
This learning paper on funding and entrepreneurship support for adaptation and resilience is a collaboration between SILP and the Climate Adaptation Innovation & Learning Initiative project. It brings together real-world insights from a community of 25 practitioners working across intermediary roles in the climate adaptation and resilience sector. Rather than presenting a fixed model, the paper explores how intermediaries operate in the “in-between spaces”, navigating tensions between funders and communities, certainty and emergence, delivery and learning. It highlights the practices already being used in these spaces, and what is needed to enable more effective, equitable, and adaptive transformation. We’ve designed this paper as a practical tool — offering insights to challenge assumptions, open dialogue, and support reflection within your own organisation or community.

Letting Go, Without Losing Coherence: How Local Agency and Donor Accountability Can Coexist - Guerzovich, Florencia (TWP CoP, University of Birmingham, 2026)
Despite widespread endorsement of locally led development, donor agencies still struggle to translate local agency into cumulative change. Donor discourse increasingly emphasises localisation and community leadership, yet operational arrangements remain structured around upward compliance. The practical question is how local agency and institutional accountability can work together so that locally generated insights lead the way while portfolio coherence is maintained. This paper examines a form of connective work that helps make this co-existence possible. Drawing on the author’s more than 20 years’ experience as a practitioner and evidence from over 150 documented cases across sectors and regions, it focuses on invisible weavers – actors who keep cooperation moving under constraint by connecting fragmented institutions, routing insights across levels, and sustaining continuity when conditions change. These functions are often essential to maintaining cooperation, insights, and continuity across fragmented actors and institutions, yet they remain largely unrecognised in formal programme design, costing, and evaluation. Despite this, donors’ informal routines allow them to act on these insights, albeit in ad hoc ways. The paper addresses three practical questions: how such actors can be recognised, how their effectiveness and resilience can be assessed, and how donors can support this connective work more intentionally without formalising it into compliance. It traces how donor staff and local counterparts use judgement, continuity, and low-overhead practices to carry locally generated insights into organisational decisions. By reframing accountability as portfolio reliability and making these patterns visible, the paper shows that donors can ‘let go’ of rigid control without losing systemic coherence. It concludes with three practical ways to make these existing ‘workarounds’ more intentional and fundable, ensuring that local intelligence leads to durable change rather than having to re-set with every project cycle.

The evaluator’s guide to generative AI - Perry, Chris; Moses, Michael; Pinter, Max (Itad, 2026)
AI’s role in evaluation: so long and thanks for all the synthesis How generative AI can enhance evaluation and analysis? We share three real-world case studies to explore what works, what doesn’t, and how to harness AI responsibly. Generative AI is everywhere—but is it really the answer to Life, the Universe and Everything? At Itad, we see AI as a powerful assistant, not a replacement for human judgment. Our new learning brief, The evaluator’s guide to generative AI, shares lessons from three real-world evaluations where we piloted AI tools to support rigorous analysis. Real-world AI use to support our work In our evaluation of the Wellcome Trust’s Climate Impacts Awards, we used Microsoft 365 Copilot to extract and synthesise data from hundreds of grant proposals. This freed up our team to focus on interpretation and recommendations. For the MacArthur Foundation’s Big Bet on Nigeria programme, we used CoLoop AI to transcribe and translate interviews, speeding up data processing. However, we found it needed careful handling to support nuanced data analysis. And in our work with a partner that supports organisations throughout Africa to generate evidence and data, we combined Copilot, ChatGPT and MaxQDA’s AI tools to analyse nearly 450 grantee reports and interviews. Each case offered valuable insights into where AI adds value and where human expertise remains essential. Lessons for rigorous AI use Across these projects, we identified seven key lessons for using AI in evaluation: 1. AI is great at extraction; humans are great at sensemaking AI tools excel at structuring large volumes of data and surfacing patterns. But interpretation, trade-offs and conclusions must come from people. In the Wellcome case, Copilot helped us quickly synthesise proposal data, but our team’s judgment was still vital to make sense of the findings. 2. Break the work into smaller steps Complex tasks can overwhelm AI. Better results can be gained by breaking analysis into clear stages, such as defining a framework, extracting evidence, classifying responses and synthesising findings. This approach improves accuracy and makes verification easier. 3. Design prompts around your framework The quality of AI output depends on how you ask the question. Structured prompts aligned to evaluation frameworks, with clear definitions and examples, helps AI stay on-topic and produce more consistent results. 4. Always verify the results Even when AI seems accurate, it’s essential to check. Build in spot-checks and scoring to assess completeness, relevance, and accuracy. In the Wellcome case, manual review of 15% of proposals confirmed high fidelity. In other cases, verification helped us identify and correct oversimplifications. 5. Limit the context for nuanced analysis AI performs better when given focused tasks. In the MacArthur evaluation, CoLoop struggled with broad thematic analysis. We found that narrowing the scope to analyse one theme or subset at a time leads to more meaningful insights. 6. Choose the right tool for the job Different tools have different strengths, and matching the right tool to the task is key. Our experience suggests that ChatGPT is best for contextual analysis, Copilot works well for secure document review and MaxQDA’s Tailwind supported synthesis of coded data. 7. Document your process To build confidence in the use of AI, a clear audit trail is essential. Keeping records of prompts, outputs and verification steps helped us build trust with clients, support internal learning and ensure transparency. Looking ahead: scaling AI in evaluation The next challenge is scaling AI use without losing rigour. This means using AI to reduce manual effort, such as document triage or evidence extraction, while keeping humans in the loop for interpretation and decision-making. At Itad, we’re building bespoke platforms that embed AI into our workflows. For example, one new tool we’re developing for a philanthropic partner organisation will help surface insights from five years of grantee reports. We’re also investing in an internal platform to make our best AI practices available across all projects, laying the groundwork for new service areas including foresight and scenario planning. As we explore these frontiers, we’re guided by a simple truth: AI can help us ask better questions and work more efficiently—but it’s still up to us to make sense of the answers.

Using rubrics to explore pathways to scale: Navigating the Integration of Graduation into Public Systems - Blog post on Better Evaluation - Pellini, Arnaldo; Torres, Kristian Paolo; Warokka, Tania Dora (Better Evaluation, 2026)
In this blog, Arnaldo Pellini, Kristian Paolo Torres, and Tania Dora Warokka reflect on their experience developing and testing a rubric to support dialogue and learning as complex initiatives are integrated into public systems.

Moving Beyond Projects: Portfolios and Systems MEL - Muhereza, Suzane; Tolmer, Zazie (UNDP Strategic Innovation, 2026)
Authors: Suzane Muhereza & Zazie Tolmer Contributor: Simone Uriartt & Bianca Tejada

Technically Strong and Politically Savvy: Enhancing Thinking and Working Politically when practicing the consevation standards at USAID - Schuttenberg, H.Z.; Stark, J.; Schueler, K. (USAID, 2026)
This supplemental guide offers practical advice for thinking and working politically (TWP) when practicing the Conservation Standards. The CS are a widely used, systematic approach for planning, monitoring, and adapting programs aiming to conserve biodiversity. TWP is an approach to international development that brings greater attunement to the political dynamics of the local programming context in order to enhance development results. Given strong evidence that political factors overwhelmingly drive the loss of biodiversity, this guide is based on the premise that conservation programs will be more effective when they skillfully integrate TWP.

Doing MEL differently: CECG’s experience of piloting Conversation-Based Reporting - SMILE, Climate (Medium, 2026)
The case for Conversation-based reporting (CBR)

Exploring participants’ experiences of and perspectives on relational development: The cases of CLARISSA SP+ and WORKFREE - Lazarus, Joel (Community Development, 2026)
There is growing engagement with “relational” approaches to community development work - interventions that give practical or even metaphysical primacy to relations and processes over identities and outcomes. But what more can we say about the nature and efficacy of such approaches? CLARISSA SP+ and WorkFREE were two major relational community development intervention and research projects that took place in Bangladesh and India respectively. In this article, I compare the experiences and perspectives of participants in CLARISSA SP+ and WorkFREE to learn more about the nature and efficacy of relational community development. My findings reveal how these projects served to meet participants’ individual and collective psychological and material needs and highlight the transformative power of unconditionality. Whilst my overall findings support the case for advancing relational approaches to community development work, I conclude that more work must be done on theoretical and operational refinement and systematic evaluation.
Keywords: Relationality; community development; needs; universal basic income

Launching a new Co-design Toolkit for Early Years Services - Adams, Ellen; Alves Calistri, Nicole (Policy Lab, UK Gov, 2026)
This blog launches a new co-design toolkit to support the development of services for early years. The co-design toolkit is a practical set of resources created to enable local authorities and communities to design public services together. It has been

Scaling democratic innovations: Features of Effective Catalyst Organisations & Future Frontiers - Claudia, Chwalisz; McKinney, Sammy (DemocracyNext, 2026)
Citizens' assemblies and other democratic innovations are spreading around the world. But they do not spread by themselves. Behind every successful scaling story sits a constellation of organisations doing the essential, often invisible work of building capacity, establishing networks, advocating with decision makers, and ensuring quality standards. These are what we call scaling catalysts: organisations that intentionally drive the expansion of democratic innovations in their regions. In this paper, we make three core contributions to the field: 1. We distil six features of effective scaling catalyst organisations, aiming to elevate the important role they play. 2. We examine critical tensions and trade-offs these organisations face, and how they can navigate these. 3. We identify five frontiers of future practice that can further accelerate the scaling of democratic innovations and promote more deliberative cultures beyond the work of individual catalyst organisations. This paper is for three key audiences: We offer insights for practitioners building similar organisations, for funders seeking to support this vital work, and for researchers identifying knowledge gaps.

Complex Systems Frameworks Collection: an illustrated collection - Finegood, Diane; Bradd, Sam; Yakimov, Chris (Frameworks Collection, 2026)

UNDP Portfolio Evaluation Guidelines - UNDP (Independent Evaluation Office of UNDP, 2026)

The Architecture of Learning: Systems, Infrastructure, & Leadership for Health System Transformation (Stanford Medicine - Department of Pediatrics, 2025)
The Architecture of Learning: Systems, Infrastructure, and Leadership for Health System Transformation Session Description Health care can improve as a system only if its parts are aligned around the shared purpose of better health for all. This talk will explore how learning health systems integrate science, data, and culture to enable continuous improvement and innovation. Through real-world examples from a single patient’s question to national learning networks, we will examine how infrastructure, leadership, and community engagement create the conditions for systems that learn and improve at scale.

Democratic deliberation as a North Star: Showcasing a framework to assess the value for money of systems-change efforts - Peterson, Heidi (Evaluation, 2025)
The growing interest in systems-change initiatives sits alongside increasing pressure to demonstrate value for money (VFM), which is challenging for emergent, interconnected and often intangible work. A new way to assess the VFM of systems-change work involves considering the value of changing system conditions and the creation of potential value for future systems transformation. This innovation combines the Water of Systems Change framework with the Cycles of Value Creation to create five types of value for systems-change work: inherent, potential, applied, realised and transformative. The VFM of systems-change efforts is then best understood, negotiated and judged by different stakeholders through democratic deliberation. A practical case study of WorkWell (a systems-change initiative to improve mental well-being in Victoria, Australia) is used to demonstrate the use of the approach and offer reflections on its rigour.

The five stages of AI adoption: A field guide for evaluators - Blog post on Better Evaluation - Patrick, Jonathan Kuhn (2025)
This blog by Jonathan Kuhn-Patrick (UK Evaluation Society Trustee and AI Working Group Lead) explores the five emotional stages many evaluators experience as they begin working with AI.

Archetypes of AI Adoption: A Snapshot in Time of Philanthropic Practice - Higdon, Grace Lyn; Raftree, Linda (The Foundation Review, 2025)
Key Points • The authors identified four archetypes describing philanthropic funders’ approach to AI: The Curious, The Doers, The Dreamers, The Skeptics. • The authors did not find major differences across foundations based on their geographic location, though there were differences found based on their mission and values. Feminist and social justice funders in the Global South demonstrated more skepticism than others. • Even in seemingly benign or straightforward applications, AI systems demonstrate unpredictable behavior, leading to most foundations proceeding with caution. • The rapid pace of AI has meant that formal AI policies are rare, though foundations align on some common “red lines” that guide internal use of AI.

AI and the Gell-Mann amnesia effect - King, Julian (Evaluation and Value for Investment, 2025)
Quality assurance strategies when working on unfamiliar topics

From paradigm blindness to paradigm shift? An integrative review and critical analysis of the regenerative paradigm - Taveras-Dalmau, Vanessa; Becken, Susanne; Westoby, Ross (Ambio, 2025)
Amid growing concerns over sustainable development failures, scholars are exploring the ‘regenerative paradigm’ as a pathway for systemic change; yet, its paradigmatic foundations remain underexamined. Using thematic analysis, we analyse the regenerative knowledge field through an integrative review of 320 cross-disciplinary articles on regenerative approaches, synthesising findings into an interactive Regenerative Paradigm Map with 7 principles, 33 themes, and 253 specific elements. We assess all Map components against an analytical framework of 14 paradigm criteria to evaluate whether regeneration constitutes a paradigm, finding it meets several criteria but not others. We interpret results through the lens of paradigm blindness, interpreted as entrenched worldviews impacting all paradigms. We conclude with a critical reflection on how select Map components may support a shift away from growth-oriented systems and conceptualise the Tensions of Paradigm Shifts to discuss blind spots in scholarly interpretations of regeneration, which may contribute to paradigm blindness within the field.
Keywords: Artefact; Neoliberalism; Paradigms; Regenerative; Sustainability; Systems change

Generative AI guide for civil society - Byoung-il, Oh; Achim, Koh; Yeo-kyung, Chang (Digital Justice Network, 2025)
The Digital Justice Network produced a guide that originated from the realisation that civil society organisations and activists need help in establishing generative AI policies and properly utilise these tools when necessary. “Some of our activists are drafting statements with ChatGPT, and I’m worried about what approach we should take.” Many organisations are likely having similar concerns. With the emergence of various generative AI services—such as chatbots like ChatGPT and Gemini, and tools that create images, music, and videos—an increasing number of citizens are using them for both professional and personal purposes. Civil society activists are no exception. However, while they are being utilised, usually based on individual judgment even for work-related tasks, almost no organisation currently has an organisational-level policy on generative AI. There are many points that civil society organisations (CSOs) must consider when using generative AI. For example, if factual inaccuracies (hallucinations) from generative AI are included in an organisation’s official documents, the organisation’s credibility can be severely damaged. Security issues may arise if personal or confidential information is uploaded to unreliable commercial services. Furthermore, the output of generative AI might contain biases that conflict with the organisation’s values. The process of drafting a statement using generative AI may exclude aspects crucial for activist capacity building and internal organisational deliberation. If activists use AI tools based on individual choice without an organisational policy, there is a high likelihood that issues beyond the organisation’s control will emerge. However, in the Korean context, there is a lack of guidelines available regarding whether it is appropriate for civil society organisations to utilise generative AI services, what principles and policies should govern their use if they choose to do so, and what guidance can be referenced from a human rights perspective. Moreover, the current status of which AI tools activists are using for which tasks has not been documented. This guide originates from the realisation that we need to help civil society organisations and activists establish generative AI policies and properly utilise these tools when necessary. To create this guide, we conducted a survey on which AI tools are actually being used for which tasks, how useful generative AI is perceived to be, and what problems users are experiencing. We gathered opinions not only from domestic activists but also from activists worldwide through the APC network. While the sample size is limited, restricting its statistical significance, we were able to confirm the real concerns and shared understanding of the issues felt by activists. Even those who use generative AI minimally responded, sharing their thoughts. Furthermore, we held workshops with civil society and labor union activists focusing on generative AI. We shared the survey results and a preliminary policy framework, allowing participants to exchange their experiences and perspectives. Through this process, we reconfirmed that the act of honestly sharing feelings and concerns is crucial, rather than simply reaching a consensus. The policy framework presented in this guide is merely a starting point; the process of each organisation creating its own policy that reflects its reality and the voices of its activists is paramount. While some activists use generative AI with interest, many others still feel uncomfortable with generative AI itself. We clearly state that this guide is not intended to encourage the use of generative AI. The fact that the development of major generative AI models and the provision of services are exclusively controlled by Big Tech companies is also a concern. Although this guide focuses on the commercial generative AI services currently in dominant use, we deeply empathize with the need to overcome these structural limitations. Despite various limitations, we hope this guide will be of some help to organisations and activists currently contemplating policies related to generative AI.

There Is No Such Thing as Impact Investing - Starr, Kevin (Stanford Social Innovation Review, 2025)
There is philanthropic investing, and there is commercial investing, and there is nothing in between.

AI in evaluation: Good practice guidelines for practitioners - UK Evaluation Society (UK Evaluation Society, 2025)
This practical guide from the UK Evaluation Society explores how evaluators can use AI tools responsibly and transparently across all stages of the evaluation process.

Track to Change | Home - Clear Horizon (Clear Horizon, 2025)
Track to Change homepage

Working between systems and complexity - a practitioners guide - Allen, Will (learningforsustainability.net, 2025)
How practitioners navigate between systems and complexity thinking. Exploring structure and emergence in real-world collaboration and adaptive work.

A critical, integrative review on evaluating systems change and transformation, Part One: 2011–2021 - Gates, Emily F.; Vidueira, Pablo; Komakhidze, Maya; Aldrich, Charlotte; Shim, Clara (Evaluation, 2025)
Calls for evaluating systems change and transformation in multiple fields present an opportunity to explore cross-field patterns. This article reports on part one of a critical, integrative review of academic and gray literature published between 2011 and 2021 (n = 102) within five areas: evaluation, health, organizational change, sustainability, and philanthropy. Questions address key definitions, how a systemic approach differs from traditional social problem-solving, leverage points to influence change, and implications for evaluation. Four findings include (1) limited normative debate about change and transformation; (2) conventional and systemic approaches contrasted as binary paths; (3) 10 shared leverage areas with the least attention on power and resources; and (4) an expanded role for evaluation that presents challenges and opportunities. Review results provide support for shifts underway within the evaluation field, including funders working in deeper collaborations, evaluators expanding their skill sets, and intermediary agencies facilitating transdisciplinary exchanges.

Moving from nudging to boosting: empowering behaviour change to address global challenges - Hertwig, Ralph; Michie, Susan; West, Robert; Reicher, Stephen (Behavioural Public Policy, 2025)
The COVID-19 pandemic provided a stark reminder that societies will struggle to address global challenges unless they are able to change behaviour at scale. The widely adopted ‘nudge’ approach epitomizes an individualistic, deficit model of human cognition and motivation that leverages or overcomes people’s weaknesses and biases to get them to do things they would otherwise not. By contrast, we argue that tackling the challenges facing humanity requires a collective, capacity-building approach – one that boosts the competences, opportunities, and motivations of individuals to act together.
Keywords: agency; behaviour change; boosting; bounded rationality; nudging

Towards a future we won't regret: A dispatch from 2035 - Simpson, Lea (Brink, 2025)
Earlier this year we spoke to 102 leaders from across the globe to ask: Ten years from now, what will we regret not having done today? This framing turned out to be a powerful device for collective horizon scanning and foresight. Across every conversation there was a sense that this moment demands more from us all, along with a strong eagerness to explore what we can own and act on. More than that, these discussions surfaced direction, hope and a way forward. From CEOs, professors, PhDs, tech founders, heads of UN agencies, frontline leaders, local innovators, public servants and practitioners we heard about some of the shifts that are already happening in pockets around the world.

Understanding A Quick Approach to PEA - Whaites, Alan (FCDO, 2025)

Polytunity - The Ideas Letter - Ang, Yuen Yuen (The ideas letter, 2025)
“Polycrisis” has become a ubiquitous descriptor for the multisector, intersecting problems afflicting the world—but the term masks the systemic nature of these crises and overwhelmingly reflects Eurocentric voices, priorities, and…

Shift the status quo: Applying a systems approach to improving adolescent health and wellbeing in cities - A guide for practitioners - Ecorys (Ecorys, 2025)
Healthy Cities for Adolescents Phase II (HCA-II) is a multi-country programme that supports projects working to improve adolescent and young people’s health and wellbeing in intermediary cities. Guided by the principle of shifting the status quo, with adolescents at the centre of this, HCA-II aims to catalyse meaningful changes in how cities function for young people. This guide aims to help you influence the systems that shape adolescent and young people’s health and wellbeing in the cities where they live. You don’t need to be a ‘systems expert’ to use it. It’s designed for anyone working on HCA-II projects (or similar) – whether you’re a project manager, Monitoring, Evaluation and Learning (MEL) lead, consortium partner or youth worker – who wants practical ideas for planning, delivering, and learning from work that aims to make cities fairer and healthier for adolescents. A core focus of this guide is helping projects shift system conditions – the underlying structures, relationships, and beliefs that shape how things work. It’s not about fixing everything at once, but about identifying where small changes, like who’s involved in decision-making, how resources are shared, or how people relate to one another, can unlock bigger, longer-term improvements. Focusing on how our work shifts system conditions helps us move beyond surface-level problems and get to the root causes, like policies, resources, relationships, and behaviours that shape adolescents’ lives. It’s a practical way of thinking and working that leads to more joinedup, fair, and lasting change. This guide is designed to be flexible; dip in and out depending on where you are in your project. It includes practical tools, examples, and ideas across five key areas of systems practice.

Evaluative Inquiry for Systemic Change: A Guide to Shift Beyond Fixes to Lasting Value - Gates, Emily F.; Vidueira, Pablo (SAGE Publications, Inc, 2025)
In a world where social, ecological, health, and other problems are influenced by multiple, intersecting systems, why do most interventions target single-factor solutions? Why does so much evaluation settle for incremental change, and how could it contribute instead to deeper, lasting, transformative change? Authors Emily F. Gates and Pablo Vidueira make two interconnected arguments in this book: they critique the "fixed" approach of traditional program evaluation and policy analysis, and they advance an alternative approach centered on changing systems and developing the value of efforts toward change. They demonstrate that systemic change that embeds evaluative inquiry is a theoretically supported, practical way to work toward fundamental social change with lasting impact.

A personal reflection on Thinking and Working Politically at USAID: - Jacobstein, David (2025)
This paper shares an insider's perspective on the spread of Thinking and Working Politically (TWP) across the US Agency for International Development (USAID), highlighting lessons learned along the way that are relevant even beyond the dissolution of the Agency in 2025.

Implementing Systems Change - Jain, Vikram; Singh, Vivek; Mundol, Hisham; Gogri, Mirik (2025)
While enthusiasm for systems change has grown steadily over the past decade, we find a significant gap persists between this enthusiasm and execution. Funders attempting to implement systems change consistently face 9 challenges, including unclear outcome attribution, long commitment horizons, and complex stakeholder coordination. When faced with these challenges, it is understandable why many funders find themselves defaulting to patching the system rather than changing it. The pressures are real: boards expect clear metrics, program timelines demand quick wins, and established processes reward predictable outcomes. These institutional realities often nudge funders toward incremental fixes, even when they recognize deeper change is needed. This report names and explores the challenges in implementing systems change and provides steps to navigate them. We draw on the authors’ 14 years of experience implementing systems change programs in the Global South across affordable housing, early childhood education, and gender equity. Foundation leaders (e.g., CEOs, principals, and other executives) can use this guide to implement systems change in their work, finding entry points that align with their foundation’s current capacity, constraints, and strategic objectives. Funders can choose to lean in and make their intent to change the system explicit. Doing so would offer the potential for permanent, transformative change that continues long after the intervention or program ends. This choice requires abandoning the comfort of clear attribution and predictable outcomes and accepting that it may be impossible to claim credit for your contributions. The communities you serve rarely need you to prove exactly which intervention created change, but they do need you to commit to changing the conditions that trap them in cycles of inequity. “Courage is not the absence of fear, but action in spite of it.” – Nelson Mandela

Outcomes & Systems: From Predicted Goods to Emergent Cohesion - Johar, Indy (Indy Johar, 2025)
Critical summary – the single through‑line of the essay

Modernizing Development: Introducing Portfolios - a new systems-way of working for today’s complexity, built for greater impact - UNDP (UNDP, 2025)

What if design could transform the way we think and make public policies? Proposing a new model: the Policy Design Journey - Gouache, Christophe (2025)
For about 15-20 years, a rather small-yet growing-community of designers started getting involved in public policy innovation and public problem solving activities. Those design activists (willing to work for the common good, public welfare and social justice) have decided to apply their design skills, tools and methods in order to help public authorities of all levels (from village, to city, metropolis, national governments, or EU commission, etc.) reset & solve public problems and/or fix public policies and public services. « Designing better policies for people, by people » could be their motto. In parallel to working on solving public problems, designers have also spend energy on training civil servants and sometimes elected officials about their design approach so as to empower them in replicating, once the designers are gone, some design-inspired techniques and tools such as co-design with users, involving citizens in policy ideation, prototyping and experimenting before implementing, etc. By doing so, designers slowly spread their know-how to equip public authorities with design skills and progressively transform them. At the same time, we started seeing the first public authorities which would offer to designers real official design positions within their own administrations. Designers would not be only consulting for public authorities but transforming public authorities from within (often linked with the creation of public policy innovation labs). If it's still too early to discuss the success and impact of this strategy on the long term, we can already observe that by applying design methods to policy making, a new way of creating policies is emerging, one inspired by design. In this paper, we'll propose to draw a new guiding approach to policymaking inspired by design. To do so, we will build upon the initial critics of the policy cycle model such as the one of Lasswell drawn in 1957 (still in use today as a reference model) which, despite its clarity and simplicity, is purely theoretical (too linear, too static and too rational) and never takes place in the « real world » and question the actionability of policy threads or streams model as proposed by Howlett in 2015. Then, we'll look into how, in practice, designers have, through practice, « tinted » policy making with their own methods to finally, extract and draw a new model for policy making, one which would build upon the design thinking & doing methodology (questioning the double diamond of the UK Design Council) but also the design « spirit » (capacity to improvise, to detour, to navigate at ease through uncertainty, etc.). Considering public policymaking from the angle of a "journey" could enable designers but also public authorities to embrace the fact that true policy design is an unwritten voyage, made of choices, options, creative detours, opportunities, trips to the unknown, etc.

Transforming Health Care - Shared Commitments for a Learning Health System - Madara, James L.; Miyamoto, Suzanne; Dowdy, Sean C.; Greene, Sarah M.; Tarrant, Julie; Margolis, Peter A. (The New England Journal of Medicine, 2025)

Using the Participation Cube to Conduct Participation Analysis of Development Projects in Zambia and Uganda - Roberts, Tony; Howard, Jo (The Journal of Development Studies, 2025)
Cooke and Kothari argue that the seductive claims of participation often provide cover for shallow and tokenistic development practices that fail to address unequal power relationships. This paper provides a novel mechanism that documents and enables a three-dimensional analysis of (a) who participates (b) at which project stages and (c) at what level of agency and power. No other mechanism was found in the literature for systematically tracing and analysing changes in power relationship along all three dimensions. The application of this ‘participation tracing’ approach to two case study projects in Zambia and Uganda identified new opportunities to address power inequities thus moving in the direction of what Hickey and Mohan refer to as transformative participation. The method has only been applied retrospectively to date. Further research is necessary applying the method proactively in project planning and during implementation as a form of participatory project assessment.
Keywords: Uganda; Zambia; analysis; cube; evaluation; participation; tracing

ChatGPT in thematic analysis: Can AI become a research assistant in qualitative research? - Nguyen-Trung, Kien (Quality & Quantity, 2025)
Despite an emerging body of scholarship on applying generative AI (GenAI) to qualitative data analysis, this area remains underdeveloped. This article evaluates how GenAI can support thematic analysis using a publicly available interview dataset from Lumivero. It introduces Guided AI Thematic Analysis (GAITA), an adaptation of King et al.’s (2018) Template Analysis. This framework positions researchers as a reflexive instrument and intellectual leader while thoroughly guiding GPT-4 in four stages: data familiarization; preliminary coding; template formation and finalization; and theme development. Additionally, the article proposes the ACTOR framework, a simple approach to combining different effective prompting techniques when working with GenAI for qualitative research purposes. Findings reveal GenAI’s capacity for analyzing the data, generating codes, subcodes, clusters, and themes, along with its adaptive learning and interactive assistance in organizing unstructured data and developing trustworthiness. However, this model has some key limitations in terms of its restricted context window for processing large datasets, its inconsistent outputs requiring multiple prompt attempts, the need to move across workspaces, and the lack of relevant training data for qualitative research purposes.
Keywords: Artificial Intelligence; Categorizing; ChatGPT; Coding; Computational Intelligence; Discourse Research; Generative AI; Qualitative Psychology; Qualitative data analysis; Reflexivity; Research Skills; Symbolic AI; Template analysis; Thematic analysis

Considerations and Practical Applications for using AI in Evaluations. Technical Note - Cekova, Diana; Corsetti, Lea; Ferretti, Silva; Vaca, Sara (CGIAR, 2025)
The CGIAR 2030 Research and Innovation Strategy commits organizational change with seven ways of working, including “Making the digital revolution central to our way of working”. In that context, Artificial Intelligence (AI), introduces both opportunities and risks to evaluation practice. Guided by the CGIAR-wide Evaluation Framework, integrating AI tools requires a governance approach to balance innovation with ethical responsibility, ensuring transparency, fairness, accountability, and inclusivity. This Technical Note encourages and guides CGIAR evaluators to ethically explore, negotiate, and experiment with AI tools: Explore: Evaluators are invited to discover how AI, especially GenAI, can enhance evaluation efficiency, from scoping and data analysis to reporting. The Note provides practical guidance on AI applications and examples to support creative yet responsible exploration. Negotiate: Integrating AI should be openly discussed with commissioners, stakeholders, and teams. The Note prioritizes jointly defining boundaries, expectations, and ethical parameters (transparency, accountability, and data sensitivity) at each phase of evaluation. Use AI Responsibly: While AI tools are evolving, evaluators are encouraged to pilot and iterate their use. The document supports experimentation through practical tips, prompt examples, and tool selection criteria, all while emphasizing documentation and learning from each use case. Effective AI governance is grounded in core principles: Transparency requires clear documentation of AI tool usage, data sources, model limitations, and decision-making processes. Accountability involves assigning responsibility for AI decisions and outputs and establishing oversight and redress mechanisms.Fairness and inclusion must proactively mitigate bias and discrimination, with particular attention to underrepresented groups and data gaps. Data privacy and security must align with applicable data protection regulations and ensure secure handling practices. Human oversight ensures that evaluators retain control over processes and can intervene as needed. In operationalizing ethical AI governance in CGIAR evaluations, due diligence is required in assessing AI tools for ethical alignment before deployment: reviewing the transparency of vendors, the documentation of models, and their intended use cases. Where relevant, components involving AI—especially those engaging human subjects or sensitive data—should undergo ethics review. AI applications must be adapted to the local and cultural contexts in which evaluations are conducted, as what is suitable in one setting may be inappropriate in another. Additionally, participants should be informed about the use of AI systems and the implications of data collection or processing to ensure informed consent. Ethical AI governance should be embedded in the entire evaluation lifecycle. During the design phase, evaluators should define AI tools to use, why they are selected, and assess risks. In data collection, AI tools should be used in ways that uphold data privacy and protection standards and avoid reinforcing harmful stereotypes or excluding groups. During the analysis phase, the role of AI in supporting interpretation should be documented, with an acknowledgment of limitations or biases. In dissemination, documentation, and reporting, AI’s contribution, limitations, and human validation should be disclosed. By rapid adaptation of content across formats, languages, and complexity levels, AI opens possibilities for broader, more inclusive communication of findings. Finally, the follow-up phases should include a reflection on the ethical implications observed and how these lessons can improve future evaluations. By embedding methodological flexibility into the evaluation processes, AI adoption would contribute to integrity, equity, and learning in an era of rapid technological advancement. This Technical Note is a conversation starter—as a “Beta” version, it will evolve based on responsible real-world experimentation and continuous reflection. Evaluators are encouraged to be responsive to stakeholder input throughout the evaluation processes, to ensure relevance, accuracy, and inclusivity.

Monitoring, Evaluation and Learning for Systems Change - McDonald, Lizzy; Simpson, Kate; Schramm, Carolin (Wasafiri, 2025)
Today’s critical challenges are complex and require system level change. Issues like ending poverty, reversing biodiversity collapse, creating more equitable and sustainable food systems, or ending health inequalities, will not respond to neat siloed technical interventions. These are not problems we can tackle head on. Rather, we need to change the conditions (systems) that create these problems. Consequently, many of the most ambitious philanthropic foundations are adopting ‘systems change’ approaches to their work. There is a critical need to better understand the impact of systems change interventions. This is for two reasons: 1) without being able to understand our impact we can’t learn about the efficacy of efforts and focus on doing more helpful action and less unhelpful action, and 2) without effective monitoring and evaluation it is hard to galvanise resources, whether as money, time, or enthusiasm. Traditional monitoring, evaluation and learning (MEL) approaches and practices struggle to respond to complexity. Many of today’s MEL approaches have been developed to serve funders keen to ‘prove’ the efficacy of their work and hold grantees accountable. This paradigm has produced approaches like ‘logical frameworks’ that seek to create a linear logic between intervention, outputs, outcomes and impact. Such approaches often pay scant regard to the activity (intentional or otherwise) of other actors in any system, and don’t adapt well to the characteristics of complex problems such as emergence, uncertainty, and interdependency. There is a fast-developing body of practice on MEL for systems change. A growing number of practitioners are experimenting with complexity informed approaches to MEL, and UNDP is pioneering an open-source set of tools and ideas.ii Wasafiri is contributing to this innovation by developing a MEL approach that builds on our (open access) Systemcraft Framework. iii Emerging approaches for systems change MEL are underpinned by some key principles. There is growing consensus on the need to be pragmatic and flexible, not just because it is a new field for many that requires a degree of mindset and behaviour change and different ways of working. MEL approaches for system change primarily focus on improving and learning and on identifying ways to adapt and pivot. This contrasts with linear approaches to implementation and measurement. Learning questions guide and co-exist alongside indicators to support continual learning and adaptation. These shifts will require a shift in relationships between funders and implementers from measuring and accountability to greater flexibility and patience, close engagement and learning together. There is pressure to show impact in less time. Reduced time horizons, increasing impatience and integration of new sets of agendas in development programming present challenges in the context of systems change. In an increasingly resource constrained context and amid growing global uncertainty (e.g. current global politics, emergencies like COVID, economic instability, rapid technological advancement), it is unclear which direction donors will take, i.e. an even greater focus on value for money and impact, and/or willingness to explore innovative approaches and learning in uncertain territory. Wasafiri’s approach for systems MEL builds on Systemcraft. This is a diagnostic and design tool for systems change, that has been used globally across a broad range of systems and interventions. This paper summarises our approach to MEL for systems change. It aims to provide a short introductory overview and form a ‘conversation starter’ to initiate a practical dialogue on current practices, experiences, and learnings with MEL for systems change. It introduces some concepts and appropriate tools for measuring systems change and poses some key questions for further discussion. We aim to engage with funders and practitioners working on complex problems who are in the process of or planning to develop their approach to and practices for MEL, who are interested to learn, share and explore. We welcome your inputs, either in response to this document, via a call, or during online webinars that we will convene in 2025.

Balancing Innovation and Rigor: Guidance for the Thoughtful Integration of Artificial Intelligence for Evaluation - Anuj, Harsh; Den Boer, Hannah; Raimondo, Estelle (IEG, The World Bank & IOE, IFAD, 2025)
One of the main questions evaluators have when using AI is: "How can I know if it performs well?" To answer this question, we have developed a guidance note aimed at capturing what we have learned so far about integrating AI into evaluation and to offer a framework for further exploration. Since we began our experiments with Large Language Models (LLMs) in the Spring of 2023, we have made significant progress in using Generative Artificial Intelligence (GenAI) for processing and analyzing large volumes of text data in our evaluations. Currently, we are running multiple experiments across IEG to test the various capabilities of LLMs in different evaluation use cases. As we continue to explore the strengths, limitations, and risks of this technology, we have partnered with others in the evaluation community to learn from each other and identify promising practices. The guidance note is the result of our collaboration with the Independent Office of Evaluation of the International Fund for Agricultural Development. Our guidance note establishes clear criteria to judge AI performance and good practices for designing meaningful experiments. We use these criteria to test AI on key tasks such as text classification, summarization, and synthesis. Assessing AI’s Performance LLMs do not always provide appropriate or veracious responses, and their output must be thoroughly validated before use, especially in evaluation practice where analytical rigor is crucial. The guidance note proposes several performance dimensions depending on the task at hand. For classification tasks, standard machine learning metrics like accuracy, precision, recall, balanced accuracy, and F1 scores are used (please see the guidance note for a description of metrics and scores). These metrics measure the overlap between machine-annotated 'predicted' labels and human-annotated 'ground-truth' labels. Separate training, validation, testing, and prediction sets are necessary to compute useful performance metrics and reduce bias. For text summarization, text synthesis, and information extraction tasks, producing human-annotated data as ground-truth is not always feasible. In such cases, we assess the quality of model responses using criteria from the field of Natural Language Generation, a subfield of AI: faithfulness, relevance, and coherence. Faithfulness checks if the AI-generated information is factually consistent with the source, relevance assesses if the content selected by AI is the most important, and coherence evaluates the overall quality of the generated sentences. To determine the minimum acceptable value for each metric, we take a context-specific approach. For example, in a text classification task for a structured literature review, recall and precision scores of 0.75 and 0.60 respectively may be sufficient for the initial literature identification, especially with highly class-imbalanced data. This was the case in our applied experiments within an ongoing IEG evaluation. However, for tasks with higher evaluative stakes, such as judging the extent to which a portfolio of activities has reached key performance targets, higher metric values are required. The guidance note provides a framework of assessment criteria with clear definitions. Other Good Practices Identifying use cases where LLMs can add significant value compared to traditional approaches is crucial. Not every task will have (net) benefits from the use of LLMs, so aligning experiments with tasks that can truly leverage their capabilities is essential. Planning workflows within use cases involves breaking down tasks into granular steps to understand where and how LLMs can be effectively applied. Translating typical evaluation flows into AI-enabled workflows is a critical step. Incorporating a modular design for the AI-enabled workflows can further enhance the process since it allows for the reuse of successful components within and across use cases. Clear understanding and agreement on necessary resources and expected outcomes are vital within multidisciplinary teams needed to carry out such work. This includes human resources, technology, timeline, and defining success for each experiment. A robust sampling strategy is needed to parse datasets into training, validation, testing, and prediction sets, facilitating effective prompt development, model evaluation, and most importantly, model responses that are helpful for the task. Iterative prompt development and validation involve testing and refining prompts, including requests for justification to gauge the model's reasoning, which itself helps with prompt refinement. Going Forward Experimenting with AI in evaluation practice involves thoughtful risk-taking, continuous learning, and adaptation. It is a continuous process of questioning, testing, learning, and refining, analogous to how models like GPT themselves learn during their training phase: through constant trial and error. Our guidance note focuses on defining and adapting evaluation workflows to include LLMs where they fit best and building trust through thorough performance testing. Further research, experimentation, and collaboration are needed to standardize and expand frameworks for assessing LLM performance in evaluation. Sharing experiences and findings from experiments across organizations and contexts is essential. Much has been written about the potential and perils of leveraging LLMs in research and analytical tasks. However, it is through concrete, practical, context-specific experimentation that we can discover what works, what does not, and under what circumstances. We are committed to exploring and sharing our findings as widely as possible.

Pando Funding. A princpled apprach to deploying capital to support system change - Muoio, Anna; Lehan, Brendan (2025)
Pando Funding is a practical, principle-based approach to pooling and deploying capital to support a system change network—collaborations of diverse actors working across strategies, sectors, and theories of change to shift a complex system for the better. A system change network brings together leaders, helps them develop a shared vision for transformational change, and shifts decision-making into their hands so they can adapt and innovate in response to emerging developments on the ground. Pando Funding is designed to finance the often invisible, under-funded but critical “roots” for transformative change, the interconnections and enabling conditions among people, projects, and practices that we know are key drivers of a system’s health. Pando Funding also finances the innovative “shoots,” the portfolio of projects launched from a system change network by leaders who see the power in working in infinitely more strategic and connected ways behind their shared vision. At the most basic level, Pando Funding supports ecologies of diverse but connected actors, organizations, and theories of change intervening over time. Appropriately building and resourcing a system change network is the necessary infrastructure for leaders to organize themselves within; and more importantly, to develop the root enabling conditions for system change such as: relationships, trust, collective system awareness, information and knowledge flows, and a shared vision that can cohere all their work. These conditions are not nice-to-haves. They are critical pre-conditions for a group to be able to take coordinated and coherent action together through a range of many specific but aligned projects. That said, Pando Funding is more than just a funding mechanism to distribute capital. It is a framework for rebalancing power, encouraging a collective approach, and fostering long-term commitment in the service of transformational change. Pando Funding creates the conditions for deeper, more resilient change by aligning the strategic use of capital with the collective intelligence and shared purpose of leaders working within a system change network.

Pando Funding: Financing System Change Networks - Ricigliano, Rob; Muoio, Anna (Medium, 2025)
By Robert Ricigliano and Anna Muoio

Fast Isn’t the Same as Forward - Brink (Brink's Cognitive Download, 2025)
What it takes to create smart velocity

Emergent Models of Place-Based Philanthropy: A Review of Six Foundations’ Initiatives - Easterling, Douglas (The Foundation Review, 2025)

Community Weaving: a framework for weaving healthy communities - Dixon, Erin; Pfortmüller, Fabian; Bachmann, Michel; Magnuson, Sita (Community Weaving, 2025)

Questions you can answer with causal mapping: Our new presentation - Powell, Steve (2025)
Hey there 👋We've created a new presentation called "Questions you can answer with causal mapping." It's designed to help you get a better handle on what causal mapping is all abou

Process Tracing Method in Program Evaluation - Beach, Derek; Raimondo, Estelle (Independent Evaluation Group, World Bank Group, 2025)
Process tracing (PT) involves a detailed analysis of the processes that link interventions to outcomes. PT is particularly useful for evaluating interventions that are difficult to quantify, such as knowledge work or institution building. It involves creating a detailed causal theory, tracking the process theory of change (pToC) by examining the observable evidence, and learning general lessons from the cases studied. PT offers two main benefits to evaluators: first, it provides a clear way to present and evaluate the strength of the evidence collected. This helps in making the evaluation process more understandable and credible. Second, it helps evaluators learn practical lessons more easily by emphasizing the explicit connections between actors, their actions, and resulting behavioral changes. However, the reliability of findings from PT depends on how well the theory matches the observable evidence. This paper explains how process tracing (PT) is a valuable tool for evaluating policy changes and program effectiveness. It details the methodology, highlights the importance of tracking processes and events linking interventions to results, and discusses PT's advantages and limitations. The paper provides a step-by-step implementation guide and examines the use of PT in an anonymized IEG evaluation as an example.

A workflow for collecting and understanding stories at scale, supported by artificial intelligence - Powell, Steve; Caldas Cabral, Gabriele; Mishan, Hannah (Evaluation, 2025)
This article presents an artificial intelligence-assisted causal mapping pipeline for gathering and analysing stakeholder perspectives at scale. Evidence relevant to constructing a programme theory, as well as evidence for the causal influences flowing through it, are both collected at the same time, without the evaluator needing to possess a prior theory. The method uses an artificial intelligence interviewer to conduct interviews, automated coding to identify causal claims in the transcripts, and causal mapping to synthesise and visualise results. The authors tested this approach by interviewing participants about problems facing the United States. Results indicate that the method can efficiently collect and process qualitative data, producing useful causal maps that capture respondents’ views. The article discusses the potential of this approach for evaluation, enabling rapid, large-scale qualitative analysis. It also notes limitations and ethical concerns, emphasising the need for human oversight and verification.

Using AI responsibly for Research on Violence Agains Women - Amazon-Brown, Isabelle; Tsui, Quito; Raftree, Linda; Dartnall, Elizabeth (MERL Tech & SVRI, 2025)
Generative AI (GenAI) is evolving rapidly, and its use in violence against women research remains new and largely experimental. While researchers are beginning to explore GenAI’s potential, uncertainty around its risks, ethical challenges, and real-world application persists. Recognising the need for clear guidance, the Sexual Violence Research Initiative (SVRI) partnered with The MERL Tech Initiative (MTI) to develop this resource. This guide aims to provide background on GenAI, strengthen researchers’ AI literacy, and offer practical advice to help the violence against women research community critically and carefully engage with GenAI while minimising risks. We start off with an overview of GenAI and potential areas where researchers might use it at different stages of the research process. We highlight some low-risk, practical applications of GenAI that can help streamline tasks while ensuring that researchers remain in control. (Appendices provide clear definitions of key AI terms and a simple explanation of how generative AI works for those less familiar with the technology.) Following these examples, we discuss some of the key risks and potential harms of using GenAI for research on violence against women, including reliability, bias, dehumanisation, data privacy, power and patriarchy, relevance, transparency and replicability, inclusion and participation, worker’s rights, inequities and environmental harms. To support informed decision-making, we offer a simple decision framework with critical questions to consider before using AI for any task: • Does AI provide a clear benefit over existing methods? • Does its use comply with ethical guidelines, survivor consent, and data protection laws? • Are humans in the loop to validate results and take responsibility? • Can potential risks be effectively managed? • Is there a process in place for making and documenting decisions about the use of GenAI? If the answer to any of these is “no,” the guidance suggests reconsidering or avoiding AI in that context. This framework can help researchers to weigh AI’s advantages against its limitations, ensuring thoughtful and responsible use. Recognising AI’s fast-changing nature, we encourage a cautious approach to GenAI. We suggest starting small—testing GenAI tools on non-sensitive, publicly available data—to better understand their capabilities and limitations. If you choose to integrate GenAI into your research, we encourage you to document and share your experiences, so we can learn collectively and refine best practices together. The guide reinforces a key message: human expertise is irreplaceable. AI can improve efficiency, but final analysis and interpretation should always lie with researchers. By experimenting in a controlled way and keeping ethical principles at the forefront, researchers can build confidence in AI while avoiding unintended harm. This guidance is intended to be a living document—SVRI will maintain and update it as the AI landscape evolves and new insights emerge. SVRI and MTI are grateful for the critical feedback received from colleagues during the production of this guide, which has helped strengthen its content. (See Appendix B for details on the methodology used.)

Relational State Capacity: Conceiving of Relationships as a Core Component of Society’s Ability to Achieve Collective Ends - Honig, Dan; Krishnamurthy, Mekhala; Sharma, Rahul Karnamadakala (Johns Hopkins University, 2025)
Relational state capacity (RSC) integrates the quality of citizen-state relationships into the concept of state capacity, emphasizing mutual recognition between citizens and state agents as foundational for societal problem-solving. Traditional state capacity definitions overemphasize technical and institutional elements, overlooking relational dynamics. RSC is a latent societal resource activated through mutual recognition in citizen-state exchanges with potential to improve outcomes across diverse domains, from public health to social cohesion. We have for too long transactionalized governance; we need instead to collectively invest in intentional structures and practices that foster relationality. Drawing on insights from existing literature, the paper explores how RSC is built, intermediated, and shaped by broader societal norms and organizational dynamics. By making RSC legible as an object of inquiry, the paper lays the groundwork for empirical investigation into RSC’s causal pathways, spillover effects, and resilience to disruption. There are many pathways for theoretical and empirical development in this emergent field; focusing on the relational in understanding state performance has the potential to enhance citizens’ welfare, strengthen democratic resilience, and improve public sector performance.

VfI backstory and overview - King, Julian (Evaluation and Value for Investment, 2025)
The post outlines the background and development of a new approach to assessing value for money (VfM) in public policies and social programs. King identifies a key problem: while there's increasing pressure to demonstrate VfM in public programs, traditional assessment methods are incomplete, with economics and evaluation operating as separate rather than integrated disciplines. Key points about the research development: 1. King developed a conceptual model for VfM evaluation, published in the American Journal of Evaluation 2. He analyzed how Cost-Benefit Analysis (CBA) measured against this model, finding significant gaps 4. He created a process model and tested it through real-world consulting work 5. The research was validated through case studies of actual programs The research made seven major contributions to evaluation: 1. Established VfM as an evaluative question requiring evidence-based judgment 2. Showed VfM as bridging evaluation and economics disciplines 3. Positioned CBA as a form of evaluation rather than a rival to it 4. Demonstrated CBA's limitations when used alone 5. Developed a stronger approach using evaluative reasoning with mixed methods 6. Applied program evaluation standards to economic evaluation 7. Demonstrated use of probative inference in validating the conceptual model The post concludes by pointing readers to various resources, including King's PhD dissertation, practical guidance documents developed with various organizations, and over 100 Substack articles exploring these concepts in detail.

Adapting ripple effects mapping for evaluating public health initiatives in complex systems: Reflections and recommendations from seven case studies - Creaser, Amy; Dowling, Lisa; Helme, Zoe; Crowther, Jamie; Casana, Lily; Williams, Rachel; Young, Emma; Nobles, James; Hall, Jennifer (Evaluation, 2025)
Ripple effects mapping is a qualitative and participatory method, developed to capture the dynamic nature and interacting elements of an initiative, and its impacts. We present our experiences of using ripple effects mapping to evaluate complex public health initiatives across seven case studies in Bradford, UK. Seven researchers engaged in qualitative reflective practice to capture their individual experiences of using ripple effects mapping within their research and practice; outlining how the method has been adapted and highlighting key reflections and recommendations for implementing ripple effects mapping in the future. We developed ten recommendations, with corresponding strategies, for implementing ripple effects mapping sessions. The recommendations outline how ripple effects mapping can be implemented to improve engagement and anticipate and overcome potential barriers. In doing so, we outline how ripple effects mapping can be used, and adapted, to evaluate various public health initiatives, in research and practice.

How do we define and support quality and rigor in Causal Pathways evaluation? - Apgar, Marina; Aston, Tom (Causal Pathways, 2025)

Funders, Do Your !!@%@#! Job - Starr, Kevin (Stanford Social Innovation Review, 2025)
Be a real advocate for those we’re trying to serve. Be accountable for impact.

Adaptive Political Economy: Toward a New Paradigm - Ang, Yuen Yuen (World Politics, 2025)
The conventional paradigm in political economy routinely treats living, complex, adaptive social systems as machine-like objects. This treatment has driven political economists to oversimplify big, complex social processes using mechanical models, or to ignore them altogether. In development, this has led to theoretical dead ends, trivial agendas, or failed public policies. This article proposes an alternative paradigm: adaptive political economy. It recognizes that social systems are complex, not complicated; complexity can be ordered, not messy; and social scientists should be developing the concepts, methods, and theories to illuminate the order of complexity, rather than oversimplifying it. The author illustrates one application of adaptive political economy by mapping the coevolution of economic and institutional change. This approach yields fresh, important conclusions that mechanical, linear models of development have missed, including that market-building institutions look and function differently from market-sustaining ones.
Keywords: adaptive; coevolution; complex adaptive systems; complexity; endogeneity; inclusive; market-building; market-sustaining; moral; political economy of development

Evaluation handbook: 2025 edition - Directorate-General for International Partnerships (European Commission); Hassnain, Hur; McHugh, Karen; Lorenzoni, Marco; Alvarez, Valentin; Augustyn, Anna Maria; Davies, Rick; Rogers, Patricia; Buchanan-Smith, Margie (Publications Office of the European Union, 2025)
The Evaluation Handbook – 2025 Edition reflects the European Commission’s steadfast commitment to promoting a culture of results, accountability, and learning across the EU’s external action. As one of the world’s leading development and international cooperation actors, the European Commission bears a particular responsibility to ensure that its interventions are informed by robust evidence, deliver tangible results, and represent the highest standards of transparency and effectiveness in the use of public funds. Building on the foundations laid by the 2024 edition— which itself constituted a major revision of the 2006 Methodology Guidance—the 2025 edition reflects the continuous evolution of the European Commission’s evaluation practice over the past two decades. It draws on international best practices, incorporates state-of-the-art methodological advances, and acknowledges the increasingly multifaceted nature of development cooperation. This Handbook is attuned to the dynamic policy, institutional, and geopolitical environment in which EU external action operates, offering an adaptive and forward-looking framework to support high-quality evaluations that are both credible and consequential. This Handbook reaffirms evaluation as an indispensable pillar of the EU’s commitment to better regulation, performance-based management, and evidence-informed decision-making. It provides practical and strategic guidance to support evaluation managers, partners, and stakeholders in navigating the full evaluation cycle—from design and implementation to dissemination and uptake of findings.
Keywords: EU action; International policy; aid evaluation; cooperation policy; coordination of aid; development aid; food security; report; the EU's international role; user guide

Theory of Change, Monitoring, and Evaluation A toolkit for developing a culture of evaluation and learning - Ecorys (West Yorkshire Combined Authority, 2025)

Education Systems Changemakers Challenge Workbook - Gluckman, Maxie; Rubio Rodríguez, Raquel (IREX, 2025)
Introduction Phase 1 - Power Up Phase 2 - Level Up

Learning about learning health systems: Strengthening theory through continuous evaluation and improvement - Gremyr, Andreas; Seid, Michael; Margolis, Peter A.; Ramaswamy, Rohit (Learning Health Systems, 2025)
Introduction Learning Health Systems (LHSs) offer potential for transforming healthcare through continuous learning and improvement. However, the current literature lacks a robust connection between theory and practice, limiting knowledge transferability across diverse healthcare environments. This article proposes a novel framework for integrating theorizing and continuous evaluation into LHSs, illustrated through the case of the Bipolar Action Network (The Network). Methods We use a process of theorizing to structure how mid-range theory, program theory and LHS design, development and evaluation can support improved practices and theories. We use the Conceptual Framework for Value-Creating Learning Health Systems as our initial mid-range theory, and the Bipolar Action Network serves as an illustrative case. Results The framework emphasizes continuous multi-level LHS theory refinement based on real-world data, ensuring that both the system and its theoretical underpinnings evolve in response to new insights and challenges by connecting four steps: (1) Selecting an initial mid-range theory, (2) Creating a program theory for a specific LHS, (3) Evaluating LHS performance using operational data, and (4) Using evaluation findings to refine both the LHS program and mid-range theory. Conclusions This article contributes to the field by offering a practical methodology for bridging the gap between LHS theory and practice. By promoting ongoing theorizing and evaluation, our framework aims to both enhance the effectiveness and adaptability of LHSs, as well as inform theory development. Challenges remain, including resource intensity for data infrastructure and potential limitations in data quality or accessibility, which must be addressed to realize the full potential of LHSs as adaptive, theory-driven systems.
Keywords: health services research; implementation science; learning health system; quality improvement

Ecosystem Cards: Deepen your understanding of the status quo. Envision desirable futures. Design strategic experiments to learn more - Lukyanova, Zoya; Polok, Darius; Shah, Indira; Martin, Naomi; Kildeev, Ruslan (resense360 & iac Berlin, 2025)
A tool to deepen your understanding of the status quo, envision desirable futures, and engage with ecosystems more intentionally.

Adaptive Management in International Development - Knowledge Base - Prieto Martin, Pedro (2025)
A curated collection of resources on Adaptive Management and MEL in International Development

Digital.gov Resources - U.Ss GSA (Digital.gov, 2025)
Essential how-to guidance for government product managers and digital teams.

Many-to-Many - Field Guide with learnings, models, insights and tools - Zucker, Michelle; Dhami., Annette (Dark Matter Lab, 2025)
A living library of practical tools, frameworks, and case studies designed to support practitioners in complex collaborations. Many-to-Many System -aims to support collaborations tackling complex, entangled challenges where disrupting norms and values, ownership, and power is essential. Solving today’s complex, interconnected problems requires what we term “complex collaborations” - bringing together many diverse groups (public, private, civic) with many new perspectives, including future generations and the natural world. While many collaborations are already doing great work, we believe that finding better ways to support how they are structured and organised them could unlock more effective, system-level change. The Many-to-Many System is focussed on unlocking the governance, organising, legal, and learning structures of complex collaborations to enable many resources, not just money, but also knowledge and relationships to flow more freely, and to foster many ways of working that embrace diverse value exchange. For each and everyone of us, our fundamental understandings of the world are influenced by ‘deep codes,’ often invisibly embedded within our creations, frameworks, and rules. The Many-to-Many System explored how these codes shape collaboration and governance, aiming to understand if they could be reimagined and how those within complex collaborators themselves can embed them into their collaboration’s infrastructures. More intentional and visible shifting of deep codes for governance and organising could help collaborations to better align with their systemic missions and offer approaches for rethinking core concepts like value, power, risk, and ownership. The Many-to-Many System distills two years of learning, prototyping, research, and practice. Here we offer a blend of elements: a core framework, practical insights and learnings, illustrative models for complex topics, and tools developed along the way. Our aspiration is to provide a range of resources that can help others embed the Many-to-Many deep code shifts into their own complex collaboration work.

The How we need now: a capacity agenda for 2025 and beyond - Pahlka, Jennifer; Greenway, Andrew (Niskanen Center, 2024)

A Synthesis of Systemic Action Research Processes in the CLARISSA Programme on Worst Forms of Child Labour - Burns, Danny; Apgar, Marina (CLARISSA Research and Evidence Paper, 2024)
This paper describes, synthesises and analyses the work of 25 Action Research Groups that were the heartbeat of the CLARISSA Systemic Action Research programme. The CLARISSA programme worked to create participatory and systemic solutions to the worst forms of child labour in the adult entertainment sector in Nepal, and the leather sector in Bangladesh. The paper explores how the action research groups were set up, and how they worked through key stages of trust and relationship building, generating evidence, building theories of change, planning and taking action, and assessing the effects of that action. The paper, which is a synthesis of the evidence and learning produced in the programme, draws together learning about the process for both ground-level action researchers and those who might fund, design or implement future programmes.

Surfacing values through rubric development. Why and how. - King, Julian (Evaluation and Value for Investment, 2024)
This article provides a comprehensive overview of rubric development in evaluation practice, focusing on stakeholder engagement and practical implementation. King explains that rubrics are essential tools for implementing evaluative reasoning, serving as frameworks that combine evidence, criteria, and standards to make transparent value judgments. The piece emphasizes that effective rubric development is inherently participatory, requiring meaningful stakeholder engagement for both ethical and practical reasons. The author presents detailed guidance for practitioners, including tips for facilitating stakeholder involvement, creating safe collaborative spaces, and maintaining simplicity while ensuring context-specificity. Through various real-world examples, King illustrates the flexible nature of rubric development processes, from two-hour workshops to multi-year evaluations. The article argues that while rubric development requires significant upfront investment, it pays dividends throughout the evaluation process by enhancing clarity, stakeholder buy-in, and ultimate utility of evaluation findings. Written as pre-workshop reading for training sessions in London, the piece serves both as an introduction for newcomers and a detailed reference for experienced evaluators.

From Local Roots to Global Reach: Global South Intermediaries Driving Just and Equitable Governance - Castells Carrión, Paula (Trust, Accountability and Inclusion Collaborative (TAI), 2024)
In an evolving international development landscape, the push for localization has gained urgency. Global South Intermediary Organizations (GSIs) are emerging as key players, offering a promising pathway to achieving localization programming. To harness their potential, it is crucial to develop a deeper understanding of their functions, structures, and strategies. However, they should complement—not replace—direct funding relationships with grassroots organizations. The Trust, Accountability, and Inclusion Collaborative (TAI) is excited to present this comprehensive analysis by consultant Paula Castell Carrión, highlighting the significant potential of GSIs to drive locally led solutions. As Global North donors seek to increase funding to frontline groups, movements, and activists in Global South contexts, GSIs emerge as strategic partners in localization programming. These organizations possess deep contextual knowledge and established networks that can help bridge the gap between international funders and local implementers. Their potential extends beyond simple fund management to include capacity building, strategic guidance, and advocacy for local priorities in grantmaking decisions. There is a growing number of GSIs fulfilling these roles, who have a track record to deliver governance-related funding to the grassroots. In this new study, TAI´s Paula Castell Carrión examines the evolving role of Global South Intermediaries in advancing localization efforts, with a particular emphasis on achieving equitable governance outcomes. Through a detailed investigation of 4 operational models and practical experiences, this paper aims to provide donors with actionable insights for a more effective engagement with GSIs. Paula's investigation centers on six carefully selected case studies and reflects on broader practices and trends within the funding ecosystem. These cases shed light on the diverse approaches, organizational structures, and strategic interventions employed by GSIs across different contexts. We hope that this research contributes to broader strategies on international development funding by: 1. Providing evidence-based insights into the effectiveness of different intermediary models. 2. Identifying best practices for donor-GSI partnerships. 3. Supporting the development of more equitable and effective governance programming

Demystifying Interoperability. Insights from our work in Ethiopia's agriculture sector - Hatcher-Mbu, Beverley; Ferreyra, Fernando; Postelnicu, Mihai; Cihai, Viorel; Getachew, Yosef; FIkre, Yohannes; Kassa, Selam; Orrell, Tom; Tewari, Khyatee (Development Gateway, 2024)
Government offices are filled with siloed, sector-specific digital systems that strain their capacity to make decisions, use data effectively, and achieve ambitious sustainable development goals. Public investments in digital development, transformation, and infrastructure can only meet citizen needs if data and systems are consolidated and interoperable. While interoperability is a sensible approach to building digital public infrastructure, transforming existing systems is easier said than done. Limited resources and expertise, lack of training, inadequate hardware and software infrastructure, and concerns over data privacy are some of the key factors that limit the impact of systems or lead to their discontinuation. The data that feeds these systems also needs to be standardized using global best practices and common languages and frameworks that can be understood across multiple platforms and adapted to the local context. In increasingly digital economies, systems and their data need to be relevant and responsive to citizens’ specific conditions and needs With emerging challenges like climate change that require more complex data, and new technologies like artificial intelligence (AI) that require large volumes of data, systems need to be agile and scalable to advance solutions and fully harness innovation. This paper discusses, in practical terms, what goes into implementing interoperable solutions in partnership with public administrations. Based on 20+ years of DG’s experience, the paper demystifies key components needed to build robust, resilient, and interoperable data systems, focusing on the “how” of data standardization, data governance, and implementing technical infrastructure.

The end of reporting - Sutton, Amy (2024)
After eight years, I am leaving Porticus. From January 2025, I'll be starting a new chapter - working in a freelance capacity to support systems change efforts through strong learning, evaluation and strategy. Having worked with the amazing Porticus UK team for the first four years, I moved to Amsterdam in 2020 to work in the central strategy and learning team, supporting colleagues around the globe to embed strategy, evaluation and learning in their work. I loved this work. I learned a huge amount about social justice, and about how philanthropy can most effectively support the many fights for change needed today. I will miss the passion and commitment of so many people I feel privileged to work with - too many to name here. One of the pieces of work I am most proud of is the design and introduction of a new practice of learning between Porticus and its partner grantees, ending the burden of written reporting. I will always appreciate my colleagues’ appetite for innovation and to living out their values with this work. Getting rid of reporting may sound radical, but static reporting about what happened and what was done doesn’t fit with a systems change approach. If we trust our grantee partners, let’s focus on working and learning with them, about what matters. Reporting about outputs, activities, actual spend vs anticipated spend, will not help us shift systems. Learning together about what’s changing, why, how, and how can we adjust or adapt as a result of this insight and learning, in real time, will. As I leave Porticus, I will be taking this approach to learning with me. And as I do so, I would like to urge the philanthropy sector to challenge itself and explore how the systems we use for evaluation, learning and reporting can be made more fit for purpose for system change, less burdensome and more equitable. Here is a reflection piece I’ve written about why this learning practice. I offer this as a personal reflection and share it in the hope that more foundations and funders will think about doing their reporting differently - and ideally, not doing reporting at all…

Working with Systems-Informed Theory of Change - A practical introduction - Vogel, Isabel; Abosenna, Karim R (Climate-KIC, 2024)
Explore Climate KIC’s S-ToC guide A practical and flexible resource designed to help teams and partners navigate the complexity of systems change in the face of the climate crisis. Our S-ToC approach integrates systems thinking to support adaptive monitoring, evaluation, and ongoing learning. This guide empowers users to analyse systems, identify leverage points, and co-create change-oriented strategies that evolve with context. Whether you’re working on a small-scale pilot or a multi-stakeholder portfolio, S-ToC can help you articulate dynamic change trajectories, catalyse impact, and document learning across interconnected efforts. Designed as a step-by-step Miro board template, the guide supports collaborative sense-making and can be used to facilitate stakeholder workshops or train others in systems-based ToC thinking.

Becoming Futures Ready: How Philanthropy Can Leverage Strategic Foresight For Democracy - Masters, Suzette Brooks; Ingram, Maya; Hooke, Hillary (2024)
In 2022, the Democracy Funders Network launched the Better Futures Project and published Imagining Better Futures for American Democracy to urge democracy donors and practitioners to move beyond reaction and critique towards creation and change. The project seeks to provide succor and inspiration at a time marked by anxiety and fear, but also great possibility. The project challenges us to consider the future of our democracy as a dynamic system — not one that revolves solely around the next election — and to activate our collective agency and imagination. With consequential elections on the horizon, it is particularly important for pro-democracy actors to cultivate and hold a longer term, affirmative orientation even as, in the near term, attention and resources continue to focus primarily on electoral issues and democracy defense. The Democracy Funders Network, through its Better Futures Project, is publishing this toolkit to build the strategic foresight skills of individual and organizational democracy donors, and to expand this field’s capacity to think longer term and more boldly about the future of American liberal democracy. The toolkit provides practical applications of strategic foresight that can be incorporated into democracy donors’ philanthropic strategies and grantmaking practices. It also showcases recent examples of how donors have integrated strategic foresight into their operations. This toolkit benefited from conversations and interviews with strategic foresight practitioners and philanthropists, as well as a review of leading publications in the field of strategic foresight and futures thinking. Contents Navigating Rapid Societal Change Why Use Strategic Foresight in Philanthropy How to Deploy Strategic Foresight and Futures Thinking in Philanthropy. An Overview of Techniques Leveraging Foresight and Futures Thinking Internally. Organizational Structure and Culture Leveraging Foresight and Futures Thinking Externally. Communities of Practice and Thought Leadership Reaching The Next Frontier. Futures-Powered Grantmaking How to Successfully Harness Foresight and Futures Thinking. Enablers and Blockers

Raising the Bar: Improving How to Assess Evidence Quality in Evaluating Systems-Change Efforts - Apgar, Marina; Aston, Thomas; Snijder, Mieke; Zwollo, Tom (The Foundation Review, 2024)

A Framework for Embedding Learning in Systems Change - Bertermann, Kecia; Coffman, Julia (Center for Evaluation Innovation, 2024)
This five-step framework, developed and tested by a foundation, embeds learning in emergent systems change strategies. It prioritizes the testing of hypotheses and assumptions, uses learning questions, and calls for examining both confirming and disconfirming evidence. --- A framework for embedding learning in systems change strategies and for testing strategic uncertainties. Learning and evaluation approaches that accompany systems change efforts need to fit with and support the emergent nature of this work. Systems change efforts call for examining, with evidence, the hypotheses and assumptions that undergird the work and regularly adapting in response to what is learned. They require staff to be highly engaged in testing their ideas by gathering evidence, interpreting it, and then acting on its implications. This brief presents a learning approach—referred to here as the Learning Framework—developed and tested by a foundation to support its systems change strategies. It is intended for other practitioners working on approaches for integrating learning with systems change efforts. Because a wealth of conceptual work exists about why learning is critical to systems change but practical advice about how to do it is much less available, we describe the approach we took and how the Framework’s core elements fit together in practice. The primary purpose of this approach is to interrogate our strategic uncertainties. This approach to learning is different from the dominant approach to learning and evaluation in philanthropy that primarily looks for evidence (often static metrics) that expected outcomes have been achieved (an approach that is more suitable for linear and predictable change strategies). This Learning Framework is driven by, and organized around, hypotheses and assumptions. This is especially important for systems change efforts where the path forward cannot be determined with complete certainty, but where robust evidence collection and regular sensemaking can increase our confidence in chosen directions and ultimately increase our chances of success.

Political Economy Analysis in Sudan: Handy Tools for Everyone? - Jones, Beverley; Oosthuizen, Daniel (TWP CoP, 2024)
Based on a case study of the Kullana Liltanmia (All of Us for Development) (KLP) Contextual Analysis (CA) course in Sudan, this paper tells the story of how training and capacity building in Political Economy Analysis (PEA) made a difference in the lives and experiences of those involved, including in particular young people. The paper highlights three critical lessons: • At the country level, the twists and turns of Sudan’s recent history reinforce the importance of people deepening their understanding of what is really going on, in order to make better informed (if not the best or least-worst) choices. The skills and mindset of both broad and localised PEA capacity can make a difference to people living in fragile and conflict-affected settings, where navigating complexity and uncertainty is a daily reality. • At the level of strategic programming, the potential of PEA remains under-utilised. There is considerable scope for capacity in PEA to support not only ordinary people across a range of different occupations and educational levels, but also prominent actors who may eventually play key roles in political processes, and whose choices can affect the longer-term prospects for stability and peace. This becomes even more imperative in the context of the war in Sudan, which now involves a plethora of external actors. • At the level of the experiment in the KLP CA course, this story reinforces the paramount importance of adapting and grounding PEA training in a common understanding among its intended participants, shaping it through their insights and experience. It also highlights how essential it is to nurture PEA skills and mindsets not only in stable contexts but also in challenging, more dangerous and uncertain ones. Supporting this vital capability during periods of hiatus, fast-moving transitions, and even active conflict is a public good; and it requires courage and creativity on the part of the donors, course designers, implementers and participants. The paper is structured as follows. Section 1 provides an introduction setting out the broad context of Sudan within which the KLP CA training took place. Section 2 describes the 30-year period of al-Bashir’s regime (1989-2019). Section 3 focuses on successive UK governance programmes in Sudan between 2015 and 2019. Section 4 describes the KLP CA course (2017-2018) and its development. Section 5 looks at the revolutionary hiatus and brief transition to civilian rule in Sudan (2017-2018). This is followed by an account in Section 6 of the 2019-2023 transitional phase and dashed hopes for a move to civilian rule with the outbreak of a civil war that has led to exceptional violence (including weaponised sexual violence), displacement and hunger. Section 7 concludes with reflections on missed opportunities to capitalise on the KLP CA experiment, and with recommendations on the need for continued support on PEA for Sudanese people and organisations.

RAAM Toolkit: A Toolkit for Reduced Access Analytical Methods - RAAM Project (USAID & Mercycorps, 2024)
The Reduced Access Analytical Methods (RAAM) toolkit is a practical resource designed to help humanitarian practitioners overcome monitoring challenges in reduced access environments. Reduced access can be caused by natural disaster, conflict, political instability, or other factors, and typically makes it difficult to conduct normal monitoring of program implementation. The RAAM toolkit offers technical and managerial tools for a menu of analytical methods that can fill information gaps and/or improve data quality during reduced access program monitoring. RAAM supplements (but does not replace) traditional Monitoring, Evaluation and Learning (MEL). The analytical methods covered by the RAAM toolkit were designed to address commonly-encountered information gaps and data quality issues in reduced access environments. The toolkit covers the following methods: Data Triangulation: Systematic comparison of multiple data sources to improve the reliability and accuracy of program data, helping to identify strengths and weaknesses in information. Context Mapping: Layering various data sources on a map to understand factors that may impact program activities, such as access changes or conflict trends. Rumor Tracking: Monitoring unverified information in communities to identify potential gaps, implementation issues, or early signs of violence, enabling timely interventions. Remote Sensing: Using satellite or aerial data to monitor physical conditions and changes in implementation areas from a distance. Transaction Analysis: Analysing digital transaction data from cash or voucher assistance programs to understand usage patterns, vendor capacity, and more. Available in French, Spanish, and Arabic.

Local Systems Position Paper - USAID (USAID, 2024)
The Local Systems Position Paper describes the key approaches USAID will use to translate systems thinking into systems practice. It focuses on ways USAID can better understand and engage local systems to support them in producing more sustainable results. Systems thinking is a mindset and set of tools that we use to understand how systems behave and produce certain results or outcomes. Systems practice is the application of systems thinking to better understand challenges and strengthen the capacity of local systems to unlock locally led, sustained progress. The shift from systems thinking to systems practice is driven by a desire to integrate systems practice throughout the Program Cycle and increase our capacity to actively and adaptively manage programming in ways that recognize complexity and help make our programs more effective and sustainable. These approaches will be utilized alongside and within the context of USAID’s policies and guidance, including technical guidance for specific sectors, as well as evidence and lessons learned from partners around the world. Systems thinking is a long-standing discipline that can serve as a powerful tool for understanding and working with local systems. It has been a consistent component of USAID’s decades-long commitment to locally led development and humanitarian assistance. USAID uses systems thinking to better understand the complex and interrelated challenges we confront – from climate change to migration to governance – and the perspectives of diverse stakeholders on these issues. When we understand challenges as complex systems – where outcomes emerge from the interactions and relationships between actors and elements in that system – we can leverage and help strengthen the local capacities and relationships that will ultimately drive sustainable progress. Recognizing this, in 2014, USAID released the Local Systems Framework, outlining ten principles to help USAID staff, partners, and local stakeholders place local systems at the center of all of our efforts to promote sustainability. Since then, development and humanitarian actors have embraced systems thinking as a concept and reiterated its importance in other policies, strategies, and tools, across sectors and regions. When USAID released its Localization Vision and Approach in 2022, it included local systems practice as a core component of adapting policies and program practices. Building on the success of the Local Systems Framework, this position paper reiterates USAID’s commitment to systems practice and presents six approaches that will help USAID strengthen and further embed systems practice in all of our work

Innovation Carve-Outs: Curating the space for innovation - Rahman, Asad; Brink (Brink, 2024)
Over the course of this paper we lay out the basis of the Carve-Out method, an approach that leverages a behavioural framework to allow organisations (even those burdened with layers of bureaucracy and entrenched ideas) to create intentionally designed environments that give people the space, resources and support they need to explore and test new ideas. Ideas that may go on to transform the core of their organisation. The Carve-Out method is based on a simple insight: That if we want to deliver real world, enduring impact then we must recognise that it’s real (often irrational, always human) people that are doing the work of innovating. So to deliver the outcomes we seek for people and our planet then we must explore and unlock the human behaviours that we need to get us there. The Carve-Out method makes this possible by creating the conditions for continuous innovation so that existing teams are empowered to explore solutions and bring those transformational ideas to fruition. To get there, first we look at the idea of the innovation lab or ‘skunkworks’: typically physical spaces, built away from business-as-usual constraints and processes where a small team can move quickly and with relative autonomy. We explore the legacy skunkworks principles have left on our contemporary innovation landscape, for example, how the iPhone was developed by a skunkworks team at Apple, as well as the proliferation of government innovation labs in the public sector. We then go on to examine some of the downsides involved in creating isolated innovation units, particularly the potential disconnect between the innovation process and core business operations, and the result this can have on products that might lack relevance to the market or practical applicability to the company’s mainline business. The second section of the paper looks at another model of organisational innovation, which has been adopted by tech companies such as Google and Netflix. This bottom-up model, exemplified by Google’s ‘20% time’ policy, gave employees the freedom and dedicated resources to explore transformative innovations. The paper looks at how these companies leveraged their employees to fuel ‘strategic risk-taking’, but also how this introduced significant pressure on individuals, and the difficulties in sustaining this level of innovation while scaling over the long term. In the third section we introduce the CarveOut method for organisational innovation: a powerful solution for organisations seeking to foster a culture of ideation experimentation, empower teams, and accelerate the development of impactful innovations and drive sustainable growth. Here we show how systematic, continuous, scalable innovation requires genuine integration and the application of behavioural science practices. This combination allows both for the creation of the space, time, and resources for innovation, as well as carving out the psychological and emotional space necessary for changemakers to thrive and grow within an organisation. Through multiple examples we show how designing and shaping the ‘innovation environment’, it is possible to create an integrated and agile system of organisational innovation that allows for the ‘carving out’ of time, resources and space, but also of cognitive bandwidth and the environmental conditions that enable individuals and teams to take creative risks, embrace new perspectives, and engage fully in the innovation process. We lay out five key principles that underpin the Carve-Out model: 1. Commit to evolving and create structures to enable it 2. Keep the people, change the mindsets 3. Learn fast, fail small 4. Invest and scale in line with confidence 5. Do in order to learn The final section of the paper looks at how the Carve-Out method has been successfully applied in real-world scenarios across a number of sectors. The first story looks at how behaviourally informed features were brought to bear on the Frontier Tech Hub, a programme that supports government officials in the UK Foreign, Commonwealth and Development Office (FCDO) to explore radical, early-stage tech ideas that might solve big development challenges and in doing so, evolve the innovation culture of a government department. In the second story we look at how, after the COVID-19 pandemic led to school closures, Lebanon-based NGO Jusoor worked alongside Brink to explore and test a new way of ensuring access to education for refugee children that would eventually shift the whole education system. And in the third story we explore how the uBoraBora fund for education implementers in sub-Saharan Africa, supported teams to take an experimental approach to exploring and testing key questions to enhance their product or service alongside their businessas-usual, and integrate what they learn with governments’ education systems. We end the paper by asking our readers to imagine what could be achieved if they could dedicate a small portion of their organisation’s time and resources to create the conditions for innovation, so their teams could operate with a different set of rules, and be empowered to explore solutions in a safe and supportive environment. We also provide links to a number of tools and resources created by Brink so others can begin to put the ideas found in this paper into action in their work.

Relationality: How Moving from Transactional to Transformational Relationships Can Reshape Our Lonely World - Jay, David (North Atlantic Books, 2024)
How moving from transactional to transformational relationships and organizations can save our democracy, nurture our connections, and make us happier and healthier. Powerful institutions, from schools to tech and social media companies, create breeding grounds for isolation by failing to invest in relational work. This obstacle stands in the way of our fight for racial equity, economic justice, and climate resilience. In Relationality, leading asexuality and relationship activist David Jay brings clarity to the crisis with a fresh perspective that expands upon the fundamental idea that all entities in the universe are connected. Jay draws from a range of vivid personal experiences, including his time spent helping tech workers and policymakers reform social media. This book is for people who believe in the power of relationships and want to see increased investment in relational work. Its scientifically grounded framework will help readers foster conversations about relational work, establish conditions for relationships to thrive, and quantify the impact of them. Equipping professionals and activists involved in nonprofit, political, and other types of relational work with the knowledge they need to fight for and utilize resources, Relationality shares valuable insight on: - The history of why institutions fail to invest in relationships - Reimagining ROI calculations to account for relational work - Using tools of prediction and emergence theory to build communities - How stories and data about relationships can help us direct resources toward relational work - Relational economics and the redistribution of wealth With isolation and loneliness on the rise in a post-lockdown world, Relationality offers a roadmap to nourish our connections toward a better, more liberated world—personally, organizationally, and in community.

Financing System Health: The Pando Fund - Ricigliano, Rob (Medium, 2024)
By Robert Ricigliano and Anna Muoio

The Evaluation Metro Map - Lemire, Sebastian (Journal of MultiDisciplinary Evaluation, 2024)
Background: The practice and profession of evaluation continue to evolve. Over the years, evaluation scholars and practitioners have developed a broad—and still broadening—range of evaluation approaches. These evaluation approaches are often combined with various study designs in real-world evaluations. To provide an overview of these approaches and designs, and inspired by my son’s love of metro trains, I created an evaluation metro map. Purpose: The purpose of this article is to describe the background, purpose, and benefits and limitations of the evaluation metro map. Setting: Not applicable. Intervention: Not applicable. Research Design: Not applicable. Data Collection and Analysis: Not applicable. Findings: Not applicable.
Keywords: Evaluation theory; evaluation metro map; evaluation theory tree; special issue; visualization; visualizing evaluation theory

The Garden of Evaluation Approaches Visualization - Montrosse-Moorhead, Bianca; Schröter, Daniela; Becho, Lyssa Wilson (Journal of MultiDisciplinary Evaluation, 2024)
Background: The Garden of Evaluation Approaches (Montrosse-Moorhead, et al., 2024a) maps evaluation approaches against eight dimensions of practice and situates them in their philosophical orientations and methodological dispositions. Purpose: The Garden’s guiding question is: How do evaluation approaches compare in terms of dimensions that facilitate use and application? Primary intended users are evaluation practitioners, and secondary intended users include ecosystem actors (e.g., clients, program participants, funders), educators, researchers, and theoreticians. Setting: Not applicable. Intervention: Not applicable. Research Design: Exploratory sequential mixed methods. Data Collection and Analysis: Data collection and analysis included locating and reading primary sources for each evaluation approach. One team member thematically coded each source for evidence of an approach’s stance on the eight dimensions of practice, its’ philosophical orientation and methodological disposition, and steps for implementing the approach in practice. This qualitative evidence was used to generate ordinal ratings for the eight dimensions using a rating scale developed by the team, and to classify the approaches' philosophical orientation and methodological disposition. Drafts of handouts summarizing qualitative and quantitative data were reviewed by the two other team members, with dialogue and deliberation among all members used to come to consensus. Findings: The first iteration of the Garden includes seven approaches: fourth generation evaluation, made in Africa, nation-to-nation evaluation, practical participatory evaluation, sistematización de experiencias, theory-driven evaluation, and transformative participatory evaluation. Each approach is represented as a flower within the Garden, with each petal corresponding to the approaches’ stance on one of the eight dimensions of practice. The colors of the flowers represent the underlying philosophical foundation of the approach. The pattern within the central disc of each flower represents the methodological disposition encouraged by the approach. Summaries of the empirical mixed-method analysis are made transparent in supporting handouts. Future work will grow flowers for additional approaches and continue to map out the Garden.
Keywords: approach; evaluation practice; model; special issue; theory; visualizing evaluation theory

The Periodic Table of Evaluation - Vaca, Sara (Journal of MultiDisciplinary Evaluation, 2024)
Evaluation is a vast field of designs and methods in continuous development. The numerous parameters and choices associated with different methodological options can be difficult to fully understand without deep study and desk review. The motivation for the Periodic Table of Evaluation was to create a one-pager visual that catalogues methodological options and parameters involved in an evaluation design.
Keywords: catalogue; criteria; data visualization; designs; evaluation; evaluation methodology; methods; objectives; paradigms; purposes; special issue; visualizing evaluation theory

A Guide to Evaluation of Value for Money in UK Public Services - King, Julian; Hurrell, Alex (Verian, 2024)
When assessing value for money (VfM), there are sound arguments for combining insights from economics and evaluation. Drawing on the complementary strengths of both disciplines supports comprehensive assessment of policies and programmes’ VfM. In this Guide we introduce an interdisciplinary approach, Value for Investment, that integrates methods and tools from both disciplines, and which has gained traction worldwide. Among the concepts and methods of evaluation that economics brings to this interdisciplinary mix is cost-benefit analysis (CBA), which evaluates whether an intervention creates more value for society than it consumes. CBA is vital in the UK policy context, acting as a key tool for evaluating public policies, programmes, and projects as set out in the Green Book, HM Government’s central guide for appraisal and evaluation, and the Magenta Book which focuses on evaluation methodologies and practices across the whole policy cycle. Together, the Green Book and Magenta Book form a robust framework for evidence-based policymaking, ensuring CBA informs sound decision-making and evaluates implemented policies’ impacts. Within this policy context, value for money (VfM) is often framed as an objective or question within an evaluation and is commonly equated with CBA. However, we argue that while CBA contributes important evidence, it may be insufficient. The approach set out in this Guide includes economic analysis while integrating multiple values (social, economic, environmental, and cultural) and diverse evidence sources (quantitative and qualitative) for a more comprehensive VfM assessment, drawing on the strengths of CBA without limiting the evaluation to economic methods and metrics alone. The Value for Investment approach uses mixed methods (integrating quantitative and qualitative evidence), employs evaluative reasoning (interpreting evidence through the lens of explicit criteria and standards – improving transparency in the rationale for evaluative judgements), and is participatory (involving stakeholders in co-design and analysis). It is designed to be intuitive and practical to use by following a logical sequence of steps. The Value for Investment approach was developed through doctoral research (King, 2017; 2019) and refined through real-world VfM evaluations in diverse contexts. Leading examples of its application are published in Assessing Value for Money – the Oxford Policy Management Approach (King, Wate, Namukasa, Hurrell, Hansford, Ward, & Faramarzifar, 2023), and Value for Investment – Application and Insights (King, Crocket, & Field, 2023). This Guide is designed to complement existing VfI guidance by focusing on the interface between CBA and other evaluation methods to provide sound answers to VfM questions in the context of UK public policy.

The Strategy Testing Workbook: Politically-Smart Adaptive Programming in Complex Environments - Nixon, Nicola; O'Donnell, Carolyn; Rajalingam, Gobie; Tweedie, Pauline; Sullivan, Elizabeth (The Asia Foundation, 2024)
The guide begins with a section on what we at TAF have learned about practicing Strategy Testing, highlighting things that we didn’t know when we started out. Some of these lessons may seem obvious, but others may strike you as less so. In any case, we believe that, when taken together, they provide a useful reminder of some of the pitfalls that arise in efforts to practice Adaptive Management. Moreover, they help to remind us that there are multiple layers to doing Strategy Testing effectively as a system instead of reducing it to its component parts and applying them mechanistically. Thus these lessons are useful to read now and to revisit later. The guide then comprises three more sections: • Part 1, the Design Stage, involves constructing the system and making sure everyone is on board; • Part 2, the Inception Phase (start-up phase), helps the implementing team to make the leap from the ideal initially set out in the design stage to the grounding, support, and frameworks that the team will need to implement Strategy Testing in a meaningful and impactful way. We’ve found that the gap between design and implementation is very often the cause of an ineffective ‘all talk and no walk’ approach to Adaptive Management. And finally; • Part 3 covers Program Implementation – the practices, knowledge, and culture needed for the feedback loop that underpins Strategy Testing. The guide can be used for individual or group learning, or it can be used directly to inform Reflection Sessions (see Section 3) through the questions we raise or the activities herein.

“Everyone is talking about it, but no one is doing it”: How implementation research is – or isn’t – informing adaptation towards scale in international education - Dowd, Amy Jo (International Journal of Educational Development, 2024)
With growing evidence of what works in education, governments and their partners strive towards scale. While implementation research to adapt solutions holds promise for testing what works, where and for whom, its use in education is limited. Sixty-three international education stakeholders from government, academia, think tanks, local and international non-governmental organizations, multi- and bi-laterals and foundations offer insight into three modes of implementation research that differ on responsiveness, timeliness and stakeholder collaboration. They speak to the primacy of government involvement, trust and curiosity in collaborative efforts. This study explores enabling factors for evidence-based implementation improvement on the pathway to scale.
Keywords: Evidence-based practice; Implementation research; International education

Exploring causal pathways amid complexity - Lynn, Jewlya; Apgar, Marina (Research Handbook on Program Evaluation, 2024)
Evaluation has a long history of using experimental and quasi-experimental designs to measure the effects of programs and strategies, and through this, to infer causality. Yet, these approaches are often not appropriate when evaluating change in complex, dynamic systems. Further, programs and strategies that seek to produce change in complex settings are increasingly common. This leads to an evaluation dilemma: if and how do evaluators attend to causality amid complexity? Too often, the answer has been to use causal thinking in the design of the evaluation and interpretation of findings, but not incorporate causal analysis and inference. In practice, this allows for assumptions about how change happens to go unchallenged. In this chapter, we explore the use of causal analysis through methods designed to attend to complexity and context, strengthened by participatory implementation, that can be implemented with rigor.

Theories of Change in Reality: Strengths, Limitations and Future Directions - Koleros, Andrew; Adrien, Marie-Hélène; Tyrrell, Tony (Routledge, 2024)
For over 50 years, evaluators have used theories of change to articulate the causal logic underpinning how an intervention is intended to bring about a desired change. From its origins in programme evaluation, the approach has been adopted more widely for purposes from program design to program management. As theories of change continue to be used for multiple purposes, it is an opportune moment for the evaluation community—where the approach originated—to provide their perspective on the strengths and limitations of the approach and its future directions. To provide these perspectives, we asked nearly 30 of the world’s leading evaluators and programme theorists to provide a short essay on the past, present, and future of theories of change. This book presents their insights organized into five main themes: the use of theories of change in broader public policy contexts; using theories of change to establish causality; developing theories of change reflective of multiple stakeholder perspectives; using theories of change to understand wider societal change processes; and applying theories of change approaches for multiple purposes. By sharing these diverse perspectives, the book aims to both provide evaluators and emerging programme theorists with critical perspectives to inform future practice. The Open Access version of this book, available at www.taylorfrancis.com, has been made available under a Creative Commons Attribution-Non Commercial-No Derivatives (CC-BY-NC-ND) 4.0 license.

Raising the Bar — Integrating Cultural Competence and Equity: Equitable Evaluation – 2014, With 2024 Prologue - Dean-Coffey, Jara; Casey, Jill; Caldwell, Leon (The Foundation Review, 2024)
Editor’s Note: This article, first published in print and online in 2014, has been republished by The Foundation Review with minor updates. Whether implicit or explicit, social justice and human rights are part of the mission of many philanthropies. Evaluation produced, sponsored, or consumed by these philanthropies that doesn’t pay attention to the imperatives of cultural competency may be inconsistent with their missions. The American Evaluation Association’s Statement on Cultural Competence provides those who produce, sponsor, and use evaluation an opportunity to examine and align their practices and policies within a context of racial and cultural equity and inclusion. The use of such a lens is paramount when evaluating a program whose goals touch on issues of equity or inclusion. This article seeks to open a discussion of how philanthropy can use an equitable-evaluation approach to apply the principles of the AEA statement, present the concept of equitable evaluation alongside an approach for building equitable-evaluation capacity, and apply equitable-evaluation capacity building to philanthropy.

Passing in the Dark: Making Visible Philanthropy’s Hidden and Conflicting Mental Models for Systems Change - Lynn, Jewlya; Coffman, Julia (The Foundation Review, 2024)
Exploring two systems change mental models in philanthropy [ https://evaluationinnovation.org/publication/systems-mental-models/ ] it is an increasingly shared truth: If we want to tackle society’s worst problems, we must bring a systems lens to our social change efforts. We define “systems change” here as the practice of confronting the causes of social problems rather than treating their symptoms. Many in philanthropy are taking this stance, and are shaping their portfolios, strategies, evaluation, and grants management approaches to recognize what is required to shift complex and dynamic systems. We support this focus and think it is critical to address the way systems keep problems stuck in place and create inequitable outcomes.* When we don’t make our mental models visible, we risk undermining our efforts. At the same time, we see that philanthropy and those that work in it (including ourselves as longtime systems change strategists and evaluators) have not been making visible our mental models about how and why systems change. By mental models, we mean the deeply held beliefs and assumptions and ways of seeing the world that we use to examine problems and generate solutions. Our mental models drive how we work and achieve impact. But sometimes the systems change approaches that program staff, strategy consultants, and evaluators use are based on competing mental models or models that are not a good fit for the systems, problems, strategies, or practices they are using. In our experience, this ultimately undermines the work. We see two mental models about systems change at play in philanthropy. While many mental models apply to systems change work, we see two dominating the philanthropic sector—the systems dynamics mental model and the systems emergence mental model.** We do not advocate for one mental model over another. While our article compares the two models and emphasizes how they differ so that their distinctions are clear, we want to avoid binary or either-or thinking about their application. We think that certain systems and problems call for the use of one mental model over the other, but there are also systems and problems where the use of both mental models simultaneously can be useful and appropriate. The systems dynamics mental model is more common in philanthropy. This model is grounded in the work of well-known thought leaders like Donella Meadows and Jay Forrester. The mental model leads to: Strategies that focus on leverage points, or places where focused effort and resources can have an outsized impact on the system. These are often part of feedback loops in the system, where one dynamic feeds into another. Strategies often focus on policy or legal changes, changes in power, narrative change, resource changes, or even changing the field of actors working on an issue. Grantmaking focused on chosen points of leverage. Theories of change that predict short- and long-term outcomes, including hypothesized causal relationships. Often these are less linear than a programmatic theory of change, but they still predict how change will happen. Evaluation that focuses on determining whether theorized change processes are playing out as expected. Use of tools like systems maps, causal loop diagrams, field-building assessments, and social network analyses, and theory-driven methods like contribution analysis and process tracing. The systems emergence mental model deserves more attention. This mental model draws on thinking from complexity, critical race, social capital, and ecological systems theories, among others. While terms here are Western in origin, the culturally embedded approaches to systems thinking in Indigenous communities align most closely with this mental model and have significantly informed this way of thinking about how systems change (see, for example, the work of Common Ground, Melanie Goodchild, and Don McIntyre, Geneva Cloutis, & Dan McCarthy). This mental model leads to: Strategies that are non-linear and decentralized, where opportunities to influence change are experimented with and discovered, released when no longer relevant, and learned from throughout. Strategies focus on parts of the system that seem ripe for attention (e.g., green shoots of innovation, underutilized or unexplored parts of the system, the most hidden aspects of the system), even when it is not clear what acting in these areas might lead to. Grantmaking that enables time-based experiments and long-term partnerships with like minded, values-aligned organizations. It engages those most affected by the problem and may attempt to change who has leadership over systems and solutions. A line of sight that describes a future worth achieving (what the system could be), but that does not make upfront predictions about a precise chain of outcomes to get there. Hypotheses and other loose storytelling about how change might happen, rather than the defined pathways featured in theories of change. Emergent outcomes, leveraging real-time and deeply embedded learning led by program staff or facilitated by a developmental evaluator. Use of tools like emergent strategy, the Cynefin framework, emergent learning, futures thinking, and participatory systems sensing. Our mental models need to be fit for purpose. Our thesis is that the mental models we use should match the systems being changed and what is known (or not) about those systems. Reasons for using a particular mental model should not be driven solely by what has been tried before in a different system (what is familiar), what tools are already available (what is easily accessed through trusted consultants), or by standardized reporting and accountability requirements that apply across the foundation regardless of strategy approach (what is organizationally normed). We see both mental models at play in systems change efforts in philanthropy, and we also see mismatches occurring between mental models and philanthropic practices. In our full article, we offer three case studies–two that illustrate how each mental model plays out in a real-life foundation strategy, and one where mental models were in conflict. Read the full article to go deeper. Systems change thinking and scholarship is heady stuff and often overly intellectualized. We try to avoid this in our article by using metaphors and storytelling and by offering concrete tools to explore your own models. We also explore how these mental models relate to our shared commitments to equity and trust-based philanthropy. We’re curious to hear from you. What are the mental models you’re using and why? Are you seeing additional models ? What happens when models are aligned or in conflict with systems change strategies or evaluations?

Reflecting Forewords - Chambers, Robert; Lewin, Tessa (Practical Action Publishing, 2024)
Reflecting Forewords is an unashamedly ambitious book. Full of lessons from the past that are relevant for the future, it is a compilation of inspiring forewords from books written by Robert Chambers; one of the most influential and prolific scholars to write about participation, poverty, and knowledge in development studies. Tessa Lewin, Robert’s colleague and an experienced practitioner and researcher, has worked with Robert to weave together the forewords alongside a series of reflections from other practitioners and scholars in the field. In our world of dramatic unforeseeable change, Reflecting Forewords seeks to draw lessons and present insights that should not be ephemeral, but can contribute on a lasting way to policy, practice and research in the rest of our 21st century. Readers will draw their own conclusions. The orientation is forward-looking into our unpredictable future, stressing the need to be alert, in touch, nimble, and guided by reflexivity. This curated collection of resurfaced gems will inspire development professionals, teachers and graduate students of development studies, policymakers, historians of development thinking and practice, activists, and all those who are concerned with a better life for all.

Bridging Learning and Action: How Did CLARISSA’s Participatory Adaptive Management Approach Foster Innovation, Effectiveness, and Stakeholder Empowerment? - Prieto Martin, Pedro; Apgar, Marina; Afroze, Jiniya; Arulanantham, Amit; Hicks, Jacqueline; Karki, Shanta; Mareschal, Sophie; Paul, Sukanta; Snijder, Mieke; Uddin, Forhad; Veitch, Helen (CLARISSA Research and Evidence Paper, 2024)
Child Labour: Action-Research-Innovation in South and South-Eastern Asia (CLARISSA) is an evidence and innovation-generation programme funded by the United Kingdom’s Foreign, Commonwealth & Development Office (FCDO), responding to the challenge of the worst forms of child labour (WFCL) in Bangladesh and Nepal. It is a challenge characterised by a poor understanding of its drivers and a lack of evidence on what works to combat it. To handle such fundamental uncertainty, the programme adopts a child-centric and participatory action research approach, which is supported by an adaptive management model to respond better to challenges and opportunities. From its inception, the programme needed to navigate shocks and challenges, such as Covid-19 lockdowns, political upheaval, and sustained budget cuts, which put its capacity to learn and evolve to the test. This paper shares insights emerging from evaluating CLARISSA’s participatory adaptive management (PAM) practices, connecting them with current discussions on adaptive management. It provides an in-depth evaluation of CLARISSA’s PAM approach, exploring how adaptive strategies were implemented and evolved throughout the programme’s life cycle. Multiple cases of adaptation and misadaptation were selected and analysed through a series of in-depth interviews and review of programme documentation, allowing us to assess whether and how the adaptive management practices have been operationalised, the degree to which they led to enhanced decision-making and effectiveness, and their empowering effect on children and other programme stakeholders. Through conceptual analysis and real-world examples, the paper examines the key stages and critical dimensions of PAM, conveying the complexities and dynamism of adaptive management in development work, while highlighting both the successes and challenges encountered in operationalising PAM within CLARISSA.

A Manifesto for the Platform Organization - Cicero, Simone; Quintarelli, Emanuele; Ruggeri, Luca (Boundaryless, 2024)
Discover the future of organizational structure with the Platform Organization Manifesto. Our guiding principles provide a blueprint for dynamic, adaptable businesses designed to thrive in a constantly changing marketplace. Harness the power of entrepreneurial spirit, cooperation, innovation, and key design principles to empower corporate evolution.

Review of current system & portfolio practices in the development sector: Organizational… - Pereverza, Kateryna (Medium, 2024)
Building on interviews with colleagues in development organizations, donor & partner governments and foundations, this article explores the adoption of system and portfolio-based approaches in the development sector focusing on organizational motivations and challenges, changes in the network of the development sector actors, and types of collaborative experiments initiated around the novel approach. The aim is to take steps towards understanding how this type of approach for system change can be institutionalized in the sector and become one of the main ways of operating for delivering transformative impact. Summary While there is a broad consensus that linear, fragmented, projectized ways of doing development & public policy are not a match for the type of context we live in today, there is less consensus about what alternatives look like. In this space, system and portfolio approaches have emerged during the last few years and a new field is beginning to slowly emerge. Currently, there is a diversity in interpretations of system and portfolio-based approaches by the development sector organizations. This is connected to different logics and theories of transformative change in complex systems. Also, the intent with the adoption of the novel approach can vary along the dimension of organizational efficiency vs place-based transformations. Organizations engage with portfolio-based approaches with different intents and in different roles. For example, OECD OPSI takes forward the innovation portfolio management for partners; the European Commission funds place-based portfolios e.g. within the M4EG program; while UNDP facilitates place-based and thematic portfolios for system transformations. When it comes to the motivations of development sector partners to experiment with system and portfolio-based approaches, these include increased effectiveness & efficiency of the use of funds for place-based transformations; response to the post-colonial critique; creation of visibility as a frontrunner; the potential of joint risk-taking; the opportunity of collective learning from an entire portfolio (coherence & dynamic evolution); and directionality added to internal and external activities. Challenges encountered by partners engaged in the experimentation with system and portfolio-based approaches are connected to resources and time constraints that emerge from the prevailing logics & constraints of current organizational structures premised on fragmentation & projectization; organizational leadership dynamics; the influence of dominant organizational culture and mindsets; the difficulties of adopting the ‘portfolio language’/jargon; prevailing funding structures and cycles; and a shortage of both time and capacity to engage with numerous local partners. The structure of the development sector ecosystem is undergoing changes driven by a rapidly changing context. The experimentation with system and portfolio-based approaches has led to new actors joining the space, such as innovative consultancies developing the logic of transformative change in systems and facilitating portfolios on the ground, and the emergence of new roles and relationships, for example, ‘learning partners’. Since 2020, three noticeable types of collaborative experiments emerged in the development sector ecosystem: systemic investments; thematic and place-based portfolio interventions; and monitoring, evaluation and learning (MEL) for the context of system change. This study revealed areas for further exploration and analysis. One is a need to co-create institutional structures to enable a co-ownership of place-based portfolios by several international partners as well as local stakeholders. Another area connects to the need to create and implement spaces and tools for collective learning from portfolios. Finally, there is a need to understand what mechanisms can enable the institutionalization of novel approaches on the level of the entire sector and their interconnection with organizational changes needed. This study is based on the interviews with representatives of ten development sector organizations conducted in spring 2023 and insights shared within portfolio practitioner gatherings and the M&E Sandbox webinars run during 2023. Introduction: The article emphasizes the importance of (re)framing challenges in portfolio-based approaches for steering system transformations. It acknowledges that the way we frame challenges significantly influences the solutions we develop, and therefore, it’s crucial to address the right problems rather than wasting resources on incorrect issues. Study Insights: The insights come from a study initiated in autumn 2021, focusing on how framing processes unfold, what needs to be framed, the open-ended nature of framing, and the roles international organizations like UNDP can play in developing inclusive and system transformation-oriented framings of urban challenges. Key Points: The study involved interviews with experts from the UNDP Istanbul and Thailand Regional hubs. It explored the experimentation with the portfolio approach in collaboration with local and national governments and partners. The article presents insights related to the framing process and highlights open questions for researchers and practitioners. System Transformations: System transformations require a deep understanding beyond surface-level issues, as existing systems often limit our imagination and prevent us from exploring the root causes of complex societal challenges. Portfolio Approaches: The article references the collaboration between the Swedish Strategic Innovation Programme “Viable Cities” and the Strategic Innovation Unit of UNDP. They focus on portfolio-based approaches for urban low-carbon transitions in Sweden, targeting the mission: “Climate-neutral cities 2030 with a good life for all within planetary boundaries.” Conclusion: The blog post concludes by highlighting the need for further research and practice in developing effective portfolio approaches to address complex societal challenges.

USAID ADS 201 - Operational Policy for the Program Cycle (Update 05/16/2024) - USAID (USAID, 2024)
The Program Cycle is USAID’s operational model for planning, delivering, assessing, and adapting development programming in a given region or country to advance U.S. foreign policy. It encompasses guidance and procedures for: 1) Making strategic decisions at the regional or country level about programmatic areas of focus and associated resources; 2) Designing supportive projects and/or activities to implement these strategic plans; and 3) Learning from performance monitoring, evaluations, and other relevant sources of information to make course corrections as needed and inform future programming.

Scaling Toolkit - A guide to scaling (digital) innovation in international cooperation - Schröder, Jan; Golbach, Pierre (GIZ, 2024)
Digital innovations are constantly redefining the boundaries of what is possible and, therefore, bring with them many challenges: it is important for organisations to scale impactful solutions and bring about positive change on a global level – a scaling toolkit is designed to help with this. With the Scaling Toolkit, a healthcare organisation, for example, can scale a digital health monitoring system, adapt it to the local language, and integrate it seamlessly into existing infrastructure. The Scaling Toolkit is a resource hub that can be applied to various projects and is designed to help organisations navigate the complicated process of scaling up the impact of digital solutions. It is based on practical examples, such as digital green innovation initiatives, and provides concrete tools and templates to guide practitioners through the iterative process of scaling digital innovations. When scaling digital solutions, technical factors and risks must be taken into account. To facilitate the scaling process, the toolkit can provide guidance and support to practitioners on how to scale successful digital innovations by integrating them locally, regionally, or globally in international cooperation. By doing so, digital solutions can achieve relevant and sustainable impact at scale.

The false dichotomy between trust and results - Lipovsek, Varja (Alliance magazine, 2024)
As the trust-based philanthropy wave is washing over the sector once again, concerns over its ability to prove impact, facilitate learning, and evaluate previous approaches are bubbling up. As a philanthropy professional with over …

Partnering with communities to co-design humanitarian health strategies: A SeeChange CommunityFirst Framework for implementation in MSF projects - Chapela Trillo, Violeta; Farber, Jessica (MSF, 2024)
The CommunityFirst Framework is intended to be implemented by field teams at MSF. The theoretical aspects and evidence presented on the importance of community engagement are intended for all MSF staff seeking to learn more about why and how to shift the way we work with communities as humanitarians. We believe this guideline, and other tools like it (including OCA’s Person-Centred Approach Guidance07, and MSF Vienna Evaluation Unit’s Guidance for Involving Communities08), to be an important contribution to the growing movement of communities and humanitarian actors who are pushing for changes in the humanitarian system that translate to dignity, health, justice, equity and self-determination for communities around the world. Specifically, the CommunityFirst Framework is intended to guide MSF teams to co-design health strategies with communities, throughout all stages of the project cycle, for exploratory missions, projects that are just opening, projects that have been running for some time, or those that are closing. At the time of publication, the CommunityFirst Framework has been tested in pilot projects in: (1) Madre de Dios, Peru (MSF OCP, August 2022), (2) Tonkolili, Sierra Leone (MSF OCA, November 2022) and (3) Anzoátegui, Venezuela (MSF OCB, February 2023) The experiences from these pilots (feedback from teams, implementation results, adaptations to each context, etc.) have informed the adaptation of the Framework. CommunityFirst builds on existing community engagement work inside MSF and contributes a practical framework for co-designing health initiatives with communities. To avoid duplicating efforts and resources around community engagement inside MSF, the appendices in this guideline largely refer to already existing MSF resources.09 This guideline is meant to be a living document that can evolve and be adapted given the experience of MSF staff and community members and diverse community contexts. This guide can be used by anyone in MSF who is interested in partnering with communities to improve the responsiveness and impact of their humanitarian programs. This is the first iteration of the document. Subsequent iterations will be published based on additional testing during future phases of the CommunityFirst TIC project.

The Radical How - Greenway, Andrew; Loosemore, Tom (Nesta and Public Digital, 2024)
Any mission-focused government should be well equipped to define, from day one, what outcomes it wants to bring about. But radically changing what the government does is only part of the challenge. We also need to change how government does things. The usual methods, we argue in this paper, are too prone to failure and delay. There’s a different approach to public service organisation, one based on multidisciplinary teams, starting with citizen needs, and scaling iteratively by testing assumptions. We’ve been arguing in favour of it for years now, and the more it gets used, the more we see success and timely delivery. We think taking a new approach makes it possible to shift government from an organisation of programmes and projects, to one of missions and services. It offers even constrained administrations an opportunity to improve their chances of delivering outcomes, reducing risk, saving money, and rebuilding public trust. The Radical How in a nutshell The struggles and shortcomings of delivering in government are well rehearsed. Many of the root causes that make it tough have been restated several times over several decades. But what to do? We believe the government can and should change how it delivers, by: organising around multidisciplinary teams embracing incremental, feedback-driven iteration focusing more on outcomes. The Radical How is a change of mindset as much as a change in organisation. It promotes methods and processes that have been shown to work, multiple times, at scale. They are the default ways of working for many of the world’s most successful companies. However, the occasions where they have been deployed are rare in government. These occasions have come about thanks to exceptional leaders, exceptional circumstances, or both. We think they’d make a big difference if they became the norm, rather than the exception. We also think that without them, mission oriented government will not become a reality. New policy ideas will remain just that, rather than translating into profound improvements to society. Central to this approach is the widespread adoption of internet-era ways of working. This paper explains both those and our thinking in more detail, with reference to real examples.

Putting Relationships First. The case for relationship-centred communities, organisations and systems - Lloyd-Rose, Matt; Robinson, David; Robinson, Immy (The Relationship Project, 2024)

In Numbers We Trust - Starr, Kevin (Stanford Social Innovation Review, 2024)
Because trust-based philanthropy shouldn’t mean blind faith.

Doing weeknotes - What weeknotes are, how weeknotes work, and how to start writing weeknotes of your own - Turnbull, Giles (Use the Human Voice, 2024)
Doing weeknotes brings together various things I’ve written about weeknotes in different places. This text expands on things I wrote in The agile comms handbook, as well as various blog posts. Quite a lot of it is brand new. - Weeknotes for beginners - Why write weeknotes - The weeknotes rules - Weeknotes within the corporate environment - What weeknotes can bring about - Examples of good weeknotes - How to write weeknotes - Weeknotes tips and tricks - Further reading

Addressing the evaluation-of-sustainability-paradox: A relational rubric for evidencing… - Guerzovich, Florencia (Medium, 2024)
Florencia Guerzovich and Alix Wadeson

Social Accountability 3.0: Engaging Citizens to Increase Systemic Responsiveness. Background Paper - Guerzovich, Florencia; Aston, Tom (Grupo Politeia, ESAG, UDESC and act4delivery, 2024)
Social accountability has become increasingly important to development programming since the 2004 World Development Report spearheaded its first generation of programming. In 2016, Thomas Carothers synthesized global experts’ perspectives on what became known as the second generation. With an evolving global context, new research and evaluations and tacit knowledge on how those assumptions played out in practice, the time is ripe for asking what the evidence since 2004 can tell us about what the social accountability’s next generation might look like? This meta-analysis takes stock of that knowledge through a systems lens and presents a possible “social accountability 3.0.” It draws on a database of 157 cases of initiatives implemented by different civil society organizations around the world and the lived experience of practitioners. Scholarly debates present practitioners’ options in binary terms, contrasting 1.0 minimalist and 2.0 maximalist options. The review finds that this binary is inaccurate and misleading; it masks practitioners’ diverse interpretations of 2.0 and range of feasibility considerations. Instead, work in the last decade looks more like a mosaic. By opening the analysis to the broader range of evidence, and reflecting more critically on scholars’ assumptions, a more adaptable middle path for social accountability 3.0 emerges. Its key outcomes include contributing to systemic responsiveness and an operational mechanism to rework the social contract.

Breaking free from the theory of change straight jacket - Apgar, Marina; Snijder, Mieke; Paul, Sukanta (CLARISSA, 2024)
The CLARISSA Social Protection Intervention was set us as an innovative social policy intervention for tackling social ills, with a...

How to be more agentic - Hall, Cate (Useful Fictions, 2024)
On a supposedly difficult thing

Exploring ‘place-based’ initiatives in philanthropic community development: A transnational examination of Canadian and US Foundations - Newton, Josh; Rabinowitz Bussell, Mirle; Martinez-Cosio, Maria; Pole, Nancy; Fontan, Jean-Marc (Local Development & Society, 2024)
Though the concept of place-based philanthropy is common parlance in community development scholarship and practice, there is still uncertainty around its meaning due to the multidisciplinary nature of its study and definitional variation among and between scholars and practitioners. This article presents findings from a recently edited volume created by a binational research team. Ten cases of philanthropic initiatives were explored to develop a transnational understanding of foundation engagement in place-based philanthropy. Place-based philanthropy was explored by considering collaboration with other stakeholders, initiative evaluation, systems change efforts, and issues of power, equity, and justice. Our findings suggest that place-based philanthropy among foundations is heterogeneous. While some foundations adhere to traditional forms of place-based philanthropy, others are pushing the boundaries of this type of work. This article urges both academics and practitioners to reimagine place-based philanthropy for transformative purposes.
Keywords: Place-based philanthropy; community development; equity and justice; systems change; transnational study

Scaling up Social Accountability in Complex Governance Systems: A Relational Approach for Evidencing Sustainability - Guerzovich, Florencia; Wadeson, Alix (World Bank, 2024)
When social accountability interventions scale up and their sustainability depends on the interactions of many agents and system components, related results are rarely observable at the end of an intervention. The 2019 OECD Development Assistance Committee’s (OECD DAC) revamped evaluations criteria for assessing sustainability acknowledges that such results are often emergent, and should be monitored and evaluated with this in mind. It therefore emphasizes a turn towards assessing complex processes prospectively. It also asks evaluations to consider how likely it is that these results are evident at the time they are monitored or evaluated. However,the social accountability field continues to have gaps regarding doing this effectively in practice. This paper presents and provides evidence from testing an innovative operational approach that has promising potential to support this aim - a sequential, relational rubric. This approach can support practitioners to monitor, evaluate and learn about the causal processes of scale up of social accountability interventions with an eye towards sustainability i.e., considering prospective sustainability. It is grounded in systems thinking, co-production and social learning theory, as well as links with collective governance and social contract theory for development. Evidence yielded from the authors’ testing of this approach on a sample of diverse projects from the Global Partnership for Social Accountability (GPSA) program revealed that the alleged ‘absence of evidence’ dilemma of social accountability scale up is due to ill-fitting concepts and methods for assessment. It challenges existing assumptions and findings that claim that social accountabilityprocesses do not scale and are unsustainable. The authors propose that by using fit-for-purpose concepts and methods with a focus on social learning and compromise – also called a ‘resonance pathway to scale’ which this paper discusses in detail – it is possible to observe loosely coordinated scale up processes at work in many (but not all) social accountability interventions and identify tangible evidence of prospective sustainability. An important caveat is that these processes, the outcomes they generate, and the corresponding evidence often look qualitatively different than the original intervention design and predictions for scale-up at that point in time. This is because the process of deliberation and compromise inherent to social accountability work in dynamic local systems introduces changes and new conditions for uptake by diverse actors in the public sector, civil society, and donor institutions. The paper concludes that even relatively small-scale localized projects of three to five years with budgets of less than one million USD, across different contexts and sectors can produce processes and outcomes which contribute to many forms of sustainability, including via scaleup.Furthermore, the cross-fertilization of learning and aggregation of results for scale-up across projects within and beyond the GPSA (and other programs) can help monitoring evaluation and learning (MEL) and social accountability practitioners alike to deliver on a program’s mandate. Doing so can also create new knowledge for the wider social accountability field that siloed interventions, lacking suitable concepts and methods for assessing scale-up and prospective sustainability, often fail to produce. The paper ends with recommendations for taking forward this approach and the associated benefits, implications and required investments.

READ ME - AM Knowledge Base - How-to user guide for the Zotero library - Prieto Martin, Pedro (Institute of Development Studies, 2024)
The library contains and categorises resources related to Adaptive Management and International Development, with some focus around the use of MEL to support it. At the time of writing the library contains 1907 resources and more than 1 Gigabyte of source documents. The documents include journal articles, books, reports, case-studies, toolkits, courses, blogposts, websites, and other grey literature. The resources are flexibly organised within a hierarchy of categories and sub-categories, which facilitate browsing and access. The knowledge base can be queried using Zotero’s powerful search functionalities, which not only include bibliographic fields such as title, authors, publisher, tags and notes, but also search within the full source text of the pdf documents. The Knowledge based can be accessed in different ways: • Through the user-friendly web interface. https://adaptdev.info • Using Zotero’s web interface. No need to register unless you want to access the pdf sources. • Using Zotero app, available for Windows, MacOS and Linux. This enables users to keep an updated copy of the library in their computer and get access to all the reference managing system’s functionalities and source documents. This illustrated “user guide” provides how-to instructions on how to use the library.

Our Data, Our Process: Re-imagining the Community-Led Monitoring and Evaluation Discussion Samahan para sa Pagsulong at Pag -unlad ng Capaculan (SPPC), Nancy Bobadilla, Gunjan Veda - SPPC, Samahan para sa Pagsulong at Pag -unlad ng Capaculan; Bobadilla, Nancy; Veda, Gunjan (Asia Pacific Journal of Evaluation, 2024)

Learning Towards Zero: Learning in + from Practice in Systems Innovation - Burkett, Ingrid (Good Shift, 2024)
In early 2021 we were asked to support two initiatives in South-East Queensland by undertaking a developmental evaluation of how they collaborated across multiple partners to reduce, prevent and ultimately end homelessness. The initiatives, Brisbane and Logan Zero, started with the ‘Advance to Zero’ methodology, and adapted this as they progressed. We were asked to track how contributors collaborated to tackle the systemic nature of this challenge. This gave us the opportunity to work with both initiatives over a two year period to reflect back to them how change was happening towards the outcome, and to track learnings focused on systemic change along the way. While the evaluation report captures the details of this exploration, there were a myriad of insights from the process that could inform other systemic innovation initiatives - so this Learnbook outlines these. So much can be learnt from these initiatives that could help others seeking to create multi-sector and multi-organisation alliances for systemic change. Systemic work is, by its very nature, complex work. Anyone who is embarking on this type of work needs to be comfortable with learning forward, and working with diverse approaches that are context dependent. We share learnings not to suggest that this is ‘the’ way to undertake systems innovation, but to share one way that is showing promise in a particular context. This Learnbook outlines four key insights that could inform systemic change initiatives. 1. Human systems are complex and dynamic. When we are working in complex domains, we need to ‘learn forward’. 2. Shifting systems requires collaboration across multiple structures, sectors and organisations. The way collaboration is organised shapes action. And if the context is complex, then organising needs to be adaptive. 3. Leading systemic shifts is about collaborating, sharing power and distributing action. Governance models need to adapt to dynamic action and collaborative leading. 4. Data is critical, but let’s be clear about the type of data that is needed and how it is used. Action towards systemic change in human systems requires human-centred, lead data.

Guidance Note: Evaluation Criteria - Chaplowe, Scott; Child, Keith; AF-TERG (Adaptation Fund, 2024)

Disentangling Government Responses: How Do We Know When Accountability Work is Gaining Traction? - Fox, Jonathan; Halloran, Brendan; Fölscher, Alta; McGee, Rosie (Working Paper, 2024)
Advocacy for public accountability aims to produce certain reactions from government officials or service providers. However, the reactions can be many and diverse, and it is not always clear to advocates how to interpret them and decide on next steps—whether to intensify efforts or back off; continue the same strategy or make adjustments. This paper presents a framework to help accountability advocates and practitioners interpret government reactions to their efforts and move forward appropriately. The framework arises from learning and reflection in the context of the International Budget Partnership (IBP)’s Strengthening Public Accountability with Results and Knowledge (SPARK) program. SPARK seeks to bolster the collective agency of marginalized communities and coalitions to advance democratic and equitable fiscal governance systems that channel public resources to services that address the priority needs of these historically excluded groups. The paper: · unpacks the broad umbrella category of government responses to citizen-led accountability initiatives, discerning within it three overlapping categories of responses, responsiveness, and accountable responsiveness (RRA), and pinpointing what distinguishes them; · maps out the public financial management (PFM) system as the critical conduit for any public service to progress from non-response through these three categories, reflecting the terrain and priorities of the SPARK program and IBP more broadly. Looking at RRA in the context of this PFM map offers key insights for the twin tasks of tracing and unblocking bottlenecks in resource management systems that hinder responsiveness and accountability, as well as demanding specific reforms that make the fiscal governance system more open to citizen engagement; and · discusses how the RRA framework can be used to take stock of partial or incremental responses and assess whether change is moving towards accountable governance, by reference to the SPARK program and other IBP efforts. The RRA framework supports accountability-claiming practice by facilitating close attention to the actions of and influences on key government stakeholders, careful interpretation of these, and consequential adjustments to advocacy strategies. Its implications for monitoring, evaluating, and learning (MEL) from practice are also significant. MEL has been evolving in the accountability field in line with the thinking and practical challenges behind the development of the RRA framework. The framework adds to a range of emerging approaches that support the application theories of change, evidence-based adaptive programming, and politically aware systems thinking in the complex task of building accountable responsiveness towards constituencies historically excluded from public goods and services, and ultimately equitable development outcomes from an intersectional perspective.
Keywords: Development studies; Public policy; Social policy

The law of the instrument: Would you rather be a theory or a nail? - Freer, Gordon (Theories of Change in Reality, 2024)
Using the hammer-and-nail analogy of the law of the instrument, often attributed to Abraham Maslow, this essay explores the minimal utilisation of theories of change within programmes despite their almost mandatory inclusion in programme proposals, designs, and evaluations. The essay then considers reasons for this lack of use and explores potential solutions to the same. The essay contends that theories of change can be used for a variety of purposes—in programme design; as complementary programme management tools; adding value to reporting, adaptation, and learning; as well as contributing more broadly to refining social science theories.

Constructing a living theory of change in fluid and volatile environments: Grounded on context, problem, and evidence - Hassnain, Hur (Theories of Change in Reality, 2024)
Theory of change is an explicit articulation of how an intervention creates an intended result to address a specific problem. This ‘theory’ is difficult to re/construct if the ‘context’ is complex, conflict-ridden, and uncertain. In such contexts, causal linkages leading to a desired outcome tend to be messy, multilevel, multidirectional, and unpredictable, as plans and strategies often do not work as anticipated. Whilst some activities lead to outcomes, others do not. Outcomes (positive or negative) could also emerge in unexpected places. This essay discusses why and how evaluations in fluid and volatile environments must aim to tackle the emergence and unpredictability of outcomes, and re/construct a theory together with key stakeholders. This ensures that it is fully grounded in the context, problem, and available evidence at the inception stage of the evaluation process and onwards. Preferably, evaluators should be engaged from the design of the intervention and its theory of change because a lack of understanding of the context results in causing harm to individuals, groups, and organisations and creates a lack of realism about the capacity of external actors to support substantive positive change. A better understanding of the context can help illustrate the causal links, hypotheses, and assumptions that make evaluations reliable and safe for the enumerators, evaluators, and the affected population. It also aids in assessing whether the intervention has had a positive or negative effect or no effect at all.

Critical systems thinking: a practitioner's guide - Jackson, Michael C. (Wiley, 2024)
"Critical systems thinking is a multiperspectival and multimethodological approach to complex problems that takes advantage of the variety of systems approaches, aware of their different strengths and weaknesses"--
Keywords: Leadership; System analysis; System theory

Strategy-level theories of change require a focus on systems change: An actor-based approach can help - Koleros, Andrew (Theories of Change in Reality, 2024)
This essay makes the case that an actor-based approach to theory of change development can assist philanthropic foundations to shift from their conventional approaches of supporting stand-alone grants or siloed programmes to strategies that focus collective efforts across a range of partners on achieving systems change. It presents the concept of an actor-based approach to theory of change development and provides an illustrative example to demonstrate its suitability. The essay then contends that evaluators can play a strong role in this process, but it will require some shifts in how evaluators typically engage with foundations. Namely, evaluators need to be engaged earlier and integrated into the strategy design process; evaluators need to adopt a design mindset as well as an evaluation mindset; and evaluators must embrace the uncertainty of engaging in systems change efforts, supporting foundation teams towards an appreciation of collective efforts and the need to generate learning to adapt and improve.

Conclusions around theories of change in reality - Koleros, Andrew; Adrien, Marie-Hélène; Tyrrell, Tony (Theories of Change in Reality, 2024)
This book sets out the responses from a group of international practitioners and experts to three questions posed by the editors around how they are using theories of change in their work, what they have found to be the limits of their use, and what further adaptations they feel are needed so that theories of change remain relevant and useful in the future. The responses we received are, given the diversity of the contributors, understandably diverse with reference to perspective, emphasis, and the context within which theories of change are used. Broadly, their perspectives indicate that theories of change have long represented a valuable approach in both programme planning as well as in evaluation design and practice; however, the authors also make clear that theories of change are often misunderstood and misused. In particular, the authors draw attention to the use of theories of change as a static product rather than an iterative, inclusive process that recognises the centrality of multiple stakeholder perspectives in evaluation design and practice. The authors also highlight the challenges involved in the application of theories of change in contexts defined by heightened complexity.

Co-Intelligence: Living and Working with AI - Mollick, Ethan (WH Allen, 2024)
**A New York Times Bestseller**'Co-Intelligence is the very best book I know about the ins, outs, and ethics of generative AI. Drop everything and read it cover to cover NOW' Angela DuckworthConsumer AI has arrived. And with it, inescapable upheaval as we grapple with what it means for our jobs, lives and the future of humanity.Cutting through the noise of AI evangelists and AI doom-mongers, Wharton professor Ethan Mollick has become one of the most prominent and provocative explainers of AI, focusing on the practical aspects of how these new tools for thought can transform our world. In Co-Intelligence, he urges us to engage with AI as co-worker, co-teacher and coach. Wide ranging, hugely thought-provoking and optimistic, Co-Intelligence reveals the promise and power of this new era.

Integrating theories of change in programme management and delivery - Oldenbeuving, Mark (Theories of Change in Reality, 2024)
Beyond a useful tool for evaluators, theories of change can serve as an effective tool for programme delivery. However, they are often not used to their full potential. This essay will cover ways in which theories of change can be more actively used in programme management to improve programme delivery. Examples of this include using theories of action in the planning, delivery, and reporting against activities, and integrating assumptions identified in the theory of change in risk management and mitigation processes, among others. It will conclude by providing suggestions on how to practically integrate theories of change in programmes’ management information software solutions.

Developing, representing, and using theories of change for interventions in complex systems - Rogers, Patricia (Theories of Change in Reality, 2024)
Theories of change for interventions in complex systems present a challenge for usual approaches to developing, representing, and using theories of change. Interventions in complex systems operate under conditions of ongoing uncertainty, not because of a lack of information but because of three features that contribute to this uncertainty: (1) numerous, diverse, and interacting components; (2) nonlinear relationships; and (3) changes brought about through self-organisation, agency, adaptation, and learning. In such situations, a different approach is needed to theories of change. Their primary intended use needs to be supporting ongoing single-loop and double-loop learning and adaptation by the various actors involved, rather than monitoring compliance with an initial plan. The underlying change theories need to be appropriate for managing complex systems—for example, focusing on developing an effective network of actors, creating an enabling environment, and supporting iterations of learning and adaptation rather than planning tightly prescribed implementation of ‘what works’.

How to ensure no-one uses your theory of change: Lessons from the front lines of theory of change facilitation and possibilities for renewal - Tangelder, Mary (Theories of Change in Reality, 2024)
This essay discusses how theory of change can be used as a design tool to help funders and their grantees shift from short-term, discrete activities to addressing deeper, systems-level change. Facilitating a theory of change design process as part of an effort to co-design a strategy should, ideally, lead to a strategy that highlights collective efforts and ultimately stimulates meaningful and powerful collaboration. Yet, engaging programme teams and grantees in theory of change has been notoriously difficult, particularly given power dynamics between funders and grantees. Drawing from experience, this essay discusses blind spots and typical stumbling blocks in theory of change design. It builds a case for why we need to rethink and reimagine how to use this tool to inspire action, build shared ownership, drive learning and adaptation, and ultimately accelerate the path to change.

Unconditional cash support sparks transformation: A story from CLARISSA Social Protection Intervention in Dhaka - Aktar, Afrin; Debnath, Raj; Kashfi, Sharmin (CLARISSA, 2023)
The CLARISSA Social Protection (SP) intervention provided six months of unconditional cash transfers to every household in the Gojmohol neighbourhood,...

Fostering Values-Driven Sustainability Through an Ex-Post Capacities Lens - Cekan/ova, Jindra (Journal of MultiDisciplinary Evaluation, 2023)
Background: Ex-post evaluation of sustainability has been done for 40 years in global development. However, it has been done far less than 1% of all global development projects, for there is little proof that “sustainable” development is or is not. Similarly, foreign aid projects are implemented to foster sustainability, but without the benefit of evidence from expost evaluations of what drove it and limited research on the benefits of robust exit strategies. Purpose: Transparency in values we hold, and evaluative capacities’ best practices that we bring to our evaluations inform how they are done, with whom, and for what. Using the evidence base from ex-post evaluations and exit strategies led to these nine checklists. Professionals in monitoring and evaluation should use them to foster longterm sustainability and learning. Setting: Drawing on primary and secondary research across 91 ex-post evaluations of foreign aid sustainability plus two major studies of exit strategies globally. Research Design: The checklists were drafted based on sustainability and exit studies and then vetted with lead researchers of the two exit studies. They were revised, and additional research was done on both values-driven evaluation and evaluation competencies. Data Collection and Analysis: Some primary data was collected during ex-post evaluations by the author, complemented by secondary research. Findings: Sustained exit commitments and conditions checklists can build evaluator capacities in evaluating sustainability. Several have been used by Tufts, USAID, the GEF, and the Adaptation Fund and verified actual sustainability and its prospects.
Keywords: Ex-post evaluation; Monitoring and evaluation; Participative M&E; capacity building; capacity development; competencies; post-project; sustainability; values

Systems-aware social accountability: Harnessing Practitioner Insights for More Responsive Governance - Guerzovich, Florencia (Pact, 2023)
The framework’s four principles, described below, are not exclusive to social accountability interventions; for example, Pact has published tools on how to use political economy analysis or integrate adaptive management principles into programming. Rather, the principles highlight how – by including the right combination of mindsets, management approaches, analytic approaches and tools – practitioners can develop and apply a deep understanding of a system to improve relationships within it, in support of greater accountability and improved service delivery. - Use a Holistic Lens - Recognize that there is always a system; understand and engage with it​: Developing a strong understanding of the system in which a project works and effectively integrating this knowledge into implementation increases the chances that local actors will sustain the interventions. - Right Fit the Approach​ - Equipped with a holistic understanding of the context, teams identify where they have a comparative advantage in a specific system. In other words, they identify where their resources, expertise and partners have the greatest potential for impact, then set realistic targets that they revise periodically. -Orchestrate Across The System -​ Practitioners take a networked, rather than a top-down, approach to capitalize on their convening power and maximize opportunity to facilitate the most-promising alignments across and within their projects, portfolios and systems. Orchestrators use their individual power, relationships and insight into the local system to enable, not impose, promising interactions and mitigate the risks of damaging ones. - Manage Adaptively - Monitor, evaluate and reflect on emerging patterns of data, evidence and learning to inform decision-making within and, equally importantly, across project cycles. Where possible, practitioners should consider how to use short-term projects to play a long-term game by linking new projects to the lessons and results of their predecessors and ensuring that they are responsive to the evolving context, particularly opportunities, limits and risks.

Practicing what we preach — our experience rethinking M&E in complex settings - Bina, Andrea; Vester, Søren (UNDP Strategic Innovation, 2023)
The UNDP has been exploring new approaches to M&E to better understand and address complex systems challenges. Traditional linear project-based planning and evaluation methods are insufficient for dealing with the unpredictable nature of such systems. The UNDP’s new Strategic Plan calls for a shift in M&E practices to align with the complexity of today’s global challenges. Key points from the blog post include: Learning and Adapting: There’s a need for continuous learning and adaptation in M&E practices to effectively address complex problems like transitioning to a circular economy. Longer Time Horizons: Substantive change takes time, and it’s often unclear what the results will look like, making it difficult to track progress. Aggregate Impact: Evaluating interconnected interventions as a portfolio is crucial because they cannot be assessed in isolation. Contribution vs. Attribution: Focusing on the contribution to broader change processes is more important than attributing specific outcomes to individual actions. The UNDP has initiated the M&E Sandbox to experiment with and learn from innovative M&E practices. This collaborative space has attracted participants from over 180 organizations, highlighting a shared interest in evolving M&E to better understand change in contexts of uncertainty. A partnership with the Bill and Melinda Gates Foundation has also been formed to focus on M&E in food systems. The blog post emphasizes the importance of embracing uncertainty, experimenting, and collaborating to contribute to positive change in complex settings1.

Insights for Influence: Understanding Impact Pathways in Crisis Response - Clark, Louise; Carpenter, Jo; Taylor, Joe (Institute of Development Studies, 2023)
The Covid-19 Responses for Equity (CORE) programme was a three-year initiative funded by the Canadian International Development Research Centre (IDRC) that brought together 20 projects from across the global South to understand the socioeconomic impacts of the Covid-19 pandemic, improve existing responses, and generate better policy options for recovery. The research covered 42 countries across Africa, Asia, Latin America, and the Middle East to understand the ways in which the pandemic affected the most vulnerable people and regions, and deepened existing vulnerabilities. Research projects covered a broad range of themes, including macroeconomic policies for support and recovery; supporting essential economic activity and protecting informal businesses, small producers, and women workers; and promoting democratic governance to strengthen accountability, social inclusion, and civil engagement. The Institute of Development Studies (IDS) provided knowledge translation (KT) support to CORE research partners to maximise the learning generated across the research portfolio and deepen engagement with governments, civil society, and the scientific community. As part of this support, the IDS KT team worked with CORE project teams to reconstruct and reflect on their impact pathways to facilitate South-South knowledge exchange on effective strategies for research impact, and share learning on how the CORE cohort has influenced policy and delivered change. This report presents an overview of these impact pathways and the lessons learnt from a selection of the projects chosen to represent the diversity of approaches to engage policymakers, civil society, and the media to generate and share evidence of the effect of the pandemic on diverse vulnerable groups.

How AI Fits into Lean Six Sigma - Holweg, Matthias; Davenport, Thomas H.; Snyder, Ken (Harvard Business Review, 2023)
AI already is being used in some areas of process improvement, and the usage of this technology — including generative AI — promises to grow. That’s because it can perform tasks faster and much less expensively than humans alone. But it will never fully replace people — and that poses management challenges.

3 things that are enabling the UNDP’s shift to portfolios - Begovic, Milica (Medium, 2023)
By Millie Begovic, Head of the UNDP Strategic Innovation Unit

Life Stories From Children Working in Bangladesh’s Leather Sector and its Neighbourhoods: Told and Analysed by Children - Sayem, Mashrique; Sayed, Sayma; Maksud, A. K. M.; Reaz Hossain, Khandaker; Afroze, Jiniya; Burns, Danny; Raw, Anna; Hacker, Elizabeth (CLARISSA Research and Evidence Paper, 2023)
CLARISSA (Child Labour: Action-Research-Innovation in South and South-Eastern Asia) has a participatory and child-centred approach that supports children to gather evidence, analyse it themselves and generate solutions to the problems they identify. The life story collection and collective analysis processes supported children engaged in the worst forms of child labour in Bangladesh to share and analyse their life stories. Over 400 life stories were collected from children who worked in the leather supply chain, or who lived and worked in leather sector neighbourhoods. Using causal mapping, 53 children who were engaged in or had experience of the worst forms of child labour collectively analysed the data. This resulted in children’s life stories becoming the evidence base for revealing macro‑level system dynamics that drive the worst forms of child labour. This paper is a record of the children’s analysis of the life stories and key themes they identified, which formed the basis of a series of seven child-led Participatory Action Research groups.

Challenge-led Innovation Workbook. Organising for Systems Innovation at Scale - Burkett, Ingrid (Griffith Centre for Systems Innovation, 2023)
Organising for Systems Innovation at Scale Our team at Griffith Centre for Systems Innovation have been experimenting with and evolving a Challenge-led Innovation Approach (based on Mission-oriented approaches developed by Mariana Mazzucato at UCL IIPP and others internationally). We are using this approach to guide the way we work internally and engage with our systems innovation partners. We’ve facilitated intensive Re:Treats, worked with government bodies, businesses and civic organisations, and engaged deeply with others exploring this work. We have a bias for developing and testing HOW such approaches could be applied to respond to both local and global challenges rather than getting too caught up in the what and why of such approaches. We decided to openly share our learnings and thinking to date in this workbook, to spark conversations and innovation in both practice and thinking amongst those exploring how we work, and to learn together to address complex systems and challenges. We see this booklet as a first step in a longer learning journey. In it we share an overview of: the principles and processes that sparked our evolution to a Challenge-led Innovation framework (from Mission-oriented). examples of our learnings from other system innovators who are experimenting. an adaptable process to help guide the learning journey. learning tools and canvases to catalyse thinking, practice, and further adaptations. Part One sets out some foundations we’ve identified as important to Challenge-led Innovation. If you want to jump straight into the mapping process, we suggest you skip to Part Two. The final section, Part Three, focuses on what we have learnt about the conditions needed and how to get started on a Challenge-led initiative.

Scaling Up Development Impact - Guerrero, Isabel; Gokhale, Siddhant; Fahsbender, Jossie (Bowker, 2023)
Today, nearly one billion people lack electricity, over three billion lack clean water, and 750 million lack basic literacy skills. Many of these challenges could be solved with existing solutions, and technology enables us to reach the last mile like never before. Yet, few solutions attain the necessary scale to match the size of these challenges. Scaling Up Development Impact offers an analytical framework, a set of practical tools, and adaptive evaluation techniques to accompany the scaling process. It presents rich organizational experiences that showcase real-world journeys toward increased impact. From the people from IMAGOgg.org

Hidden Potential: The Science of Achieving Greater Things - Grant, Adam (WH Allen, 2023)
#1 New York Times bestseller "This brilliant book will shatter your assumptions about what it takes to improve and succeed. I wish I could go back in time and gift it to my younger self. It would've helped me find a more joyful path to progress."-Serena Williams, 23-time Grand Slam singles tennis championThe #1 New York Times bestselling author of Think Again illuminates how we can elevate ourselves and others to unexpected heights.We live in a world that’s obsessed with talent. We celebrate gifted students in school, natural athletes in sports, and child prodigies in music. But admiring people who start out with innate advantages leads us to overlook the distances we ourselves can travel. We can all improve at improving. And when opportunity doesn’t knock, there are ways to build a door.Hidden Potential offers a new framework for raising aspirations and exceeding expectations. Adam Grant weaves together groundbreaking evidence, surprising insights, and vivid storytelling that takes us from the classroom to the boardroom, the playground to the Olympics, and underground to outer space. He shows that progress depends less on how hard you work than how well you learn. Growth is not about the genius you possess – it’s about the character you develop. Grant explores how to build the character skills and motivational structures to realize our own potential, and how to design systems that create opportunities for those who have been underrated and overlooked.This book reveals how anyone can rise to achieve greater things. The true measure of your potential is not the height of the peak you’ve reached, but how far you’ve climbed to get there.

Anatomy of Social Accountability 3.0: A working framework for moving forward - Guerzovich, Florencia; Aston, Tom (Florianopolis, SC, Brazil: Grupo Politeia, ESAG, University of the State of Santa Catarina and act4delivery, 2023)
This note presents the key components of a potential next generation of social accountability programming. The key aspects outlined emerge from a review of over 150 cases as well as tacit knowledge from dozens of practitioners (Guerzovich and Aston, 2023).The main thread of social accountability 3.0 and what distinguishes it from previous generations is a focus on its contribution towards more responsive systems and accountable social contracts. In particular, social accountability should be considered as an operational means to rework social contracts.Over the last two decades, the field has been defined by a supposed dichotomy between minimalist (i.e., “widget” based) approach or a maximalist (“go big or go home”) approaches. Yet, most practice sits somewhere between these extremes. The paper presents various options derived from practice; practitioners should choose whichever best fits their own particular circumstances.Those hoping to improve and expand their practice with a systems-lens may benefit from considering how they approach the key elements of social accountability 3.0.
Keywords: accountability; development; responsiveness; social accountability; social contract; systems

Storytelling and Evaluation: Can Pathways be the Theory Missing in Explanatory Narratives? - Guerzovich, Florencia (Medium, 2023)
Florencia Guerzovich

Changing theories - Aston, Thomas (Medium, 2023)
You may, or may not, be surprised to hear that many theories of change lack what we might generally understand as a theory.

How adaptive M&E from the peace sector can help demonstrate the value of aid - Kratzer, Sebastian (2023)
Over the last decade, the peace sector has been developing and adapting Monitoring and Evaluation (M&E) systems and tools to fit their contexts and ways of working. This evolution may hold some insights for the aid community in how to go beyond more traditional, backwards-looking M&E to navigate today’s volatile, interest-based world of politics and aid.

Why are effective feedback mechanisms in cash transfers so important? - Jannat, Mahiratul (CLARISSA, 2023)
The CLARISSA Social Protection Intervention was set us as an innovative social policy intervention for tackling social ills, with a...

Participatory evaluation design and sense-making - making meaning through co-constructed lenses - King, Julian; Elliott, Kerry; Hollingsworth, Hilary (Australian Evaluation Society International Evaluation Conference, 2023)
Based on a presentation at Australian Evaluation Society International Evaluation Conference, Brisbane, 28 September 2023. The importance of participatory approaches to evaluation is well-understood. Involving the right people and balancing voices at the stakeholder table can help to make evaluations more valid, credible, useful, and actually used. Not all evaluations, however, are done in participatory ways. This presentation shares the experience of three evaluators in two recent projects with clients and stakeholders who were unaccustomed to participatory evaluation. In both cases, new ways of working were introduced, requiring participants to come into the evaluation tent, collaborate and learn with us. Engagement processes deliberately focused on incrementally building deep stakeholder input into evaluation design - including key evaluation questions, context analysis, program theory, rubrics, methods of data gathering, and reporting formats. The resulting evaluation frameworks represent an agreed, shared set of lenses for making sense of the evidence. Through this process, relationships and trust were cemented, stakeholders understood the evaluation process and endorsed the resulting frameworks. The client and sector have become more interested, more engaged stakeholders and more savvy evaluation participants and consumers. As the evaluations progress, the evaluation design builds in ongoing stakeholder involvement in reviewing emergent footprints of system change and making evaluative judgements. Participatory approaches are many, varied, and flexible. We will share our reflections and generalisable learning from the participatory approaches we applied in these evaluations. We will invite the audience to reflect on their own practices and experiences, and provoke them to consider whether participatory evaluation is 'just' a methodological option or a foundational element of good evaluation.

Foreign Aid and Its Unintended Consequences - Koch, Dirk-Jan (Routledge, 2023)
Foreign aid and international development frequently bring with it a range of unintended consequences, both negative and positive. This book delves into these consequences, providing a fresh and comprehensive guide to understanding and addressing them. The book starts by laying out a theoretical framework based on complexity thinking, before going on to explore the ten most prevalent kinds of unintended effects of foreign aid: backlash effects, conflict effects, migration and resettlement effects, price effects, marginalization effects, behavioural effects, negative spillover effects, governance effects, environmental effects, and ripple effects. Each chapter revolves around a set of concrete case studies, analysing the mechanisms underpinning the unintended effects and proposing ways in which policymakers, practitioners, and evaluators can tackle negative side effects and maximize positive side effects. The book also includes personal testimonies, a succinct overview of unintended effects, and suggestions for further reading. Providing a clear overview of what side effects to anticipate when planning, executing, and evaluating aid, this book will be an important resource for students, development practitioners, and policymakers alike.

The curious case of the realist trial: Methodological oxymoron or unicorn? - Nielsen, Steffen Bohni; Jaspers, Sofie Østergaard; Lemire, Sebastian (Evaluation, 2023)
Realist evaluation and experimental designs are both well-established approaches to evaluation. Over the past 10 years, realist trials—evaluations purposefully combining realist evaluation and experimental designs—have emerged. Informed by a comprehensive review of published realist trials, this article examines to what extent and how realist trials align with quality standards for realist evaluations and randomized controlled trials and to what extent and how the realist and trial aspects of realist trials are integrated. We identified only few examples that met high-quality standards for both experimental and realist studies and that merged the two designs.

Causal mapping for evaluators - Powell, Steve; Copestake, James; Remnant, Fiona (Evaluation, 2023)
Evaluators are interested in capturing how things causally influence one another. They are also interested in capturing how stakeholders think things causally i...

Sense-making with stakeholders and rubrics - King, Julian (Evaluation and Value for Investment, 2023)
Principles and processes

A Journey Into Equitable Practice: Doing More, Doing Differently, and Doing Better - Bode, Bree; Panken, Sarah; Murphy, Annie; Scott, Marci (The Foundation Review, 2023)
This article describes the journey of the Research and Evaluation team at the Annie E. Casey Foundation to develop an approach that would allow us to rethink and deepen how we, as funders of research and evaluation, center equity in our practice. In particular, we explain how, through this process, we began to focus on what it means to orient research and evaluation toward participant owners and came to examine the assumptions, expectations, habits, and values that we held. These experiences have presented us with opportunities to learn and be open to new ways of engaging in our work. We illustrate what this has looked like through vignettes involving five different Casey Foundation research and evaluation projects and conclude with some of the core lessons that we have learned: View tension as an opportunity for creativity; build in and protect space and time for reflection; and develop rapport and trust to strengthen practice.

In Conversation: Two Community Foundations in Dialogue About Their Equitable Evaluation Framework™ Practice - Brandt, Madeline; Casey, Kelly; Callan, Jean-Marie; Hicks-Rivera, Joel; Leonard, Kim; Nguyen, Madeline; Ragusa, Elena Tamanas; Stancil, Cierra; Salmond, Kimberlee; Seel, Becky; Szczerbacki, Kate (The Foundation Review, 2023)
This conversation between staff at the Oregon Community Foundation and the Hartford Foundation for Public Giving shares how we are infusing the Equitable Evaluation Framework™ into our practice as we aim to be less extractive, shift power, and honor all ways of knowing and being as valid. In sharing this conversation, we want to pull the curtain back and offer a behind-the-scenes view into the conversations, realities, and challenges involved in doing this kind of work. We sat down together for 90 minutes on a Wednesday afternoon, and the following is a rough transcript of our time together. The intention is not for the article to be an exemplar, but instead to demonstrate the complexities in applying the EEF in real settings and the patience and persistence it takes for organizational (and personal) transformation. We started our conversation with these questions: 1) Why is applying the EEF principles and challenging orthodoxies important for your practice? How is this approach helping us show up authentically? 2) What does applying EEF principles/ challenging orthodoxies look and feel like for you now? How are you infusing it in your work? Where does it feel easier or harder, and why? 3) Where do you hope to go from here? What does applying the EEF look like for you in the future? Along the way, the conversation gives the opportunity to see that our practice and takeaways as Equitable Evaluation partners and practitioners vary depending on who we are and our role in the foundation. We speak to the many ways that this work can “look” and the ways that the journey of each foundation is shaped by its starting place, mission, and culture. We hope readers will see themselves within the dialogue and understand that there are many ways to start, engage in, and further this work.

The Practice of the Equitable Evaluation Framework™: Context and Introduction to the Special Issue - Coné, Marcia; Dean-Coffey, Jara (The Foundation Review, 2023)
Welcome to the special issue of The Foundation Review. For many, this is an introduction to the Equitable Evaluation Framework™, and how some folks in U.S. philanthropy are reimagining evaluation, learning, and research through its practice. For others, you’ve been in practice of the EEF alongside us and other individuals and organizations and are, thus, represented in the offerings shared from your colleagues. Over the past three years, in partnership with many, we’ve engaged in exploring, puzzling together, and sharing what it means to “be in practice of the EEF.” While impossible to convey the depth, breadth, and richness of this endeavor, we hope what follows here provides glimpses and glimmers of what is and might be possible.

Is the problem driven iterative adaptation approach (PDIA) a panacea for public financial management reform? Evidence from six African countries - Lawson, Andrew; Harris, Jamelia (World Development Perspectives, 2023)
This article assesses the application of the problem driven iterative adaptation (PDIA) approach to public financial management reform in six African countries. It draws on primary data collected using a mix of interviews, overt participation observations and a short survey. PDIA responds to shortcomings in orthodox approaches to reform and technical assistance in developing countries. It stresses local solutions to local problems, achieved through experimentation and adaptation. The principles of PDIA are appealing, but its empirical assessment is in its infancy. This study aims to fill part of this gap. Findings show that PDIA delivers results in the short-term, particularly in cases where there is an influential authorising agent and dedicated team. Progress was less forthcoming for reforms that required high level political buy-in from senior officials. The approach does exceptionally well to develop staff capability, transferable skills, and local empowerment to solve local problems, thus potentially benefitting future reforms.

Learning, Unlearning, and Sprinkling In: Our Journey with Equitable Evaluation - Mosley, Jane; Quarles, Leigh; Williams, Jason (The Foundation Review, 2023)
The Health Forward Foundation recently completed a two-year journey with the Equitable Evaluation Initiative as a practicing partner. This partnership provided us with the support to push for change that better aligned with our new focus, prioritizing racial equity and economic advancement. The partnership also allowed us to explore a number of questions fundamental to our work in learning and evaluation: what we really know about the impact philanthropy is making in our communities; how we can explain that to board members, and how we honor the personal experiences of the people we serve. In this article we discuss our journey with the initiative’s Equitable Evaluation FrameworkTM and how we experimented with some of its key concepts through a project called Stories in Power. A collection of voices detailing how members of a community have experienced their own power — or their lack of it — in making decisions that affect their lives, Stories in Power provided a concrete example of how our work can be different and helped us to lay a foundation for new ways of thinking about evaluation.

Fractal approaches to scaling transformations to sustainability - O’Brien, Karen; Carmona, Rosario; Gram-Hanssen, Irmelin; Hochachka, Gail; Sygna, Linda; Rosenberg, Milda (Ambio, 2023)
Responses to sustainability challenges are not delivering results at the scale and speed called for by science, international agreements, and concerned citizens. Yet there is a tendency to underestimate the large-scale impacts of small-scale, local, and contextualized actions, and particularly the role of individuals in scaling transformations. Here, we explore a fractal approach to scaling sustainability transformations based on “universal values.” Universal values are proposed as intrinsic characteristics that connect humans and nature in a coherent, acausal way. Drawing on the Three Spheres of Transformation framework, we consider how enacting universal values can generate fractal-like patterns of sustainability that repeat recursively across scales. Fractal approaches shift the focus from scaling through “things” (e.g., technologies, behaviors, projects) to scaling through a quality of agency based on values that apply to all. We discuss practical steps involved in fractal approaches to scaling transformations to sustainability, provide examples, and conclude with questions for future research.
Keywords: Agency; Fractals; Relational paradigms; Scaling; Three Spheres of Transformation; Universal values

Learning Circles as a Tool for Participant- Owned Evaluation - Roncaglione, Virginia; Brown, Chan; James, Jennifer; Huff, Courtney (The Foundation Review, 2023)
Learning circles are an approach where individuals with a common interest meet regularly to learn from each other about a self-identified topic in a format chosen by the group. Honoring a group’s collective wisdom, centering participants’ learning needs, and prioritizing relationships and trust are all features of learning circles. This practice is of increasing interest to funders and evaluators as a tool for practicing learning and evaluation aligned with the Equitable Evaluation Framework™. Kansas Health Foundation and its strategic learning partners, Innovation Network and Harder+Company Community Research, are exploring learning circles in two of the foundation’s initiatives: Integrated Voter Engagement and the Kansas Digital Equity and Inclusion Collaborative. The foundation further applied the learning circle approach to the evaluation of these initiatives as part of its own practice in equitable grantmaking. By doing so, it is strategically following the lead of and yielding power to initiative partners. This article shares the authors’ individual experiences and collective reflections on being in the EEF practice, striving to shift ownership of the evaluation from foundations to participants. We offer considerations on 1) the foundation’s transition to equitable grantmaking and evaluation, 2) the formation and evolution of each learning circle project, and 3) what it takes to practice this approach in line with the EEF.

A Team's Journey Toward More Equitable Philanthropic Research and Evaluation Practices - Spring, Kimberly; Mata, Maria Fernanda; Poirier, Jeffrey; Holmes, Allison; François, Amir (The Foundation Review, 2023)
This article describes the journey of the Research and Evaluation team at the Annie E. Casey Foundation to develop an approach that would allow us to rethink and deepen how we, as funders of research and evaluation, center equity in our practice. In particular, we explain how, through this process, we began to focus on what it means to orient research and evaluation toward participant owners and came to examine the assumptions, expectations, habits, and values that we held. These experiences have presented us with opportunities to learn and be open to new ways of engaging in our work. We illustrate what this has looked like through vignettes involving five different Casey Foundation research and evaluation projects and conclude with some of the core lessons that we have learned: View tension as an opportunity for creativity; build in and protect space and time for reflection; and develop rapport and trust to strengthen practice.

Whole Philantrhopy: Consciousness + Connectedness = With, not For - Chicago Beyond (2023)

Applying the River of Life Method to Support Reflection and Learning in Terre des hommes Nepal - Karki, Shanta; Giri, Roju; Neupane, Sudarshan; Snijder, Mieke; Apgar, Marina (CLARISSA Learning Note, 2023)
The RoL method is a visual narrative method that helps people tell stories of the past, present, and future. Individuals can use this method to introduce themselves in a fun and descriptive way. A group can use it to understand and reflect on the past and imagine the future of a project. Besides, it can also be used to build a shared view of a process over time while acknowledging different and perhaps contradictory perspectives. The method uses drawings rather than text, making it useful in groups that do not share a common language. Metaphors from a river are used to explore aspects of a story – such as whirlpools depicting challenges or lakes suggesting a sense of calm etc. When used in a group, it is an active method, engaging people in the process of storytelling and listening through visualising their experiences and using metaphors to explore in depth. In CLARISSA, we adapted the RoL method to document our collective understanding of the story of implementation of the programme as part of the programme’s monitoring, evaluation and learning component. The purpose was to surface the details of our process of the systemic Action Research that we are undertaking with children in the worst forms of child labour and business owners. We used the same river metaphors as is often applied when the method is used with individuals.

Gestión de la complejidad para el impacto sistémico: respuestas a entornos VUCA y BANI - Peña-López, Ismael (SociedadRed, 2023)
Research on the Information Society, the Digital Divide and Information and Communication Technologies for development

From mouthset to mindset shifts in co-creating systems change - Blomkamp, Emma; Snow, Thea; Burkett, Ingrid (Griffith Centre for Systems Innovation, 2023)

Adaptive Management Across Project Cycles: Look into Coherence in Time - Guerzovich, Florencia (Medium, 2023)
This is a series about Monitoring, Evaluating and Learning (MEL) whether sets of interventions/portfolios are adding more together than each one would produce on their own. In post 1, I pointed to coherence, the new OECD-DAC evaluation criteria as a way to bridge the ambition of bringing bigger change with the MEL world. In post 2, I shared 3 of 4 practical lessons I’ve learned in experimenting with MEL systems and exercises that focus explicitly on interactions of interventions/portfolios. In the third post, I bring Paul Pierson’s groundbreaking argument for social science to MEL. Paraphrasing, most contemporary MEL takes a “snapshot” view of interventions and portfolios, distorting their effects and meaning by ripping them from their temporal context. Instead, we should place in time interventions/portfolios with the ambition to add more than the sum of the part by constructing MEL systems looking at “moving pictures” rather than taking snapshots. ...

Three out of Four Practical Lessons about right fitting MEL for Additive Effects - Guerzovich, Florencia (Medium, 2023)
This is a series about Monitoring, Evaluating and Learning (MEL) whether sets of interventions/portfolios are adding more together than each one would produce on their own. In post 1, I pointed to coherence, the new OECD-DAC evaluation criteria as a way to bridge the ambition of bringing bigger change with the MEL world. In post 2, I shared 3 of 4 practical lessons I’ve learned in experimenting with MEL systems and exercises that focus explicitly on interactions of interventions/portfolios. In the third post, I bring Paul Pierson’s groundbreaking argument for social science to MEL. Paraphrasing, most contemporary MEL takes a “snapshot” view of interventions and portfolios, distorting their effects and meaning by ripping them from their temporal context. Instead, we should place in time interventions/portfolios with the ambition to add more than the sum of the part by constructing MEL systems looking at “moving pictures” rather than taking snapshots.

Lasting Roots: Naatal Mbay and the Integrated Finance Model in Senegal - Fowler, Ben; Courbois, Laura (MSP Ex-Post Study, 2023)
This report addresses the well-recognized evidence gap1 on the longer-term impacts created by marketdriven programming; specifically, programming influenced by market systems development (MSD) principles. It does so by presenting the findings of an ex-post study conducted three and a half years after the close of USAID’s Feed the Future Senegal Naatal Mbay Activity (hereafter Naatal Mbay) in 2019. It examines the scale and sustainability of changes resulting from Naatal Mbay’s introduction of an integrated finance model (IFM) – described in Error! Reference source not found. below – in the domestic rice sector. This study is one in a series of ex-post evaluations that are being conducted between 2023-2026 on USAID-funded MSD interventions around the world. This study focused on four questions, noted below in Figure 1. These were addressed using a mix of desk research, 122 key informant interviews with market actors and other stakeholders remotely and in Senegal, focus group discussions with 26 rice producers networks in Senegal, and a validation workshop with USAID/Senegal, implementing partner staff and market actors. Findings were analyzed leveraging the Disrupting System Dynamics (DSD) framework (see Figure 4 in the body of the report) as an analytical tool for understanding systems change.

Does the whole add more than the SUM of its parts? - Guerzovich, Florencia (Medium, 2023)
This is a series about Monitoring, Evaluating and Learning (MEL) whether sets of interventions/portfolios are adding more together than each one would produce on their own. In post 1, I pointed to coherence, the new OECD-DAC evaluation criteria as a way to bridge the ambition of bringing bigger change with the MEL world. In post 2, I shared 3 of 4 practical lessons I’ve learned in experimenting with MEL systems and exercises that focus explicitly on interactions of interventions/portfolios. In the third post, I bring Paul Pierson’s groundbreaking argument for social science to MEL. Paraphrasing, most contemporary MEL takes a “snapshot” view of interventions and portfolios, distorting their effects and meaning by ripping them from their temporal context. Instead, we should place in time interventions/portfolios with the ambition to add more than the sum of the part by constructing MEL systems looking at “moving pictures” rather than taking snapshots.

Different kinds of rubrics - King, Julian (Evaluation and Value for Money, 2023)
Generic, analytic, holistic, orbic, asterisc, escalieric, cephalopodic

Agile on steroids through the RenDanHeYi - Quintarelli, Emanuele (Boundaryless, 2023)
This article offers initial insights into the overlap, positioning, shared values, and convergences between RenDanHeYi compared to agile practices.

Rigorous Evaluation Versus Trust-Based Learning: Is This a Valid Dichotomy? - Solorzano, Brenda (The Center for Effective Philanthropy, 2023)
Social change takes time, and it is never a linear process. It also happens within an ecosystem, and rarely comes down to a single grantee partner or foundation. If we […]

Using a ‘Partnership Rubric’ in Participatory Evaluations - Snijder, Mieke; Hicks, Jacky; Paul, Sukanta; Arulanantham, Amit; Apgar, Marina; Afroze, Jiniya; Karki, Shanta; Mareschal, Sophie; Prieto Martin, Pedro; Uddin, Forhad; Veitch, Helen (CLARISSA Learning Note, 2023)
Programmes that aim to tackle complex societal issues, such as the worst forms of child labour, require rich partnerships that bring together different perspectives. CLARISSA’s consortium partnership adopts an empowerment approach to the interventions we deliver and our ways of working together. Part of this approach involves ongoing reflection and learning about how we work together in our partnership, and how this can be adapted if needed. This learning note focuses on a method used in CLARISSA to both reflect on and strengthen how we work in partnerships – the partnership rubric. We found that using the rubric flexibly was key to mitigating some of the challenges of such a complex consortium. This included using it in different sizes of forum, with different levels of preparation. Periodically adapting it for country context and as new partners came on board also helped ensure a shared sense of our preferred ways of working as the project progressed.

What Does Evaluation Look Like in a Trust-Based Context? - Trust-Based Philanthropy Project (TBP Project, 2023)
The fifth session of the Demystifying Trust-Based Philanthropy webinar series took place on Wednesday, July 12th 2023. There is a misperception that trust-based philanthropy is at odds with rigorous impact evaluation because of its focus on relationships over project-specific metrics. Trust-based evaluation is indeed rigorous, but it differs from the sector standard in that it prioritizes a learning stance over a proving stance. This ultimately helps illuminate a more nuanced understanding of the barriers and catalyzers of social change, as well as useful information that can help grantmakers improve their strategies toward advancing impact. This session unpacked these important nuances and highlighted examples of how grantmakers are relying on rigor and continuous learning to understand impact. Special thanks to our speakers for this session: Brenda Solorzano, CEO, Headwaters Foundation Sarah Walczyk, Executive Director, Satterberg Foundation Shari Turitz, Vice President for Programs, American Jewish World Service Gratitude to our philanthropic-serving organization partners for bringing this session and discussion space to their grantmaker members. Learn more about the Trust-Based Philanthropy Project: Website: https://www.trustbasedphilanthropy.org/ Follow us on Twitter:   / trustbasedphil Follow us on LinkedIn:   / trust-based-philanthropy

Fast Forwarding to Systems Maps of Corruption: Getting to Usable Analysis More Quickly - Woodrow, Peter (CJL, 2023)
For AC practitioners, systems mapping is essential but also difficult. To lessen obstacles, Peter Woodrow proposes a scaffolded approach. An Experiment in “Fast Forwarding” Drawing upon our near-decade work on corruption, we recently decided to try an experiment: we would present “common patterns” of corruption as tentative models to adapt and add to—rather than try to teach people to do systems mapping from scratch. In this teaching experiment, each common pattern would function as a kind of “scaffolding” or framework that participants can build on to generate a cogent systems map as the basis for identifying possible points of intervention and subsequent program planning.

Theory of change software - Better Evaluation (2023)
There are a number of options when it comes to using software to help create a logic model. These range from generic word processing tools (Word, Powerpoint, or their Google Doc or Mac equivalents), to software that has been specifically tailored for visualising Theories of Change, like TOCO or Miradi. You should consider what resources you have to invest in software, both in terms of cost and in time to learn and use the features. If you only have a short timeframe and have simple needs, then a basic tool may suit you better than some of the more complex software available. It's important to investigate a few options and see what is going to be best for you.

Practitioners Guidance to Assessing Systems Change: Co-Authors Preview - Gover, Dun; Nasution, Zulka; Okutu, David; Bolder, Meghan; Henao, Lina (USAID, 2023)
Check out this video to see what’s inside our new resource: Practitioners' Guidance to Assessing Systems Change, developed by MEL Managers for MEL Managers. (Check out the Guidance here https://bit.ly/MSPMELClinics.) Hear from the authors about which parts they love the most and how this guide challenges MEL managers to assess systems change as an ongoing aspect of implementation, generating feedback that teams need to better understand and catalyze change, for more impact.

Guide to Innovation Partnerships – A United Nations Ghide - Halse, Michelle; Ginsberg-Fletcher, Gabriella; Balbi, Luisa (UN Global Pulse, 2023)

Applying Adaptive Management in a Fragile Context – Case Study - Lonsdale, Jane; Green, Duncan; Robertson, Kelly (DT Global, 2023)
DT Global is proud to introduce our new Guidance Note: Practical Introduction to Adaptive Management There is a growing consensus around adaptive management as an effective (even necessary) approach when programs are tackling complex development problems. While there is no standard definition of adaptive management, there is general agreement that such programs need to routinely engage with and respond to program context; constantly test what works in that context; and adjust approaches, plans, and activities based on continuous learning. However, there remains a more limited body of evidence about what this looks like in practice—the enabling conditions, systems, resourcing, skills, and attitudes to effectively operationalise adaptive management. There is also limited guidance around when adaptive management is required, and to what extent—both critical and often overlooked considerations when planning for successful adaptive management. This Guidance Note draws together lessons and good practice in adaptive management from across DT Global’s diverse portfolio of donor-funded programs. It outlines our conceptual framework for adaptive management, with practical guidance on how it can be applied by our program teams. It is also designed to help our teams distinguish adaptive management from good (non adaptive) project management, consider when adaptive management is most useful on a program, and how adaptive a program (or part of a program) should be.

Applied political economy analysis for human rights programs and campaigns: A guide for practitioners (second edition) - Pact (Pact, 2023)
This updated guide provides practical guidance to practitioners in the human rights sector and beyond on how to integrate Applied Political Economy Analysis

Knowledge Retention and Transfer (KRT) - Model overview presentation - USAID (USAID, 2023)
What problem is the KRT model trying to solve? For workforces that experience continuous staff turnover, the lack of systematic knowledge transfer can often lead to: - Loss of programmatic momentum, - Duplication of efforts and frustration, and - Wasted time and resources. The Knowledge Retention and Transfer (KRT) model provides tools, processes, and practices to individuals, teams, offices, and organizations to improve knowledge handover, which in turn improves efficiency and programmatic and operational learning.

Evaluative reasoning in complexity - King, Julian (Evaluation and Value for Investment, 2023)

When Complex is as Simple as it Gets: Guide for Recasting Policy and Management in the Anthropocene - Roe, Emery (IDS Working Paper, 2023)
Many readers recognise and understand that complex is about as simple as it gets for major policy and management. This guide is for those unwilling in the Anthropocene to shrink back into the older platitudes about ‘keep it simple’ and ‘not to worry, we’ll scale up the analysis later on’. This guide offers key concepts, methods, counternarratives, and analogies that recast major policy and management issues in ways that do not deny their complexity but help render them more tractable for action.

How to Build Movements with Cyclical Patterns in Mind - Non Profit News | Nonprofit Quarterly - Algoso, Dave; Guerzovich, Florencia; Gattoni, Soledad (Nonprofit Quarterly, 2023)
The world changes too much for anyone who is invested in social change work to imagine that this work is linear and predictable. Opportunities come and go, whether caused by a pandemic or political shifts. This much most social movement leaders and activists intuitively understand. But what can be done with this realization? How might movement groups better prepare for moments of opportunity? We want to explore how we can create the changes we want to see by responding to the changes that are outside our control.

Mechanisms of Techno-Moral Change: A Taxonomy and Overview - Danaher, John; Sætra, Henrik Skaug (Ethical Theory and Moral Practice, 2023)
The idea that technologies can change moral beliefs and practices is an old one. But how, exactly, does this happen? This paper builds on an emerging field of inquiry by developing a synoptic taxonomy of the mechanisms of techno-moral change. It argues that technology affects moral beliefs and practices in three main domains: decisional (how we make morally loaded decisions), relational (how we relate to others) and perceptual (how we perceive situations). It argues that across these three domains there are six primary mechanisms of techno-moral change: (i) adding options; (ii) changing decision-making costs; (iii) enabling new relationships; (iv) changing the burdens and expectations within relationships; (v) changing the balance of power in relationships; and (vi) changing perception (information, mental models and metaphors). The paper also discusses the layered, interactive and second-order effects of these mechanisms.

CLARISSA Cash Plus: Innovative Social Protection in Bangladesh - Roelen, Keetie; Howard, Neil; Afroze, Jiniya; Aktar, Afrin; Ton, Giel; Huq, Lopita (CLARISSA Design Note, 2023)
Social protection, and cash transfers especially, have been found to have many positive impacts on families’ lives and are now widely recognised as a cornerstone of any prosperous, fair society. The CLARISSA Cash Plus intervention is an innovative social protection scheme for tackling social ills, including the worst forms of child labour (WFCL). Combining community mobilisation, case work and cash transfers, it aims to support people in a poor neighbourhood in Dhaka to build their individual, family, and group capacities to meet their needs. An increase in capacities is expected to lead to a corresponding decrease in deprivation and community-identified social issues that negatively affect wellbeing, including WFCL.

Participation for Humanitarian Innovation - Background Paper - Smith, Amy; Thompson, Martha; Saida Benhayoune; Crespo Cardona, Omar (Elrha, 2023)
A resource designed to help organisations, teams and individuals manage innovation journeys responsibly and successfully. We have partnered with MIT D-Lab to develop a new resource to drive greater diversity and inclusion within project design and implementation. The Participation for Humanitarian Innovation (PfHI) toolkit sets out a robust approach to setting expectations for and monitoring the degree of participation within research and innovation projects for, with, and by people affected by crisis. The PfHI toolkit is composed of five tools: Opportunity Adviser: Identify and prioritise the desired benefits of participation while weighing the potential barriers. Participation Matrix: Agree on the precise degree of participation to target at a given project stage. Resource Navigator: Select tools and processes to address the needs of stakeholders, the project objectives and context. Quality Guidance: Ensure engagements are delivered to the highest standard. Assessment Matrix: Collectively evaluate the degree of participation achieved during an activity or project phase, learn and adapt. By applying these tools before, during and after a research and/or innovation project, implementors can ensure that stakeholders/end-users are included and participating at the highest possible degree. We encourage users of the PfHI toolkit to consider how else to integrate the tools into existing practices. For instance, the Evaluation Matrix could be used to supplement existing MEAL activities to: Establish baselines to measure changes in participation over time. Track the degree of participation/engagement of stakeholders. Regularly assess the degree of participation. Seek feedback from participants about the degree of their participation. Monitor the progress of activities related to stakeholder engagement. Our Participation for Humanitarian Innovation toolkit represents our ongoing commitment to responsible research and innovation across our portfolio of grants and for the humanitarian sector more broadly. We hope you will consider downloading and using the toolkit on your next project.

Participation for Humanitarian Innovation - Toolkit - Smith, Amy; Thompson, Martha; Saida Benhayoune; Crespo Cardona, Omar (Elrha, 2023)
A resource designed to help organisations, teams and individuals manage innovation journeys responsibly and successfully. We have partnered with MIT D-Lab to develop a new resource to drive greater diversity and inclusion within project design and implementation. The Participation for Humanitarian Innovation (PfHI) toolkit sets out a robust approach to setting expectations for and monitoring the degree of participation within research and innovation projects for, with, and by people affected by crisis. The PfHI toolkit is composed of five tools: Opportunity Adviser: Identify and prioritise the desired benefits of participation while weighing the potential barriers. Participation Matrix: Agree on the precise degree of participation to target at a given project stage. Resource Navigator: Select tools and processes to address the needs of stakeholders, the project objectives and context. Quality Guidance: Ensure engagements are delivered to the highest standard. Assessment Matrix: Collectively evaluate the degree of participation achieved during an activity or project phase, learn and adapt. By applying these tools before, during and after a research and/or innovation project, implementors can ensure that stakeholders/end-users are included and participating at the highest possible degree. We encourage users of the PfHI toolkit to consider how else to integrate the tools into existing practices. For instance, the Evaluation Matrix could be used to supplement existing MEAL activities to: Establish baselines to measure changes in participation over time. Track the degree of participation/engagement of stakeholders. Regularly assess the degree of participation. Seek feedback from participants about the degree of their participation. Monitor the progress of activities related to stakeholder engagement. Our Participation for Humanitarian Innovation toolkit represents our ongoing commitment to responsible research and innovation across our portfolio of grants and for the humanitarian sector more broadly. We hope you will consider downloading and using the toolkit on your next project.

Transformative Communication for Empowerment: Silence and Participatory Development in a Marginalized Community in Nueva Ecija, Philippines - Lejarde, Valerie Anne M. (University of the Philippines - Open University, 2023)
This study explored how the Outreach Philippines, Inc. (OPI) implements the participatory human development (PHD) approach using transformative communication, in a community-based organization (CBO), Lawag ti Caridad Norte Association (LCNA), in Caridad Norte, Llanera, Nueva Ecija. A phenomenological lens examined the lived experiences of six participants interviewed on December 10 and 13, 2022, and through my own reflections as a community development worker of OPI. Specifically, I analyzed the CBO leaders’ perception of silence, transformative communication engagements in the PHD process, and changes among the stakeholders. The PHD process itself is transformative communication that allows the mutual development of both the CBO leaders and the PHD facilitator. The PHD process involves and establishes consistent dialogic communication and collaborative actions between the facilitator and the community members, hence transforming both into more empowered agencies. But transformative communication must consider different constructs of cultural silence in engaging the community, especially in facilitating participatory development.

USAID ADS 201 - Operational Policy for the Program Cycle (Update 05/22/2023) - USAID (USAID, 2023)
The Program Cycle is USAID’s operational model for planning, delivering, assessing, and adapting development programming in a given region or country to advance U.S. foreign policy. It encompasses guidance and procedures for: 1) Making strategic decisions at the regional or country level about programmatic areas of focus and associated resources; 2) Designing supportive projects and/or activities to implement these strategic plans; and 3) Learning from performance monitoring, evaluations, and other relevant sources of information to make course corrections as needed and inform future programming.

The Equitable Evaluation Framework - May 2023 Expansion - EEI (Equitable Evaluation Inititiative, 2023)

The adoption of innovation in international development organisations: lessons for development co-operation - Kumpf, Benjamin; Jhunjhunwala, Parnika (OECD Development Co-operation Working Papers, 2023)
Addressing 21st century development challenges requires investments in innovation, including the use of new approaches and technologies. Currently, many development organisations prioritise investments in isolated innovation pilots that leverage a specific approach or technology rather than pursuing a strategic approach to expand the organisation’s toolbox with innovations that have proven their comparative advantage over what is currently used. This Working Paper addresses this challenge of adopting innovations. How can development organisations institutionalise a new way of working, bringing what was once novel to the core of how business is done? Analysing successful adoption efforts across five DAC agencies, the paper lays out a proposed process for the adoption of innovations. The paper features five case-studies and concludes with a set of lessons and recommendations for policy makers on innovation management generally, and adoption of innovation in particular.

Portfolio approach - Primer: Rethinking policy & development in times of uncertainty - UNDP (2023)

Participatory action research - Cornish, Flora; Breton, Nancy; Moreno-Tabarez, Ulises; Delgado, Jenna; Rua, Mohi; de-Graft Aikins, Ama; Hodgetts, Darrin (Nature Reviews Methods Primers, 2023)
Participatory action research (PAR) is an approach to research that prioritizes the value of experiential knowledge for tackling problems caused by unequal and harmful social systems, and for envisioning and implementing alternatives. PAR involves the participation and leadership of those people experiencing issues, who take action to produce emancipatory social change, through conducting systematic research to generate new knowledge. This Primer sets out key considerations for the design of a PAR project. The core of the Primer introduces six building blocks for PAR project design: building relationships; establishing working practices; establishing a common understanding of the issue; observing, gathering and generating materials; collaborative analysis; and planning and taking action. We discuss key challenges faced by PAR projects, namely, mismatches with institutional research infrastructure; risks of co-option; power inequalities; and the decentralizing of control. To counter such challenges, PAR researchers may build PAR-friendly networks of people and infrastructures; cultivate a critical community to hold them to account; use critical reflexivity; redistribute powers; and learn to trust the process. PAR’s societal contribution and methodological development, we argue, can best be advanced by engaging with contemporary social movements that demand the redressingl of inequities and the recognition of situated expertise.

A new pathway: how can funders support meaningful monitoring, evaluation, and learning practice in the field? - Acevedo, Andrea; Colnar, Megan (Better Evaluation, 2023)
How can donors and grantees work together to create effective monitoring, evaluation, and learning (MEL) practices that drive field-wide transformation?

Building Capacity for Strategic Innovation: an Emerging Competency Framework for Portfolio Work - Begovic, Milica; Colville, Jenniffer; Quaggiotto, Giulio; Vester, Søren; Naatujuna, Deborah; Oprunenco, Alex; Sadiku; Lejla; Uusikyla, Ida; Uriartt, Simone; Zorouali, Kawtar (Medium, 2023)
In a previous post we reflected on how a key learning from our “project to portfolio” journey to date is that it is ultimately about mustering the organizational will to transform. If the early days of our innovation work were about demonstrating results quickly and creating space for experimentation, now the challenge is of a different order. Eventually this means helping UNDP transition to a different value proposition and business model, as eloquently articulated by Gerd Trogemann: “No matter how well conceived and relevant in their own right, projects tend to pursue single point rather than systemic solutions, limit strategic space and the ability to adapt continuously and to connect the dots systemically. Systemic solutions need adaptive ways of working, strategic space, iterative learning, and radical collaboration.”

Revealing the Relational Mechanisms of Research for Development Through Social Network Analysis - Apgar, Marina; Fournie, Guillaume; Haesler, Barbara; Higdon, Grace Lyn; Kenny, Leah; Oppel, Annalena; Pauls, Evelyn; Smith, Matthew; Snijder, Mieke; Vink, Daan; Hossain, Mazeda (The European Journal of Development Research, 2023)
Achieving impact through research for development programmes (R4D) requires engagement with diverse stakeholders across the research, development and policy divides. Understanding how such programmes support the emergence of outcomes, therefore, requires a focus on the relational aspects of engagement and collaboration. Increasingly, evaluation of large research collaborations is employing social network analysis (SNA), making use of its relational view of causation. In this paper, we use three applications of SNA within similar large R4D programmes, through our work within evaluation of three Interidsiplinary Hubs of the Global Challenges Research Fund, to explore its potential as an evaluation method. Our comparative analysis shows that SNA can uncover the structural dimensions of interactions within R4D programmes and enable learning about how networks evolve through time. We reflect on common challenges across the cases including navigating different forms of bias that result from incomplete network data, multiple interpretations across scales, and the challenges of making causal inference and related ethical dilemmas. We conclude with lessons on the methodological and operational dimensions of using SNA within monitoring, evaluation and learning (MEL) systems that aim to support both learning and accountability.

Evaluating Research for Development: Innovation to Navigate Complexity - Apgar, Marina; Snijder, Mieke; Higdon, Grace Lyn; Szabo, Sylvia (The European Journal of Development Research, 2023)
Large publicly funded programmes of research continue to receive increased investment as interventions aiming to produce impact for the world’s poorest and most marginalized populations. At this intersection of research and development, research is expected to contribute to complex processes of societal change. Embracing a co-produced view of impact as emerging along uncertain causal pathways often without predefined outcomes calls for innovation in the use of complexity-aware approaches to evaluation. The papers in this special issue present rich experiences of authors working across sectors and geographies, employing methodological innovation and navigating power as they reconcile tensions. They illustrate the challenges with (i) evaluating performance to meet accountability demands while fostering learning for adaptation; (ii) evaluating prospective theories of change while capturing emergent change; (iii) evaluating internal relational dimensions while measuring external development outcomes; (iv) evaluating across scales: from measuring local level end impact to understanding contributions to systems level change. Taken as a whole, the issue illustrates how the research for development evaluation field is maturing through the experiences of a growing and diverse group of researchers and evaluators as they shift from using narrow accountability instruments to appreciating emergent causal pathways within research for development.

Theory of Change in complex Research for Development programmes: challenges and solutions from the Global Challenges Research Fund - Chapman, Sarah; Boodhoo, Adiilah; Duffy, Carren; Goodman, Suki; Michalopoulou, Maria (The European Journal of Development Research, 2023)
The United Kingdom Research and Innovation (UKRI) Global Challenges Research Fund (GCRF) aimed to address global challenges to achieve the United Nations (UN) Sustainable Development Goals through 12 interdisciplinary research hubs. This research documents key lessons learned around working with Theory of Change (ToC) to guide Monitoring, Evaluation and Learning (MEL) within these complex research for development hubs. Interviews and document reviews were conducted in ten of the research hubs. The results revealed that only one hub invested in an explicit visual system mapping approach, and that funder timelines, budget constraints and issues with capacity and expertise limited the application of these approaches across all hubs. In contrast, many hubs attempted to deal with visual complexity by means of ether constructing multiple, nested ToCs, or a conscious simplification of complexity through reducing their ToC towards a straightforward and uncomplicated chain model or spherical model. While the former approach had some value, most hubs struggled to find capacity to support the full articulation of nested ToCs. In contrast, the latter approach resulted in ToCs which lacked detail or mechanism articulation, but which nevertheless were often ‘fit for purpose’ in ensuring effective communication and coherence across diverse stakeholders and sub-projects. We conclude that in instances where the reporting, funding and management cycles of complex research for development programmes cannot be adapted to properly support learning-based approaches to ToC development, imposing simplicity in the ToC might be fit for purpose. This might also be preferable to more complex visual approaches that are only partially realised.

Cost–Benefit Analysis (CBA) or the Highway? An Alternative Road to Investigating the Value for Money of International Development Research - Peterson, Heidi (The European Journal of Development Research, 2023)
Research for development (R4D) funding is increasingly expected to demonstrate value for money (VfM). However, the dominance of positivist approaches to evaluating VfM, such as cost-benefit analysis, do not fully account for the complexity of R4D funds and risk undermining efforts to contribute to transformational development. This paper posits an alternative approach to evaluating VfM, using the UK’s Global Challenges Research Fund and the Newton Fund as case studies. Based on a constructivist approach to valuing outcomes, this approach applies a collaboratively developed rubric-based peer review to a sample of projects. This is more appropriate for the complexity of R4D interventions, particularly when considering uncertain and emergent outcomes over a long timeframe. This approach could be adapted to other complex interventions, demonstrating that our options are not merely “CBA or the highway” and there are indeed alternative routes to evaluating VfM.

How are Research for Development Programmes Implementing and Evaluating Equitable Partnerships to Address Power Asymmetries? - Snijder, Mieke; Steege, Rosie; Callander, Michelle; Wahome, Michel; Rahman, M. Feisal; Apgar, Marina; Theobald, Sally; Bracken, Louise J.; Dean, Laura; Mansaray, Bintu; Saligram, Prasanna; Garimella, Surekha; et al. (The European Journal of Development Research, 2023)
The complexity of issues addressed by research for development (R4D) requires collaborations between partners from a range of disciplines and cultural contexts. Power asymmetries within such partnerships may obstruct the fair distribution of resources, responsibilities and benefits across all partners. This paper presents a cross-case analysis of five R4D partnership evaluations, their methods and how they unearthed and addressed power asymmetries. It contributes to the field of R4D partnership evaluations by detailing approaches and methods employed to evaluate these partnerships. Theory-based evaluations deepened understandings of how equitable partnerships contribute to R4D generating impact and centring the relational side of R4D. Participatory approaches that involved all partners in developing and evaluating partnership principles ensured contextually appropriate definitions and a focus on what partners value.

Laying the Foundations for Impact: Lessons from the GCRF Evaluation - Vogel, Isabel; Barnett, Chris (The European Journal of Development Research, 2023)
Research for development (R4D) aims to make a tangible difference to development challenges, but these effects typically take years to emerge. Evaluation (especially impact evaluation) often takes place before there is evidence of development impact. In this paper, we focus on opportunities for assessing the potential for impact at earlier stages in the research and innovation process. We argue that such a focus can help research programme managers and evaluators learn about the pre-conditions for impact and adjust accordingly. Using the Global Challenges Research Fund (GCRF) as a large-scale case of R4D evaluation, we identify and explore some of the building blocks that can increase impact potential. Guided by GCRF’s theory of change, we explore emerging evidence that highlights the importance of ways of working that supports positioning for impact. We conclude by drawing out a unifying construct around standards of development excellence; to sit alongside notions of scientific excellence for research intended to have an impact. Standards can help programme managers, researchers and evaluators learn and adapt to increase the likelihood of impact.

The Last 10 Per Cent: Why the World Needs a Leaner, More Innovative and Pragmatic Development Sector, Today - Harper, Erica (Routledge India, 2023)
Criticism that the development sector has not delivered in terms of eliminating extreme poverty, fast-tracking growth and preventing conflict, is neither new nor surprising. In fact, it may be the one thing that scholars, donors and practitioners agree on. While many of these concerns are valid, this book makes a case that the sector is closer to unlocking the gates to more effective and efficient development outcomes than is popularly believed. Specifically, it argues that by overturning a few myths, making better use of evidence and employing some different rules, practitioners, policy specialists and donors can foster the changes in the development architecture that are needed to reach the 10 percent of the world’s population still living in extreme poverty.Engaging, provocative and clear sighted, the book provides insight into interventions around democratic governance, refugee response, counterterrorism, gender mainstreaming, environmental protection and private sector engagement. It is instructive reading for professionals across the development sector, think tanks and NGOs.

Berkana Two Loop Model: A quick introduction to an accessible model for systems change - Proud, Emma (LinkedIn - Emma Proud, 2023)
A model I've been using a lot recently, to bring systems thinking to life, is the Berkana Two Loop model. It doesn't describe complexity or systems thinking. But it does describe systems change in a way that's simple and oriented to action.

Process Tracing Innovations in Practice: Finding the Middle Path - Aston, Thomas; Wadeson, Alix (CDI Practice Paper, 2023)
Evaluation practitioners in the international development sector have given considerable attention in recent years to process tracing as a method for evaluating impact, including discussion of how to assess the relative importance of causal factors. Despite the increasing interest, there is a relative dearth of examples of practical learning and evidence of applying process tracing in practice. This CDI Practice Paper draws on comparative learning from applying three different types of process tracing in international development initiatives. It argues in favour of a ‘middle path’ of applying evidence tests and rubrics to structure evaluative judgements rather than formal Bayesian updating or looser forms of process tracing. It also calls attention to the potential added value of taking a participatory approach, offering practical recommendations for how to do this effectively.

The Secret Sauce of Development Professionals: Tools for Assessing TOR Potential to Source Scalable Learning Interventions - Barton, Adam (Research on Improving Systems of Education (RISE), 2023)
Terms of reference (TORs) play an outsized role in driving scalable educational programming. These procurement documents shape, constrain, and signal programme priorities and possibilities. Successful funders and implementers across the globe hold rich processual knowledge about this documentation, which they use to draft and assess TORs. This project explores such best-practice knowledge around TOR review, seeking to support the design and implementation of educational programmes that can improve learning at scale in developing contexts.

Reflections on Ten Years of USAID’s Experience with Political Economy Analysis and Thinking and Working Politically - Brinkerhoff, Derick; Cassidy, Marc (TWP CoP, 2023)
The global Thinking and Working Politically (TWP) Community of Practice (CoP), the Washington DC TWP CoP, and USAID organised a webinar on 5 December 2022 to take stock of how USAID and its partners have used PEA to inform programme strategy, design and implementation, and support TWP. This paper synthesises the key points arising from the webinar, including observations on the impacts, opportunities, challenges, and prospects for PEA/TWP to become more deeply adopted and sustained as a development methodology and approach across sectors. It starts by defining key concepts. It then highlights insights from the discussions of the impact of the application of PEA and TWP principles across sectors. The paper concludes by looking at progress achieved to date, as well as constraints and opportunities to increase the uptake of both thinking and working politically in USAID-sponsored programming.