Understanding stakeholders and political networks - Practical Guidance Note - Piron, Laure-Hélène; Mwamba, Wilfred (FCDO and TWP CoP, 2026)
In 2023, the UK Foreign, Commonwealth & Development Office and the Thinking and Working Politically Community of Practice published Understanding political economy analysis and thinking and working politically. This complementary note sets out more detailed guidance on how to undertake a dynamic Stakeholder Analysis and Political Network Mapping, which are core elements of political economy analysis and other forms of politically informed context analysis. This technical note is for use alongside the UK Foreign, Commonwealth & Development Office (FCDO) guide to understanding political economy analysis (PEA) and thinking and working politically and is part of a series of guides on the use of PEA in practice. It sets out more detailed guidance on how to undertake a dynamic stakeholder analysis and political network mapping, both of which can be used to support international cooperation and development partnerships. Stakeholder analysis considers the key actors relevant to an area of work, political network mapping goes further, analysing the relationships between actors. Both tools help practitioners move from a general understanding towards specific, evidence-based decisions. These approaches can contribute to the design and implementation of a strategy, policy dialogue or development programme, and to the management of risks This guide has been produced jointly with the Thinking and Working Politically Community of Practice (TWP CoP). The TWP CoP is a global network of practitioners and researchers in international cooperation committed to helping practitioners understand how change happens and why. The TWP CoP Secretariat has been hosted by the University of Birmingham with support from FCDO.

Regenerative Evaluation - Influences, Pathways & Practices - Boone, Katie; Collins, Kirsten Bording; Kaiser, Jannik; Martins, Laila; Reppart, Mateah Spencer; Strasser, Tim; Van den Daele, Geetha (Uniity Effect, 2026)
Regenerative Evaluation: Evaluation in Service of Life To foster change today means navigating a gap between two realities: the reality of reporting (demanding accountability and proof of impact) and the reality of the planet (calling for regeneration, complexity, and interconnectedness). For too long, evaluation has forced a choice, functioning as an extractive exercise—mining data to count the "fruits" of a project while ignoring the health of the "soil" that sustains them. Regenerative Evaluation bridges this gap. Moving beyond extractive measurement, it is an evolving practice that embraces rigor while actively nourishing the systems it assesses. This document outlines its core components: ● The 9 Guiding Principles: The internal values and mindsets that orient our judgment and align our work with living systems. ● The 7 Core Practices: A practical, three-phased cycle of specific methods and habits that translate these principles into tangible evidence and action. ● The 3 Essential Evaluation Strands: The evaluation lineages (systemic, power-conscious, and ecological) that this practice weaves together. ● Applications and Future Pathways: Entry points for bringing this work to life across projects, organizational vitality, funding, and policy.

In-between Spaces - Intermediaries Rethinking Funding Practices. A learning pap er on Funding and Entrepreneurship Supp ort for Adaptation & Resilience - CAIL (Climate Adaptation Innovation Learning and SILP, 2026)
This learning paper on funding and entrepreneurship support for adaptation and resilience is a collaboration between SILP and the Climate Adaptation Innovation & Learning Initiative project. It brings together real-world insights from a community of 25 practitioners working across intermediary roles in the climate adaptation and resilience sector. Rather than presenting a fixed model, the paper explores how intermediaries operate in the “in-between spaces”, navigating tensions between funders and communities, certainty and emergence, delivery and learning. It highlights the practices already being used in these spaces, and what is needed to enable more effective, equitable, and adaptive transformation. We’ve designed this paper as a practical tool — offering insights to challenge assumptions, open dialogue, and support reflection within your own organisation or community.

Letting Go, Without Losing Coherence: How Local Agency and Donor Accountability Can Coexist - Guerzovich, Florencia (TWP CoP, University of Birmingham, 2026)
Despite widespread endorsement of locally led development, donor agencies still struggle to translate local agency into cumulative change. Donor discourse increasingly emphasises localisation and community leadership, yet operational arrangements remain structured around upward compliance. The practical question is how local agency and institutional accountability can work together so that locally generated insights lead the way while portfolio coherence is maintained. This paper examines a form of connective work that helps make this co-existence possible. Drawing on the author’s more than 20 years’ experience as a practitioner and evidence from over 150 documented cases across sectors and regions, it focuses on invisible weavers – actors who keep cooperation moving under constraint by connecting fragmented institutions, routing insights across levels, and sustaining continuity when conditions change. These functions are often essential to maintaining cooperation, insights, and continuity across fragmented actors and institutions, yet they remain largely unrecognised in formal programme design, costing, and evaluation. Despite this, donors’ informal routines allow them to act on these insights, albeit in ad hoc ways. The paper addresses three practical questions: how such actors can be recognised, how their effectiveness and resilience can be assessed, and how donors can support this connective work more intentionally without formalising it into compliance. It traces how donor staff and local counterparts use judgement, continuity, and low-overhead practices to carry locally generated insights into organisational decisions. By reframing accountability as portfolio reliability and making these patterns visible, the paper shows that donors can ‘let go’ of rigid control without losing systemic coherence. It concludes with three practical ways to make these existing ‘workarounds’ more intentional and fundable, ensuring that local intelligence leads to durable change rather than having to re-set with every project cycle.

The evaluator’s guide to generative AI - Perry, Chris; Moses, Michael; Pinter, Max (Itad, 2026)
AI’s role in evaluation: so long and thanks for all the synthesis How generative AI can enhance evaluation and analysis? We share three real-world case studies to explore what works, what doesn’t, and how to harness AI responsibly. Generative AI is everywhere—but is it really the answer to Life, the Universe and Everything? At Itad, we see AI as a powerful assistant, not a replacement for human judgment. Our new learning brief, The evaluator’s guide to generative AI, shares lessons from three real-world evaluations where we piloted AI tools to support rigorous analysis. Real-world AI use to support our work In our evaluation of the Wellcome Trust’s Climate Impacts Awards, we used Microsoft 365 Copilot to extract and synthesise data from hundreds of grant proposals. This freed up our team to focus on interpretation and recommendations. For the MacArthur Foundation’s Big Bet on Nigeria programme, we used CoLoop AI to transcribe and translate interviews, speeding up data processing. However, we found it needed careful handling to support nuanced data analysis. And in our work with a partner that supports organisations throughout Africa to generate evidence and data, we combined Copilot, ChatGPT and MaxQDA’s AI tools to analyse nearly 450 grantee reports and interviews. Each case offered valuable insights into where AI adds value and where human expertise remains essential. Lessons for rigorous AI use Across these projects, we identified seven key lessons for using AI in evaluation: 1. AI is great at extraction; humans are great at sensemaking AI tools excel at structuring large volumes of data and surfacing patterns. But interpretation, trade-offs and conclusions must come from people. In the Wellcome case, Copilot helped us quickly synthesise proposal data, but our team’s judgment was still vital to make sense of the findings. 2. Break the work into smaller steps Complex tasks can overwhelm AI. Better results can be gained by breaking analysis into clear stages, such as defining a framework, extracting evidence, classifying responses and synthesising findings. This approach improves accuracy and makes verification easier. 3. Design prompts around your framework The quality of AI output depends on how you ask the question. Structured prompts aligned to evaluation frameworks, with clear definitions and examples, helps AI stay on-topic and produce more consistent results. 4. Always verify the results Even when AI seems accurate, it’s essential to check. Build in spot-checks and scoring to assess completeness, relevance, and accuracy. In the Wellcome case, manual review of 15% of proposals confirmed high fidelity. In other cases, verification helped us identify and correct oversimplifications. 5. Limit the context for nuanced analysis AI performs better when given focused tasks. In the MacArthur evaluation, CoLoop struggled with broad thematic analysis. We found that narrowing the scope to analyse one theme or subset at a time leads to more meaningful insights. 6. Choose the right tool for the job Different tools have different strengths, and matching the right tool to the task is key. Our experience suggests that ChatGPT is best for contextual analysis, Copilot works well for secure document review and MaxQDA’s Tailwind supported synthesis of coded data. 7. Document your process To build confidence in the use of AI, a clear audit trail is essential. Keeping records of prompts, outputs and verification steps helped us build trust with clients, support internal learning and ensure transparency. Looking ahead: scaling AI in evaluation The next challenge is scaling AI use without losing rigour. This means using AI to reduce manual effort, such as document triage or evidence extraction, while keeping humans in the loop for interpretation and decision-making. At Itad, we’re building bespoke platforms that embed AI into our workflows. For example, one new tool we’re developing for a philanthropic partner organisation will help surface insights from five years of grantee reports. We’re also investing in an internal platform to make our best AI practices available across all projects, laying the groundwork for new service areas including foresight and scenario planning. As we explore these frontiers, we’re guided by a simple truth: AI can help us ask better questions and work more efficiently—but it’s still up to us to make sense of the answers.

Quality Guidance for Contribution Analysis in Practice - Ton, Giel; Delahais, Thomas; Koleros, Andrew; Apgar, Marina (IEG, The World Bank, 2026)

Technically Strong and Politically Savvy: Enhancing Thinking and Working Politically when practicing the consevation standards at USAID - Schuttenberg, H.Z.; Stark, J.; Schueler, K. (USAID, 2026)
This supplemental guide offers practical advice for thinking and working politically (TWP) when practicing the Conservation Standards. The CS are a widely used, systematic approach for planning, monitoring, and adapting programs aiming to conserve biodiversity. TWP is an approach to international development that brings greater attunement to the political dynamics of the local programming context in order to enhance development results. Given strong evidence that political factors overwhelmingly drive the loss of biodiversity, this guide is based on the premise that conservation programs will be more effective when they skillfully integrate TWP.

Scaling democratic innovations: Features of Effective Catalyst Organisations & Future Frontiers - Claudia, Chwalisz; McKinney, Sammy (DemocracyNext, 2026)
Citizens' assemblies and other democratic innovations are spreading around the world. But they do not spread by themselves. Behind every successful scaling story sits a constellation of organisations doing the essential, often invisible work of building capacity, establishing networks, advocating with decision makers, and ensuring quality standards. These are what we call scaling catalysts: organisations that intentionally drive the expansion of democratic innovations in their regions. In this paper, we make three core contributions to the field: 1. We distil six features of effective scaling catalyst organisations, aiming to elevate the important role they play. 2. We examine critical tensions and trade-offs these organisations face, and how they can navigate these. 3. We identify five frontiers of future practice that can further accelerate the scaling of democratic innovations and promote more deliberative cultures beyond the work of individual catalyst organisations. This paper is for three key audiences: We offer insights for practitioners building similar organisations, for funders seeking to support this vital work, and for researchers identifying knowledge gaps.

Complex Systems Frameworks Collection: an illustrated collection - Finegood, Diane; Bradd, Sam; Yakimov, Chris (Frameworks Collection, 2026)

UNDP Portfolio Evaluation Guidelines - UNDP (Independent Evaluation Office of UNDP, 2026)

Generative AI guide for civil society - Byoung-il, Oh; Achim, Koh; Yeo-kyung, Chang (Digital Justice Network, 2025)
The Digital Justice Network produced a guide that originated from the realisation that civil society organisations and activists need help in establishing generative AI policies and properly utilise these tools when necessary. “Some of our activists are drafting statements with ChatGPT, and I’m worried about what approach we should take.” Many organisations are likely having similar concerns. With the emergence of various generative AI services—such as chatbots like ChatGPT and Gemini, and tools that create images, music, and videos—an increasing number of citizens are using them for both professional and personal purposes. Civil society activists are no exception. However, while they are being utilised, usually based on individual judgment even for work-related tasks, almost no organisation currently has an organisational-level policy on generative AI. There are many points that civil society organisations (CSOs) must consider when using generative AI. For example, if factual inaccuracies (hallucinations) from generative AI are included in an organisation’s official documents, the organisation’s credibility can be severely damaged. Security issues may arise if personal or confidential information is uploaded to unreliable commercial services. Furthermore, the output of generative AI might contain biases that conflict with the organisation’s values. The process of drafting a statement using generative AI may exclude aspects crucial for activist capacity building and internal organisational deliberation. If activists use AI tools based on individual choice without an organisational policy, there is a high likelihood that issues beyond the organisation’s control will emerge. However, in the Korean context, there is a lack of guidelines available regarding whether it is appropriate for civil society organisations to utilise generative AI services, what principles and policies should govern their use if they choose to do so, and what guidance can be referenced from a human rights perspective. Moreover, the current status of which AI tools activists are using for which tasks has not been documented. This guide originates from the realisation that we need to help civil society organisations and activists establish generative AI policies and properly utilise these tools when necessary. To create this guide, we conducted a survey on which AI tools are actually being used for which tasks, how useful generative AI is perceived to be, and what problems users are experiencing. We gathered opinions not only from domestic activists but also from activists worldwide through the APC network. While the sample size is limited, restricting its statistical significance, we were able to confirm the real concerns and shared understanding of the issues felt by activists. Even those who use generative AI minimally responded, sharing their thoughts. Furthermore, we held workshops with civil society and labor union activists focusing on generative AI. We shared the survey results and a preliminary policy framework, allowing participants to exchange their experiences and perspectives. Through this process, we reconfirmed that the act of honestly sharing feelings and concerns is crucial, rather than simply reaching a consensus. The policy framework presented in this guide is merely a starting point; the process of each organisation creating its own policy that reflects its reality and the voices of its activists is paramount. While some activists use generative AI with interest, many others still feel uncomfortable with generative AI itself. We clearly state that this guide is not intended to encourage the use of generative AI. The fact that the development of major generative AI models and the provision of services are exclusively controlled by Big Tech companies is also a concern. Although this guide focuses on the commercial generative AI services currently in dominant use, we deeply empathize with the need to overcome these structural limitations. Despite various limitations, we hope this guide will be of some help to organisations and activists currently contemplating policies related to generative AI.

AI in evaluation: Good practice guidelines for practitioners - UK Evaluation Society (UK Evaluation Society, 2025)
This practical guide from the UK Evaluation Society explores how evaluators can use AI tools responsibly and transparently across all stages of the evaluation process.

Towards a future we won't regret: A dispatch from 2035 - Simpson, Lea (Brink, 2025)
Earlier this year we spoke to 102 leaders from across the globe to ask: Ten years from now, what will we regret not having done today? This framing turned out to be a powerful device for collective horizon scanning and foresight. Across every conversation there was a sense that this moment demands more from us all, along with a strong eagerness to explore what we can own and act on. More than that, these discussions surfaced direction, hope and a way forward. From CEOs, professors, PhDs, tech founders, heads of UN agencies, frontline leaders, local innovators, public servants and practitioners we heard about some of the shifts that are already happening in pockets around the world.

Understanding A Quick Approach to PEA - Whaites, Alan (FCDO, 2025)

Shift the status quo: Applying a systems approach to improving adolescent health and wellbeing in cities - A guide for practitioners - Ecorys (Ecorys, 2025)
Healthy Cities for Adolescents Phase II (HCA-II) is a multi-country programme that supports projects working to improve adolescent and young people’s health and wellbeing in intermediary cities. Guided by the principle of shifting the status quo, with adolescents at the centre of this, HCA-II aims to catalyse meaningful changes in how cities function for young people. This guide aims to help you influence the systems that shape adolescent and young people’s health and wellbeing in the cities where they live. You don’t need to be a ‘systems expert’ to use it. It’s designed for anyone working on HCA-II projects (or similar) – whether you’re a project manager, Monitoring, Evaluation and Learning (MEL) lead, consortium partner or youth worker – who wants practical ideas for planning, delivering, and learning from work that aims to make cities fairer and healthier for adolescents. A core focus of this guide is helping projects shift system conditions – the underlying structures, relationships, and beliefs that shape how things work. It’s not about fixing everything at once, but about identifying where small changes, like who’s involved in decision-making, how resources are shared, or how people relate to one another, can unlock bigger, longer-term improvements. Focusing on how our work shifts system conditions helps us move beyond surface-level problems and get to the root causes, like policies, resources, relationships, and behaviours that shape adolescents’ lives. It’s a practical way of thinking and working that leads to more joinedup, fair, and lasting change. This guide is designed to be flexible; dip in and out depending on where you are in your project. It includes practical tools, examples, and ideas across five key areas of systems practice.

A personal reflection on Thinking and Working Politically at USAID: - Jacobstein, David (2025)
This paper shares an insider's perspective on the spread of Thinking and Working Politically (TWP) across the US Agency for International Development (USAID), highlighting lessons learned along the way that are relevant even beyond the dissolution of the Agency in 2025.

Implementing Systems Change - Jain, Vikram; Singh, Vivek; Mundol, Hisham; Gogri, Mirik (2025)
While enthusiasm for systems change has grown steadily over the past decade, we find a significant gap persists between this enthusiasm and execution. Funders attempting to implement systems change consistently face 9 challenges, including unclear outcome attribution, long commitment horizons, and complex stakeholder coordination. When faced with these challenges, it is understandable why many funders find themselves defaulting to patching the system rather than changing it. The pressures are real: boards expect clear metrics, program timelines demand quick wins, and established processes reward predictable outcomes. These institutional realities often nudge funders toward incremental fixes, even when they recognize deeper change is needed. This report names and explores the challenges in implementing systems change and provides steps to navigate them. We draw on the authors’ 14 years of experience implementing systems change programs in the Global South across affordable housing, early childhood education, and gender equity. Foundation leaders (e.g., CEOs, principals, and other executives) can use this guide to implement systems change in their work, finding entry points that align with their foundation’s current capacity, constraints, and strategic objectives. Funders can choose to lean in and make their intent to change the system explicit. Doing so would offer the potential for permanent, transformative change that continues long after the intervention or program ends. This choice requires abandoning the comfort of clear attribution and predictable outcomes and accepting that it may be impossible to claim credit for your contributions. The communities you serve rarely need you to prove exactly which intervention created change, but they do need you to commit to changing the conditions that trap them in cycles of inequity. “Courage is not the absence of fear, but action in spite of it.” – Nelson Mandela

Modernizing Development: Introducing Portfolios - a new systems-way of working for today’s complexity, built for greater impact - UNDP (UNDP, 2025)

Considerations and Practical Applications for using AI in Evaluations. Technical Note - Cekova, Diana; Corsetti, Lea; Ferretti, Silva; Vaca, Sara (CGIAR, 2025)
The CGIAR 2030 Research and Innovation Strategy commits organizational change with seven ways of working, including “Making the digital revolution central to our way of working”. In that context, Artificial Intelligence (AI), introduces both opportunities and risks to evaluation practice. Guided by the CGIAR-wide Evaluation Framework, integrating AI tools requires a governance approach to balance innovation with ethical responsibility, ensuring transparency, fairness, accountability, and inclusivity. This Technical Note encourages and guides CGIAR evaluators to ethically explore, negotiate, and experiment with AI tools: Explore: Evaluators are invited to discover how AI, especially GenAI, can enhance evaluation efficiency, from scoping and data analysis to reporting. The Note provides practical guidance on AI applications and examples to support creative yet responsible exploration. Negotiate: Integrating AI should be openly discussed with commissioners, stakeholders, and teams. The Note prioritizes jointly defining boundaries, expectations, and ethical parameters (transparency, accountability, and data sensitivity) at each phase of evaluation. Use AI Responsibly: While AI tools are evolving, evaluators are encouraged to pilot and iterate their use. The document supports experimentation through practical tips, prompt examples, and tool selection criteria, all while emphasizing documentation and learning from each use case. Effective AI governance is grounded in core principles: Transparency requires clear documentation of AI tool usage, data sources, model limitations, and decision-making processes. Accountability involves assigning responsibility for AI decisions and outputs and establishing oversight and redress mechanisms.Fairness and inclusion must proactively mitigate bias and discrimination, with particular attention to underrepresented groups and data gaps. Data privacy and security must align with applicable data protection regulations and ensure secure handling practices. Human oversight ensures that evaluators retain control over processes and can intervene as needed. In operationalizing ethical AI governance in CGIAR evaluations, due diligence is required in assessing AI tools for ethical alignment before deployment: reviewing the transparency of vendors, the documentation of models, and their intended use cases. Where relevant, components involving AI—especially those engaging human subjects or sensitive data—should undergo ethics review. AI applications must be adapted to the local and cultural contexts in which evaluations are conducted, as what is suitable in one setting may be inappropriate in another. Additionally, participants should be informed about the use of AI systems and the implications of data collection or processing to ensure informed consent. Ethical AI governance should be embedded in the entire evaluation lifecycle. During the design phase, evaluators should define AI tools to use, why they are selected, and assess risks. In data collection, AI tools should be used in ways that uphold data privacy and protection standards and avoid reinforcing harmful stereotypes or excluding groups. During the analysis phase, the role of AI in supporting interpretation should be documented, with an acknowledgment of limitations or biases. In dissemination, documentation, and reporting, AI’s contribution, limitations, and human validation should be disclosed. By rapid adaptation of content across formats, languages, and complexity levels, AI opens possibilities for broader, more inclusive communication of findings. Finally, the follow-up phases should include a reflection on the ethical implications observed and how these lessons can improve future evaluations. By embedding methodological flexibility into the evaluation processes, AI adoption would contribute to integrity, equity, and learning in an era of rapid technological advancement. This Technical Note is a conversation starter—as a “Beta” version, it will evolve based on responsible real-world experimentation and continuous reflection. Evaluators are encouraged to be responsive to stakeholder input throughout the evaluation processes, to ensure relevance, accuracy, and inclusivity.

Monitoring, Evaluation and Learning for Systems Change - McDonald, Lizzy; Simpson, Kate; Schramm, Carolin (Wasafiri, 2025)
Today’s critical challenges are complex and require system level change. Issues like ending poverty, reversing biodiversity collapse, creating more equitable and sustainable food systems, or ending health inequalities, will not respond to neat siloed technical interventions. These are not problems we can tackle head on. Rather, we need to change the conditions (systems) that create these problems. Consequently, many of the most ambitious philanthropic foundations are adopting ‘systems change’ approaches to their work. There is a critical need to better understand the impact of systems change interventions. This is for two reasons: 1) without being able to understand our impact we can’t learn about the efficacy of efforts and focus on doing more helpful action and less unhelpful action, and 2) without effective monitoring and evaluation it is hard to galvanise resources, whether as money, time, or enthusiasm. Traditional monitoring, evaluation and learning (MEL) approaches and practices struggle to respond to complexity. Many of today’s MEL approaches have been developed to serve funders keen to ‘prove’ the efficacy of their work and hold grantees accountable. This paradigm has produced approaches like ‘logical frameworks’ that seek to create a linear logic between intervention, outputs, outcomes and impact. Such approaches often pay scant regard to the activity (intentional or otherwise) of other actors in any system, and don’t adapt well to the characteristics of complex problems such as emergence, uncertainty, and interdependency. There is a fast-developing body of practice on MEL for systems change. A growing number of practitioners are experimenting with complexity informed approaches to MEL, and UNDP is pioneering an open-source set of tools and ideas.ii Wasafiri is contributing to this innovation by developing a MEL approach that builds on our (open access) Systemcraft Framework. iii Emerging approaches for systems change MEL are underpinned by some key principles. There is growing consensus on the need to be pragmatic and flexible, not just because it is a new field for many that requires a degree of mindset and behaviour change and different ways of working. MEL approaches for system change primarily focus on improving and learning and on identifying ways to adapt and pivot. This contrasts with linear approaches to implementation and measurement. Learning questions guide and co-exist alongside indicators to support continual learning and adaptation. These shifts will require a shift in relationships between funders and implementers from measuring and accountability to greater flexibility and patience, close engagement and learning together. There is pressure to show impact in less time. Reduced time horizons, increasing impatience and integration of new sets of agendas in development programming present challenges in the context of systems change. In an increasingly resource constrained context and amid growing global uncertainty (e.g. current global politics, emergencies like COVID, economic instability, rapid technological advancement), it is unclear which direction donors will take, i.e. an even greater focus on value for money and impact, and/or willingness to explore innovative approaches and learning in uncertain territory. Wasafiri’s approach for systems MEL builds on Systemcraft. This is a diagnostic and design tool for systems change, that has been used globally across a broad range of systems and interventions. This paper summarises our approach to MEL for systems change. It aims to provide a short introductory overview and form a ‘conversation starter’ to initiate a practical dialogue on current practices, experiences, and learnings with MEL for systems change. It introduces some concepts and appropriate tools for measuring systems change and poses some key questions for further discussion. We aim to engage with funders and practitioners working on complex problems who are in the process of or planning to develop their approach to and practices for MEL, who are interested to learn, share and explore. We welcome your inputs, either in response to this document, via a call, or during online webinars that we will convene in 2025.

Balancing Innovation and Rigor: Guidance for the Thoughtful Integration of Artificial Intelligence for Evaluation - Anuj, Harsh; Den Boer, Hannah; Raimondo, Estelle (IEG, The World Bank & IOE, IFAD, 2025)
One of the main questions evaluators have when using AI is: "How can I know if it performs well?" To answer this question, we have developed a guidance note aimed at capturing what we have learned so far about integrating AI into evaluation and to offer a framework for further exploration. Since we began our experiments with Large Language Models (LLMs) in the Spring of 2023, we have made significant progress in using Generative Artificial Intelligence (GenAI) for processing and analyzing large volumes of text data in our evaluations. Currently, we are running multiple experiments across IEG to test the various capabilities of LLMs in different evaluation use cases. As we continue to explore the strengths, limitations, and risks of this technology, we have partnered with others in the evaluation community to learn from each other and identify promising practices. The guidance note is the result of our collaboration with the Independent Office of Evaluation of the International Fund for Agricultural Development. Our guidance note establishes clear criteria to judge AI performance and good practices for designing meaningful experiments. We use these criteria to test AI on key tasks such as text classification, summarization, and synthesis. Assessing AI’s Performance LLMs do not always provide appropriate or veracious responses, and their output must be thoroughly validated before use, especially in evaluation practice where analytical rigor is crucial. The guidance note proposes several performance dimensions depending on the task at hand. For classification tasks, standard machine learning metrics like accuracy, precision, recall, balanced accuracy, and F1 scores are used (please see the guidance note for a description of metrics and scores). These metrics measure the overlap between machine-annotated 'predicted' labels and human-annotated 'ground-truth' labels. Separate training, validation, testing, and prediction sets are necessary to compute useful performance metrics and reduce bias. For text summarization, text synthesis, and information extraction tasks, producing human-annotated data as ground-truth is not always feasible. In such cases, we assess the quality of model responses using criteria from the field of Natural Language Generation, a subfield of AI: faithfulness, relevance, and coherence. Faithfulness checks if the AI-generated information is factually consistent with the source, relevance assesses if the content selected by AI is the most important, and coherence evaluates the overall quality of the generated sentences. To determine the minimum acceptable value for each metric, we take a context-specific approach. For example, in a text classification task for a structured literature review, recall and precision scores of 0.75 and 0.60 respectively may be sufficient for the initial literature identification, especially with highly class-imbalanced data. This was the case in our applied experiments within an ongoing IEG evaluation. However, for tasks with higher evaluative stakes, such as judging the extent to which a portfolio of activities has reached key performance targets, higher metric values are required. The guidance note provides a framework of assessment criteria with clear definitions. Other Good Practices Identifying use cases where LLMs can add significant value compared to traditional approaches is crucial. Not every task will have (net) benefits from the use of LLMs, so aligning experiments with tasks that can truly leverage their capabilities is essential. Planning workflows within use cases involves breaking down tasks into granular steps to understand where and how LLMs can be effectively applied. Translating typical evaluation flows into AI-enabled workflows is a critical step. Incorporating a modular design for the AI-enabled workflows can further enhance the process since it allows for the reuse of successful components within and across use cases. Clear understanding and agreement on necessary resources and expected outcomes are vital within multidisciplinary teams needed to carry out such work. This includes human resources, technology, timeline, and defining success for each experiment. A robust sampling strategy is needed to parse datasets into training, validation, testing, and prediction sets, facilitating effective prompt development, model evaluation, and most importantly, model responses that are helpful for the task. Iterative prompt development and validation involve testing and refining prompts, including requests for justification to gauge the model's reasoning, which itself helps with prompt refinement. Going Forward Experimenting with AI in evaluation practice involves thoughtful risk-taking, continuous learning, and adaptation. It is a continuous process of questioning, testing, learning, and refining, analogous to how models like GPT themselves learn during their training phase: through constant trial and error. Our guidance note focuses on defining and adapting evaluation workflows to include LLMs where they fit best and building trust through thorough performance testing. Further research, experimentation, and collaboration are needed to standardize and expand frameworks for assessing LLM performance in evaluation. Sharing experiences and findings from experiments across organizations and contexts is essential. Much has been written about the potential and perils of leveraging LLMs in research and analytical tasks. However, it is through concrete, practical, context-specific experimentation that we can discover what works, what does not, and under what circumstances. We are committed to exploring and sharing our findings as widely as possible.

Pando Funding. A princpled apprach to deploying capital to support system change - Muoio, Anna; Lehan, Brendan (2025)
Pando Funding is a practical, principle-based approach to pooling and deploying capital to support a system change network—collaborations of diverse actors working across strategies, sectors, and theories of change to shift a complex system for the better. A system change network brings together leaders, helps them develop a shared vision for transformational change, and shifts decision-making into their hands so they can adapt and innovate in response to emerging developments on the ground. Pando Funding is designed to finance the often invisible, under-funded but critical “roots” for transformative change, the interconnections and enabling conditions among people, projects, and practices that we know are key drivers of a system’s health. Pando Funding also finances the innovative “shoots,” the portfolio of projects launched from a system change network by leaders who see the power in working in infinitely more strategic and connected ways behind their shared vision. At the most basic level, Pando Funding supports ecologies of diverse but connected actors, organizations, and theories of change intervening over time. Appropriately building and resourcing a system change network is the necessary infrastructure for leaders to organize themselves within; and more importantly, to develop the root enabling conditions for system change such as: relationships, trust, collective system awareness, information and knowledge flows, and a shared vision that can cohere all their work. These conditions are not nice-to-haves. They are critical pre-conditions for a group to be able to take coordinated and coherent action together through a range of many specific but aligned projects. That said, Pando Funding is more than just a funding mechanism to distribute capital. It is a framework for rebalancing power, encouraging a collective approach, and fostering long-term commitment in the service of transformational change. Pando Funding creates the conditions for deeper, more resilient change by aligning the strategic use of capital with the collective intelligence and shared purpose of leaders working within a system change network.

Community Weaving: a framework for weaving healthy communities - Dixon, Erin; Pfortmüller, Fabian; Bachmann, Michel; Magnuson, Sita (Community Weaving, 2025)

Process Tracing Method in Program Evaluation - Beach, Derek; Raimondo, Estelle (Independent Evaluation Group, World Bank Group, 2025)
Process tracing (PT) involves a detailed analysis of the processes that link interventions to outcomes. PT is particularly useful for evaluating interventions that are difficult to quantify, such as knowledge work or institution building. It involves creating a detailed causal theory, tracking the process theory of change (pToC) by examining the observable evidence, and learning general lessons from the cases studied. PT offers two main benefits to evaluators: first, it provides a clear way to present and evaluate the strength of the evidence collected. This helps in making the evaluation process more understandable and credible. Second, it helps evaluators learn practical lessons more easily by emphasizing the explicit connections between actors, their actions, and resulting behavioral changes. However, the reliability of findings from PT depends on how well the theory matches the observable evidence. This paper explains how process tracing (PT) is a valuable tool for evaluating policy changes and program effectiveness. It details the methodology, highlights the importance of tracking processes and events linking interventions to results, and discusses PT's advantages and limitations. The paper provides a step-by-step implementation guide and examines the use of PT in an anonymized IEG evaluation as an example.

Using AI responsibly for Research on Violence Agains Women - Amazon-Brown, Isabelle; Tsui, Quito; Raftree, Linda; Dartnall, Elizabeth (MERL Tech & SVRI, 2025)
Generative AI (GenAI) is evolving rapidly, and its use in violence against women research remains new and largely experimental. While researchers are beginning to explore GenAI’s potential, uncertainty around its risks, ethical challenges, and real-world application persists. Recognising the need for clear guidance, the Sexual Violence Research Initiative (SVRI) partnered with The MERL Tech Initiative (MTI) to develop this resource. This guide aims to provide background on GenAI, strengthen researchers’ AI literacy, and offer practical advice to help the violence against women research community critically and carefully engage with GenAI while minimising risks. We start off with an overview of GenAI and potential areas where researchers might use it at different stages of the research process. We highlight some low-risk, practical applications of GenAI that can help streamline tasks while ensuring that researchers remain in control. (Appendices provide clear definitions of key AI terms and a simple explanation of how generative AI works for those less familiar with the technology.) Following these examples, we discuss some of the key risks and potential harms of using GenAI for research on violence against women, including reliability, bias, dehumanisation, data privacy, power and patriarchy, relevance, transparency and replicability, inclusion and participation, worker’s rights, inequities and environmental harms. To support informed decision-making, we offer a simple decision framework with critical questions to consider before using AI for any task: • Does AI provide a clear benefit over existing methods? • Does its use comply with ethical guidelines, survivor consent, and data protection laws? • Are humans in the loop to validate results and take responsibility? • Can potential risks be effectively managed? • Is there a process in place for making and documenting decisions about the use of GenAI? If the answer to any of these is “no,” the guidance suggests reconsidering or avoiding AI in that context. This framework can help researchers to weigh AI’s advantages against its limitations, ensuring thoughtful and responsible use. Recognising AI’s fast-changing nature, we encourage a cautious approach to GenAI. We suggest starting small—testing GenAI tools on non-sensitive, publicly available data—to better understand their capabilities and limitations. If you choose to integrate GenAI into your research, we encourage you to document and share your experiences, so we can learn collectively and refine best practices together. The guide reinforces a key message: human expertise is irreplaceable. AI can improve efficiency, but final analysis and interpretation should always lie with researchers. By experimenting in a controlled way and keeping ethical principles at the forefront, researchers can build confidence in AI while avoiding unintended harm. This guidance is intended to be a living document—SVRI will maintain and update it as the AI landscape evolves and new insights emerge. SVRI and MTI are grateful for the critical feedback received from colleagues during the production of this guide, which has helped strengthen its content. (See Appendix B for details on the methodology used.)

Relational State Capacity: Conceiving of Relationships as a Core Component of Society’s Ability to Achieve Collective Ends - Honig, Dan; Krishnamurthy, Mekhala; Sharma, Rahul Karnamadakala (Johns Hopkins University, 2025)
Relational state capacity (RSC) integrates the quality of citizen-state relationships into the concept of state capacity, emphasizing mutual recognition between citizens and state agents as foundational for societal problem-solving. Traditional state capacity definitions overemphasize technical and institutional elements, overlooking relational dynamics. RSC is a latent societal resource activated through mutual recognition in citizen-state exchanges with potential to improve outcomes across diverse domains, from public health to social cohesion. We have for too long transactionalized governance; we need instead to collectively invest in intentional structures and practices that foster relationality. Drawing on insights from existing literature, the paper explores how RSC is built, intermediated, and shaped by broader societal norms and organizational dynamics. By making RSC legible as an object of inquiry, the paper lays the groundwork for empirical investigation into RSC’s causal pathways, spillover effects, and resilience to disruption. There are many pathways for theoretical and empirical development in this emergent field; focusing on the relational in understanding state performance has the potential to enhance citizens’ welfare, strengthen democratic resilience, and improve public sector performance.

How do we define and support quality and rigor in Causal Pathways evaluation? - Apgar, Marina; Aston, Tom (Causal Pathways, 2025)

Are We Making the Same Movie - Cabaj, Mark (Tamarack Institute, 2025)
The aim of this paper is to highlight several frameworks that changemakers can employ to assist them in thinking about, planning, and evaluating their efforts to change or transform systems. The paper is based on the idea that changemakers may have an easier time getting on the same page about what they are trying to accomplish if they have some common language about the different degrees, orders, or levels of change they seek. The paper goes further in exploring different types of change than the Primer (Getting our Heads Around Transformation) by showcasing frameworks that focus on slightly different ways of understanding change.

Transformation and Systems Change: a primer - Cabaj, Mark (Tamarack Institute, 2025)
This foundation paper introduces the core ideas behind systems transformation and summarizes the most current thinking about how deep, structural change actually unfolds. It integrates insights from systems science, social innovation, sustainability transitions, and evaluation to clarify the mechanisms, patterns, and conditions that distinguish transformative change from more incremental forms of improvement. Designed as an accessible primer, it helps change-makers build a shared understanding of what “transformation” means—and what it demands—from those seeking to pursue it seriously.

Education Systems Changemakers Challenge Workbook - Gluckman, Maxie; Rubio Rodríguez, Raquel (IREX, 2025)
Introduction Phase 1 - Power Up Phase 2 - Level Up

Credible Claims in Complexity - Building Narratives and Evaluating Systemic Change with the Impace Garden Framework - Kaiser, Jannik; Godesberg, Erinda Panen (Unity Effect, 2025)

Ecosystem Cards: Deepen your understanding of the status quo. Envision desirable futures. Design strategic experiments to learn more - Lukyanova, Zoya; Polok, Darius; Shah, Indira; Martin, Naomi; Kildeev, Ruslan (resense360 & iac Berlin, 2025)
A tool to deepen your understanding of the status quo, envision desirable futures, and engage with ecosystems more intentionally.

Many-to-Many - Field Guide with learnings, models, insights and tools - Zucker, Michelle; Dhami., Annette (Dark Matter Lab, 2025)
A living library of practical tools, frameworks, and case studies designed to support practitioners in complex collaborations. Many-to-Many System -aims to support collaborations tackling complex, entangled challenges where disrupting norms and values, ownership, and power is essential. Solving today’s complex, interconnected problems requires what we term “complex collaborations” - bringing together many diverse groups (public, private, civic) with many new perspectives, including future generations and the natural world. While many collaborations are already doing great work, we believe that finding better ways to support how they are structured and organised them could unlock more effective, system-level change. The Many-to-Many System is focussed on unlocking the governance, organising, legal, and learning structures of complex collaborations to enable many resources, not just money, but also knowledge and relationships to flow more freely, and to foster many ways of working that embrace diverse value exchange. For each and everyone of us, our fundamental understandings of the world are influenced by ‘deep codes,’ often invisibly embedded within our creations, frameworks, and rules. The Many-to-Many System explored how these codes shape collaboration and governance, aiming to understand if they could be reimagined and how those within complex collaborators themselves can embed them into their collaboration’s infrastructures. More intentional and visible shifting of deep codes for governance and organising could help collaborations to better align with their systemic missions and offer approaches for rethinking core concepts like value, power, risk, and ownership. The Many-to-Many System distills two years of learning, prototyping, research, and practice. Here we offer a blend of elements: a core framework, practical insights and learnings, illustrative models for complex topics, and tools developed along the way. Our aspiration is to provide a range of resources that can help others embed the Many-to-Many deep code shifts into their own complex collaboration work.

The How we need now: a capacity agenda for 2025 and beyond - Pahlka, Jennifer; Greenway, Andrew (Niskanen Center, 2024)

A Synthesis of Systemic Action Research Processes in the CLARISSA Programme on Worst Forms of Child Labour - Burns, Danny; Apgar, Marina (Institute of Development Studies, 2024)
This paper describes, synthesises and analyses the work of 25 Action Research Groups that were the heartbeat of the CLARISSA Systemic Action Research programme. The CLARISSA programme worked to create participatory and systemic solutions to the worst forms of child labour in the adult entertainment sector in Nepal, and the leather sector in Bangladesh. The paper explores how the action research groups were set up, and how they worked through key stages of trust and relationship building, generating evidence, building theories of change, planning and taking action, and assessing the effects of that action. The paper, which is a synthesis of the evidence and learning produced in the programme, draws together learning about the process for both ground-level action researchers and those who might fund, design or implement future programmes.

From Local Roots to Global Reach: Global South Intermediaries Driving Just and Equitable Governance - Castells Carrión, Paula (Trust, Accountability and Inclusion Collaborative (TAI), 2024)
In an evolving international development landscape, the push for localization has gained urgency. Global South Intermediary Organizations (GSIs) are emerging as key players, offering a promising pathway to achieving localization programming. To harness their potential, it is crucial to develop a deeper understanding of their functions, structures, and strategies. However, they should complement—not replace—direct funding relationships with grassroots organizations. The Trust, Accountability, and Inclusion Collaborative (TAI) is excited to present this comprehensive analysis by consultant Paula Castell Carrión, highlighting the significant potential of GSIs to drive locally led solutions. As Global North donors seek to increase funding to frontline groups, movements, and activists in Global South contexts, GSIs emerge as strategic partners in localization programming. These organizations possess deep contextual knowledge and established networks that can help bridge the gap between international funders and local implementers. Their potential extends beyond simple fund management to include capacity building, strategic guidance, and advocacy for local priorities in grantmaking decisions. There is a growing number of GSIs fulfilling these roles, who have a track record to deliver governance-related funding to the grassroots. In this new study, TAI´s Paula Castell Carrión examines the evolving role of Global South Intermediaries in advancing localization efforts, with a particular emphasis on achieving equitable governance outcomes. Through a detailed investigation of 4 operational models and practical experiences, this paper aims to provide donors with actionable insights for a more effective engagement with GSIs. Paula's investigation centers on six carefully selected case studies and reflects on broader practices and trends within the funding ecosystem. These cases shed light on the diverse approaches, organizational structures, and strategic interventions employed by GSIs across different contexts. We hope that this research contributes to broader strategies on international development funding by: 1. Providing evidence-based insights into the effectiveness of different intermediary models. 2. Identifying best practices for donor-GSI partnerships. 3. Supporting the development of more equitable and effective governance programming

Demystifying Interoperability. Insights from our work in Ethiopia's agriculture sector - Hatcher-Mbu, Beverley; Ferreyra, Fernando; Postelnicu, Mihai; Cihai, Viorel; Getachew, Yosef; FIkre, Yohannes; Kassa, Selam; Orrell, Tom; Tewari, Khyatee (Development Gateway, 2024)
Government offices are filled with siloed, sector-specific digital systems that strain their capacity to make decisions, use data effectively, and achieve ambitious sustainable development goals. Public investments in digital development, transformation, and infrastructure can only meet citizen needs if data and systems are consolidated and interoperable. While interoperability is a sensible approach to building digital public infrastructure, transforming existing systems is easier said than done. Limited resources and expertise, lack of training, inadequate hardware and software infrastructure, and concerns over data privacy are some of the key factors that limit the impact of systems or lead to their discontinuation. The data that feeds these systems also needs to be standardized using global best practices and common languages and frameworks that can be understood across multiple platforms and adapted to the local context. In increasingly digital economies, systems and their data need to be relevant and responsive to citizens’ specific conditions and needs With emerging challenges like climate change that require more complex data, and new technologies like artificial intelligence (AI) that require large volumes of data, systems need to be agile and scalable to advance solutions and fully harness innovation. This paper discusses, in practical terms, what goes into implementing interoperable solutions in partnership with public administrations. Based on 20+ years of DG’s experience, the paper demystifies key components needed to build robust, resilient, and interoperable data systems, focusing on the “how” of data standardization, data governance, and implementing technical infrastructure.

The end of reporting - Sutton, Amy (2024)
After eight years, I am leaving Porticus. From January 2025, I'll be starting a new chapter - working in a freelance capacity to support systems change efforts through strong learning, evaluation and strategy. Having worked with the amazing Porticus UK team for the first four years, I moved to Amsterdam in 2020 to work in the central strategy and learning team, supporting colleagues around the globe to embed strategy, evaluation and learning in their work. I loved this work. I learned a huge amount about social justice, and about how philanthropy can most effectively support the many fights for change needed today. I will miss the passion and commitment of so many people I feel privileged to work with - too many to name here. One of the pieces of work I am most proud of is the design and introduction of a new practice of learning between Porticus and its partner grantees, ending the burden of written reporting. I will always appreciate my colleagues’ appetite for innovation and to living out their values with this work. Getting rid of reporting may sound radical, but static reporting about what happened and what was done doesn’t fit with a systems change approach. If we trust our grantee partners, let’s focus on working and learning with them, about what matters. Reporting about outputs, activities, actual spend vs anticipated spend, will not help us shift systems. Learning together about what’s changing, why, how, and how can we adjust or adapt as a result of this insight and learning, in real time, will. As I leave Porticus, I will be taking this approach to learning with me. And as I do so, I would like to urge the philanthropy sector to challenge itself and explore how the systems we use for evaluation, learning and reporting can be made more fit for purpose for system change, less burdensome and more equitable. Here is a reflection piece I’ve written about why this learning practice. I offer this as a personal reflection and share it in the hope that more foundations and funders will think about doing their reporting differently - and ideally, not doing reporting at all…

Working with Systems-Informed Theory of Change - A practical introduction - Vogel, Isabel; Abosenna, Karim R (Climate-KIC, 2024)
Explore Climate KIC’s S-ToC guide A practical and flexible resource designed to help teams and partners navigate the complexity of systems change in the face of the climate crisis. Our S-ToC approach integrates systems thinking to support adaptive monitoring, evaluation, and ongoing learning. This guide empowers users to analyse systems, identify leverage points, and co-create change-oriented strategies that evolve with context. Whether you’re working on a small-scale pilot or a multi-stakeholder portfolio, S-ToC can help you articulate dynamic change trajectories, catalyse impact, and document learning across interconnected efforts. Designed as a step-by-step Miro board template, the guide supports collaborative sense-making and can be used to facilitate stakeholder workshops or train others in systems-based ToC thinking.

Becoming Futures Ready: How Philanthropy Can Leverage Strategic Foresight For Democracy - Masters, Suzette Brooks; Ingram, Maya; Hooke, Hillary (2024)
In 2022, the Democracy Funders Network launched the Better Futures Project and published Imagining Better Futures for American Democracy to urge democracy donors and practitioners to move beyond reaction and critique towards creation and change. The project seeks to provide succor and inspiration at a time marked by anxiety and fear, but also great possibility. The project challenges us to consider the future of our democracy as a dynamic system — not one that revolves solely around the next election — and to activate our collective agency and imagination. With consequential elections on the horizon, it is particularly important for pro-democracy actors to cultivate and hold a longer term, affirmative orientation even as, in the near term, attention and resources continue to focus primarily on electoral issues and democracy defense. The Democracy Funders Network, through its Better Futures Project, is publishing this toolkit to build the strategic foresight skills of individual and organizational democracy donors, and to expand this field’s capacity to think longer term and more boldly about the future of American liberal democracy. The toolkit provides practical applications of strategic foresight that can be incorporated into democracy donors’ philanthropic strategies and grantmaking practices. It also showcases recent examples of how donors have integrated strategic foresight into their operations. This toolkit benefited from conversations and interviews with strategic foresight practitioners and philanthropists, as well as a review of leading publications in the field of strategic foresight and futures thinking. Contents Navigating Rapid Societal Change Why Use Strategic Foresight in Philanthropy How to Deploy Strategic Foresight and Futures Thinking in Philanthropy. An Overview of Techniques Leveraging Foresight and Futures Thinking Internally. Organizational Structure and Culture Leveraging Foresight and Futures Thinking Externally. Communities of Practice and Thought Leadership Reaching The Next Frontier. Futures-Powered Grantmaking How to Successfully Harness Foresight and Futures Thinking. Enablers and Blockers

A Framework for Embedding Learning in Systems Change - Bertermann, Kecia; Coffman, Julia (Center for Evaluation Innovation, 2024)
This five-step framework, developed and tested by a foundation, embeds learning in emergent systems change strategies. It prioritizes the testing of hypotheses and assumptions, uses learning questions, and calls for examining both confirming and disconfirming evidence. — A framework for embedding learning in systems change strategies and for testing strategic uncertainties. Learning and evaluation approaches that accompany systems change efforts need to fit with and support the emergent nature of this work. Systems change efforts call for examining, with evidence, the hypotheses and assumptions that undergird the work and regularly adapting in response to what is learned. They require staff to be highly engaged in testing their ideas by gathering evidence, interpreting it, and then acting on its implications. This brief presents a learning approach—referred to here as the Learning Framework—developed and tested by a foundation to support its systems change strategies. It is intended for other practitioners working on approaches for integrating learning with systems change efforts. Because a wealth of conceptual work exists about why learning is critical to systems change but practical advice about how to do it is much less available, we describe the approach we took and how the Framework’s core elements fit together in practice. The primary purpose of this approach is to interrogate our strategic uncertainties. This approach to learning is different from the dominant approach to learning and evaluation in philanthropy that primarily looks for evidence (often static metrics) that expected outcomes have been achieved (an approach that is more suitable for linear and predictable change strategies). This Learning Framework is driven by, and organized around, hypotheses and assumptions. This is especially important for systems change efforts where the path forward cannot be determined with complete certainty, but where robust evidence collection and regular sensemaking can increase our confidence in chosen directions and ultimately increase our chances of success.

Political Economy Analysis in Sudan: Handy Tools for Everyone? - Jones, Beverley; Oosthuizen, Daniel (TWP CoP, 2024)
Based on a case study of the Kullana Liltanmia (All of Us for Development) (KLP) Contextual Analysis (CA) course in Sudan, this paper tells the story of how training and capacity building in Political Economy Analysis (PEA) made a difference in the lives and experiences of those involved, including in particular young people. The paper highlights three critical lessons: • At the country level, the twists and turns of Sudan’s recent history reinforce the importance of people deepening their understanding of what is really going on, in order to make better informed (if not the best or least-worst) choices. The skills and mindset of both broad and localised PEA capacity can make a difference to people living in fragile and conflict-affected settings, where navigating complexity and uncertainty is a daily reality. • At the level of strategic programming, the potential of PEA remains under-utilised. There is considerable scope for capacity in PEA to support not only ordinary people across a range of different occupations and educational levels, but also prominent actors who may eventually play key roles in political processes, and whose choices can affect the longer-term prospects for stability and peace. This becomes even more imperative in the context of the war in Sudan, which now involves a plethora of external actors. • At the level of the experiment in the KLP CA course, this story reinforces the paramount importance of adapting and grounding PEA training in a common understanding among its intended participants, shaping it through their insights and experience. It also highlights how essential it is to nurture PEA skills and mindsets not only in stable contexts but also in challenging, more dangerous and uncertain ones. Supporting this vital capability during periods of hiatus, fast-moving transitions, and even active conflict is a public good; and it requires courage and creativity on the part of the donors, course designers, implementers and participants. The paper is structured as follows. Section 1 provides an introduction setting out the broad context of Sudan within which the KLP CA training took place. Section 2 describes the 30-year period of al-Bashir’s regime (1989-2019). Section 3 focuses on successive UK governance programmes in Sudan between 2015 and 2019. Section 4 describes the KLP CA course (2017-2018) and its development. Section 5 looks at the revolutionary hiatus and brief transition to civilian rule in Sudan (2017-2018). This is followed by an account in Section 6 of the 2019-2023 transitional phase and dashed hopes for a move to civilian rule with the outbreak of a civil war that has led to exceptional violence (including weaponised sexual violence), displacement and hunger. Section 7 concludes with reflections on missed opportunities to capitalise on the KLP CA experiment, and with recommendations on the need for continued support on PEA for Sudanese people and organisations.

RAAM Toolkit: A Toolkit for Reduced Access Analytical Methods - RAAM Project (USAID & Mercycorps, 2024)
The Reduced Access Analytical Methods (RAAM) toolkit is a practical resource designed to help humanitarian practitioners overcome monitoring challenges in reduced access environments. Reduced access can be caused by natural disaster, conflict, political instability, or other factors, and typically makes it difficult to conduct normal monitoring of program implementation. The RAAM toolkit offers technical and managerial tools for a menu of analytical methods that can fill information gaps and/or improve data quality during reduced access program monitoring. RAAM supplements (but does not replace) traditional Monitoring, Evaluation and Learning (MEL). The analytical methods covered by the RAAM toolkit were designed to address commonly-encountered information gaps and data quality issues in reduced access environments. The toolkit covers the following methods: Data Triangulation: Systematic comparison of multiple data sources to improve the reliability and accuracy of program data, helping to identify strengths and weaknesses in information. Context Mapping: Layering various data sources on a map to understand factors that may impact program activities, such as access changes or conflict trends. Rumor Tracking: Monitoring unverified information in communities to identify potential gaps, implementation issues, or early signs of violence, enabling timely interventions. Remote Sensing: Using satellite or aerial data to monitor physical conditions and changes in implementation areas from a distance. Transaction Analysis: Analysing digital transaction data from cash or voucher assistance programs to understand usage patterns, vendor capacity, and more. Available in French, Spanish, and Arabic.

Local Systems Position Paper - USAID (USAID, 2024)
The Local Systems Position Paper describes the key approaches USAID will use to translate systems thinking into systems practice. It focuses on ways USAID can better understand and engage local systems to support them in producing more sustainable results. Systems thinking is a mindset and set of tools that we use to understand how systems behave and produce certain results or outcomes. Systems practice is the application of systems thinking to better understand challenges and strengthen the capacity of local systems to unlock locally led, sustained progress. The shift from systems thinking to systems practice is driven by a desire to integrate systems practice throughout the Program Cycle and increase our capacity to actively and adaptively manage programming in ways that recognize complexity and help make our programs more effective and sustainable. These approaches will be utilized alongside and within the context of USAID’s policies and guidance, including technical guidance for specific sectors, as well as evidence and lessons learned from partners around the world. Systems thinking is a long-standing discipline that can serve as a powerful tool for understanding and working with local systems. It has been a consistent component of USAID’s decades-long commitment to locally led development and humanitarian assistance. USAID uses systems thinking to better understand the complex and interrelated challenges we confront – from climate change to migration to governance – and the perspectives of diverse stakeholders on these issues. When we understand challenges as complex systems – where outcomes emerge from the interactions and relationships between actors and elements in that system – we can leverage and help strengthen the local capacities and relationships that will ultimately drive sustainable progress. Recognizing this, in 2014, USAID released the Local Systems Framework, outlining ten principles to help USAID staff, partners, and local stakeholders place local systems at the center of all of our efforts to promote sustainability. Since then, development and humanitarian actors have embraced systems thinking as a concept and reiterated its importance in other policies, strategies, and tools, across sectors and regions. When USAID released its Localization Vision and Approach in 2022, it included local systems practice as a core component of adapting policies and program practices. Building on the success of the Local Systems Framework, this position paper reiterates USAID’s commitment to systems practice and presents six approaches that will help USAID strengthen and further embed systems practice in all of our work

Innovation Carve-Outs: Curating the space for innovation - Rahman, Asad; Brink (Brink, 2024)
Over the course of this paper we lay out the basis of the Carve-Out method, an approach that leverages a behavioural framework to allow organisations (even those burdened with layers of bureaucracy and entrenched ideas) to create intentionally designed environments that give people the space, resources and support they need to explore and test new ideas. Ideas that may go on to transform the core of their organisation. The Carve-Out method is based on a simple insight: That if we want to deliver real world, enduring impact then we must recognise that it’s real (often irrational, always human) people that are doing the work of innovating. So to deliver the outcomes we seek for people and our planet then we must explore and unlock the human behaviours that we need to get us there. The Carve-Out method makes this possible by creating the conditions for continuous innovation so that existing teams are empowered to explore solutions and bring those transformational ideas to fruition. To get there, first we look at the idea of the innovation lab or ‘skunkworks’: typically physical spaces, built away from business-as-usual constraints and processes where a small team can move quickly and with relative autonomy. We explore the legacy skunkworks principles have left on our contemporary innovation landscape, for example, how the iPhone was developed by a skunkworks team at Apple, as well as the proliferation of government innovation labs in the public sector. We then go on to examine some of the downsides involved in creating isolated innovation units, particularly the potential disconnect between the innovation process and core business operations, and the result this can have on products that might lack relevance to the market or practical applicability to the company’s mainline business. The second section of the paper looks at another model of organisational innovation, which has been adopted by tech companies such as Google and Netflix. This bottom-up model, exemplified by Google’s ‘20% time’ policy, gave employees the freedom and dedicated resources to explore transformative innovations. The paper looks at how these companies leveraged their employees to fuel ‘strategic risk-taking’, but also how this introduced significant pressure on individuals, and the difficulties in sustaining this level of innovation while scaling over the long term. In the third section we introduce the CarveOut method for organisational innovation: a powerful solution for organisations seeking to foster a culture of ideation experimentation, empower teams, and accelerate the development of impactful innovations and drive sustainable growth. Here we show how systematic, continuous, scalable innovation requires genuine integration and the application of behavioural science practices. This combination allows both for the creation of the space, time, and resources for innovation, as well as carving out the psychological and emotional space necessary for changemakers to thrive and grow within an organisation. Through multiple examples we show how designing and shaping the ‘innovation environment’, it is possible to create an integrated and agile system of organisational innovation that allows for the ‘carving out’ of time, resources and space, but also of cognitive bandwidth and the environmental conditions that enable individuals and teams to take creative risks, embrace new perspectives, and engage fully in the innovation process. We lay out five key principles that underpin the Carve-Out model: 1. Commit to evolving and create structures to enable it 2. Keep the people, change the mindsets 3. Learn fast, fail small 4. Invest and scale in line with confidence 5. Do in order to learn The final section of the paper looks at how the Carve-Out method has been successfully applied in real-world scenarios across a number of sectors. The first story looks at how behaviourally informed features were brought to bear on the Frontier Tech Hub, a programme that supports government officials in the UK Foreign, Commonwealth and Development Office (FCDO) to explore radical, early-stage tech ideas that might solve big development challenges and in doing so, evolve the innovation culture of a government department. In the second story we look at how, after the COVID-19 pandemic led to school closures, Lebanon-based NGO Jusoor worked alongside Brink to explore and test a new way of ensuring access to education for refugee children that would eventually shift the whole education system. And in the third story we explore how the uBoraBora fund for education implementers in sub-Saharan Africa, supported teams to take an experimental approach to exploring and testing key questions to enhance their product or service alongside their businessas-usual, and integrate what they learn with governments’ education systems. We end the paper by asking our readers to imagine what could be achieved if they could dedicate a small portion of their organisation’s time and resources to create the conditions for innovation, so their teams could operate with a different set of rules, and be empowered to explore solutions in a safe and supportive environment. We also provide links to a number of tools and resources created by Brink so others can begin to put the ideas found in this paper into action in their work.

A Guide to Evaluation of Value for Money in UK Public Services - King, Julian; Hurrell, Alex (Verian, 2024)
When assessing value for money (VfM), there are sound arguments for combining insights from economics and evaluation. Drawing on the complementary strengths of both disciplines supports comprehensive assessment of policies and programmes’ VfM. In this Guide we introduce an interdisciplinary approach, Value for Investment, that integrates methods and tools from both disciplines, and which has gained traction worldwide. Among the concepts and methods of evaluation that economics brings to this interdisciplinary mix is cost-benefit analysis (CBA), which evaluates whether an intervention creates more value for society than it consumes. CBA is vital in the UK policy context, acting as a key tool for evaluating public policies, programmes, and projects as set out in the Green Book, HM Government’s central guide for appraisal and evaluation, and the Magenta Book which focuses on evaluation methodologies and practices across the whole policy cycle. Together, the Green Book and Magenta Book form a robust framework for evidence-based policymaking, ensuring CBA informs sound decision-making and evaluates implemented policies’ impacts. Within this policy context, value for money (VfM) is often framed as an objective or question within an evaluation and is commonly equated with CBA. However, we argue that while CBA contributes important evidence, it may be insufficient. The approach set out in this Guide includes economic analysis while integrating multiple values (social, economic, environmental, and cultural) and diverse evidence sources (quantitative and qualitative) for a more comprehensive VfM assessment, drawing on the strengths of CBA without limiting the evaluation to economic methods and metrics alone. The Value for Investment approach uses mixed methods (integrating quantitative and qualitative evidence), employs evaluative reasoning (interpreting evidence through the lens of explicit criteria and standards – improving transparency in the rationale for evaluative judgements), and is participatory (involving stakeholders in co-design and analysis). It is designed to be intuitive and practical to use by following a logical sequence of steps. The Value for Investment approach was developed through doctoral research (King, 2017; 2019) and refined through real-world VfM evaluations in diverse contexts. Leading examples of its application are published in Assessing Value for Money – the Oxford Policy Management Approach (King, Wate, Namukasa, Hurrell, Hansford, Ward, & Faramarzifar, 2023), and Value for Investment – Application and Insights (King, Crocket, & Field, 2023). This Guide is designed to complement existing VfI guidance by focusing on the interface between CBA and other evaluation methods to provide sound answers to VfM questions in the context of UK public policy.

The Strategy Testing Workbook: Politically-Smart Adaptive Programming in Complex Environments - Nixon, Nicola; O'Donnell, Carolyn; Rajalingam, Gobie; Tweedie, Pauline; Sullivan, Elizabeth (The Asia Foundation, 2024)
The guide begins with a section on what we at TAF have learned about practicing Strategy Testing, highlighting things that we didn’t know when we started out. Some of these lessons may seem obvious, but others may strike you as less so. In any case, we believe that, when taken together, they provide a useful reminder of some of the pitfalls that arise in efforts to practice Adaptive Management. Moreover, they help to remind us that there are multiple layers to doing Strategy Testing effectively as a system instead of reducing it to its component parts and applying them mechanistically. Thus these lessons are useful to read now and to revisit later. The guide then comprises three more sections: • Part 1, the Design Stage, involves constructing the system and making sure everyone is on board; • Part 2, the Inception Phase (start-up phase), helps the implementing team to make the leap from the ideal initially set out in the design stage to the grounding, support, and frameworks that the team will need to implement Strategy Testing in a meaningful and impactful way. We’ve found that the gap between design and implementation is very often the cause of an ineffective ‘all talk and no walk’ approach to Adaptive Management. And finally; • Part 3 covers Program Implementation – the practices, knowledge, and culture needed for the feedback loop that underpins Strategy Testing. The guide can be used for individual or group learning, or it can be used directly to inform Reflection Sessions (see Section 3) through the questions we raise or the activities herein.

Bridging Learning and Action: How Did CLARISSA’s Participatory Adaptive Management Approach Foster Innovation, Effectiveness, and Stakeholder Empowerment? - Prieto Martin, Pedro; Apgar, Marina; Afroze, Jiniya; Arulanantham, Amit; Hicks, Jacqueline; Karki, Shanta; Mareschal, Sophie; Paul, Sukanta; Snijder, Mieke; Uddin, Forhad; Veitch, Helen (Institute of Development Studies, 2024)
Child Labour: Action-Research-Innovation in South and South-Eastern Asia (CLARISSA) is an evidence and innovation-generation programme funded by the United Kingdom’s Foreign, Commonwealth & Development Office (FCDO), responding to the challenge of the worst forms of child labour (WFCL) in Bangladesh and Nepal. It is a challenge characterised by a poor understanding of its drivers and a lack of evidence on what works to combat it. To handle such fundamental uncertainty, the programme adopts a child-centric and participatory action research approach, which is supported by an adaptive management model to respond better to challenges and opportunities. From its inception, the programme needed to navigate shocks and challenges, such as Covid-19 lockdowns, political upheaval, and sustained budget cuts, which put its capacity to learn and evolve to the test. This paper shares insights emerging from evaluating CLARISSA’s participatory adaptive management (PAM) practices, connecting them with current discussions on adaptive management. It provides an in-depth evaluation of CLARISSA’s PAM approach, exploring how adaptive strategies were implemented and evolved throughout the programme’s life cycle. Multiple cases of adaptation and misadaptation were selected and analysed through a series of in-depth interviews and review of programme documentation, allowing us to assess whether and how the adaptive management practices have been operationalised, the degree to which they led to enhanced decision-making and effectiveness, and their empowering effect on children and other programme stakeholders. Through conceptual analysis and real-world examples, the paper examines the key stages and critical dimensions of PAM, conveying the complexities and dynamism of adaptive management in development work, while highlighting both the successes and challenges encountered in operationalising PAM within CLARISSA.
Accepted: 2024-05-10T08:21:46Z

USAID ADS 201 - Operational Policy for the Program Cycle (Update 05/16/2024) - USAID (USAID, 2024)
The Program Cycle is USAID’s operational model for planning, delivering, assessing, and adapting development programming in a given region or country to advance U.S. foreign policy. It encompasses guidance and procedures for: 1) Making strategic decisions at the regional or country level about programmatic areas of focus and associated resources; 2) Designing supportive projects and/or activities to implement these strategic plans; and 3) Learning from performance monitoring, evaluations, and other relevant sources of information to make course corrections as needed and inform future programming.

Scaling Toolkit - A guide to scaling (digital) innovation in international cooperation - Schröder, Jan; Golbach, Pierre (GIZ, 2024)
Digital innovations are constantly redefining the boundaries of what is possible and, therefore, bring with them many challenges: it is important for organisations to scale impactful solutions and bring about positive change on a global level – a scaling toolkit is designed to help with this. With the Scaling Toolkit, a healthcare organisation, for example, can scale a digital health monitoring system, adapt it to the local language, and integrate it seamlessly into existing infrastructure. The Scaling Toolkit is a resource hub that can be applied to various projects and is designed to help organisations navigate the complicated process of scaling up the impact of digital solutions. It is based on practical examples, such as digital green innovation initiatives, and provides concrete tools and templates to guide practitioners through the iterative process of scaling digital innovations. When scaling digital solutions, technical factors and risks must be taken into account. To facilitate the scaling process, the toolkit can provide guidance and support to practitioners on how to scale successful digital innovations by integrating them locally, regionally, or globally in international cooperation. By doing so, digital solutions can achieve relevant and sustainable impact at scale.

Partnering with communities to co-design humanitarian health strategies: A SeeChange CommunityFirst Framework for implementation in MSF projects - Chapela Trillo, Violeta; Farber, Jessica (MSF, 2024)
The CommunityFirst Framework is intended to be implemented by field teams at MSF. The theoretical aspects and evidence presented on the importance of community engagement are intended for all MSF staff seeking to learn more about why and how to shift the way we work with communities as humanitarians. We believe this guideline, and other tools like it (including OCA’s Person-Centred Approach Guidance07, and MSF Vienna Evaluation Unit’s Guidance for Involving Communities08), to be an important contribution to the growing movement of communities and humanitarian actors who are pushing for changes in the humanitarian system that translate to dignity, health, justice, equity and self-determination for communities around the world. Specifically, the CommunityFirst Framework is intended to guide MSF teams to co-design health strategies with communities, throughout all stages of the project cycle, for exploratory missions, projects that are just opening, projects that have been running for some time, or those that are closing. At the time of publication, the CommunityFirst Framework has been tested in pilot projects in: (1) Madre de Dios, Peru (MSF OCP, August 2022), (2) Tonkolili, Sierra Leone (MSF OCA, November 2022) and (3) Anzoátegui, Venezuela (MSF OCB, February 2023) The experiences from these pilots (feedback from teams, implementation results, adaptations to each context, etc.) have informed the adaptation of the Framework. CommunityFirst builds on existing community engagement work inside MSF and contributes a practical framework for co-designing health initiatives with communities. To avoid duplicating efforts and resources around community engagement inside MSF, the appendices in this guideline largely refer to already existing MSF resources.09 This guideline is meant to be a living document that can evolve and be adapted given the experience of MSF staff and community members and diverse community contexts. This guide can be used by anyone in MSF who is interested in partnering with communities to improve the responsiveness and impact of their humanitarian programs. This is the first iteration of the document. Subsequent iterations will be published based on additional testing during future phases of the CommunityFirst TIC project.

The Radical How - Greenway, Andrew; Loosemore, Tom (Nesta and Public Digital, 2024)
Any mission-focused government should be well equipped to define, from day one, what outcomes it wants to bring about. But radically changing what the government does is only part of the challenge. We also need to change how government does things. The usual methods, we argue in this paper, are too prone to failure and delay. There’s a different approach to public service organisation, one based on multidisciplinary teams, starting with citizen needs, and scaling iteratively by testing assumptions. We’ve been arguing in favour of it for years now, and the more it gets used, the more we see success and timely delivery. We think taking a new approach makes it possible to shift government from an organisation of programmes and projects, to one of missions and services. It offers even constrained administrations an opportunity to improve their chances of delivering outcomes, reducing risk, saving money, and rebuilding public trust. The Radical How in a nutshell The struggles and shortcomings of delivering in government are well rehearsed. Many of the root causes that make it tough have been restated several times over several decades. But what to do? We believe the government can and should change how it delivers, by: organising around multidisciplinary teams embracing incremental, feedback-driven iteration focusing more on outcomes. The Radical How is a change of mindset as much as a change in organisation. It promotes methods and processes that have been shown to work, multiple times, at scale. They are the default ways of working for many of the world’s most successful companies. However, the occasions where they have been deployed are rare in government. These occasions have come about thanks to exceptional leaders, exceptional circumstances, or both. We think they’d make a big difference if they became the norm, rather than the exception. We also think that without them, mission oriented government will not become a reality. New policy ideas will remain just that, rather than translating into profound improvements to society. Central to this approach is the widespread adoption of internet-era ways of working. This paper explains both those and our thinking in more detail, with reference to real examples.

Putting Relationships First. The case for relationship-centred communities, organisations and systems - Lloyd-Rose, Matt; Robinson, David; Robinson, Immy (The Relationship Project, 2024)

Doing weeknotes - What weeknotes are, how weeknotes work, and how to start writing weeknotes of your own - Turnbull, Giles (Use the Human Voice, 2024)
Doing weeknotes brings together various things I’ve written about weeknotes in different places. This text expands on things I wrote in The agile comms handbook, as well as various blog posts. Quite a lot of it is brand new. - Weeknotes for beginners - Why write weeknotes - The weeknotes rules - Weeknotes within the corporate environment - What weeknotes can bring about - Examples of good weeknotes - How to write weeknotes - Weeknotes tips and tricks - Further reading

Social Accountability 3.0: Engaging Citizens to Increase Systemic Responsiveness. Background Paper - Guerzovich, Florencia; Aston, Tom (Grupo Politeia, ESAG, UDESC and act4delivery, 2024)
Social accountability has become increasingly important to development programming since the 2004 World Development Report spearheaded its first generation of programming. In 2016, Thomas Carothers synthesized global experts’ perspectives on what became known as the second generation. With an evolving global context, new research and evaluations and tacit knowledge on how those assumptions played out in practice, the time is ripe for asking what the evidence since 2004 can tell us about what the social accountability’s next generation might look like? This meta-analysis takes stock of that knowledge through a systems lens and presents a possible “social accountability 3.0.” It draws on a database of 157 cases of initiatives implemented by different civil society organizations around the world and the lived experience of practitioners. Scholarly debates present practitioners’ options in binary terms, contrasting 1.0 minimalist and 2.0 maximalist options. The review finds that this binary is inaccurate and misleading; it masks practitioners’ diverse interpretations of 2.0 and range of feasibility considerations. Instead, work in the last decade looks more like a mosaic. By opening the analysis to the broader range of evidence, and reflecting more critically on scholars’ assumptions, a more adaptable middle path for social accountability 3.0 emerges. Its key outcomes include contributing to systemic responsiveness and an operational mechanism to rework the social contract.

Scaling up Social Accountability in Complex Governance Systems: A Relational Approach for Evidencing Sustainability - Guerzovich, Florencia; Wadeson, Alix (World Bank, 2024)
When social accountability interventions scale up and their sustainability depends on the interactions of many agents and system components, related results are rarely observable at the end of an intervention. The 2019 OECD Development Assistance Committee’s (OECD DAC) revamped evaluations criteria for assessing sustainability acknowledges that such results are often emergent, and should be monitored and evaluated with this in mind. It therefore emphasizes a turn towards assessing complex processes prospectively. It also asks evaluations to consider how likely it is that these results are evident at the time they are monitored or evaluated. However,the social accountability field continues to have gaps regarding doing this effectively in practice. This paper presents and provides evidence from testing an innovative operational approach that has promising potential to support this aim - a sequential, relational rubric. This approach can support practitioners to monitor, evaluate and learn about the causal processes of scale up of social accountability interventions with an eye towards sustainability i.e., considering prospective sustainability. It is grounded in systems thinking, co-production and social learning theory, as well as links with collective governance and social contract theory for development. Evidence yielded from the authors’ testing of this approach on a sample of diverse projects from the Global Partnership for Social Accountability (GPSA) program revealed that the alleged ‘absence of evidence’ dilemma of social accountability scale up is due to ill-fitting concepts and methods for assessment. It challenges existing assumptions and findings that claim that social accountabilityprocesses do not scale and are unsustainable. The authors propose that by using fit-for-purpose concepts and methods with a focus on social learning and compromise – also called a ‘resonance pathway to scale’ which this paper discusses in detail – it is possible to observe loosely coordinated scale up processes at work in many (but not all) social accountability interventions and identify tangible evidence of prospective sustainability. An important caveat is that these processes, the outcomes they generate, and the corresponding evidence often look qualitatively different than the original intervention design and predictions for scale-up at that point in time. This is because the process of deliberation and compromise inherent to social accountability work in dynamic local systems introduces changes and new conditions for uptake by diverse actors in the public sector, civil society, and donor institutions. The paper concludes that even relatively small-scale localized projects of three to five years with budgets of less than one million USD, across different contexts and sectors can produce processes and outcomes which contribute to many forms of sustainability, including via scaleup.Furthermore, the cross-fertilization of learning and aggregation of results for scale-up across projects within and beyond the GPSA (and other programs) can help monitoring evaluation and learning (MEL) and social accountability practitioners alike to deliver on a program’s mandate. Doing so can also create new knowledge for the wider social accountability field that siloed interventions, lacking suitable concepts and methods for assessing scale-up and prospective sustainability, often fail to produce. The paper ends with recommendations for taking forward this approach and the associated benefits, implications and required investments.

READ ME - AM Knowledge Base - How-to user guide for the Zotero library - Prieto Martin, Pedro (Institute of Development Studies, 2024)
The library contains and categorises resources related to Adaptive Management and International Development, with some focus around the use of MEL to support it. At the time of writing the library contains 1907 resources and more than 1 Gigabyte of source documents. The documents include journal articles, books, reports, case-studies, toolkits, courses, blogposts, websites, and other grey literature. The resources are flexibly organised within a hierarchy of categories and sub-categories, which facilitate browsing and access. The knowledge base can be queried using Zotero’s powerful search functionalities, which not only include bibliographic fields such as title, authors, publisher, tags and notes, but also search within the full source text of the pdf documents. The Knowledge based can be accessed in different ways: • Through the user-friendly web interface. https://adaptdev.info • Using Zotero’s web interface. No need to register unless you want to access the pdf sources. • Using Zotero app, available for Windows, MacOS and Linux. This enables users to keep an updated copy of the library in their computer and get access to all the reference managing system’s functionalities and source documents. This illustrated “user guide” provides how-to instructions on how to use the library.

Learning Towards Zero: Learning in + from Practice in Systems Innovation - Burkett, Ingrid (Good Shift, 2024)
In early 2021 we were asked to support two initiatives in South-East Queensland by undertaking a developmental evaluation of how they collaborated across multiple partners to reduce, prevent and ultimately end homelessness. The initiatives, Brisbane and Logan Zero, started with the ‘Advance to Zero’ methodology, and adapted this as they progressed. We were asked to track how contributors collaborated to tackle the systemic nature of this challenge. This gave us the opportunity to work with both initiatives over a two year period to reflect back to them how change was happening towards the outcome, and to track learnings focused on systemic change along the way. While the evaluation report captures the details of this exploration, there were a myriad of insights from the process that could inform other systemic innovation initiatives - so this Learnbook outlines these. So much can be learnt from these initiatives that could help others seeking to create multi-sector and multi-organisation alliances for systemic change. Systemic work is, by its very nature, complex work. Anyone who is embarking on this type of work needs to be comfortable with learning forward, and working with diverse approaches that are context dependent. We share learnings not to suggest that this is ‘the’ way to undertake systems innovation, but to share one way that is showing promise in a particular context. This Learnbook outlines four key insights that could inform systemic change initiatives. 1. Human systems are complex and dynamic. When we are working in complex domains, we need to ‘learn forward’. 2. Shifting systems requires collaboration across multiple structures, sectors and organisations. The way collaboration is organised shapes action. And if the context is complex, then organising needs to be adaptive. 3. Leading systemic shifts is about collaborating, sharing power and distributing action. Governance models need to adapt to dynamic action and collaborative leading. 4. Data is critical, but let’s be clear about the type of data that is needed and how it is used. Action towards systemic change in human systems requires human-centred, lead data.

Guidance Note: Evaluation Criteria - Chaplowe, Scott; Child, Keith; AF-TERG (Adaptation Fund, 2024)

Disentangling Government Responses: How Do We Know When Accountability Work is Gaining Traction? - Fox, Jonathan; Halloran, Brendan; Fölscher, Alta; McGee, Rosie (Accountability Research Center, 2024)
Advocacy for public accountability aims to produce certain reactions from government officials or service providers. However, the reactions can be many and diverse, and it is not always clear to advocates how to interpret them and decide on next steps—whether to intensify efforts or back off; continue the same strategy or make adjustments. This paper presents a framework to help accountability advocates and practitioners interpret government reactions to their efforts and move forward appropriately. The framework arises from learning and reflection in the context of the International Budget Partnership (IBP)’s Strengthening Public Accountability with Results and Knowledge (SPARK) program. SPARK seeks to bolster the collective agency of marginalized communities and coalitions to advance democratic and equitable fiscal governance systems that channel public resources to services that address the priority needs of these historically excluded groups. The paper: · unpacks the broad umbrella category of government responses to citizen-led accountability initiatives, discerning within it three overlapping categories of responses, responsiveness, and accountable responsiveness (RRA), and pinpointing what distinguishes them; · maps out the public financial management (PFM) system as the critical conduit for any public service to progress from non-response through these three categories, reflecting the terrain and priorities of the SPARK program and IBP more broadly. Looking at RRA in the context of this PFM map offers key insights for the twin tasks of tracing and unblocking bottlenecks in resource management systems that hinder responsiveness and accountability, as well as demanding specific reforms that make the fiscal governance system more open to citizen engagement; and · discusses how the RRA framework can be used to take stock of partial or incremental responses and assess whether change is moving towards accountable governance, by reference to the SPARK program and other IBP efforts. The RRA framework supports accountability-claiming practice by facilitating close attention to the actions of and influences on key government stakeholders, careful interpretation of these, and consequential adjustments to advocacy strategies. Its implications for monitoring, evaluating, and learning (MEL) from practice are also significant. MEL has been evolving in the accountability field in line with the thinking and practical challenges behind the development of the RRA framework. The framework adds to a range of emerging approaches that support the application theories of change, evidence-based adaptive programming, and politically aware systems thinking in the complex task of building accountable responsiveness towards constituencies historically excluded from public goods and services, and ultimately equitable development outcomes from an intersectional perspective.
Artwork Size: 936692 Bytes
Keywords: Development studies; Public policy; Social policy

Systems-aware social accountability: Harnessing Practitioner Insights for More Responsive Governance - Guerzovich, Florencia (Pact, 2023)
The framework’s four principles, described below, are not exclusive to social accountability interventions; for example, Pact has published tools on how to use political economy analysis or integrate adaptive management principles into programming. Rather, the principles highlight how – by including the right combination of mindsets, management approaches, analytic approaches and tools – practitioners can develop and apply a deep understanding of a system to improve relationships within it, in support of greater accountability and improved service delivery. - Use a Holistic Lens - Recognize that there is always a system; understand and engage with it​: Developing a strong understanding of the system in which a project works and effectively integrating this knowledge into implementation increases the chances that local actors will sustain the interventions. - Right Fit the Approach​ - Equipped with a holistic understanding of the context, teams identify where they have a comparative advantage in a specific system. In other words, they identify where their resources, expertise and partners have the greatest potential for impact, then set realistic targets that they revise periodically. -Orchestrate Across The System -​ Practitioners take a networked, rather than a top-down, approach to capitalize on their convening power and maximize opportunity to facilitate the most-promising alignments across and within their projects, portfolios and systems. Orchestrators use their individual power, relationships and insight into the local system to enable, not impose, promising interactions and mitigate the risks of damaging ones. - Manage Adaptively - Monitor, evaluate and reflect on emerging patterns of data, evidence and learning to inform decision-making within and, equally importantly, across project cycles. Where possible, practitioners should consider how to use short-term projects to play a long-term game by linking new projects to the lessons and results of their predecessors and ensuring that they are responsive to the evolving context, particularly opportunities, limits and risks.

Insights for Influence: Understanding Impact Pathways in Crisis Response - Clark, Louise; Carpenter, Jo; Taylor, Joe (Institute of Development Studies, 2023)
The Covid-19 Responses for Equity (CORE) programme was a three-year initiative funded by the Canadian International Development Research Centre (IDRC) that brought together 20 projects from across the global South to understand the socioeconomic impacts of the Covid-19 pandemic, improve existing responses, and generate better policy options for recovery. The research covered 42 countries across Africa, Asia, Latin America, and the Middle East to understand the ways in which the pandemic affected the most vulnerable people and regions, and deepened existing vulnerabilities. Research projects covered a broad range of themes, including macroeconomic policies for support and recovery; supporting essential economic activity and protecting informal businesses, small producers, and women workers; and promoting democratic governance to strengthen accountability, social inclusion, and civil engagement. The Institute of Development Studies (IDS) provided knowledge translation (KT) support to CORE research partners to maximise the learning generated across the research portfolio and deepen engagement with governments, civil society, and the scientific community. As part of this support, the IDS KT team worked with CORE project teams to reconstruct and reflect on their impact pathways to facilitate South-South knowledge exchange on effective strategies for research impact, and share learning on how the CORE cohort has influenced policy and delivered change. This report presents an overview of these impact pathways and the lessons learnt from a selection of the projects chosen to represent the diversity of approaches to engage policymakers, civil society, and the media to generate and share evidence of the effect of the pandemic on diverse vulnerable groups.
Accepted: 2023-11-10T12:56:06Z

Life Stories From Children Working in Bangladesh’s Leather Sector and its Neighbourhoods: Told and Analysed by Children - Sayem, Mashrique; Sayed, Sayma; Maksud, A. K. M.; Reaz Hossain, Khandaker; Afroze, Jiniya; Burns, Danny; Raw, Anna; Hacker, Elizabeth (Institute of Development Studies, 2023)
CLARISSA (Child Labour: Action-Research-Innovation in South and South-Eastern Asia) has a participatory and child-centred approach that supports children to gather evidence, analyse it themselves and generate solutions to the problems they identify. The life story collection and collective analysis processes supported children engaged in the worst forms of child labour in Bangladesh to share and analyse their life stories. Over 400 life stories were collected from children who worked in the leather supply chain, or who lived and worked in leather sector neighbourhoods. Using causal mapping, 53 children who were engaged in or had experience of the worst forms of child labour collectively analysed the data. This resulted in children’s life stories becoming the evidence base for revealing macro‑level system dynamics that drive the worst forms of child labour. This paper is a record of the children’s analysis of the life stories and key themes they identified, which formed the basis of a series of seven child-led Participatory Action Research groups.
Accepted: 2023-11-07T09:14:57Z

Challenge-led Innovation Workbook. Organising for Systems Innovation at Scale - Burkett, Ingrid (Griffith Centre for Systems Innovation, 2023)
Organising for Systems Innovation at Scale Our team at Griffith Centre for Systems Innovation have been experimenting with and evolving a Challenge-led Innovation Approach (based on Mission-oriented approaches developed by Mariana Mazzucato at UCL IIPP and others internationally). We are using this approach to guide the way we work internally and engage with our systems innovation partners. We’ve facilitated intensive Re:Treats, worked with government bodies, businesses and civic organisations, and engaged deeply with others exploring this work. We have a bias for developing and testing HOW such approaches could be applied to respond to both local and global challenges rather than getting too caught up in the what and why of such approaches. We decided to openly share our learnings and thinking to date in this workbook, to spark conversations and innovation in both practice and thinking amongst those exploring how we work, and to learn together to address complex systems and challenges. We see this booklet as a first step in a longer learning journey. In it we share an overview of: the principles and processes that sparked our evolution to a Challenge-led Innovation framework (from Mission-oriented). examples of our learnings from other system innovators who are experimenting. an adaptable process to help guide the learning journey. learning tools and canvases to catalyse thinking, practice, and further adaptations. Part One sets out some foundations we’ve identified as important to Challenge-led Innovation. If you want to jump straight into the mapping process, we suggest you skip to Part Two. The final section, Part Three, focuses on what we have learnt about the conditions needed and how to get started on a Challenge-led initiative.

Anatomy of Social Accountability 3.0: A working framework for moving forward - Guerzovich, Florencia; Aston, Tom (Florianopolis, SC, Brazil: Grupo Politeia, ESAG, University of the State of Santa Catarina and act4delivery, 2023)
This note presents the key components of a potential next generation of social accountability programming. The key aspects outlined emerge from a review of over 150 cases as well as tacit knowledge from dozens of practitioners (Guerzovich and Aston, 2023).The main thread of social accountability 3.0 and what distinguishes it from previous generations is a focus on its contribution towards more responsive systems and accountable social contracts. In particular, social accountability should be considered as an operational means to rework social contracts.Over the last two decades, the field has been defined by a supposed dichotomy between minimalist (i.e., “widget” based) approach or a maximalist (“go big or go home”) approaches. Yet, most practice sits somewhere between these extremes. The paper presents various options derived from practice; practitioners should choose whichever best fits their own particular circumstances.Those hoping to improve and expand their practice with a systems-lens may benefit from considering how they approach the key elements of social accountability 3.0.
Keywords: accountability; development; responsiveness; social accountability; social contract; systems

Whole Philantrhopy: Consciousness + Connectedness = With, not For - Chicago Beyond (2023)

Applying the River of Life Method to Support Reflection and Learning in Terre des hommes Nepal - Karki, Shanta; Giri, Roju; Neupane, Sudarshan; Snijder, Mieke; Apgar, Marina (Institute of Development Studies, 2023)
The RoL method is a visual narrative method that helps people tell stories of the past, present, and future. Individuals can use this method to introduce themselves in a fun and descriptive way. A group can use it to understand and reflect on the past and imagine the future of a project. Besides, it can also be used to build a shared view of a process over time while acknowledging different and perhaps contradictory perspectives. The method uses drawings rather than text, making it useful in groups that do not share a common language. Metaphors from a river are used to explore aspects of a story – such as whirlpools depicting challenges or lakes suggesting a sense of calm etc. When used in a group, it is an active method, engaging people in the process of storytelling and listening through visualising their experiences and using metaphors to explore in depth. In CLARISSA, we adapted the RoL method to document our collective understanding of the story of implementation of the programme as part of the programme’s monitoring, evaluation and learning component. The purpose was to surface the details of our process of the systemic Action Research that we are undertaking with children in the worst forms of child labour and business owners. We used the same river metaphors as is often applied when the method is used with individuals.
Accepted: 2023-09-18T10:13:55Z

Lasting Roots: Naatal Mbay and the Integrated Finance Model in Senegal - Fowler, Ben; Courbois, Laura (USAID, 2023)
This report addresses the well-recognized evidence gap1 on the longer-term impacts created by marketdriven programming; specifically, programming influenced by market systems development (MSD) principles. It does so by presenting the findings of an ex-post study conducted three and a half years after the close of USAID’s Feed the Future Senegal Naatal Mbay Activity (hereafter Naatal Mbay) in 2019. It examines the scale and sustainability of changes resulting from Naatal Mbay’s introduction of an integrated finance model (IFM) – described in Error! Reference source not found. below – in the domestic rice sector. This study is one in a series of ex-post evaluations that are being conducted between 2023-2026 on USAID-funded MSD interventions around the world. This study focused on four questions, noted below in Figure 1. These were addressed using a mix of desk research, 122 key informant interviews with market actors and other stakeholders remotely and in Senegal, focus group discussions with 26 rice producers networks in Senegal, and a validation workshop with USAID/Senegal, implementing partner staff and market actors. Findings were analyzed leveraging the Disrupting System Dynamics (DSD) framework (see Figure 4 in the body of the report) as an analytical tool for understanding systems change.

Using a ‘Partnership Rubric’ in Participatory Evaluations - Snijder, Mieke; Hicks, Jacky; Paul, Sukanta; Arulanantham, Amit; Apgar, Marina; Afroze, Jiniya; Karki, Shanta; Mareschal, Sophie; Prieto Martin, Pedro; Uddin, Forhad; Veitch, Helen (Institute of Development Studies, 2023)
Programmes that aim to tackle complex societal issues, such as the worst forms of child labour, require rich partnerships that bring together different perspectives. CLARISSA’s consortium partnership adopts an empowerment approach to the interventions we deliver and our ways of working together. Part of this approach involves ongoing reflection and learning about how we work together in our partnership, and how this can be adapted if needed. This learning note focuses on a method used in CLARISSA to both reflect on and strengthen how we work in partnerships – the partnership rubric. We found that using the rubric flexibly was key to mitigating some of the challenges of such a complex consortium. This included using it in different sizes of forum, with different levels of preparation. Periodically adapting it for country context and as new partners came on board also helped ensure a shared sense of our preferred ways of working as the project progressed.
Accepted: 2023-07-13T10:20:02Z

Guide to Innovation Partnerships – A United Nations Ghide - Halse, Michelle; Ginsberg-Fletcher, Gabriella; Balbi, Luisa (UN Global Pulse, 2023)

Applying Adaptive Management in a Fragile Context – Case Study - Lonsdale, Jane; Green, Duncan; Robertson, Kelly (DT Global, 2023)
DT Global is proud to introduce our new Guidance Note: Practical Introduction to Adaptive Management There is a growing consensus around adaptive management as an effective (even necessary) approach when programs are tackling complex development problems. While there is no standard definition of adaptive management, there is general agreement that such programs need to routinely engage with and respond to program context; constantly test what works in that context; and adjust approaches, plans, and activities based on continuous learning. However, there remains a more limited body of evidence about what this looks like in practice—the enabling conditions, systems, resourcing, skills, and attitudes to effectively operationalise adaptive management. There is also limited guidance around when adaptive management is required, and to what extent—both critical and often overlooked considerations when planning for successful adaptive management. This Guidance Note draws together lessons and good practice in adaptive management from across DT Global’s diverse portfolio of donor-funded programs. It outlines our conceptual framework for adaptive management, with practical guidance on how it can be applied by our program teams. It is also designed to help our teams distinguish adaptive management from good (non adaptive) project management, consider when adaptive management is most useful on a program, and how adaptive a program (or part of a program) should be.

Applied political economy analysis for human rights programs and campaigns: A guide for practitioners (second edition) - Pact (Pact, 2023)
This updated guide provides practical guidance to practitioners in the human rights sector and beyond on how to integrate Applied Political Economy Analysis

Knowledge Retention and Transfer (KRT) - Model overview presentation - USAID (USAID, 2023)
What problem is the KRT model trying to solve? For workforces that experience continuous staff turnover, the lack of systematic knowledge transfer can often lead to: - Loss of programmatic momentum, - Duplication of efforts and frustration, and - Wasted time and resources. The Knowledge Retention and Transfer (KRT) model provides tools, processes, and practices to individuals, teams, offices, and organizations to improve knowledge handover, which in turn improves efficiency and programmatic and operational learning.

When Complex is as Simple as it Gets: Guide for Recasting Policy and Management in the Anthropocene - Roe, Emery (Institute for Development Studies, 2023)
Many readers recognise and understand that complex is about as simple as it gets for major policy and management. This guide is for those unwilling in the Anthropocene to shrink back into the older platitudes about ‘keep it simple’ and ‘not to worry, we’ll scale up the analysis later on’. This guide offers key concepts, methods, counternarratives, and analogies that recast major policy and management issues in ways that do not deny their complexity but help render them more tractable for action.
Accepted: 2023-06-06T09:51:20Z

CLARISSA Cash Plus: Innovative Social Protection in Bangladesh - Roelen, Keetie; Howard, Neil; Afroze, Jiniya; Aktar, Afrin; Ton, Giel; Huq, Lopita (Institute of Development Studies, 2023)
Social protection, and cash transfers especially, have been found to have many positive impacts on families’ lives and are now widely recognised as a cornerstone of any prosperous, fair society. The CLARISSA Cash Plus intervention is an innovative social protection scheme for tackling social ills, including the worst forms of child labour (WFCL). Combining community mobilisation, case work and cash transfers, it aims to support people in a poor neighbourhood in Dhaka to build their individual, family, and group capacities to meet their needs. An increase in capacities is expected to lead to a corresponding decrease in deprivation and community-identified social issues that negatively affect wellbeing, including WFCL.
Accepted: 2023-06-30T11:14:55Z

Participation for Humanitarian Innovation - Background Paper - Smith, Amy; Thompson, Martha; Saida Benhayoune; Crespo Cardona, Omar (Elrha, 2023)
A resource designed to help organisations, teams and individuals manage innovation journeys responsibly and successfully. We have partnered with MIT D-Lab to develop a new resource to drive greater diversity and inclusion within project design and implementation. The Participation for Humanitarian Innovation (PfHI) toolkit sets out a robust approach to setting expectations for and monitoring the degree of participation within research and innovation projects for, with, and by people affected by crisis. The PfHI toolkit is composed of five tools: Opportunity Adviser: Identify and prioritise the desired benefits of participation while weighing the potential barriers. Participation Matrix: Agree on the precise degree of participation to target at a given project stage. Resource Navigator: Select tools and processes to address the needs of stakeholders, the project objectives and context. Quality Guidance: Ensure engagements are delivered to the highest standard. Assessment Matrix: Collectively evaluate the degree of participation achieved during an activity or project phase, learn and adapt. By applying these tools before, during and after a research and/or innovation project, implementors can ensure that stakeholders/end-users are included and participating at the highest possible degree. We encourage users of the PfHI toolkit to consider how else to integrate the tools into existing practices. For instance, the Evaluation Matrix could be used to supplement existing MEAL activities to: Establish baselines to measure changes in participation over time. Track the degree of participation/engagement of stakeholders. Regularly assess the degree of participation. Seek feedback from participants about the degree of their participation. Monitor the progress of activities related to stakeholder engagement. Our Participation for Humanitarian Innovation toolkit represents our ongoing commitment to responsible research and innovation across our portfolio of grants and for the humanitarian sector more broadly. We hope you will consider downloading and using the toolkit on your next project.

Participation for Humanitarian Innovation - Toolkit - Smith, Amy; Thompson, Martha; Saida Benhayoune; Crespo Cardona, Omar (Elrha, 2023)
A resource designed to help organisations, teams and individuals manage innovation journeys responsibly and successfully. We have partnered with MIT D-Lab to develop a new resource to drive greater diversity and inclusion within project design and implementation. The Participation for Humanitarian Innovation (PfHI) toolkit sets out a robust approach to setting expectations for and monitoring the degree of participation within research and innovation projects for, with, and by people affected by crisis. The PfHI toolkit is composed of five tools: Opportunity Adviser: Identify and prioritise the desired benefits of participation while weighing the potential barriers. Participation Matrix: Agree on the precise degree of participation to target at a given project stage. Resource Navigator: Select tools and processes to address the needs of stakeholders, the project objectives and context. Quality Guidance: Ensure engagements are delivered to the highest standard. Assessment Matrix: Collectively evaluate the degree of participation achieved during an activity or project phase, learn and adapt. By applying these tools before, during and after a research and/or innovation project, implementors can ensure that stakeholders/end-users are included and participating at the highest possible degree. We encourage users of the PfHI toolkit to consider how else to integrate the tools into existing practices. For instance, the Evaluation Matrix could be used to supplement existing MEAL activities to: Establish baselines to measure changes in participation over time. Track the degree of participation/engagement of stakeholders. Regularly assess the degree of participation. Seek feedback from participants about the degree of their participation. Monitor the progress of activities related to stakeholder engagement. Our Participation for Humanitarian Innovation toolkit represents our ongoing commitment to responsible research and innovation across our portfolio of grants and for the humanitarian sector more broadly. We hope you will consider downloading and using the toolkit on your next project.

USAID ADS 201 - Operational Policy for the Program Cycle (Update 05/22/2023) - USAID (USAID, 2023)
The Program Cycle is USAID’s operational model for planning, delivering, assessing, and adapting development programming in a given region or country to advance U.S. foreign policy. It encompasses guidance and procedures for: 1) Making strategic decisions at the regional or country level about programmatic areas of focus and associated resources; 2) Designing supportive projects and/or activities to implement these strategic plans; and 3) Learning from performance monitoring, evaluations, and other relevant sources of information to make course corrections as needed and inform future programming.

The Equitable Evaluation Framework - May 2023 Expansion - EEI (Equitable Evaluation Inititiative, 2023)

The adoption of innovation in international development organisations: lessons for development co-operation - Kumpf, Benjamin; Jhunjhunwala, Parnika (OECD Publishing, 2023)
Addressing 21st century development challenges requires investments in innovation, including the use of new approaches and technologies. Currently, many development organisations prioritise investments in isolated innovation pilots that leverage a specific approach or technology rather than pursuing a strategic approach to expand the organisation’s toolbox with innovations that have proven their comparative advantage over what is currently used. This Working Paper addresses this challenge of adopting innovations. How can development organisations institutionalise a new way of working, bringing what was once novel to the core of how business is done? Analysing successful adoption efforts across five DAC agencies, the paper lays out a proposed process for the adoption of innovations. The paper features five case-studies and concludes with a set of lessons and recommendations for policy makers on innovation management generally, and adoption of innovation in particular.

Process Tracing Innovations in Practice: Finding the Middle Path - Aston, Thomas; Wadeson, Alix (Institute of Development Studies, 2023)
Evaluation practitioners in the international development sector have given considerable attention in recent years to process tracing as a method for evaluating impact, including discussion of how to assess the relative importance of causal factors. Despite the increasing interest, there is a relative dearth of examples of practical learning and evidence of applying process tracing in practice. This CDI Practice Paper draws on comparative learning from applying three different types of process tracing in international development initiatives. It argues in favour of a ‘middle path’ of applying evidence tests and rubrics to structure evaluative judgements rather than formal Bayesian updating or looser forms of process tracing. It also calls attention to the potential added value of taking a participatory approach, offering practical recommendations for how to do this effectively.
Accepted: 2023-03-27T15:20:09Z

The Secret Sauce of Development Professionals: Tools for Assessing TOR Potential to Source Scalable Learning Interventions - Barton, Adam (Research on Improving Systems of Education (RISE), 2023)
Terms of reference (TORs) play an outsized role in driving scalable educational programming. These procurement documents shape, constrain, and signal programme priorities and possibilities. Successful funders and implementers across the globe hold rich processual knowledge about this documentation, which they use to draft and assess TORs. This project explores such best-practice knowledge around TOR review, seeking to support the design and implementation of educational programmes that can improve learning at scale in developing contexts.

Reflections on Ten Years of USAID’s Experience with Political Economy Analysis and Thinking and Working Politically - Brinkerhoff, Derick; Cassidy, Marc (TWP CoP, 2023)
The global Thinking and Working Politically (TWP) Community of Practice (CoP), the Washington DC TWP CoP, and USAID organised a webinar on 5 December 2022 to take stock of how USAID and its partners have used PEA to inform programme strategy, design and implementation, and support TWP. This paper synthesises the key points arising from the webinar, including observations on the impacts, opportunities, challenges, and prospects for PEA/TWP to become more deeply adopted and sustained as a development methodology and approach across sectors. It starts by defining key concepts. It then highlights insights from the discussions of the impact of the application of PEA and TWP principles across sectors. The paper concludes by looking at progress achieved to date, as well as constraints and opportunities to increase the uptake of both thinking and working politically in USAID-sponsored programming.

Adaptive Evaluation: A Complexity-based approach to Systematic Learning for Innovation and Scaling in Development - Gokhale, Siddhant; Walton, Michael (Center for International Development, Harvard University, 2023)
Nearly all challenges in international development tend to be complex because they depend on constantly evolving human behaviour, systems, and contexts, involving multiple actors, entities, and processes. As a result, both the discovery and scaling of innovations to address challenges in development often involve changes in system behaviour or even system-level transformation. This is rarely a linear process over time and can result in unexpected outcomes. Existing evaluation techniques commonly used in international development, including Randomized Control Trials (RCT) and quasi-experimental methods, are good at assessing specific effects of interventions but are not designed for the change processes inherent to innovation and scaling within a system. There is a need to reconstruct how we use existing measurement tools, techniques, and methodologies so that they capture the complexity of the environment in which an intervention or change occurs. We introduce Adaptive Evaluation, designed to learn at various levels of complexity while supporting the transformation needed to foster sustainable change. An Adaptive Evaluation uses three main approaches to work with complex questions—systems diagnosis, theorybased assessment of change processes, and iterative designs. An Adaptive Evaluation typically builds hypotheses from field-based interactions, emphasizes learning over testing, advocates open-mindedness with techniques, and appreciates the value of dialogue and participation in navigating complex processes. It can use RCT or similar techniques to analyse specific processes within a system or a development cycle, but these are embedded in a broader approach to assessment and interpretation. It is designed to be flexible and adjust to shifting contexts. Finally, an Adaptive Evaluation can be applied at any stage in a complex intervention's lifecycle, from the interpretation of the system and change processes to rapid experimentation, prototyping, and testing of select interventions, and then adaptation to different settings for impact at scale. This paper provides the theoretical basis for an Adaptive Evaluation—the main approaches, core ideology, process, and applications.

Glass Half Full: Civic Space and Contestation in Bangladesh, Sri Lanka and Nepal - Surie, Mandakini D.; Saluja, Sumaya; Nixon, Nicola (The Asia Foundation, 2023)
The past decade has witnessed a surge of interest in and concern over the global trend toward democratic regression. In South Asia, regulatory and institutional frameworks have become increasingly restrictive, curbing the ability of citizens and civil society organizations to occupy and use civic spaces to organize, express themselves, and participate in decisions that affect the lives of people whose interests they serve. Of course, this is only one half of the story. The other half is how citizens, collectives, and organizations adapt by carving out spaces where they can maintain—or even expand—the boundaries of their engagement in local and national civic spaces. In this paper, we examine how civic spaces are evolving in three South Asian countries—Bangladesh, Sri Lanka, and Nepal—drawing on the experiences and perspectives of civil society representatives. The paper concludes with recommendations for donors and development practitioners on how they can make relevant investments that will strengthen civic spaces and support democratic resilience in the region.

Celebrating Adaptive Delivery: A View from the Frontline in Myanmar - Barnes, Katrina; Lonsdale, Jane (Institute of Development Studies, 2023)
The conversation on adaptive management has grown fast amongst development actors. These conversations often focus on designing, commissioning, and managing large-scale development programmes. Exactly how this impacts the frontline, the implementers, and day-to-day project delivery is still being debated. Yet, perspectives drawn directly from practice are often largely missing within these debates. This paper is written by two development practitioners. Through this paper, we reflect on the difference between adaptive management and adaptive delivery, and how this interacts with risk and aid accountability, particularly in contexts of fragility. Drawing on examples of Oxfam in Myanmar work and our personal insights in relation to delivering programming across humanitarian, peace-building, and development, we suggest that in complex, conflict-affected, and highly political environments adaptive delivery already happens far more regularly than is currently recognised, as a necessity to get activities delivered. However, it happens despite the system, not because of it, and is therefore often hidden and carried out ‘under the radar’ rather than celebrated as a success in difficult environments. This paper was written in 2019, before the military seized control of Myanmar in February 2021. Whilst it draws on examples from pre-2021 Myanmar to illustrate real life cases, it is a contribution to a broader global debate on adaptive management in practice, specifically in fragile contexts. This is not specifically aimed at practitioners working in Myanmar at present, who are now working in a protracted crisis. This paper makes tangible recommendations on steps that donors, international non-governmental organisations, local staff, and partners could take to promote a system of encouraging and celebrating adaptability in programme delivery in fragile contexts.
Accepted: 2023-02-02T14:48:03Z

Two Loops Guide - Colchester, Joss (Si Network, 2023)
The Two Loop model is a nonlinear theory of change based upon the ideas of living systems created by Margaret Wheatley & Deborah Frieze. It provides us with insight into the simultaneous growth and decline process that are underway within a system during a transition period. In this way, it provides a heuristic for us to better try and recognize and connect the past to the future during a change process. It likewise gives us the possibility to try and come to some consensus about where we might be as a group in this process of change and the best actions to take at different stages. This guide will be of relevance for anyone involved in a complex organizational change process.

Understanding Political Economy Analysis and Thinking and Working Politically - Whaites, Alan; Piron, Laure-Hélène; Menocal, Alina Rocha; Teskey, Graham (FCDO and TWP CoP, 2023)
This guide is adapted from work by the UK Foreign, Commonwealth and Development Office (FCDO) with inputs from members of the Thinking and Working Politically Community of Practice (TWP CoP). It outlines how to understand and use a set of analytical tools that are collectively known as Political Economy Analysis (PEA). The guide aims to equip practitioners to act in an informed manner, given that development objectives are invariably politically complex, and entail engaging with counterparts’ political incentives and preferences. The guide summarises different types of tools – from very light-touch to more in-depth approaches – and provides advice on how development professionals can decide what is most appropriate in a given context, with illustrations based on the experiences of teams working on these issues. This guide will help development professionals and others to make use of PEA and to apply it to their own specific needs. The first part of the guide offers a general picture of the approach. The second part provides more specific guidance for those who are tasked with deploying a PEA. Contents –> Main audience What is PEA, its role and purpose (Section 2) –> General information for all readers The main elements of PEA (Section 3) Thinking and Working Politically (Section 4) –> Core information for teams planning and using PEA How to ensure quality (Section 5) –> Essential reading for those directly responsible for a PEA Important concepts and terminology (Annex) –> General information for all readers

Creative Activities for Work Teams and Communities of Practice. A guide for designing participatory, productive, and engaging sessions - AKF (2023)
This free guide aims to build the skills and confidence of people designing and facilitating participatory, productive, and engaging workshop sessions for work teams and communities of practice by providing them with a series of creative activities that can be run in-person or virtually. By the time they have gone through this guide, session leads will be able to: Describe ways in which creative activities can be of value to teams and communities of practice, Design both in-person and virtual workshop sessions that incorporate creative activities, Facilitate participatory, productive and engaging workshop sessions that incorporate one or several creative activities, and Lead an effective debrief following a creative activity. This toolkit was developed by the Aga Khan Foundation.

Virtual Facilitation Techniques for Work Teams and Communities of Practice. A guide for designing and running dynamic and engaging remote sessions and meetings - AKF (2023)
This free guide aims to build the skills and confidence of people designing and facilitating virtual meetings, including online community of practice sessions. By the time they have gone through this guide, community of practice facilitators will be able to: Design an engaging and participatory virtual session, Develop an effective, varied and realistic agenda for a virtual session, Apply a wide range of virtual facilitation techniques, and Using technology to facilitate virtual sessions. This toolkit was developed by the Aga Khan Foundation.

Problem Framing Canvas Handbook: Making sense of problems for better responses - Burkett, Ingrid (Griffith Centre for Systems Innovation, 2023)
This canvas and workbook were borne out of frustration and a sense of possibility. We’ve worked in various tricky and complex contexts where we were confronted with a raft of responses ranging from: ‘quick fixes’, ‘offthe-shelf solutions’, ‘transplanted solutions’, to ‘we’re stuck’. We’ve seen the effects of analysis paralysis, diving down wombat holes (the Australian alternative to rabbit holes!), and getting lost in problems that were not really the problem. So, we started to explore ways in which we could work with people to ‘zoom out’ and focus on how they could frame and name problems more effectively, recognise different types of problems, and start to ‘learn’ their way into responding. We developed some of our own ways of helping people, and we borrowed some of the great tools and frameworks others have used to support better framings and responses to problems. We drew these together in a canvas to help people spend more time framing problems rather than jumping headlong into problem ‘solving’. We found that this approach helped people to develop better skills to analyse problems and to reflect, test, and learn about possible responses more effectively. The response from teams we’ve worked with has been very positive – and we’ve refined the canvas via feedback and testing. In 2022 we shared the canvas in one of our social media posts and were inundated with requests for deeper explanations of each of the segments. In response, in 2023 we are making our Problem Framing Canvas workshop materials open source through this workbook.

Si Guides on System Innovation - Colchester, Joss; Si Network (Si Network, 2023)
So you have just hear about systems thinking and systems innovation and are keen to know more, Ok sparky let's get started.We have structured all the content into four main areas that we think you ...

Setting new standards for better MEL. Lessons for grantees & funders - Colnar, Megan; Azevedo, Andrea; Tolmie, Courtney; Caddick, Hannah (Open Society Foundations, 2023)
How can donors and grantees work together to create effective monitoring, evaluation, and learning (MEL) practices that drive field-wide transformation? The Open Society Foundation’s Fiscal Governance Program found success by focusing on six key approaches, including empowering grantees and relinquishing power. In 2021, an external close-out evaluation by Intention to Impact of the program (which ran for 7 years and gave over $150 million in grants) revealed something pretty remarkable—the program’s deliberate focus on strengthening field-wide monitoring, evaluation, and learning practices was a success. Substantial capacity increases were observed across key institutions and grantees, new complexity-sensitive practices and methods were being actively championed and deployed, and a growing community of better-connected practitioners were exchanging tips and tricks on how to apply smart, context-specific MEL across fiscal governance issues. What’s more, in this evaluation, most grantees gave high praise to these efforts. So, how did this come about? We detail the six different approaches we used in our new publication Setting new standards for better MEL: Lessons for funders and grantees. The approaches range from checking power dynamics to growing skills for evaluative thinking and seeding peer learning and field-wide research. The publication is paired with a toolkit and showcases resources we used and iterated on across the various approaches.

The Art of Scaling Deep - Research in Summary - Fraser, Tatiana (The Systems Sanctuary, 2023)
Over the last 15 years the concept of scale has become a foundational part of the apparatus of the social and environmental change sector. A business mindset of growth has been seamlessly transferred to the social and environmental problems we are collectively trying to shift in the world. Scaling up, (influencing policy) has been considered the strategic pathway to systems change. Scaling out (spreading new models) is seen as a pathway to success. The allure of these scaling theories lies, in part, in their tangibility, and the easy way in which they can be measured. However this focus on growth has shifted our attention away from a series of messy truths. Sometimes bigger isn't better. Endless growth is not sustainable and our urgency to try to fix the problem and seek solutions may be part of the crisis we are in. One unintended consequence of this has been that another type of scale has been devalued and as a result, under-resourced. It’s a scale that values the slow steady work of deepening relationships. It recognizes the significance of context, building connections that bridge diverse communities and it prioritizes inner work and healing as integral components of the scaling process. We call this type of scale ‘Scaling Deep’ and we believe that adequately supporting it, and funding it, holds the greatest potential for long lasting systemic change. The purpose of this research has been to delve deeper into the art and craft of Scaling Deep. Ultimately, our goal is for it to become firmly ingrained within the recognized realm of social change, alongside the well-established concepts of scaling up and scaling out. We want practitioners who are Scaling Deep to be able to harness the wisdom and power of this work and to talk about it openly, with confidence and credibility. We want understanding of this approach to flourish and evolve and for it to be appropriately celebrated and supported. We want decision-makers to be informed and inspired by the principles and practices of scaling deep. For it to be embedded in theories of change as an essential component of decision-making processes within the wider change discourse. Importantly we want leaders who are Scaling Deep to have access to sufficient resources and to receive the care from the field that they need to thrive. We would like to see organisations that have the power to invest, to align their efforts with the profound impact that scaling deep aspires to cultivate in the world. Before we begin, let us be crystal clear. We are not opposed to scaling up and out as strategies for change. Innovation and scaling what is working is part of how we evolve as humanity. As social entrepreneurs ourselves, we have both done this twice over. We value it and we know it’s important. We recognize these are strategies for creating widespread impact and effecting positive change. Our intention here is not to dismiss or undermine the value of scaling up and out, but rather to encourage a broader and more holistic perspective that includes other dimensions of scale. There is a need to understand how the different scaling approaches can work together, rather than seeing them as hierarchical and disconnected. Our ultimate goal is to equip the change sector with a more inclusive, expansive, and powerful approach to tackling the myriad challenges we face, by exploring the potential of Scaling Deep as a transformational strategy for systems change.

Building a team culture for Adaptive Management in MSD: 5 Strategies MEL Managers Say Work - Gover, Dun; Nasution, Zulka; Okutu, David; Bolder, Meghan; Henao, Lina (USAID, 2023)
Keywords: Bolder Meghan; Gover Dun; Henao Lina; Nasution Zulka; Okutu David

Enhancing partner and system-level learning: 8 Tips from MEL Managers - Gover, Dun; Nasution, Zulka; Okutu, David; Bolder, Meghan; Henao, Lina (USAID, 2023)
Effective learning is a key driver of market systems change, with the potential to enhance system competitiveness, resilience, and inclusiveness. Shifting the Locus of Learning: Catalyzing Private Sector Learning to Drive Systemic Change recently outlined a rationale for enhancing the scale and quality of learning in a system and identifying 10 strategies programs can contextualize to catalyze learning. These strategies are also backed with robust examples from 13 programs doing this work across 11 countries. To deepen insights on what MSD Monitoring, Evaluation and Learning (MEL) Managers have experienced in putting several of those strategies into practice, the Feed the Future Market Systems and Partnerships (MSP) Activity convened a series of peer discussions as part of a larger initiative (see Figure 1). This brief shares the collective learning and experience on this topic of three senior MEL Managers who were interested in and had experience with this topic. The group represented full-time, program-based MEL Leads working on MSD programs funded by USAID and DFAT, based in Fiji, Albania, and Kosovo working for Adam Smith International, SwissContact, and DT Global, respectively. From those discussions, this paper synthesizes eight tips from MEL Managers for practically enhancing partner and system-level learning: 1. Identify the right decision-maker(s) at potential partners. 2. Use a co-creation process to identify learning opportunities. 3. Use diagnostics and assessments to strengthen partner and system capacity for actionable learning. 4. Use a phased capacity strengthening process tied to behavior change. 5. Measure partners’ continued investment in and use of learning—not the continuation of specific learning activities. 6. Work with sector-level institutions for scale but be aware of risks. 7. Leverage informal communities of practice to share learning. 8. Use the right terminology to talk about partner and system-focused learning.

Practioners Guide to Assessing Systems Change - Gover, Dun; Nasution, Zulka; Okutu, David; Bolder, Meghan; Henao, Lina (USAID, 2023)

Value for Investment: Application and Insights. Youth Primary Mental Health and Addictions Evaluation - King, Julian; Crocket, Alicia; Field, Adrian (Dovetail Consulting Ltd., 2023)
This document describes key principles of the Value for Investment evaluation approach and provides guidance on how to use it to design and implement evaluations, using the evaluation of the Youth Primary Mental Health and Addictions (Youth PMHA) initiative as an illustrative example.

Assessing Value for Money: the Oxford Policy Management Approach - King, Julian; Daniel Wate; Esther Namukasa; Alex Hurrell; Frances Hansford; Patrick Ward; Shiva Faramarzifar (Oxford Policy Management, 2023)
This document offers practical guidance for assessing the Value for Money (VfM) of government- and donor-financed programmes and policy interventions. In line with OPM’s focus and mission, it has been predominantly applied in the international development sector, but the approach upon which it is based is also used in the context of domestic public policy and programmes.1 There is increasing scrutiny on VfM in international development, but a lack of appropriate methods to support its assessment. There is a risk of reaching invalid conclusions if VfM evaluation is tied to a narrow set of indicators devoid of any evaluative judgement—for example, by emphasising the most readily quantifiable measures rather than the most important (but harder to quantify) aspects of performance, or by focusing on the quantification of outputs and outcomes at the expense of more nuanced consideration of their quality, value, and importance. The approach presented in this guide combines theory and practice from evaluation and economics to respond to requirements for accountability and good resource allocation, as well as to support reflection, learning, and adaptive management. It involves developing and implementing a framework for: • organising evidence of performance and VfM; • interpreting the evidence on an agreed basis; and • presenting a clear and robust performance story. This guide sets out a framework for making and presenting judgements in a way that opens both the reasoning process and the evidence to scrutiny. The approach is designed to be used in alignment with broader monitoring, evaluation, and learning (MEL) systems—both for efficiency's sake, and to ensure conceptual coherence between VfM evaluation and wider MEL work. The VfM framework achieves these aims by: • using explicit criteria (aspects of performance) and standards (levels of performance) to provide a transparent basis for making sound judgements about performance and VfM; • combining quantitative and qualitative forms of evidence to support a richer and more nuanced understanding than can be gained from the use of indicators alone; • accommodating economic evaluation (where feasible and appropriate) without limiting the analysis to economic methods and metrics alone; and • incorporating and building on an approach to VfM evaluation which is familiar to international aid donors.

Laudes Rubrics - Measurement and Learning Approach - Laudes Foundation (Laudes Foundation, 2023)
How can we measure and learn when promoting systems change? It’s a challenge that has inspired Laudes Foundation to develop a rubrics-based methodology to help us, our partners and the wider field of philanthropy, understand our contribution to change, while learning and adapting to new and unforeseen circumstances. Change cannot be captured by numbers alone because metrics put the focus on what can be counted, not always what’s most important. Rubrics are a framework that set a standard for what ‘good’ looks like – and create a shared language for describing and assessing it using both quantitative and qualitative evidence. At Laudes Foundation, the rubrics are integrated into our grantmaking processes – from the design phase through to measurement, evaluation and learning. The four Laudes Foundation rubrics categories Laudes Foundation has developed 21 rubrics that work across different levels, from processes to long-term impact. When measuring a specific initiative, a smaller set of relevant rubrics are chosen and assessed on a rating scale from ‘harmful’ to ‘thrivable’. The 21 rubrics are categorised into four groups, with some natural overlap between categories B, C and D. Category A focuses on the process-related aspects of initiatives, including design, implementation, monitoring, communication and learning, and organisation and network effectiveness. Category B focuses on the early and later changes that need to happen to create the right conditions to achieve the 2025 outcomes. Category C captures the 2025 outcomes, focusing on how policymakers, financiers, business leaders, and workers and producers behave. Category D captures the 2030 impacts, describing the new reality created as a result of sustained efforts.

On the Right Tack: Reflections on Coalition-Building Initiatives across The Asia Foundation - Nixon, Nicola; Yates, Peter; Saluja, Sumaya; Yi, Su Lae; Lucas, Miranda; Bain, Katherine (The Asia Foundation, 2023)
Coalitions—groups of organizations and individuals that work together to pursue a common policy goal or reform—are crucial to development. Some of The Asia Foundation’s longest-standing and most successful development programs and portfolios have used coalition-building as an implementation modality. This paper examines successful initiatives in Bangladesh, Nepal, Philippines, Sri Lanka, Thailand, and Timor-Leste. By delving into each of these, we shed light on this coalition-building modality, sharing insights about how externally supported coalitions emerge and function and contributing ideas about how such support can be adapted to other contexts. We hope that this paper and these case studies contribute to innovations in contemporary development theory and help practitioners use and move beyond some formulaic and technocratic modalities, such as traditional capacity development, knowledge transfer, and technical assistance. People, relationships, and working partnerships can then take center stage, and sustained cooperation will gain primacy over short-term outputs and deliverables.

Developing a Systems Thinking Lens for Collective Leadership - O’Donnell, Joan (Collective Leadership for Scotland, 2023)
Systems thinking approaches are gaining traction as an effective way of understanding and working with increasing complexity. They are being put forward by the Organisation for Economic Co-operation and Development as well as the World Health Organisation as a way to tackle the complex and unpredictable environments we now operate in. As the world has become increasingly interconnected, national or local boundaries cannot isolate and control social problems. The climate emergency, war and political instability have become everyday realities that accompany an ever-widening gap between those who have and control resources, and those who do not have enough to meet basic needs. The usual responses are becoming more and more ineffective; all interventions have unseen consequences or emergent properties that cannot be predicted using an inputoutput outcomes model. Designing interventions therefore requires a fresh lens to manage our way through turbulence and uncertainty. Most complex situations benefit from a complementary mix of a systematic (linear) decisive intervention as much as a systemic (whole-view) understanding of a proposed intervention. In this sense, adopting a systems thinking approach requires both framing the situation using a systems lens, as well as a commitment to take action and learn forward in realtime. When utilised in this way, a systems thinking lens complements rather than replaces traditional management tools. It draws on well-tested concepts and tools and relies on the systems leader to develop their practice, which includes the art of knowing what is needed and when. It is about threading traditional planning methods together with an understanding of the interrelationships, multiple perspectives and boundary judgements that influence the framing of a situation. It is becoming increasingly clear that effective leadership is embedded and invested: objective management science has no place in a world bound by interconnectedness and unintended consequences. Leaders are, as we shall see, an inherent part of both the problem and the proposed intervention. The purpose of this paper is to share some concepts informed by systems thinking to support you, as leaders in your organisations, to bring a systems-informed lens to your work. It puts you as a practitioner-leader at the centre of your own practice and encourages you to reflect critically on your positionality, as well as the lens you use to understand and intervene in complex issues. This paper has been developed on foot of training piloted with public service leaders in Scotland in the summer of 2022, and acts as an introduction to developing your systems literacy skills. While we are all born with a systems sensibility, many of us lose that sense of connectedness and interdependence with the world as we grow up. This paper acts as a prompt to bring a complexity informed approach to your work practice. It also contains some pointers for further resources and tools that may support your learning.

A Guide to Participatory Outcome-Focused Monitoring with Most Significant Change, Outcome Harvesting, and Outcome Mapping - Smith, Richard (CARE, 2023)
Audience This guidance is primarily aimed at development and humanitarian professionals who design or manage monitoring and evaluation processes to support design and implementation of qualitative monitoring using Most Significant Change (MSC), Outcome Harvesting (OH), or Outcome Mapping (OM), which are participatory, outcome-focused qualitative methods. Purpose This guidance is a resource to strengthen qualitative monitoring, specifically addressing commonplace gaps or weakness in the monitoring of complex development and humanitarian projects. Importantly, these gaps include: 1. Lack of evidence about unexpected, as well as expected, changes or outcomes, especially during project implementation. 2. Limited documentation about whether and how a project contributed to change. Such information is vital to learning about what worked and how to build on successes. 3. Limited data to inform the review of theories of change, which limits their function and relevance. Addressing these monitoring gaps has the potential to contribute to more informed and effective adaptive management and learning. This guidance can be used to support the design and implementation of monitoring exercises using MSC, OH, and OM. Specifically, it supports users in choosing among MSC, OH, and OM or blending elements from these approaches. Additionally, it provides practical tips and examples - including expertise gained during the two pilot qualitative monitoring activities conducted in Niger and Bangladesh - as well as other publicly available materials. 1. Relevant CARE guidance and tools include: 1. Qualitative Methods Guidance Note for Monitoring, Evaluation and Learning (MEL) at CARE; 2. Outcome Mapping Manual: Development of a monitoring system for the Right to Food project using Outcome Mapping, CARE Nepal; 3. Tipping Point MEL Methods Brief: Outcome Mapping; 4. The Rolling Profile Tool.

Evaluating place-based approaches. A review of methods used - Smith, Stephanie; Irving, Michelle; Mann, Georgina; Bjorndal, A.; Lewis, Jane (Youth Endowment Fund, 2023)

Monitoring, Evaluating and Learning for place-based approaches A toolkit for the Victorian Public Service - Victoria State Government (Department of Jobs, Skills, Industry and Regions, 2023)

How Political Contexts Influence Education Systems: Patterns, Constraints, Entry Points - Levy, Brian (Research on Improving Systems of Education (RISE), 2022)
This paper synthesises the findings of a set of country studies commissioned by the RISE Programme to explore the influence of politics and power on education sector policymaking and implementation. The synthesis groups the countries into three political-institutional contexts: - Dominant contexts, where power is centred around a political leader and a hierarchical governance structure. As the Vietnam case details, top-down leadership potentially can provide a robust platform for improving learning outcomes. However, as the case studies of Ethiopia, Indonesia, Nigeria, and Tanzania illustrate, all-too-often dominant leaders’ goals vis-à-vis the education sector can veer in other directions. - In impersonal competitive contexts, a combination of strong formal institutions and effective processes of resolving disagreements can, on occasion, result in a shared commitment among powerful interests to improve learning outcomes—but in none of the case studies is this outcome evident. In Peru, substantial learning gains have been achieved despite messy top-level politics. But the Chilean, Indian, and South African case studies suggest that the all-too-common result of rule-boundedness plus unresolved political contestation over the education sector’s goals is some combination of exaggerated rule compliance and/or performative isomorphic mimicry. - Personalised competitive contexts (Bangladesh, Ghana, and Kenya for example) lack the seeming strengths of either their dominant or their impersonal competitive contexts; there are multiple politically-influential groups and multiple, competing goals—but no credible framework of rules to bring coherence either to political competition or to the education bureaucracy. The case studies show that political and institutional constraints can render ineffective many specialised sectoral interventions intended to improve learning outcomes. But they also point to the possibility that ‘soft governance’ entry points might open up some context-aligned opportunities for improving learning outcomes. In dominant contexts, the focus might usefully be on trying to influence the goals and strategies of top-level leadership. In impersonal competitive contexts, it might be on strengthening alliances between mission-oriented public officials and other developmentally-oriented stakeholders. In personalised competitive contexts, gains are more likely to come from the bottom-up—via a combination of local-level initiatives plus a broader effort to inculcate a shared sense among a country’s citizenry of ‘all for education’.

How and Why Practitioners Think and Work Politically - Evidence from Chemonics Programming Across Sectors - Kantelberg, Renee; Swift-Morgan, Jennifer; Watson, Bryce (Chemonics International, 2022)
Most development practitioners have long recognized that deep contextual knowledge is crucial to understanding how projects interact with their local systems and, in turn, to navigating these systems. Moreover, this knowledge must complement projects' technical solutions, or they will fall flat and may even undercut project objectives as they clatter down. What, then, explains practitioners' particular interest in TWP as an explicit strategy and more than just "doing good development"? This report responds to that question and to the many calls for a more comprehensive picture of TWP by presenting new evidence of the various forms that TWP may take in practice. The evidence comes from a 2022 study that Chemonics undertook to foster more robust learning about TWP. Specifically, we closely examined Chemonics implemented projects that used or are using various forms of TWP in nine countries: Bangladesh, Iraq, Mozambique, the Philippines, Pakistan, Syria, Timor-Leste, and Tunisia. In conducting the study, we interviewed multiple staff from these projects. We complemented what we learned from these projects with a review of eight additional Chemonics- implemented projects applying TWP that had received dedicated support from Chemonics’ Center for Politically Informed Programming (the Center). We consider these findings alongside those of the recent (2022) USAID-Chemonics study on political economy analysis (PEA) usage to identify and articulate what is different and more effective about PEA processes and TWP practices that have received more support.

Study on application of Adaptive Management (AM) in Education Out Loud (EOL) project implementation in the Horn, Eastern adn Southern Africa (HESA) - ANHRE (Oxfam, 2022)
This study mainly aimed to look at adaptive management in the program cycle (throughout Program Cycle processes, including strategy, project, and activity design and implementation) and to look at the Enabling Conditions: how an organization’s culture, business processes, and resource allocation support adaptive management. It explores elements of adaptive management that correspond to the objectives of the study as follows: 1) Assess grantees’ level of understanding of adaptive management as a programming approach and a method of risk mitigation Methodology. 2) Examine how adaptive management has been applied in EOL project implementation thus far. 3) Explore ways in which an adaptive management approach can be used as a source of learning during project implementation. 4) Propose ways in which the application of adaptive management can be improved in EOL project implementation. Objective 1: Assess grantees’ level of understanding of adaptive management as a programming approach and a method of risk mitigation Methodology: The researchers asessed whether the EOL program, GMU, and HESA RMU have a culture of openness, continuous learning, reflection, and improvement. This assessment looked at actual practices and perceptions of field-based staff and project officers. In addition, they looked at processes to assess knowledge management and decision-making process – the ability to distill, share and disseminate knowledge and the awareness among grantees/ employees of the decision-making process. Summary of findings: Understanding of adaptive management as a programming approach among the selected sample of grantees was linked to continuous learning, the ability to adjust implementation and inform decision making to reach the intended objectives based on analysis and assessment of the context and flexibility in management. RMU HESA was perceived to have a culture of openness where grantees feel comfortable sharing their opinion and ideas and is perceived to be receptive to proposed changes to projects and encourage grantees to provide feedback. The HESA RMU-Oxfam Denmark is also seen as an agency that is very “flexible“. Recommendations: Enhance Capacity building on adaptive management tools, particularly for finance and admin teams with a focus on pro-active adaptive management which scans the horizon for opportunities and risks, and it adapts to take advantage of them. It explicitly plans for experimentation and regular upgrading of the strategies; it considers learning and the reduction of uncertainty and imperfect knowledge as one of the key objectives of the management effort. Objective 2: Examine how adaptive management has been applied in EOL project implementation thus far Methodology: Two elements were assessed : Collaboration and how internal and external collaboration of grantees and grant agent, assessing the ability to collaborate with other departments within the entity, the ability to collaborate with learning partners, other grantees, to what extent they consult their stakeholders and their ability to build coalitions. In addition, researchers looked at Adaptation and the best practices and lessons learned in adopting change in the design, implementation, budgeting or monitoring of EOL programs and the process followed to adapt the desired change. Summary of findings: Adaptive management practices are seen to be adapting to context changes, constant learning, and flexibility. The management and coordination of EOL program is deemed to be collaborative, adaptive, and understanding of the emerging needs of EOL Program implementation. There is a simple process in place for grantees to make changes to the program implementation. Most grantees do take the time to pause and reflect on what is going well and what could be improved and to share the knowledge gained with their coalition members and regional coalitions. HESA RMU was found to be encouraging learning practices through organizing periodic online and offline regional learning and reflection workshops. Compared to other donors, the majority of grantees believe that EOL provided the highest level of flexibility and ability to adapt which is seen to be a necessity for advocacy work. However, it is important to enforce structure to tighten the feedback loop between data and action: adaptive management aims for double learning more than just correcting deviations in planned versus actual. Recommendations: Introducing outcome harvesting tool as one of the mechanisms to capture outcomes, institutionalization of adaptive management further by developing context analysis tools/ regular reviews of TOC. Further use of Scenario planning as a tool to map and plan different scenarios on how change may happen, and to examine the program’s role in making future events happen, enforcing structure to tighten the feedback loop between data and action. In addition to the introduction of quarterly planning cycles where data and information gathered in the previous quarter can be used to inform the next planning cycle and update the TOC, project design and activity implementation strategies. Objective 3: To explore ways in which an adaptive management approach can be used as a source of learning during project implementation Methodology: The team assessed the grantees’ ability to use evidence to build their pathways for change, their ability to use learning to inform decision making, the frequency and habit of reviewing their theory of change, the risks and opportunities through scenario planning, how they assess the efficiency of their data collection tools and monitoring and evaluation data quality, on whether they have a mechanism to monitor unintended outcomes (outcome harvesting) and reflect. Summary of findings: This study revealed that there is a culture of collaborating, learning and openness among grantees and HESA RMU. The EOL program has adopted Re-active Adaptive Management that relies on regular monitoring and reflection activities to detect unpredicted challenges and, when needed, to adjust planned actions to remain on track to achieve the desired program outcomes. EOL provided learning opportunities, platforms for knowledge sharing and encouraged peer learning among grantees. Recommendations: Encouraging further engagement on knowledge sharing hubs and investing in knowledge transfer on adaptive management and learning to other partners in targeted countries (EOL to become an influencer in this field), encouraging further engagement from learning partners to provide mentorship and support during implementation and continuous learning on adaptive management tools. RISP and GISP could be further engaged in the monitoring process. Based on the stated objective, what was assessed and findings, it would also be interesting to enable grantees distill learnings from AM practices and applying that knowledge to improve project implementation. This means AM will be expected to go beyond changing implementation strategies and budgets and document learnings from such changes and apply them to improve the remaining part of the project. Objective 4: Propose ways in which the application of adaptive management can be improved in EOL project implementation Summary of findings: Grantees were able to articulate well their understanding of adaptive management but were less able to provide concrete examples of how adaptive management is implemented in practice. There is a culture of collaborating, learning and openness among grantees and HESA RMU. The EOL program relies on regular monitoring and reflection activities to detect unpredicted challenges and, when needed, to adjust planned actions to remain on track to achieve the desired program outcomes. EOL provided learning opportunities, platforms for knowledge sharing and encouraged peer learning among grantees. However, it is important to enforce structure to tighten the feedback loop between data and action and to institutionalize practicing adaptive management. Recommendations: A roadmap that looks at reviewing seven steps for improving adaptive management is recommended as a systematic tool to introduce improvement. This roadmap is useful in a multi-organizational, multi-country set-up where efficient design is needed to align top-down structuring and requirements with bottom-up diversity and divergence. This roadmap of seven steps is introduced by HIVOS. These 7 steps can come in a different order and it depends on the user to assess which of the seven steps needs to be undertaken. 1. Annual Review results/ outcome statements: using outcome harvesting to continuously monitor intended/ unintended outcomes (whether positive or negative or within the sphere of control (outputs) or within the sphere of influence (outcomes). 2. Learning questions: Learning from the monitoring data; Learning about the critical Make-or-Break assumptions in the Theory of Change; and Developing learning topics/ questions and learning agenda at each level 3. Changes in context: Identifying possible trends and signs about emerging shifts in the environment, followed by analyzing and collective sense-making of possible implications for the Theory of Change (is there a need to revise intervention strategies, desired outcomes, pathways, assumptions? 4. Theory of change: Reviewing the theory of change was not seen as a frequently recommended practice. Constant validation of pathways to change and evidence in the TOC is important. 5. Capacity assessment and development: What capacities are available amongst the alliance members and partner organizations in relation to the (revised) Theory of Change; What would be Capacity Development needs in view of the capacity assessment results? 6. Strategies and interventions: With the outcomes achieved and learning so far, what can you say about your interventions and strategies? What worked? Why? Is there a need for adjustment? 7. Annual reflection and Planning Who should take part in the annual planning and reflection?

The path to a preferable future: Investing in System Innovation - Leadbeater, Charles; Smith, Shawn; Winhall, Jennie (The Rockwool Foundation, 2022)
Acknowledgements 3 1. Introduction 4 1.1. System Innovation 6 1.2. Systems Investing 7 2. News from the Frontier – An Emerging Field 10 2.1. Foundations 11 2.2. Private Capital 12 2.3. Government 12 2.4. Community Capital 13 2.5. Movements 14 2.6. Framers 14 3. Systems Investment Strategy and Intent 15 3.1. Improvement or Transformation 16 3.2. Solving Problems or Opening Up Possibilities 18 3.3. Pragmatic Steps or Transformational Intent 20 3.4. Four Keys to System Innovation 25 4. Unlocking the Potential of Systems Investing 27 4.1. Orchestrate Opportunities, Convene Capital 27 4.2. Create Attractive Systems Investment Propositions 31 4.2.1. Get In Early 32 4.2.2. Build a Backbone 34 4.2.3. Invest in Ecosystems, Not Single Interventions 34 4.2.4. Create Conducive Conditions for System Shifts 35 4.3. Create Accessible Investment Vehicles 36 4.4. Set Better Measures 40 4.5. Craft New Portfolios of Systems Change Interventions 42 5. Conclusions 43 Appendix 1. Learning Process and Contributors 47 Appendix 2. Systems Investing Portfolio Matrix 50

Beneficiary feedback mechanisms - UK Aid (FCDO, 2022)
This guidance seeks to ensure that UK Aid Direct applicants and grant holders understand what the Foreign, Commonwealth & Development Office (FCDO) means by beneficiary feedback mechanisms, and more specifically, that they: • Understand the terms used that relate to beneficiary feedback mechanisms in UK Aid Direct guidance and templates • Understand beneficiary feedback mechanisms and why they are a useful tool for project monitoring and learning • Learn how to use beneficiary feedback mechanisms during project implementation. • Can demonstrate that using beneficiary feedback mechanisms can lead to greater accountability.

Everyday patterns for shifting systems - GCSI (GCSI, 2022)

The Art and Craft of Bricolage in Evaluation - Aston, Thomas; Apgar, Marina (Institute of Development Studies, 2022)
This CDI Practice Paper by Tom Aston and Marina Apgar makes the case for ‘bricolage’ in complexity-aware and qualitative evaluation methods. It provides a framework based on a review of 33 methods to support evaluators to be more intentional about bricolage and to combine the component parts of relevant methods more effectively. It discusses two cases from practice to illustrate the value added of taking a more intentional approach. It further argues that navigating different forms of power is a critical skill for bricolage, and that doing so can help to ensure rigour.
Accepted: 2022-10-14T10:56:39Z

Learning from Life Story Collection and Analysis With Children Who Work in the Worst Forms of Child Labour in Nepal - Karki, Shanta; Apgar, Marina; Snijder, Mieke; Sharma, Ranjana (Institute of Development Studies, 2022)
The CLARISSA Nepal team collected and analysed 400 life stories of children and young people engaged in or affected by the worst forms of child labour (WFCL), particularly in the “Adult Entertainment” sector in Nepal, which includes children working in Dohoris (restaurants playing folk music), dance bars, spa-massage parlours, khaja ghars (tea/snack shop) and guest houses. Stories were also collected from children in CLARISSA’s focus neighbourhoods, children in this category include street connected children and those working in transportation, party palaces, domestic labour and construction sites. Of the 400 stories collected, 350 were collected by adult researchers and 50 were collected by children themselves.
Accepted: 2022-11-04T12:22:13Z

How to do Process Tracing: A Method for Testing “How Change Happened” in Complex and Dynamic Settings - Lynn, Jewlya; Stachowiak, Sarah; Beyers, Jennifer (ORS Impact, 2022)
Process tracing is a causal methodology that can help people understand how a particular large-scale change actually happened within a complex dynamic environment. Much of the existing literature provides important information about the method; we wrote this brief to help more people operationalize the concepts and learn about practical steps for using this method more easily, with quality, and toward a more equitable world. This piece was written based on our experiences implementing process tracing when our experience showed that existing materials on the method had a lot more conceptual than practical information. We’ve approached this as people with some successful (and some unsuccessful) experience with the method itself, alongside deep experience in evaluating initiatives and strategies in complex and dynamic settings. We focus not on the Bayesian side of process tracing but rather on how this can be implemented in a way that’s more participatory and lifts up the experiences and wisdom of those closest to the work and the problems being tackled. We hope this contributes to and helps make more approachable the important work of political scientists and methodologists upon which this work sits.

UNDP Digital Leadership Learning Modules - UNDP (United Nations Development Programme, 2022)
In order to support the digital transformation of government operations Digital Learning Modules for Civil Servants are available, an off-the-shelf package of capacity development in form of replicable training modules to empower public servants at both the local and central government level to be leaders of digital transformation for delivering better public services. The modules cover a multitude of fundamental areas: comprehending digital government and services, human-centered design for inclusivity and agile learning cycles; feature the importance of security and privacy, the value of data and how to manage data and technology related risks; spotlight the key role of supportive leadership and offer practical tools for assessing and overcoming main barriers to ensure a successful digital transformation journey.

Local Capacity Strengthening Policy - USAID (USAID, 2022)
Sustainable development depends on local actors leading efforts to improve their communities and working inclusively and collectively to see those efforts through. For this reason, local capacity strengthening is and has been a foundational component of USAID programming. Effective local capacity strengthening can propel inclusive economic growth; advance improvements in essential health, food and nutrition, and education services and systems; and cultivate democratic governance. Local capacity strengthening also can address underlying factors of fragility, bolster local humanitarian response systems, and enhance resilience to shocks and stresses. As a result, effective local capacity strengthening supports countries to prevent, mitigate, and recover from crises. Ultimately, the capacity of local actors is a key determinant of the success of USAID and its partners in achieving and sustaining humanitarian and development gains around the world. Moreover, across the global humanitarian and development landscape, a consensus has emerged about the importance of capacity strengthening—which is inclusive and locally led—in contributing to sustainable development. However, despite this emerging consensus, USAID has never had a unifying and authoritative Agency policy on local capacity strengthening. This policy fills that gap and affirms USAID global leadership by providing the Agency with a vision for effective local capacity strengthening that builds on the consensus across the development landscape, feedback from local actors and partners, and years of implementation experience and evidence. This new vision will be grounded in a commitment to partnerships based on mutual respect and reciprocity and through which local actors from all backgrounds and cultures have their voices heard, exercise their unique capabilities, and lead their country’s development. USAID support for local capacity strengthening must be programmed in a way that builds on the existing strengths of local actors and systems, responds to dynamic country and regional contexts, and aligns with USAID strategic priorities. This means that approaches to local capacity strengthening will vary. To support coherent implementation of local capacity strengthening across sectors and diverse contexts, USAID’s Local Capacity Strengthening Policy establishes an Agency-wide vision based on a shared commitment to a set of principles that will guide all relevant USAID humanitarian assistance and development programming.

The Haves and the Have Nots: Civic Technologies and the Pathways to Government Responsiveness - Mellon, Jonathan; Peixoto, Tiago C; Sjoberg, Fredrik M (World Bank, 2022)
“Civic tech” broadly refers to the use of digital technologies to support a range of citizen engagement processes. From allowing individuals to report problems to local government to enabling the crowdsourcing of national legislation, civic tech aims to promote better policies and services – while contributing to more inclusive democratic institutions. But could civic tech affect public issues in a way that benefits some and excludes others? Over the decades, the question of who participates in and who is excluded from participation mediated by technology has been the focus of both civic tech critics and proponents. The latter tend to argue that, by enabling citizens to participate without constraints of time and distance, civic tech facilitates the participation of those who usually abstain from engaging with public issues, leading to more inclusive processes. Critics argue that, given the existing digital divide, unequal access to technology will tend to empower the already empowered, further deepening societal differences. Yet both critics and proponents do tend to share an intuitive assumption: the socio-economic profile of who participates is the primary determinant of who benefits from digitally mediated civic participation. For instance, if more men participate, outcomes will favor male preferences, and if more young people participate, outcomes will be more aligned with the concerns of the youth. In a new paper, we show that the link between the demographics of those participating through digital channels, and the beneficiaries of the participation process, is not necessarily as straightforward as commonly assumed. We review four civic tech cases where data allow us to trace the full participatory chain through: the initial digital divide the participant’s demographics the demands made through the process the policy outcomes We examine online voting in the Brazilian state of Rio Grande do Sul’s participatory budgeting process, the local problem reporting platform Fix My Street (FMS) in the United Kingdom, Iceland’s online crowdsourced constitution process, and the global petitioning platform Change.org. Counterintuitive findings Change.org has been used by nearly half a billion people around the globe. Using a dataset of 3.9 million signers of online petitions in 132 countries, we examine the number of successful petitions and assess whether petitions created by women have more success than those submitted by men. Our analysis shows that, even if women create fewer online petitions than men, their petitions are more likely to be successful. All else equal, when online petitions have an impact on government policy, the agenda being implemented is much closer to the issues women choose to focus on. In Rio Grande do Sul’s digital participatory budgeting (PB), we show that despite important demographic differences between online and offline voters, these inequalities do not affect which types of projects are selected for funding – a consequence of PB’s unique institutional design, which favors redistributive effects. In fact, of all the cases analyzed, none reflect the standard assumption that inequalities in who participates translate directly into inequalities in who benefits from the policy outcomes. Our results suggest that the socio-economic profile of participants predicts only in part who benefits from civic tech. Just as important to policy outcomes is how the platform translates civic participation into policy demands, and how the government responds to those demands. While civic tech practitioners pay a lot of attention to design from a technological perspective, our findings highlight the importance of considering how civic tech platforms function as political institutions that encourage certain types of behavior while discouraging others. Civic tech, it seems, is not inherently good nor bad for democratic institutions. Instead, its effect is a combination of who participates on digital platforms and the choices of platform designers and governments.

How do we know we are doing good work? - Sharp, Cathy; McLaughlin, Dot; Whitley, Janet; Lawson, Karen (Collective Leadership for Scotland, 2022)
Collective leadership feels timely and important in an uncertain, fast changing, and challenging world. This report comes at this heightened moment of urgency and appetite for renewal, bringing potential to do things differently in public services and communities. The opportunities and challenges of true collaboration-in-practice, in the absence of blueprints, mean that it has never been more necessary to change ways of working and foreground learning. As outcomes remain important, and time and resources are scarcer than ever, the approach of collective leadership makes the creation of impact a shared, conscious, and actionable choice. A pathway to ultimate impact at scale is co-created through dialogue about expectations and contributions, and by design, not assuming change will happen because we have good intentions. Collective Leadership for Scotland (CLfS) has a strong vision and enjoys continuing active interest, drawing together participants from a variety of agencies working in public services. People are looking for fresh thinking, space and time for reflection, connection with others, a chance to think about how to tackle difficulties, and to test out what it takes to do, and continue to do, the work of collaborative public service. These motivations are deepened and brought into sharper focus by the pandemic, with an added interest in developing skills in online facilitation. CLfS contributes to building a critical mass for system change, to help to sustain the ambitions of the Christie Commission and the delivery of the National Outcomes for Scotland. There remains further potential to realise wider and deeper impact amongst organisations, communities, and wider systems. The conclusions of this report are likely to have wider resonance beyond interests in the CLfS programmes. This report deepens understanding of some of the challenges of commissioning, convening, and the scope for deeper impact through building reflective and relational leadership practices. It also outlines social and experiential sensemaking and facilitation practices to strengthen the action inquiry approach as a deliberate learning strategy, building cultures that support new forms of collaborative inquiry and systemic action research.

Designing Contribution Analysis of Participatory Programming to Tackle the Worst Forms of Child Labour - Apgar, Marina; Snijder, Mieke; Prieto Martin, Pedro; Ton, Giel; Macleod, Shona; Kakri, Shanta; Paul, Sukanta (Institute of Development Studies, 2022)
This Research and Evidence Paper presents the theory-based and participatory evaluation design of the Child Labour: Action-Research- Innovation in South and South-Eastern Asia (CLARISSA) programme. The evaluation is embedded in emergent Participatory Action Research with children and other stakeholders to address the drivers of the worst forms of child labour (WFCL). The report describes the use of contribution analysis as an overarching approach, with its emphasis on crafting, nesting and iteratively reflecting on causal theories of change. It illustrates how hierarchically-nested impact pathways lead to specific evaluation questions and mixing different evaluation methods in response to these questions, critical assumptions, and agreement on causal mechanisms to be examined in depth. It also illustrates how realist evaluation can be combined with contribution analysis to deeply investigate specific causal links in the theory of change. It reflects on learning from the use of causal hotspots as a vehicle for mixing methods. It offers considerations on how to navigate relationships and operational trade-offs in making methodological choices to build robust and credible evidence on how, for whom, and under what conditions participatory programming can work to address complex problems such as child labour.
Accepted: 2022-08-18T11:05:51Z

Integrating Local Knowledge in Development Programming - Johnson, Madelyn; Maunder, Ishan; Pinga, Andie (USAID, 2022)
The objective of this report, Integrating Local Knowledge in Development Programming is to share knowledge of how development donors and implementing organizations leverage local knowledge to inform programming. In a recent speech at Georgetown University, United States Agency for International Development (USAID) Administrator Samantha Power said, “As Americans with a fraught history living up to our own values, we’ve got to approach this work with intention and humility. But the entire development community needs to interrogate the traditional power dynamics of donor-driven development and look for ways to amplify the local voices of those who too often have been left out of the conversation.” To that end, USAID’s Agency Knowledge Management and Organizational Learning (KMOL) function facilitated conversations with multilateral and bilateral donors and local organizations to understand how organizations define, utilize, and incorporate local knowledge into their programmatic and operational activities. Using qualitative tools to gather data for this report, the research team explored five overarching themes: 1. Local Knowledge Nomenclature and Definitions 2. Best Practices 3. Outcomes 4. Ethics and Power Dynamics 5. Challenges

Evidence-led adaptive programming: Lessons from MUVA - Sharp, Samuel; Riemenschneider, Nils; Selvester, Kerry (ODI, 2022)
Calls for more ‘adaptive programming’ have been prominent in international development practice for over a decade. Learning-by-doing is a crucial element of this, but programmes have often found it challenging to become more learning oriented. Establishing some form of reflective practice, against countervailing incentives, is difficult. Incorporating data collection processes that generate useful, timely and practical information to inform these reflections is even more so.This paper explores MUVA - an adaptive female economic empowerment programme in Mozambique. MUVA, we suggest, is atypically evidence-led. It combines systematic, inclusive reflective practice with extensive real-time data collection. We describe the fundamental features of MUVA’s monitoring, evaluation and learning (MEL) approach that supported this. One, how data collection and analysis are synchronised with set cycles for learning and adapting projects. Two, how MEL systems are designed to prioritise actionable learning, with data collection oriented more to the needs of implementing staff than to the reporting requirements of funders.This approach was enabled by building collective ownership over the programme’s objectives and the purpose of MEL from the outset. Implementers are asked about their motivations, and these are related to the programme’s Theory of Change. The evidence culture is supported by the proximity of MEL staff to implementing staff; and through structuring upwards accountability to funders around justifying evidence-based adaptations instead of reporting on more narrow indicators. We conclude by considering the relevance, or not, of MUVA’s approach to programmes in other contexts or issue areas trying to replicate a similarly evidence-informed approach to adaptive management. Key messages Learning-by-doing is essential to adaptive programming, but it can be challenging to establish data collection processes that generate useful, timely and practical information. MUVA – a female economic empowerment programme in Mozambique – has an atypically evidence-led adaptive management approach. This has two fundamental features. One, data collection and analysis are synchronised with set cycles for learning and adapting projects. Two, data collection is oriented more to the needs of implementing staff than to the reporting requirements of funders. Monitoring, evaluation and learning (MEL) systems are designed to prioritise actionable learning. This approach was enabled by building collective ownership over the programme’s objectives and the purpose of MEL from the outset. Implementers are asked about their motivations, and these are related to the programme’s Theory of Change. The evidence culture is supported by the proximity of MEL staff to implementing staff; and through structuring upwards accountability to funders around justifying evidence-based adaptations instead of reporting on more narrow indicators.

Thinking and Working Politically on Health Systems Resilience: Learning from the experience of Cameroon, Nepal and South Africa during COVID-19 - Williams, Gareth (TWP Community of Practice, 2022)

Integrated Governance: Achieving Governance Results and Contributing to Sector Outcomes - Frazer, Sarah; Granius, Mark; Brinkerhoff, Derick W.; McGregor, Lisa (RTI Press, 2022)
Achieving broad-based socio-economic development requires interventions that bridge disciplines, strategies, and stakeholders. Effective sustained progress requires more than simply an accumulation of sector projects, and poverty reduction, individual wellbeing, community development, and societal advancement do not fall neatly into sectoral categories. However, researchers and practitioners recognize key operational challenges to achieving effective integration that stem from the structures and processes associated with the current practice of international development. Integration calls for the intentional linking of intervention designs, implementation, and evaluation across sectors and disciplines to achieve mutually reinforcing outcomes. In this report, we summarize the results of a study we conducted to explore the challenges facing governance programs that integrate with sector interventions to achieve governance outcomes and contribute to sector outcomes. Through a review of policy documents and project reports from recent integrated governance programs and interviews with donor and practitioner staff, we found three integrated governance programming variants, an emphasis on citizen and government collaboration to improve service delivery, interventions that serve as the glue between sectors, and a balancing act for indicators to measure contribution to sectoral outcomes. Our analysis identified four key success factors: contextual readiness, the application of learning and adapting approaches, donor support, and recognition of the limitations of integrated governance. We then discuss recommendations and implications and for answering the challenge of integrating governance and sector programming to achieve development outcomes.

Evaluating complex interventions: What are appropriate methods? - Masset, Eduardo (CEDIL, 2022)
In the CEDIL Methods brief, ‘Evaluating complex interventions: What are appropriate methods?’ we identify four types of complex development interventions: long causal chain interventions, multicomponent interventions, portfolio interventions, and system-level interventions. These interventions are characterised by multiple activities, multiple outcomes, multiple components, a high level of interconnectedness, and non-linear outcomes.
Section: Publications

Seeing the Combined Effects of Aid Programmes - Burge, Richard; Nadelman, Rachel; McGee, Rosie; Fox, Jonathan; Anderson, Colin (Institute of Development Studies, 2022)
Multiple aid agencies often try to support change in the same places, at the same time, and with similar actors. Surprisingly, their interactions and combined effects are rarely explored. This Policy Briefing describes findings from research conducted on recent aid programmes that overlapped in Mozambique, Nigeria, and Pakistan, and from a webinar with UK Foreign, Commonwealth & Development Office (FCDO) advisors and practitioners. The research found three distinct categories of ‘interaction effects’: synergy, parallel play, and disconnect. We explore how using an ‘interaction effects’ lens in practice could inform aid agency strategies and programming.
Accepted: 2022-05-10T09:42:35Z

Locally driven development: Overcoming the obstacles - Ingram, George (Brookings Institution, 2022)
Locally led development is a complex process that the development community, in the U.S. and around the world, has spent several decades trying to get right. Yet, despite all the experience and lessons learned, it feels like we are barely beyond the starting line. This publication aims to contribute to the ongoing dialogue on locally led development, especially as to how the United States can address the obstacles posed by U.S. law, regulation, policy, and practice. It consists of two parts: • An essay by George Ingram that notes the path that has taken us to this point, identifies key obstacles, and invites a discussion of how to overcome impediments and move forward. • A set of 15 commentaries written by development experts that add a range of perspectives and nuances to the discussion

Measuring and Monitoring Adaptive Learning: A Landscape Review - LaFond, Anne; Adrian, Haley (USAID MOMENTUM Knowledge Accelerator, 2022)
This landscape review on measuring and monitoring adaptive learning highlights the learning from five adaptive programming guidelines and toolkits and one implementation science framework to inform the monitoring and evaluation of adaptive learning. The introduction of adaptive learning processes and skillsets in global health programming is part of an emerging strategy to advance a learning culture within projects and teams to improve health program performance. The monitoring and evaluation of adaptive learning is an emergent field aiming to monitor how adaptive learning processes have been introduced, how they are used, and whether they are having the intended results. Although there is a growing body of literature on adaptive programming more generally, there is a limited knowledge base on the monitoring and evaluation of adaptive learning interventions and their impacts. Unlike other implementation strategies or program management approaches, there are no standard metrics or a monitoring and evaluation framework to track the integration, implementation, and effectiveness of adaptive learning in health programming.

Capacity Development in a Participatory Adaptive Programme: the Case of the Clarissa Consortium - Widmer, Mireille; Apgar, Marina; Afroze, Jiniya; Malla, Sudhir; Healey, Jill; Constant, Sendrine (Institute of Development Studies, 2022)
Doing development differently rests on deliberate efforts to reflect and learn, not just about what programmes are doing and achieving, but about how they are working. This is particularly important for an action research programme like Child Labour: Action- Research-Innovation in South and South-Eastern Asia (CLARISSA), which is implemented by a consortium of organisations from across the research and development spectrum, during a rapidly changing global pandemic. Harnessing the potential of diverse skills and complementary strengths across partners in responding to the complex challenge of the worst forms of child labour, requires capacity to work together in novel ways. This Research and Evidence Paper documents how CLARISSA approached capacity development, and what we learnt from our challenges and successes. From the start, the programme incorporated a capacity development strategy resting on self-assessment of a wide range of behavioural and technical competencies that were deemed important for programme implementation, formal training activities, and periodic review of progress through an after-action review (AAR) process. An inventory of capacity development activities that took place during the first year of implementation reveals a wide range of additional, unplanned activities, enabled by the programme’s flexibility and adaptive management strategy. These are organised into eight modalities, according to the individual or collective nature of the activity, and its sequencing – namely, whether capacity development happens prior to, during, or after (from) implementation. We conclude with some reflections on the emergent nature of capacity development. Planning capacity development in an adaptive programme provides a scaffolding in terms of time, resources, and legitimacy that sustains adaptiveness. We also recognise the gaps that remain to be addressed, particularly on scaling up individual learning to collective capabilities, and widening the focus from implementation teams to individuals working at consortium level.
Accepted: 2022-04-25T10:25:54Z

System Change: A Guidebook for Adopting Portfolio Approaches - Wellsch, Brent (UNDP, 2022)
This guidebook codifies the principles and methods of applying systems change and portfolio approaches to complex development challenges with practical tools and examples. It is based on the empirical learning generated from the collaborative initiatives in UNDP Country Offices in Bhutan, Pakistan, the Philippines, and Viet Nam with support from Regional Innovation Centre for Asia and the Pacific.

Key Issues in Digitalisation and Governance - Roberts, Tony; Hernandez, Kevin; Faith, Becky; Prieto Martin, Pedro (SDC Governance Network, 2022)
Digitalisation is perhaps the most important strategic challenge that governance will face over the coming decade. The process is delivering digital dividends as well as new exclusions and injustices, with the rapid but uneven increase in access to mobile and internet technologies transforming how social and economic life takes place. This report highlights the key opportunities and challenges arising from digitalisation.
Accepted: 2022-07-25T11:31:42Z

Theory of Change Workbook: A Step-by-Step Process for Developing or Strengthening Theories of Change - Salib, Monalisa (USAID, 2022)
While over time theories of change have become synonymous with simple if/then statements, a strong theory of change should actually be a much more detailed, context-specific articulation of how we *theorize* change will happen under a program.

Systems Thinking and Practice: A Guide to Concepts, Principles and Tools for FCDO and Partners - Woodhill, Jim; Millican, Juliet (Institute of Development Studies, 2022)
This guide is a basic reference on systems thinking and practice tailored to the context and needs of the UK Government’s Foreign, Commonwealth & Development Office (FCDO). It is an output of the FCDO Knowledge for Development Programme (K4D), which facilitated a Learning Journey on Systems Thinking and Practice with FCDO staff during 2021 and 2022. The guide offers a common language and shared framing of systems thinking for FCDO and its partners. It explores what this implies for working practices, business processes and leadership. It also offers links to additional resources and tools on systems thinking. We hope it can support systems thinking to become more commonplace within the culture and practices of FCDO and working relations with partner organisations.
Accepted: 2023-02-03T15:47:58Z

The Difference Learning Makes - Factors that enable and inhibit adaptive programming - Gray, Stephen; Carl, Andy (Christian Aid, 2022)
Executive Summary When Christian Aid (CA) Ireland devised its multi-country and multi-year Irish Aid funded Programme Grant II (2017-2022), they opted to move away from a linear programme management approach and to explore an adaptive one. Across seven countries: Angola, Colombia, El Salvador, Guatemala, Israel and the occupied Palestinian territory, Sierra Leone, and Zimbabwe, CA and partner organisations support marginalised communities to realise their rights, reduce violence and address gender inequality. Since 2019, Adapt Peacebuilding has accompanied CA Ireland, CA country teams and partner organisations as they experimented with using a deliberate adaptive approach. The authors were also asked to follow up on an initial study by CA Ireland and Overseas Development Institute in 2018, which described the rationale for adopting this new approach and included early lessons from its first year of implementation. The aim of this study is to help deepen CA Ireland, CA country teams’ and partners’ understanding of (a) whether their application of adaptive programming has resulted in better development outcomes, and (b) how they can better understand the factors that enabled or inhibited the effectiveness of using this approach. Over the past three years, this study has found evidence and multiple examples that show adaptive programming contributed to better development outcomes. The main reasons cited were that these were made possible both from improvements to programming strategies based on proactive reflection and learning, as well as those that stem from the reactive capacity of adaptive programmes to change course in response to unanticipated changes in operating conditions. This study found that adaptive programming has enabled better development practice where organisations are enhancing their skills to better respond and be flexible to contextual challenges. 72% of partners surveyed described adaptive programming as the most useful approach to programme management that they have used. The programme approach has meant that CA and partner staff were better able to explore the significance of change in the context and their contributions to them. It also enabled spaces for meaningful engagement with communities in learning and programme planning processes and encouraged opportunities for experimentation in programming. The study also found that adaptive programming has supported flexible delivery. This led to better outcomes that would not have been possible were the programme not able to make flexible adjustments. The main focus has been the analysis of nine factors that can determine the effectiveness and impact (or otherwise) of using an adaptive approach, flagging important issues for understanding. These factors are identified as: 1) Leadership; 2) Organisational culture; 3) Conceptual understanding; 4) Staff capacities; 5) Partnership approaches; 6) Participation; 7) Methods and tools; 8) Administrative procedures; and 9) The operating context. Together these can provide an analytical framework for assessing an organisation’s ‘adaptive scope’, which can be used as a tool for better understanding an organisation’s potential to generate improved development outcomes via adaptive programming and how to strengthen them. The study concludes with several recommendations for CA Ireland, all of which have relevance for a broader community of donors and implementing organisations interested in the potential of adaptive programming.

Guidance Note for Adaptive Management - Ministry of Foreign Affairs of Denmark (Government of Denmark, 2022)
This note explains what adaptive aid management is; why and when it should be considered; and how it should be applied. It covers all Danish development support channels and modalities, including bilateral country assistance, assistance to and through civil society, the private sector and to and through multilateral organisations. This guide has three chapters. Chapter 1 provides an executive overview of what adaptive management is. Chapter 2 goes deeper into five key operational principles of adaptive management. Chapter 3 details the main tenets of adaptive management processes during the programming cycle.

The Patterns of Possibility - How to Recast Relationships to Create Healthier Systems and Better Outcomes - Winhall, Jennie; Leadbeater, Charles (The Rockwool Foundation, 2022)
In Building Better Systems, we introduced four keys to unlock system innovation: purpose and power, relationships and resource flows. These four keys make up a set. Systems are often hard to change because power, relationships, and resource flows are locked together in a reinforcing pattern to serve the system’s current purpose. Systems start to change fundamentally when this pattern is disrupted and opened up so that a new configuration can emerge, serving a new purpose. In this article series we delve deeper into these four keys and provide practical advice on how they can be put to use. This article is about relationships. Systems are defined by the patterns of interactions between their parts: their relationships. Those interactions generate the outcomes of the system as a whole. Transforming the outcomes of a system requires remaking its relationships and then unlocking the other keys to system innovation: purpose, power and resources. This shift in relationships allows all those in the system to learn faster, to be more creative. System innovators redesign the relationships in the system to allow dramatically enhanced learning across the system, and thereby generate far better outcomes.

Complexity Leadership Theory (CLT) - Abbaszada, Ramiz (2022)
The present literature review was conducted in order to investigate the role of Complexity Leadership Theory (CLT) in modern public administration. We use data from different academic articles to decompose the value within the framework of this theoretical perspective. In addition to explaining how extend, Complexity Leadership Theory (CLT) is crucial in the “information age” and its main factors will be the main priorities of the study. Traditional top-down conceptions of leadership are at best unduly simplistic, as twenty-first-century management continues to stress decentralized organizing structures and co-evolutionary ecologies of enterprises, institutions, and markets. There's also a growing recognition that effective leadership isn't always defined by the leader's symbolic, motivating, or charismatic behaviors. But, if leadership isn't "in" or "done by" a leader, how can we understand what leadership is and where it comes from? A unique method to answering these concerns is based on complexity theory, specifically the idea that leadership emerges from relational interactions among actors. Leadership, in this view, is more than a talent, a transaction, or a symbol; it comes from dynamic relationships. The role of leadership in accelerating those processes in companies where interdependent acts among many persons merge into a collective venture is investigated by "complexity leadership theory." Complexity leadership is a combination of the three forms of leadership described below: (1) administrative leadership based on tight control and a large bureaucratic hierarchy (2) adaptive leadership, which is built on creative issue solution, resonating with changing situations, and learning, and (3) action-centered leadership, which entails crisis decision-making and dynamic productivity.

Market systems change rubric - Loveridge, Donna (Oxford Policy Management, 2022)
This systems change rubric describes different performance levels according to various systems elements, such as policy (formal rules), practices and relationships and connections. Programmes can use the rubric to assess the performance of systems to help decide where and how to intervene, or during and post-implementation to conduct progress assessments, and assess the effectiveness of interventions and type, breadth and depth of systems change. Each performance level description highlights the type of data and information that needs to be collected. One analysis is completed, users can compare this to the performance descriptions to see which level best matches the analysis. This helps programmes draw conclusions about systems changes. The rubric was developed in 2020 and tested in 2021 and builds on systems change thinking and frameworks from two previous FSG publications. It can be used as: pre-intervention to conduct an assessment during an intervention to conduct progress assessments and reflect on the effectiveness of interventions to change systems and inform decision making post intervention to make judgements about whether interventions were valuable given the resources, time and effort spent Useful for: Implementation managers to determine the effectiveness of interventions, as well as by Monitoring, Evaluation and Learning (MEL) teams to track progress against expected outcomes.

MOTION HANDBOOK - Developing a transformative Theory of Change - Palavicino, Carla Alvial; Matti, Cristian; Witte, Jenny (TIPC, Utrecht University, 2022)
A step-by-step guide on how to develop a Transformative Theory of Change, for innovation projects, programmes and organisations working on systems transformation. The MOTION project was initiated with one key question in mind: how can we help projects and organisations be more transformative, using the framework and concept provided by the multi-level perspective? And what kind of tools, methods and frameworks can we co-design that translate scientific concepts into practises relevant for policy practitioners? This led us into a co-creation journey during which researchers from the Transformative Innovation Policy Consortium (TIPC) and EIT Climate-KIC project partners experimented, reflected and learned from each other in building the approach that we share in this handbook. Through this journey, we gained a deeper understanding of what the portfolio approach means in a transformative system change context and which skills and competences are needed to facilitate processes of co-creation in the science-policy-practice interface. We had the opportunity to configure the key building blocks of our theoretical approach, the Transformative Outcomes, into practical insights and actions that can easily be applied by innovation organisations at many levels. This handbook is the culmination of the journey as it translates key learnings from the MOTION project into practical insights that are relevant to practitioners working on systems transformation.

Sensemaking Workshop Preparation Guide and Facilitator Guide and Sensemaking Training - UNDP (UNDP, 2022)
Based on experience from running Sensemaking workshops for UNDP offices and government partners, the Asia-Pacific Regional Innovation Centre developed the Sensemaking Preparation Guide and Facilitator Guide to share its knowledge with teams and organization that are interested in using the Sensemaking process.

Social Accountability and Service Delivery Effectiveness: What is the Evidence for the Role of Sanctions? Background Paper - Aston, Tom; Zimmer Santos, Grazielli (2022)
Executive Summary Understanding how civil society can get government to respond to their needs, preferences and demands, and deliver goods and services to citizens is a central concern in social accountability initiatives. It is widely argued that sanctions make a key difference to service delivery outcomes, and that without them, transparency and accountability interventions are less likely to be effective and less likely to be sustainable (Anderson et al., 2020; Arugay, 2016; Fox, 2020; Goetz and Jenkins, 2005; Grandvoinnet et al., 2015; McGee and Gaventa, 2011; Molina et al., 2017; Joshi, 2010; Joshi, 2017; Tsai et al. 2019). In this paper, sanctions refer to the threat or imposition of a punishment for transgressing a rule or norm. Yet, what evidence is there to support the claim that sanctions are king? How much do we actually know about social and formal sanctions and their effectiveness in improving service delivery? Looking at 11 meta-reviews and 35 cases, this background paper sheds light on these questions and the conditions under which sanctions promoted within social accountability interventions may have contributed to improved service delivery. Sanctions, both social and formal, feature very prominently in the scholarly literature related to accountability, so much so that some scholars have argued that it has become synonymous with punishment (Mansbridge, 2014; Schedler, 1999). While it may not be true that these concepts are fully synonymous, this trend in scholarship significantly influenced the thinking of donors such as the Department for International Development (DFID) and the World Bank over the last two decades (Grandvoinnet et al., 2015; Malena and McNeil, 2010; Moore and Teskey, 2006; World Bank, 2003). As a result, sanctions also feature in the dominant models, or theories of change, in the social accountability sector. Yet, in reviewing 11 meta-reviews in the transparency and accountability sector we found that while there is plenty of theoretical argument asserting the potential effectiveness of social and formal sanctions, there was limited empirical evidence to support the claim that sanctions were key. Our review of 35 cases revealed five mechanisms of change related to social and formal sanctions. These were: (i) “sticks” – response to punishment; (ii) “big brother is watching” – response to threat of formal punishment; (iii) diagonal accountability – response to threat of formal punishment by horizontal accountability agencies; (iv) litigation – response to legal investigations or lawsuits supported by community paralegals and legal aid organizations and; (v) response to “naming and shaming” by civil society and/or media. We challenge the dominant view in scholarship that harder social sanctions and enforcement of stronger formal sanctions are either necessary to the achievement of higher-level service delivery outcomes or that they will deliver better and more sustainable outcomes. We find that both social and formal sanctions can contribute to improving service delivery outcomes across a variety of country contexts. In half of the 35 cases reviewed, we were able to establish a likely link between social or formal sanctions and intermediate effects over the short term with some degree of confidence. These effects ranged from increased service provider awareness and motivation, increased availability of funding, staff, and materials, to improved infrastructure quality, and in a minority of cases impact level changes such as improved test scores. However, the role of sanctions in delivering outcomes was often unclear, outcomes were almost never sustainable, and in close to half of the 35 cases reviewed there were substantial negative effects. These effects ranged from reducing transparency and funding, to discrediting, relocating, and reprisals for advocates and whistle-blowers, threats of violence to collaborating government actors, damaging staff morale, reducing attendance, and generating conflict among staff and between staff and community members and between staff and patients, damaging trust. There are therefore some serious ethical dilemmas associated with sanctioning efforts which need to be carefully considered. We argue that imposing sanctions without building relationships or systems to promote good behavior is unlikely to improve service delivery outcomes in a sustainable way. Another, perhaps surprising, finding was that there are some actors that are regular targets of sanctions, and in many cases, these actors are a lot weaker than is commonly assumed. Three broad types of actors were the most common targets of punishment: (i) absentee nurses or teachers who had their pay or allowances reduced; (ii) offending officials who were either suspended, relocated, or fired; and (iii) contractors who had to cover the cost of rejected materials or faced lawsuits, alongside civil servants involved in contracting. We suggest that closer relationships may perhaps reduce stakeholders’ appetite to impose sanctions. Particularly in the health sector, we found that more proximate relationships created disincentives for confrontation, and in such circumstances, a “policing” approach to monitoring was also deemed inappropriate and counterproductive. Conversely, it seems that actors generally prefer to sanction “others,” i.e., when an actor/organization was outside the group. Short-term consultants, contractors, and suppliers were easy (and quite vulnerable) targets for harder sanctions. It has also been argued that there may be productive combinations of collaborative and confrontational tactics — i.e., hybrids. We found that many supposed confrontational and collaborative hybrids were, in fact, dislocated in time and space. Many so-called “inside-outside” strategies, therefore, seem to be a potential mischaracterization. We also found, as Fung and Wright (2003) argued nearly two decades ago, that adversarial forms of engagement cannot easily be redeployed for collaborative purposes. It is widely argued that supposedly ‘weaker’ forms of citizen engagement are less effective than those with ‘strong enforceability (McGee and Gaventa, 2011).’ We found no compelling evidence to support this contention. A quarter of cases reviewed were collaborative for certain periods or in certain locations. On average, these were slightly more successful when compared with periods or locations when imposing sanctions was a key strategic emphasis. So, soft power can also be powerful. Rather than one approach necessarily being superior (confrontational, collaborative, or hybrid) however, we argue that the best approach is likely to be the one most appropriate to the context. While we were unable to identify strong trends of contextual factors which enabled social and formal sanctions to play a role in enhancing service delivery, we were able to identify several conditions which we believe offer the greatest promise. These conditions were: (i) supportive leaders who played a role as champions; opening doors or accompanying civil society efforts; (ii) high capacity and legal authority of oversight agencies; (iii) competitive elections, which provided windows of opportunity for CSOs to combine political and social accountability efforts, and; (iv) vulnerable public servants and service providers already in relatively precarious situations and are thus easy targets. Overall, our study finds that sometimes sanctions can be effective, but punishment is not the answer to all the world’s problems. Given these limitations, we recommend that it is time to reconsider “carrots” and enquire further into the enabling conditions for bureaucrats and service providers. Relatedly, scholars, evaluators, and program teams should look more closely at service providers’ or civil servants’ motivations and take context into account more seriously. To uncover these contextual and motivational features, we believe that scholars, evaluators, and prprogrameams should also make better use of theory-based and participatory methods. And perhaps most importantly, donors and practitioners should carefully consider and mitigate the potential for backlash from sanctions-based approaches.

Guidance Note: Practical introduction to adaptive management - DT Global (DT Global, 2022)
DT Global is proud to introduce our new Guidance Note: Practical Introduction to Adaptive Management There is a growing consensus around adaptive management as an effective (even necessary) approach when programs are tackling complex development problems. While there is no standard definition of adaptive management, there is general agreement that such programs need to routinely engage with and respond to program context; constantly test what works in that context; and adjust approaches, plans, and activities based on continuous learning. However, there remains a more limited body of evidence about what this looks like in practice—the enabling conditions, systems, resourcing, skills, and attitudes to effectively operationalise adaptive management. There is also limited guidance around when adaptive management is required, and to what extent—both critical and often overlooked considerations when planning for successful adaptive management. This Guidance Note draws together lessons and good practice in adaptive management from across DT Global’s diverse portfolio of donor-funded programs. It outlines our conceptual framework for adaptive management, with practical guidance on how it can be applied by our program teams. It is also designed to help our teams distinguish adaptive management from good (non adaptive) project management, consider when adaptive management is most useful on a program, and how adaptive a program (or part of a program) should be.

Systems change: from theory to practice - Kessler, J J; Molenaar, J W (Aid Environment, 2022)
In recent years concepts such as market or sector transformation and systems change have become part of the mainstream development discourse. The aim is to address root causes that undermine the performance of sectors (e.g. agriculture, mining or energy) in order to achieve long-term and widescale impact. It seems, however, that it is not always clear to everyone using these concepts what is meant with these concepts and how it differs from ‘business as usual.’ Adopting a systems change approach is often seen as an incremental change from ‘business as usual,’ while in most cases it implies a much more fundamental shift to the ‘way of working.’ Moreover, even where systems change has been more or less clearly defined, it remains a challenge to actually achieve it and to monitor it. To provide practitioners with relevant input, this paper provides an overview of relevant insights from literature and AidEnvironment’s own experience in conceptualizing, implementing and monitoring systems change. The insights have been brought together in a number of ‘systems change characteristics (‘what is it?’), and systems change principles (‘how to do it?’), with actionable details. These are meant to support organisations in understanding and implementing systems change.

Human Learning Systems: A practical guide for the curious - Lowe, Toby; Padmanabhan, Chandrima; McCart, Des; McNeill, Karen; Brogan, Andy; Smith, Mark (Centre for Public Impact, 2022)
Our new guide provides practical advice to help any organisation working in public service apply the Human Learning Systems approach to their work. In doing so, they will be better equipped to explore, learn and respond to the unique strengths and needs of each person, family and community they serve.

Context-monitoring for adaptive management - Pickwick, Sarah (World Vision, 2022)

Learning to change: The case for systemic learning strategies in the humanitarian sector - Ramalingam, Ben; Mitchell, John (ALNAP/ODI, 2022)
This paper presents the case for systemic organisational change in the humanitarian system. The paper firstly shows that that organisational learning has tended to reinforce existing ways of working and has not been able to shift a culture that values action over reflection. As a result, the rest of the paper asks about the most significant changes in the humanitarian sector

Learning from Life Story Collection and Analysis with Children who Work in the Leather Sector in Bangladesh - Sayem, Mashrique; Paul, Sukanta; Apgar, Marina; Snijder, Mieke (Institute of Development Studies, 2022)
CLARISSA (Child Labour: Action-Research-Innovation in South and South-eastern Asia) is a participatory evidence and innovation generating programme. We are generating evidence on the drivers of the worst forms of child labour (WFCL) and exploring how to address them through participatory Action Research (PAR) with children and other stakeholders in the leather supply chain in Dhaka, Bangladesh. Our main intervention modality is Systemic Action Research (SAR) (Burns, 2007), of which life story collection and analysis (LSC&A) is the first step in our participatory design that will inform child-led PAR groups which will become the engines of innovative responses to WFCL. The LSC&A methodology is a storytelling and story listening methodology and was chosen because of the universal power of stories to make sense of complex realities and seek new futures. People across the world like to tell their stories; they like to feel listened to and they are interested in how their story connects and compares to others. By collecting and analysing stories from hundreds of children in WFCL we can visualise the bigger system that each individual story is connected to. We hypothesize that through engagement in the process of telling, listening, collecting, and analysing life stories, children engaged in harmful and hazardous work will use their understanding of systems dynamics to move into creating their own solutions to the drivers of WFCL. In 2021 in Bangladesh, we collected 405 life stories from children living in Hazaribagh, Hemayetpur, Lalbagh, and Bhairab in Dhaka, with more than one hundred of these stories collected by children themselves. Following the story collection and transcription, children were supported by the CLARISSA implementation team to collectively analyse the stories through identifying critical ‘factors’ (events which have causes and consequences) and understanding how they causally relate to each other. The analysis of the 405 life stories resulted in the development of large system maps that illustrate all the causal dynamics that underpin lived experiences of WFCL. Based on the systemic analysis process the children identified themes of the PAR to be set up in their localities. Our experience with the LSC&A methodology is the first in the context of children in WFCL. The methodology has been used in one other project in Bangladesh to date. We therefore intentionally aimed to learn from the implementation process and to evaluate if and how the telling of, listening to, collecting and analysing of life stories is empowering and in turn whether it leads to increased ownership of the problems which motivates collective action (Burns, 2021). In this learning note we share our methodological learning and reflect on operational implications for designing and facilitating an LSC&A process with children which we hope will support adaptation and use of the methodology by others working in participatory programming with children.
Accepted: 2022-07-21T08:43:41Z

Storytelling for Systems Change: insights from the field - Snow, Thea; Murikumthara, David; Dusseldorp, Teya; Fyfe, Rachel; Wolff, Lila; McCracken, Jane (Centre for Public Impact, 2022)
Capturing the impact of community-led work The Centre for Public Impact, Dusseldorp Forum, and Hands Up Mallee have been exploring how stories can be used to more effectively communicate the impact of community-led systems change work. Community-led place based initiatives are modelling new ways of working - shifting away from top down, program-focussed approaches towards an approach grounded in systems thinking and community-led innovations. However, while these stories of change are sitting in communities, they’re often not being told or celebrated. We wanted to understand why this is, and what might be done to better enable these stories to be shared and heard. The story of storytelling We talked to a range of people to uncover the story of storytelling - including collective impact backbone team members, community members, storytelling experts, and those working in and around community-led systems change initiatives across Australia. We explored the roles stories play in different communities; what good storytelling looks like; what barriers to storytelling might be; and what role stories can play in supporting systems change. Our findings We have learned through this project that stories can be used both to change the system and to evaluate, understand and showcase the change that is occurring in communities. We have heard that different stories require different approaches – stories that are seeking to enable change look different to those that are seeking to celebrate change.

Trust-Based Philanthropy - Self-Reflection Tool - TBP Project (Trust-based philanthropy project, 2022)
Trust-based philanthropy is a holistic approach that requires alignment in a grantmaking organization’s four interrelated areas: 1) culture, 2) structure, 3) leadership, and 4) practices. This tool is designed to help funders at various stages of the trust-based journey examine how trust shows up across your organization, and identify areas that may need more inquiry, refinement, or deepening. This is intended as a reflection tool rather than a prescriptive tool.

What is the Outcomes Star? - Triangle (Triangle, 2022)
The Outcomes Stars are a suite of evidence-based outcomes measurement and keywork tools, which drive an ‘enabling help’ approach to service delivery. They support a person-centred, collaborative and trauma informed approach and give service users, workers, managers and commissioners vital information about needs and progress. Since the first version was published in 2006, the unique features of the Outcomes Star contributed to its popularity and widespread use, with over 1,000 organisations with licenses to use the Star including over 500 charities and 170 local authorities

An introductory systems thinking toolkit for civil servants - UK Government (UK Government Office for Science, 2022)
This document is a an Introductory Toolkit for for civil servants. It is one component of a suite of documents that aims to act as a springboard into systems thinking for civil servants unfamiliar with this approach. These documents introduce a small sample of systems thinking concepts and tools, chosen due to their accessibility and alignment to civil service policy development, but which is by no means comprehensive. They are intended to act as a first step towards using systems thinking approaches to solve complex problems and the reader is encourage to explore the wider systems thinking field further.

The UNDP Accelerator Labs enter a year of maturity: let a thousand flowers bloom! - Annual report 2022 - UNDP accelerator labs (UNDP, 2022)
The UNDP Accelerator Labs were designed as an agile and dynamic Network to allow communication and information transfer between 91 Accelerator Labs in 115 countries, and with the global innovation ecosystem, UNDP as a whole, and thousands of partners including grassroots innovators and their communities. The accumulated knowledge of this Network creates new pathways to the solutions that hold the key to sustainable development problems. What it looked like and how it unfolded last year will be illuminated in this annual report: The UNDP Accelerator Labs enter a year of maturity: let a thousand flowers bloom!

CLA Maturity Tool - Card Deck (Implementing Partners version 1) - USAID LEARN (USAID, 2022)
USAID’s Bureau for Policy, Planning and Learning (PPL) and its support mechanism, LEARN, have developed a Collaborating, Learning, and Adapting (CLA) Framework and Maturity tool to help USAID missions think more deliberately about how to plan for and implement CLA approaches that fit the mission’s context and assist them in achieving their development objectives. While the tool is intended primarily for USAID audiences to be used in participatory self-assessment workshops, the CLA Framework and maturity spectrum are relevant to a wider audience. USAID’s CLA Framework identifies key components and subcomponents of daily work that may be opportunities for intentional, systematic, and resourced CLA. The framework recognizes the diversity of what CLA can look like in various organizations and programs while also giving CLA structure, clarity, and coherence across two key dimensions: • CLA in the Program Cycle: how CLA is incorporated throughout Program Cycle processes, including strategy, project, and activity design and implementation; and • Enabling Conditions: how an organization’s culture, business processes, and resource allocation support CLA integration. Recognizing that CLA is not binary—it’s not an issue of “doing it or not doing it”—PPL and LEARN have developed a spectrum of practice for each of the 16 subcomponents in the framework. The spectrum offers examples of what integration might look like at different stages: Not Yet Present, Emergent, Expanding, Advanced and Institutionalized. The maturity stage descriptions are only illustrative and are intended to spark reflection on current practice and opportunities for improvements. In this resource, each CLA subcomponent page describes the key concepts for that topic and includes a description of the maturity stages. Although the descriptions were originally developed for USAID, the majority of the concepts easily transfer or have equivalents in the partner community. For example, although organizations outside of USAID may not hold “Portfolio Reviews” (part of the Pause & Reflect subcomponent), the majority hold some type of meeting to review programmatic progress. This is the seventh version of the CLA Framework and maturity spectrum. PPL and LEARN will continue reviewed and periodically update them based on user feedback, so if you have comments about the content, please email learning@usaid.gov. We would also love to hear how you’ve used this content with your team or organization.

Collaborating, Learning and Adapting (CLA) Maturity Spectrum (v7) - USAID LEARN (USAID, 2022)
USAID’s Bureau for Policy, Planning and Learning (PPL) and its support mechanism, LEARN, have developed a Collaborating, Learning, and Adapting (CLA) Framework and Maturity tool to help USAID missions think more deliberately about how to plan for and implement CLA approaches that fit the mission’s context and assist them in achieving their development objectives. While the tool is intended primarily for USAID audiences to be used in participatory self-assessment workshops, the CLA Framework and maturity spectrum are relevant to a wider audience. USAID’s CLA Framework identifies key components and subcomponents of daily work that may be opportunities for intentional, systematic, and resourced CLA. The framework recognizes the diversity of what CLA can look like in various organizations and programs while also giving CLA structure, clarity, and coherence across two key dimensions: • CLA in the Program Cycle: how CLA is incorporated throughout Program Cycle processes, including strategy, project, and activity design and implementation; and • Enabling Conditions: how an organization’s culture, business processes, and resource allocation support CLA integration. Recognizing that CLA is not binary—it’s not an issue of “doing it or not doing it”—PPL and LEARN have developed a spectrum of practice for each of the 16 subcomponents in the framework. The spectrum offers examples of what integration might look like at different stages: Not Yet Present, Emergent, Expanding, Advanced and Institutionalized. The maturity stage descriptions are only illustrative and are intended to spark reflection on current practice and opportunities for improvements. In this resource, each CLA subcomponent page describes the key concepts for that topic and includes a description of the maturity stages. Although the descriptions were originally developed for USAID, the majority of the concepts easily transfer or have equivalents in the partner community. For example, although organizations outside of USAID may not hold “Portfolio Reviews” (part of the Pause & Reflect subcomponent), the majority hold some type of meeting to review programmatic progress. This is the seventh version of the CLA Framework and maturity spectrum. PPL and LEARN will continue reviewed and periodically update them based on user feedback, so if you have comments about the content, please email learning@usaid.gov. We would also love to hear how you’ve used this content with your team or organization.

Collaborating, Learning, and Adapting Framework & Key Concepts (Implementing Partner Version 1) - USAID LEARN (USAID, 2022)
Although collaborating, learning, and adapting (CLA) are not new to USAID and its implementing partners, they often do not happen regularly or systematically and are not intentionally resourced. The CLA Framework above identifies components and subcomponents to help you think more deliberately about what approach to CLA might be best tailored to your organizational or project context. The framework recognizes the diversity of what CLA can look like in various organizations and projects while also giving CLA structure, clarity, and coherence across two key dimensions: - CLA in the Program Cycle: how CLA is incorporated into planning and design processes throughout the Program Cycle in order to improve their effectiveness; and - Enabling Conditions: how an organization’s culture, daily operating processes, and resource allocation support CLA integration. Organizations need both integrated CLA practices appropriate for their context and conducive enabling conditions to become stronger learning organizations capable of managing adaptively. The framework stresses the holistic and integrated nature of the various components of CLA to reinforce the principle that CLA is not a separate workstream—it should be integrated into existing processes to strengthen the discipline of development and improve aid effectiveness.

Change & Complexity: Vector Theory of Change - a theory of change for complex systems - Doyle, Linda (Cynefin Centre, 2021)

Evaluating complex interventions in international development - Masset, Edoardo; Shrestha, S.; Juden, M. (CEDIL, 2021)
This paper reviews promising methods for the evaluation of complex interventions that are new or have been used in a limited way. It offers a taxonomy of complex interventions in international development and draws on literature to discuss several methods that can be used to evaluate these interventions. Complex interventions are those that are characterised by multiple components, multiple stakeholders, or multiple target populations. They may also be interventions that incorporate multiple processes of behavioural change. While such interventions are very common and receive a large proportion of development aid budgets, they are rarely subject to rigorous evaluations. The CEDIL Methods Working Paper, ‘Evaluating Complex Interventions in International Development’, reviews promising methods for the evaluation of complex interventions that are new or have been used in a limited way. It offers a taxonomy of complex interventions in international development and draws on literature to discuss several methods that can be used to evaluate these interventions. The paper focuses its attention on methods that address causality and allow us to state conclusively whether an intervention works or not. It shows that several rigorous methods developed in different disciplines can be adapted and used to evaluate complex interventions in international development.
Section: Publications

Thinking and working politically: What have we learned since 2013 - Teskey, Graham (TWP Community of Practice, 2021)
The Thinking and Working Politically (TWP) Community of Practice (CoP) was established at a small meeting tacked on at the end of a meeting of Governance Advisers working for the United Kingdom’s Department of International Development (DFID) on South and South-East Asian countries, held in Delhi in November 2013. Since then, a number of meetings have been held throughout the world, each addressing different issues; ‘TWP’ has entered the lexicon of mainstream development; the CoP has expanded to more than 300 people; a Washington DC chapter has been established; and the International CoP has been granted modest funding from DFID’s successor, the Foreign, Commonwealth and Development Office (FCDO) and from Australia’s Department of Foreign Affairs and Trade (DFAT). It is legitimate to ask, however, what has been achieved operationally: how have the ideas underpinning TWP affected operational practice? This short paper traces the evolution of the idea and practice of TWP from 2013 to late 2021, and identifies what we have learned. What has been successful, and what has not? I asked in 2017 whether TWP had become a second orthodoxy (Teskey, 2017). Did this represent hubris or was 2017 in some ways the apogee of what might rather grandly be called the TWP ‘movement’?

Understanding Governance from the Margins: What Does It Mean In Practice? - Barnes, Katrina, Katrina, Barnes; Anderson, Colin; de Chassy, Stephanie; Ahmed, Affaf; Ali, Mudabbir; Aung, Myo Min; Chaimite, Egidio; Joshi, Anuradha; Khan, Danyal; Loureiro, Miguel; Posse, Lucio; Rowlands, Jo; et al. (Institute of Development Studies (IDS), 2021)
What does governance look like ‘from below’ – from the perspectives of poor and marginalised households? How do patterns of conflict affect that? These were the questions at the heart of the Governance at the Margins research project. Over three years from 2017-2020 we worked to explore this through in-depth study in conflict-affected areas of Mozambique, Myanmar, and Pakistan. Our research teams interviewed the same people regularly over that time, finding out how they resolved problems and interacted with authorities. In this paper we connect what we found to the realities and complexities of development practice, drawing on the input of 20 experienced practitioners working in bilateral and multilateral development agencies and international NGOs, who generously gave their time to help us think through the practical implications of our wealth of findings.

Getting Real about Unknowns in Complex Policy Work - Andrews, Matt (Research on Improving Systems of Education (RISE), 2021)
As with all public policy work, education policies are demanding. Policy workers need to ‘know’ a lot—about the problems they are addressing, the people who need to be engaged, the promises they can make in response, the context they are working in, and the processes they will follow to implement. Most policy workers answer questions about such issues within the structures of plan and control processes used to devise budgets and projects. These structures limit their knowledge gathering, organization and sense-making activities to up-front planning activities, and even though sophisticated tools like Theories of Change suggest planners ‘know’ all that is needed for policy success, they often do not. Policies are often fraught with ‘unknowns’ that cannot be captured in passive planning processes and thus repeatedly undermine even the best laid plans. Through a novel strategy that asks how much one knows about the answers to 25 essential policy questions, and an application to recent education policy interventions in Mozambique, this paper shows that it is possible to get real about unknowns in policy work. Just recognizing these unknowns exist—and understanding why they do and what kind of challenge they pose to policy workers—can help promote a more modest and realistic approach to doing complex policy work.

DPPD Handbook. A step-by-step guide for development practitioners to apply the Data Powered Positive Deviance method - DPPD (DPPD Initiative, 2021)
The Method Positive Deviance (PD) is based on the observation that in every community or organization, there are a few individuals who achieve significantly better outcomes than their peers, despite having similar challenges and resources. These individuals are referred to as positive deviants, and adopting their solutions is what is referred to as the PD approach¹. The method described in this Handbook follows the same logic as the PD approach but uses pre-existing, non-traditional data sources instead of — or in conjunction with — traditional data sources. Non-traditional data in this context broadly refers to data that is digitally captured (e.g. mobile phone records and financial data), mediated (e.g. social media and online data), or observed (e.g. satellite imagery). The integration of such data to complement traditional data sources generally used in PD is what we refer to as Data Powered Positive Deviance² (DPPD). The digital data opportunity Recent developments in the availability of digital data provide an opportunity to look for positive deviants³ in new ways and in unprecedented geographical and on temporal scales. A number of studies⁴ have described the challenges related to the application of the PD approach in development. Given these challenges, there are obvious opportunities for innovation in PD and our particular interest here is in the innovative opportunities offered by non-traditional data, following the increasing “datafication” of development and the growing availability of big datasets in a variety of development sectors⁵. DPPD builds on this and expands our ability to extract value from non-traditional digital data while providing a systematic process for leveraging local know-how and the collective wisdom of communities. Data Powered Positive Deviance The DPPD method described in this Handbook emerged from a process of research and testing and follows the same stages as the PD approach. The difference is that DPPD integrates pre-existing, non-traditional data across the five stages, requiring a series of new and specific methods and practices that are not required in the PD approach. The first stage is also somewhat different because it not only defines the problem, but it also checks if it is suitable and feasible to use the DPPD method for the proposed project. Table 1 lists the five stages of the DPPD method. This Handbook dedicates a section to each stage. Stage 1 Assess problem-method fit Stage 2 Determine positive deviants Stage 3 Discover underlying factors Stage 4 Design and implement interventions Stage 5 Monitor and evaluate

Strengthening Innovation Ecosystems - IDIA (International Development Innovation Alliance, 2021)
This report presents the findings of that work and includes an analysis of current definitions of ecosystem strengthening approaches as well as insights and learning from consultations with IDIA agencies and innovation ecosystem actors in Asia, Africa and Latin America.

System Innovation on Purpose - Leadbeater, Charles; Winhall, Jennie (The Rockwool Foundation, 2021)
In Building Better Systems, we introduced four keys to unlock system innovation: purpose and power, relationships and resource flows. These four keys make up a set. Systems are often hard to change because power, relationships, and resource flows are locked together in a reinforcing pattern to serve the system’s current purpose. Systems start to change fundamentally when this pattern is disrupted and opened up. Then a new configuration can emerge, serving a new purpose. In this essay series we delve deeper into these four keys and provide practical advice on how they can be put to use. This essay is about the role that purpose plays in orchestrating complex systems and how system innovators can create a new system around a new sense of purpose.

Are we there yet? Localisation as the journey towards locally led practice: models, approaches and challenges - Baguios, Arbie; King, Maia; Martins, Alex; Pinnington, Rose (ODI, 2021)
Localisation and locally led international development practice has long been discussed, but has still not been delivered. Systemic barriers have posed challenges, and the term itself is contested. Now, the last tumultuous 18 months could provide a critical juncture to finally move forward with this crucial agenda. The pandemic has highlighted structural inequalities in the global system, and disrupted ways of working in the international development sector. The Black Lives Matter movement has brought conversations about racism and colonialism to the fore. And the climate crisis has highlighted the need for global action on humanity’s challenges that remain rooted in local realities. The emerging analysis in this review aims to set out the key issues in this agenda, building on a wealth of existing knowledge. It aims to span sectors, highlighting many new and existing models and approaches in the humanitarian, development, philanthropic and private sectors. It reviews the barriers and challenges to localisation and locally led practice, with a view to informing a campaign for systemic change to move forward with this agenda. The review is based on: a rapid review of the literature and evidence on localisation and locally led practice; two consultations with over 100 total participants, targeted at Global South actors; and analysis of 28 existing models and approaches.

Building a culture of learning at scale: learning networks for systems change. - McKenzie, Fiona (Orange Compass & Paul Ramsay Foundation, 2021)
This scoping paper explores the question ‘what would it take to build a culture of learning at scale?’. It focuses on systems-wide learning that can help to inform systems change efforts in complex contexts. To answer this question, literature was reviewed from across diverse disciplines and the realms of education, innovation systems, systems thinking and knowledge management. This inquiry was also supported by in-depth interviews with numerous specialists from the for-purpose sector and the examination of several case studies of learning across systems. The goal was to derive common patterns to inform a ‘learning for systems change’ framework. Learning for systems change is critical when working with complexity. The dynamic nature of complex adaptive systems requires an ability to continually sense and learn from the system and adapt accordingly. This is because the nature of the challenge and ‘what works’ to meet the challenge is continually shifting (Lowe and Plimmer, 2019). This requires an ongoing process of iterative inquiry that draws upon wisdom and insights from across the system. Such learning challenges traditional siloes of expertise and organisational boundaries (Clarke et al., 2019). Learning is not simply a nice to have. It is critical for greater impact and improved outcomes, particularly in mission-driven organisations and initiatives (Price et al., 2019). In this paper, a ‘learning networks’ approach is proposed, one that draws upon individual, group and systems-wide learning to build capacity and resilience for systems change in uncertain environments. This fills a gap in the literature where the focus is largely on learning within organisations. Instead, the focus here is on what is required to support learning to occur across scales and boundaries - from the individual to system-wide. A simple meta-framework for developing learning networks is proposed that includes high level guidance on the enabling conditions - the mindsets, relationships, processes and structures- that would enable learning networks to flourish.

Lessons Learned from Applying the Data Powered Positive Deviance - Pawelke, Andreas; Glücker, Andreas; Albanna, Basma; Boy, Jeremy (DPPD, 2021)
This report presents six learnings from four pilot projects conducted by the Data Powered Positive Deviance (DPPD) initiative, a global collaboration between the GIZ Data Lab, the UNDP Accelerator Labs Network, the University of Manchester Center for Digital Development, and UN Global Pulse Lab Jakarta. The pilots seek out grassroots solutions to development challenges that range from the interaction between livestock farming and deforestation to gender-based violence and insecurity in dense urban environments in Ecuador, Mexico, Niger and Somalia. The learnings relate to the early stages of the DPPD method, originally proposed by Albanna & Heeks [1], and focus mainly on the access to, and use of digital data. They are summarized as follows: 1. Remain flexible in the face of data unavailability 2. Leverage existing partnerships for data access 3. Map and fill know-how gaps early 4. Scale with caution 5. Look at deviance over time 6. Look beyond individual or community practices and behavior The report is written for development practitioners, data analysts, domain experts, and more generally anyone interested in using new data sources and technologies to uncover successful local solutions to development challenges.

Twenty years of UK governance programmes in Nigeria - Piron, Laure-Hélène; Cummings, Clare; Williams, Gareth; Derbyshire, Helen; Hadley, Sierd (ODI, 2021)
This Flagship report analyses 20 years of governance programmes in Nigeria funded by the UK Department for International Development (DFID) and the Foreign, Commonwealth and Development Office (FCDO) in the North-western states of Jigawa (since 2001), Kano (since 2005) and Kaduna (since 2006), as well as the North-eastern state of Yobe (since 2011). The report’s main research question is whether, how, under what conditions and for whom UK-funded state-level governance programmes in Nigeria have contributed to sustained changes in governance, and related changes in health and education. ... The report concludes with the following recommendations: To international development partners: 1. Invest for the long term – 10 to 20 years – combining support for both state and nonstate actors. 2. Ensure programmes have the strategic-level mandate, managerial capacity and frontline staff skills to pursue politically savvy opportunities. 3. Take PEA to the next level by unpacking causal mechanisms, understanding incentives and designing interventions to make change happen. 4. Give governance programmes the ability to flex between core governance and service delivery issues. 5. Incentivise greater collaboration between governance and sector programmes. 6. Incentivise greater attention to gender, and to social inclusion beyond disability issues, in governance programming. To FCDO: 7. Empower and resource FCDO teams to enable TWP programmes, ensuring decision-making by country teams to respond to local priorities. 8. Re-imagine TWP for FCDO Nigeria, giving implementers the space to operate in TWP ways. 9. Incentivise stronger collaboration between PERL, Lafiya (health programme) and the Partnership for Learning for All in Nigerian Education. 10. Invest in impact data analysis. To partner governments in Nigeria and beyond: 11. Explicitly set out the objectives for which you would like to receive assistance. 12. Use TWP principles to decide how development partners can support your political objectives and the scope for politically-feasible and mutually-beneficial collaboration. 13. Invest in the coordination of development partners. To non-state partners in Nigeria and beyond: 14. Join coalitions to achieve your priorities. 15. Select development partners which can strengthen your skills, not just fund your activities.

Supply Chains, the Informal Economy, and the Worst Forms of Child Labour - Aked, Jody (Institute of Development Studies, 2021)
As a cohort of people, ‘children in work’ have become critical to the everyday functioning of diverse supply chain systems. This Working Paper considers diverse commodity chains (leather, waste, recycling and sex) to explore the business realities that generate child labour in its worst forms. A review of the literature finds that occurrence of the worst forms of child labour (WFCL) in supply chain systems is contingent on the organising logics and strategies adopted by actors in both the formal and informal economies. Piecing together the available evidence, the paper hypothesises that a supply chain system is sensitive to the use of WFCL when downward pressure to take on business risk cannot be matched by the economic resilience to absorb that risk. Emergencies and persistent stressors may increase risk and reduce resilience, shifting norms and behaviour. There is a need for further work to learn from business owners and workers in the informal economy.
Accepted: 2021-07-26T11:16:44Z

How Does Participatory Action Research Generate Innovation? Findings from a Rapid Realist Review - Snijder, Mieke; Apgar, J. Marina (Institute of Development Studies, 2021)
This Emerging Evidence Report shares evidence of how, for whom, and under what circumstances, Participatory Action Research (PAR) leads to innovative actions. A rapid realist review was undertaken to develop programme theories that explain how PAR generates innovation. The methodology included peer-reviewed and grey literature and moments of engagement with programme staff, such that their input supported the development and refinement of three resulting initial programme theories (IPTs) that we present in this report. Across all three IPTs, safe relational space, group facilitation, and the abilities of facilitators, are essential context and intervention components through which PAR can generate innovation. Implications from the three IPTs for evaluation design of the CLARISSA programme are identified and discussed. The report finishes with opportunities for the CLARISSA programme to start building an evidence base of how PAR works as an intervention modality, such as evidencing group-level conscientisation, the influence of intersecting inequalities, and influence of diverse perspectives coming together in a PAR process.
Accepted: 2021-07-23T08:48:45Z

Designing a Participatory Programme at Scale: Phases 1 and 2 of the CLARISSA Programme on Worst Forms of Child Labour - Burns, Danny; Apgar, Marina; Raw, Anna (Institute of Development Studies, 2021)
CLARISSA (Child Labour: Action-Research-Innovation in South and South-Eastern Asia) is a large-scale Participatory Action Research programme which aims to identify, evidence, and promote effective multi-stakeholder action to tackle the drivers of the worst forms of child labour in selected supply chains in Bangladesh, Nepal, and Myanmar. CLARISSA places a particular focus on participants’ own ‘agency’. In other words, participants’ ability to understand the situation they face, and to develop and take actions in response to them. Most of CLARISSA’s participants are children. This document shares the design and overarching methodology of the CLARISSA programme, which was co-developed with all consortium partners during and since the co-generation phase of the programme (September 2018–June 2020). The immediate audience is the CLARISSA programme implementation teams, plus the Foreign, Commonwealth & Development Office (FCDO). This design document is also a useful reference point for other programmes trying to build large-scale participatory processes. It provides a clear overview of the CLARISSA programmatic approach, the design, and how it is being operationalised in context.

FCDO Programme Operating Framework - FCDO (FCDO, 2021)
The Programme Operating Framework (PrOF) sets the standard for how the Foreign, Commonwealth & Development Office (FCDO) delivers its programmes and projects.

Energy Governance in Developing Countries — A New Approach - Mcculloch, Neil (The Policy Practice & Chemonics, 2021)
In 2015, leaders from around the world agreed to 17 Sustainable Development Goals (SDGs) to be achieved by 2030. The seventh goal (SDG7) is: “Ensure access to affordable, reliable, sustainable and modern energy for all.” In the same year, the world’s leaders concluded the Paris Agreement to tackle climate change, which will require a global transition in the energy sector away from the use of fossil fuels. Yet, despite growing investments in clean energy in many developing countries, the transition is happening much more slowly than needed. The central reason for this is poor energy governance. This technical brief shows how poor energy governance damages energy access and efforts to improve the quality and reliability of power. It explains the political reasons why energy governance is so bad in many countries and contrasts this with the current system of procuring technical assistance, which largely ignores the energy governance challenge. It shows that a new approach to tackling energy governance is emerging that is better matched to the nature of the problems faced and provides recommendations on how to implement it.

Why Tackling Energy Governance in Developing Countries Needs a Different Approach - McCulloch, Neil (The Policy Practice & Chemonics, 2021)
Global efforts to improve energy access and quality and to tackle climate change need a different approach to addressing poor energy governance. In 2015, leaders from around the world agreed to 17 Sustainable Development Goals (SDGs) to be achieved by 2030.1 The seventh goal (SDG7) is “Ensure access to affordable, reliable, sustainable and modern energy for all.” In the same year, the world’s leaders concluded the Paris Agreement to tackle climate change, which will require a global transition of the energy sector away from the use of fossil fuels. Yet, in many developing countries, despite growing investments in clean energy, the transition is happening much more slowly than needed to achieve SDG7 and avert damaging climate change. The central reason for this is poor energy governance. This paper outlines the size and nature of the energy challenge, with a focus on electricity. It describes the investments that are currently being made to improve the quality of power and access to electricity — and the growing evidence that investments often fail due to poor energy governance. The paper then delves more deeply into how bad governance influences the quality of and access to electricity, with specific country examples. It shows the importance of understanding how electricity fits into the political settlement of a country and how this affects the incentives of key actors in the sector. Unfortunately, donor projects designed to widen electricity access or to support reform of the power sector in developing countries often pay too little attention to the problem’s political nature; the same is true of measures to improve energy efficiency or to promote renewables. The paper outlines a new way of thinking about energy governance and shows how interventions can be better matched to the different governance challenges that they face. It concludes with recommendations for donors on how energy programs can be better designed and procured — as well as recommendations for implementors on how to improve the chances of successful implementation by adapting to the political realities of the contexts in which they operate.

UNDP Evaluation Guidelines - June 2021 update - UNDP (Independent Evaluation Office of UNDP, 2021)

Learning to adapt & adapting to learn - Using elements of outcome mapping in the ‘Resilient Adolescents in the Syria Crisis’ programme - van Ongevalle, Jan; Kvintradze, Ana; Rennesson, Gaël; Miller, David (Save the Children, 2021)
This learning paper highlights how elements of outcome mapping were used by Save the Children Sweden in a project (2018-2020) that supports adolescents, affected by the Syria crisis, to become more resilient. The paper first outlines how the spheres of influence framework has been applied to develop an actor focused theory of change. It then describes how progress markers, as an alternative to SMART indicators, were formulated to monitor the programme’s results. The paper also outlines how long lists of progress markers were categorised in a more realistic and practical results framework. The paper then continues to elaborate how outcome journals, qualitative data analysis techniques and regular review meetings and reflection workshops were utilised for data collection, for collective learning among programme stakeholders and for informing planning and programme adjustment. Various practical guidelines and tips on how to implement elements of outcome mapping are provided. The final part of the paper explores to what extent outcome mapping was able to foster several key enablers of adaptive programme management and highlights some of the challenges that programme stakeholders faced. Practical recommendations towards the use of outcome mapping in future programmes are also proposed.

Applying Systems Thinking to Education: The RISE Systems Framework - Spivack, Marla (Research on Improving Systems of Education (RISE), 2021)
Many education systems in low- and middle-income countries are experiencing a learning crisis. Many efforts to address this crisis do not account for the system features of education, meaning that they fail to consider the ways that interactions and feedback loops produce outcomes. Thinking through the feedback relationships that produce the education system can be challenging. The RISE Education Systems Framework, which is sufficiently structured to give boundaries to the analysis but sufficiently flexible to be adapted to multiple scenarios, can be helpful. The RISE Framework identifies four key relationships in an education system: politics, compact, management, and voice and choice; and five features that can be used to describe these relationships: delegation, finance, information, support, and motivation. This Framework can be a useful approach for characterising the key actors and interactions in the education system, thinking through how these interactions produce systems outcomes, and identifying ways to intervene that can shift the system towards better outcomes.

Impact Evidence and Beyond: Using Evidence to Drive Adoption of Humanitarian Innovations - Dodgson, Kate; Crowley, Catie (Elrha, 2021)
This learning paper provides guidance to humanitarian innovators on how to use evidence to enable and drive adoption of innovation. Innovation literature and practice show time and time again that it is difficult to scale innovations. Even when an innovation is demonstrably impactful, better than the existing solution and good value for money, it does not automatically get adopted or used in mainstream humanitarian programming. Why do evidence-based innovations face difficulties in scaling and how can innovators best position their innovation to scale? This learning paper is for innovators who want to effectively use evidence to support and enable their journey to scale. It explores the underlying social, organisational and behavioural factors that stifle uptake of innovations. It also provides guidance on how to use, prioritise and communicate evidence to overcome these barriers. The paper aims to help innovators generate and present their evidence in more tailored and nuanced ways to improve adoption and scaling of their innovations.

Understanding Strategic Capacity in Constituency-Based Organizations - Jane Booth-Tobin; Kal Munis; Lynsy Smithson-Stanley; Hahrie Han (The P3 Lab, Johns Hopkins University, 2021)
Movement organizations work in inherently uncertain political environments. Whether an organization is advocating for a new minimum wage, working to close a private prison, or seeking to influence an election, the terrain they are operating on shifts nearly every day. That is increasingly true as political uncertainty rises in the 21st century, particularly for historically race-class subjugated communities. Any movement-based organization seeking to build, exercise, and win political power must have sophisticated strategic capacities to be able to navigate these uncertain, dynamic, and constantly shifting political environments. Yet, our knowledge of how movements can nurture the kind of strategic capacities that allows them to build constituencies and leadership that can operate in the flexible ways needed for these dynamic circumstances is limited. This report seeks to synthesize what is currently known about organizations that successfully build and wield strategic capacity, with a particular eye toward how it might apply to constituency-based organizations. The report concludes with an assessment and facilitated conversation guide to support movements and movement organizations in understanding how developed (or not) their strategic capacities are.

Let’s Take the Con Out of Randomized Control Trials in Development - Pritchett, Lant (Center for International Development, Harvard University, 2021)
The enthusiasm for the potential of RCTs in development rests in part on the assumption that the use of the rigorous evidence that emerges from an RCT (or from a small set of studies identified as rigorous in a “systematic” review) leads to the adoption of more effective policies, programs or projects. However, the supposed benefits of using rigorous evidence for “evidence based” policy making depend critically on the extent to which there is external validity. If estimates of causal impact or treatment effects that have internal validity (are unbiased) in one context (where the relevant “context” could be country, region, implementing organization, complementary policies, initial conditions, etc.) cannot be applied to another context then applying evidence that is rigorous in one context may actually reduce predictive accuracy in other contexts relative to simple evidence from that context—even if that evidence is biased (Pritchett and Sandefur 2015). Using empirical estimates from a large number of developing countries of the difference in student learning in public and private schools (just as one potential policy application) I show that commonly made assumptions about external validity are, in the face of the actual observed heterogeneity across contexts, both logically incoherent and empirically unhelpful. Logically incoherent, in that it is impossible to reconcile general claims about external validity of rigorous estimates of causal impact and the heterogeneity of the raw facts about differentials. Empirically unhelpful in that using a single (or small set) of rigorous estimates to apply to all other actually leads to a larger root mean square error of prediction of the “true” causal impact across contexts than just using the estimates from non-experimental data from each country. In the data about private and public schools, under plausible assumptions, an exclusive reliance on the rigorous evidence has RMSE three times worse than using the biased OLS result from each context. In making policy decisions one needs to rely on an understanding of the relevant phenomena that encompasses all of the available evidence.

Beyond Net Zero - A Systemic Design Framework - DC (Design Council, 2021)
What is systemic design? Systemic design is the acknowledgement of complexity and interconnectedness throughout the design thinking and doing process. It is both a mindset and a methodology - considering the structures and beliefs that underpin a challenge. It asks both designers and non-designers to radically reimagine and create new ways of living. The climate and biodiversity crises, and associated social injustice, are the biggest challenges facing humanity today, and so design must evolve to include these: acting more systemically, shifting towards alternative regenerative systems. Systemic Design Framework Our overall, high-level methodology for Design for Planet. It keeps the core premise of the Double Diamond (divergent and convergent thinking) and asks designers to come together to Explore, Reframe, Create and Catalyse. It expands the design process to encompass the ‘invisible activities’ that sit around it: Orientation and vision setting, Connections and relationships, Leadership and storytelling, and Continuing the journey. It also includes six principles that underpin Systemic Design, and the four key roles needed to help work in this way.

Systemic Design Toolkit - DC (Design Council, 2021)
Systemic Design Toolkit Our way of putting the Systemic Design Framework into practice. It is for people who want to use design to address the climate emergency, but who find that their current methods and ways of working are insufficient for the complexity of the challenge. The toolkit contains 11 tools that sit across the systemic design stages, helping you to adopt a systemic approach.

Implementing adaptive management: A front-line effort — Is there an emerging practice? - Teskey, Graham; Tyrrel, Lavinia (Abt Associates, 2021)
Among the many principles that currently inform donor-funded development initiatives, three appear to stand out: they should be politically informed, locally led, and adaptive. There is as yet little practical guidance for aid implementers regarding how to operationalise these approaches. What will it take to shift practice away from linear and planned approaches, towards models which foster local leadership and which engage with emergent and complex systems? This paper suggests that the answer is not to throw out the discipline of the logical framework, results frameworks, or theories of change. Rather they need to be handled rather more reflectively and ‘elastically’. The purpose of this paper is to set out how this can be achieved, and to propose 15 tools for donors, implementors and front-line staff to apply adaptive management (AM) in practice, at critical stages of the project cycle and within the dominant aid paradigm. This is what we are calling PILLAR: politically informed, locally led and adaptive responses. We are framing PILLAR to cover the full project cycle (design, implementation and review), hence the nomenclature of an ‘end to end’ approach. Our hope is that these tools will eventually replace the current planned, log-frame driven and top-down approach to aid design and delivery which dominates the development sector.

Lessons learned from PERL and partners' response to the COVID-19 crisis - Sharp, Samuel; Nwachukwu, Tochukwu; Srivatsa, Sripriya Iyengar (PERL Programme, 2021)
The first case of COVID-19 in Nigeria was confirmed on 27 February 2020, with the first lockdown orders issued on 30 March 2020. The pandemic and resultant containment measures have had farreaching socio-cultural, economic, financial and political implications, globally as well as in Nigeria. For the Partnership to Engage, Reform and Learn (PERL) and its partners, the pandemic has required considerable adaptation of their strategic approach and working practices. This report reflects on how COVID-19 changed the operating context for PERL’s partners, how PERL responded and what lessons have been learned across delivery teams. For government partners, the most substantial impacts have been to budgets, struck by falling oil prices and reduced economic activity. Universally, states have had to adjust budgets and reforecast, revising budgets downwards and shifting the focus of expenditure towards healthcare. The World Bank’s State Fiscal Transparency, Accountability and Sustainability (SFTAS) Programme has generated powerful incentives for this budget revision, which PERL has been able to work alongside. A range of new governance structures – such as public response committees and task forces – have been established to deal with various aspects of COVID-19 policy, and PERL has had to grapple to maintain its ongoing engagement with these. For civil society organisations (CSOs), the closure of offices from 30 March 2020 has changed the nature of engagement with government. CSOs often developed innovative approaches to maintaining access, including use of social media and direct calls. But the shift to virtual working has been challenging for many CSOs, both in terms of covering the costs of data for virtual meetings and the risks of disengagement and marginalisation for some organisations. In response to this changed context, from March 2020 PERL began to restrategise. The flexible nature of the programme’s workplans, progress markers and budgets enabled activities to be adjusted in a relatively timely manner, with a new workplan approved by the end of April 2020. Central PERL management developed a COVID-19 response strategy which provided a broad framework for adaptations, but allowed substantial autonomy to state and regional teams to lead on reprioritisation according to their understanding of the local context. This was valued by both management and delivery staff. Challenges manifested themselves more in effectively delivering on these adapted workplans than in the process of restrategising – due to two rounds of budget cuts, the merger of DFID and the FCO to form the Foreign, Commonwealth and Development Office (FCDO) and the difficulties of engaging partners virtually. Overall, the pandemic undoubtedly delayed activities (by roughly three months for deprioritised areas of work), but resulted in an array of new, tailored interventions under its broad categories of work. Interventions relating to the health sector became more prevalent, as did work supporting budget adjustments. Domestic resource mobilisation and education interventions were often adjusted to be more relevant to the COVID context or experienced delay. The report provides short illustrative case studies of PERL’s adaptations to: support budget revisions; work with media partners on COVID-19 sensitisation; tracking and advocacy for palliative distribution; and support for the introduction of tax relief. There is some evidence, albeit partial, that PERL was able to take advantage of windows of opportunity offered by the pandemic to drive ahead with certain ongoing reform initiatives.

A Guide to Assessing the Political Economy of Domestic Climate Change Governance - Worker, Jesse; Palmer, Niki (World Resources Institute, 2021)
This paper discusses how understanding the domestic political economy of climate governance is critical for developing informed strategies to build and sustain political ambition. It provides guidance and a methodology for domestic stakeholders to determine the types of institutional reforms, incentives, coalitions, and policy design that can entrench long-term political support for climate ambition. The assessment can also help users identify political barriers to more equitable climate action and identify reforms that may strengthen inclusion and accountability. EXECUTIVE SUMMARY Highlights ▪ There is overwhelming evidence of the social, economic, and environmental case to reduce greenhouse gas (GHG) emissions and rapidly scale up adaptation. Yet, despite a proliferation of climate laws and policies over the last 10–15 years, emissions are still rising, and adaptation needs remain urgent. ▪ This calls for a more sophisticated assessment of the political economy factors that may enable or constrain implementation of policies and actions and sustain political commitment at the country level. ▪ This guide offers an assessment methodology to understand how structural factors, rules and norms, stakeholders and interests, and ideas and narratives influence the political economy of climate action in a given country context. ▪ The methodology was developed on the basis of climate policy, governance, and political economy literature with contributions from subject matter experts. ▪ We intend the assessment to support civil society coalitions, reform-minded civil servants and politicians, international organizations, and other stakeholders. ▪ The resulting analysis should deepen the understanding of context while informing the advocacy, policy design, coalition building, capacity building, and communications of domestic stakeholders.

Theory-Based Approaches to Evaluation: Concepts and Practices - Treasury Board of Canada Secretariat (Government of Canada, 2021)
This document introduces some of the key concepts of theory-based approaches to evaluation. It is hoped that readers will be encouraged by the information and advice provided in this document and will explore the use (e.g., through pilot evaluations) of theory-based approaches to evaluation in a federal setting. To support this, Sections 1.0 to 8.0 of the document describe the general application of theory-based approaches to evaluation, and Sections 9.0 and 10.0 discuss the potential application of theory-based approaches to a range of federal programs. This document is neither an exhaustive training program in theory-based evaluation nor a step-by-step guide to undertaking a theory-based evaluation. Evaluators who wish to integrate theory-based approaches into their practices are encouraged to pursue additional readings (including those referenced in this document) and, as appropriate, to seek additional support in undertaking a theory-based evaluation.
Last Modified: 2021-03-22

The Future of Development: “Make Happen” with Portfolios of Options - CHÔRA Foundation (CHÔRA Foundation, 2021)
This Green Paper intends to review key elements of the problem that Development actors will confront as a new decade opens up ahead of us. It will articulate a solution that we believe should become an inherent feature of Development programs and initiatives. This is the outcome of an intense period of experiences and reflections in the Development space across different geographies and institutional mandates and activities, during which the Foundation has collaborated with institutions such as the UNDP and Grand Challenges Canada. At the heart of our collaborations is a Strategic Innovation and System Transformation Framework, with its associated concepts, working definitions, processes, tools and people. Sourced from CHÔRA Foundation’s knowledge and practice assets, this is a capability we intend to make relevant, customise, scale up and distribute to our partners and stakeholders. We are looking to create with them a space that offers the world a transformational capability. Central to this capability we see a distinctive and robust practice: the design and dynamic management of Portfolios of Strategic Innovation and System Transformation Options. These Portfolios are unique, context relevant, embedded mechanisms for learning, sensemaking and problem solving that social systems leverage to have an impact on themselves and their problems, and to induce the transformations that are necessary to them. It is our view that Portfolios of Options are the most effective means by which human social systems can supply themselves with budgets of possibility that ensure choice and create opportunity. They will also support pragmatic evolutionary outcomes and enable resilience.

FCDO Beneficiary Engagement - FCDO (FCDO, 2021)
This guide has been developed to help build confidence and capability, distilling useful tips and considerations that may help teams think through programme delivery issues and interpret elements of the PrOF Rules. This PrOF Guide lays out: - The definition of beneficiary engagement. - The case for beneficiary engagement. - FCDO’s approach to beneficiary engagement. - Practical tips for how to integrate beneficiary engagement throughout the programme cycle, including guiding questions to ask, rules of thumb to apply, tools to use and challenges and special topics to consider during Design, Mobilisation, Delivery and Closure phases. At its core, beneficiary engagement is about processes that recognise the dignity and support the agency of the people whose lives we are trying to improve. It is about beneficiaries and programme constituents having a say over what assistance they receive and how they receive it. It is about engaging beneficiaries and programme constituents as people with valuable insights and capabilities, rather than a compliance exercise. It is about empowering all beneficiaries and programme constituents to improve their lives by engaging them in helping us make better design and delivery decisions for the programmes that affect their lives. It’s about ensuring that a diverse set of voices are heard. Harnessing the power of beneficiary engagement can also improve outcomes and help programmes reach them more efficiently. It helps define and promote Value for Money, improve transparency and ensure that beneficiaries are safe from harm and empowered to speak out wherever harm does occur. Beneficiary engagement is supported by FCDO policy commitments, PrOF Rules, internal guidance and key international commitments. Beneficiary engagement is applicable to, and valuable in, a range of contexts, including humanitarian contexts. Beneficiary engagement requires time and resources, but programmes can help ensure the benefits of engagement outweigh the costs by Doing No Harm, Engaging Early and Closing the Loop. It is ultimately the Programme Responsible Owner’s responsibility to determine what beneficiary engagement is suitable and feasible for a programme. Quality beneficiary engagement is not about applying the one “right” approach but rather thoughtfully considering key questions and applying key principles and proven tools to the programme’s context to achieve a programme that supports the dignity and agency of all beneficiaries as much as possible.

LearnAdapt: a synthesis of our work on adaptive programming with DFID/FCDO (2017–2020) - Laws, Ed; Pett, Jamie; Proud, Emma; Rocha Menocal, Alina (ODI, 2021)
Key takeaways. • Development is not linear or straightforward, but rather complex, uncertain and context-specific. This calls for international development actors to work differently, in ways that are based on deliberate experimentation, learning and adaptation, to inform decisions and drive effective development. • Although it might go by different names, adaptive programming has been used in a variety of areas and fields in both the public and private sectors. Development practitioners have much to learn from and contribute to these different approaches and experiences. • Trust and relationship-building across all relevant stakeholders are among the most critical enabling factors for adaptive management. They are essential to give partners the space, autonomy and authority needed to try, test, reflect, iterate and feed back at the frontline of implementation, and to give donors the confidence that decisions are being made on the basis of evidence and learning to improve effectiveness. • There is an urgent need to rethink how accountability requirements, results frameworks, value for money considerations, performance markers, procurement and contracting mechanisms and other processes are understood and applied so that they are better aligned with and can support adaptive management more effectively. • The role of senior managers leading adaptive programmes from the donor side should be to create a space for experimentation and learning. Funders should hold their partners accountable for learning and how it feeds into effective programming, rather than for delivering on predetermined targets. • While formal guidance is important, leadership, champions, institutional incentives, a supportive management culture and appropriate mindsets are essential to encourage adaptive ways of working. • Adaptive management is resource-intensive. It requires skill, commitment, time for building trust and investments in learning. It is a journey, not an immediate destination – so it calls for patience, open-mindedness and a more nuanced approach to risk.

Value for money and adaptive programming - Approaches, measures and management - Laws, Ed; Valters, Craig (ODI, 2021)
- The United Kingdom’s Foreign, Commonwealth and Development Office (FCDO)’s standard economy, efficiency, effectiveness/cost-effectiveness and equity (4E) framework is still relevant for approaching, measuring and managing value for money (VfM) for adaptive programmes. • However, this framework needs to be reframed to capture and incentivise flexibility, learning and adaptation. • VfM appraisal and reporting should be done in a way that draws on beneficiary feedback and informs good decision-making, rather than just being a compliance exercise. • If VfM appraisal and reporting cannot be done appropriately for adaptive programmes, it should be avoided or minimised. There is a risk of diverting time and resources from more suitable tools and methods.

20 years of Outcome Mapping - Evolving practices for transformative change - OM Learning Community (Outcome Mapping Learning Community, 2021)
To mark 20 years of Outcome Mapping, this paper presents the core principles and concepts that are foundational to using the approach. It also presents a set of guiding practices to support transformative change. The OMLC Stewards presented the paper at a special webinar on 29th April 2021 - see the link below for the recording.

Adaptive Learning Guide: A pathway to stronger collaboration, learning, and adapting - Ross, Joey; Karlage, James; Etheridge, James; Alade, Mayowa; Fifield, Jocelyn; Goodwin, Christian; Semrau, Katherine; Hirschhorn, Lisa (USAID MOMENTUM Knowledge Accelerator, 2021)
The purpose of this Adaptive Learning Guide is to provide MOMENTUM project teams with the information and resources to integrate adaptive learning into the design, implementation, and improvement of MNCH/FP/RH programs. This guide provides a conceptual introduction to adaptive learning using links to existing resources and real-world examples of how adaptive learning can drive continuous learning and improvement in project work. The guide is built upon three foundational assumptions: We work in dynamic, often unpredictable environments. Unexpected turns of events will occur. Progress is rarely, if ever, linear. Integrating the principles and practices of USAID’s Collaborating, Learning, and Adapting Toolkit into projects and initiatives requires designing for learning and adaptation. We intend the guide to serve as a “starting point” for interested individuals and teams to begin or strengthen the processes that support the integration of adaptive learning into project work.

Opportunities and challenges for DAC members in ‘adapting to context’ - Sharp, Samuel; Wild, Leni (2021)
Key Messages New principles for OECD DAC members on ‘Managing for Sustainable Development Results’ emphasise tailoring result management approaches to different contexts; balancing internal compliance with empowerment at ground level; and adapting implementation in the pursuit of long-term outcomes. However development organisations face numerous challenges in aligning with these principles in practice. Reporting and evidence collection processes do not consistently encourage adaptive practice, reflecting their orientation towards accountability over learning. Context analysis is common during programme design, but used less on an ongoing basis. Popular tools –such as logical frameworks and theories of change - are often intepreted in linear ways, not as ‘living documents’ that react and change over time. Organisations need to meaningfully empower staff to work adaptively, including examining incentives and cultures that can make staff more comfortable with traditional results management. Even when senior leadership is supportive of adaptive ways of working, they can lack a clear understanding of the resourcing required and appropriate governance and management processes. Development organisations and their partners have attempted to address these challenges through the use of different monitoring and evaluation tools and methods, changes to reporting frameworks and templates, and initiatives to create positive incentives and motivate staff, leadership and partners at different levels.
Keywords: _tablet

Peer reviews - Guidance for facilitators and participants - Wadley, Ian (Centre for Humanitarian Dialogue, 2021)
Essential points for practitioners and donors • Mediation offers a cost-effective and proven method for resolving armed conflict. Between 1985 and 2015, 75 per cent of armed conflicts in the world were resolved through agreement rather than by force. In most cases these processes will have involved third party facilitation or support. • Professional mediators understand the high stakes involved in their work to prevent, mitigate and resolve armed conflict. In addition, they and their financial supporters are increasingly required to demonstrate ‘value-for-money’ to ensure continued funding. • However, traditional monitoring and evaluation (M&E) methods are not well suited to this task, typically imposing artificially linear project models on a dynamic conflict situation, as well as compliance reporting that moves attention away from real value. • Traditional M&E methods tend to focus on documenting the past and generating vast amounts of data, rather than enabling timely adaptation of the project in the present. • Traditional M&E approaches rely heavily on external evaluation consultants. Even in the best of cases this may interfere with the mediation process and impose a heavy time burden on the project team, leading to low acceptance of traditional M&E approaches by mediation practitioners. • In contrast, an ideal M&E approach for mediation should deliver useful insights in even the most dynamic and sensitive mediation environments, impose a light reporting burden, and be readily accepted by mediation teams. It should protect discretion and trust, enable rapid adaptation, and also provide some assurance that donor funds are being well spent.

A Political Economy Analysis Framework for EdTech Evidence Uptake - Pellini, Arnaldo; Nicolai, Susan; Magee, Arran; Sharp, Sam; Wilson, Sam (The EdTech Hub, 2021)

Adaptive Management: Learning and Action Approaches to Implementing Norms-shifting Interventions - McLarnon, Courtney; Gayles, Jennifer; Deepan, Prabu (USAID / Passages, 2021)
What Passages has Learned about Adaptive Management: • Be reflective about information that is collected and create a culture of learning. • Be systematic about establishing monitoring and learning systems. • Be strategic about data sources and analysis, prioritizing areas for learning and addressing issues raised. • Be inclusive about information collection: who is collecting what, how, and how is it being used.

USAID ADS 201 - Operational Policy for the Program Cycle (Update 01/23/2021) - USAID (USAID, 2021)
The Program Cycle is USAID’s operational model for planning, delivering, assessing, and adapting development programming in a given region or country to advance U.S. foreign policy. It encompasses guidance and procedures for: 1) Making strategic decisions at the regional or country level about programmatic areas of focus and associated resources; 2) Designing projects and supportive activities to implement strategic plans; and 3) Learning from performance monitoring, evaluations, and other relevant sources of information to make course corrections as needed and inform future programming.

Engage. Empower. Enact. - Citizen Engagement & Democratic Innovation Programme White Paper - Doyle, Linda; Smith, Bethan (Cognitive Edge, 2021)
The Cynefin Centre’s Citizen Engagement & Democratic Innovation programme provides tools for collective sense-making in the areas of community development and youth work; civic engagement and democratic innovation; collaborative service/policy design and evaluation; housing/tenant engagement; futures and planning; shared learning and peer to peer knowledge exchange.

Shifting the Evaluation Paradigm: The Equitable Evaluation Framework - EEI (Equitable Evaluation Inititiative, 2021)
This publication provides an overview of the impetus for the Equitable Evaluation Framework™ (EEF) and attempts to document early moments and first steps of engagement with U.S. philanthropic institutions — most often their research, evaluation and learning staff — whom we refer to as foundation partners throughout this publication. The themes shared in this publication surfaced through conversations with a group of foundation staff who have been part of the Equitable Evaluation Project, now referred to as the Equitable Evaluation Initiative (EEI), since 2017 as advisors, investment partners1 and/or practice partners. These are not case studies but insights and peeks behind the curtains of six foundation practice partners. It is our hope that, in reading their experiences, you will find something that resonates, be it a point of view, a mindset or a similar opportunity in your place of work. But as your colleagues share here, once you understand and see how narrowly these constructs have been defined, it cannot be unseen. The only option is to be, think and do evaluative work differently. In keeping with the EEI intention to shift the evaluation paradigm, this publication does not read in a linear fashion, nor does it follow “traditional” evaluation report form and flow. If we are to get someplace new and different, we must embrace new ways of reflecting what we are hearing, gleaning and sharing from those who are in the work. As you read along, notice commonalities in experience, perspective and positioning among the stories, and also consider the ways in which this conversation pushes your own thinking about how evaluative practice can serve its highest aim in U.S. philanthropy. They are thinking about how this influence manifests in all foundation operations and are wondering how others began to think about this reality in the context of evaluation practice. We went back and forth on naming names. We recognize the tendency to be either too specific or too general when telling stories about or from within institutional philanthropy. Because this work is both individual and organizational, we opted to keep names. We think it is important to understand who, with all their identities, took the early steps and were willing to share their experiences. Last, this is not a primer. While we reference concepts essential to the EEF, one must do the work to truly understand them. One might ask, what does it mean to “do the work”? There is a strong tendency for folks to be more aware of and reactive to how we think we should be engaging in evaluative practice based on how others are doing the work or how the work of evaluation has always been done. It is highly intellectualized and decontextualized. In the practice of the EEF, the work begins by acknowledging our own complicities in and contributions to the current paradigm, and accepting that we are the tool for change. The “work” begins with you, right now, where you are. Shifting the evaluation paradigm takes intentionality, discipline and practice. With that in mind, approach this publication with a learning mindset and open heart, and identify the ideas, perspectives and opportunities that resonate and compel you to pursue more practices aligned with the EEF at your organization.

Don’t Build It: A Guide For Practitioners In Civic Tech / Tech For Development - Jordan, Luke (Grassroot and MIT Governance Lab, 2021)
If you just remember these... If you can avoid building it, don’t build it; if you have to build it, hire a CTO, ship early, and mature long; and no matter what, draw on a trusted crew, build lean and fast, and get close to and build with your users as soon as possible. — This guide aims to help you avoid bad projects, structure the team right, ship and learn quicker, and mature longer. The guide starts with project selection, including why the best project to select is no project at all. It moves on to team structure, and the extreme importance of a full-time senior tech lead (or chief technology officer (CTO), understood as an excellent engineering manager). It then covers timelines, emphasizing shipping early but having enormous patience getting to maturity, above all in finding product-use-fit, and avoiding vanity metrics. The guide then goes into some detail on hiring, covering the CTO role, senior contractors, designers and young engineers. The longest section, by some distance, is that on hiring. Hiring is the one thing considered critical in every piece of the lore, by founders and investors and managers alike, across all sectors. It is also the field in which I think I got it mostly right, and for reasons I can explain in ways that I believe will be helpful.

Small Foundation Network Partner Evaluation Toolkit Network Evaluation for Network Coordinators Frequently Asked Questions - Small Foundation (Small Foundation, 2021)
This Frequently Asked Questions (FAQ) document is written for Small Foundation network partners. It accompanies Network Partner Evaluation Toolkit, which outlines four evaluation tools that Small Foundation is encouraging, and supporting, its network partners to use to improve their operations and, ultimately, increase their impact. This FAQ document answers typical questions about these evaluation tools and provides additional detailed recommendations for how network coordinators can effectively implement them. This document has five sections: 1) A brief introduction to network theory and the value of evaluation for network coordinators. 2) An overview of the four evaluation tools and answers general questions about incorporating evaluation into network coordination duties. 3) Recommendations regarding the Network Coordinator Administrative Information (tool #1). 4) Information about Social Network Analysis (tool #2). 5) Recommendations regarding the Network Participant Survey (tool #3). 6) Information about the Collaborative Activity Dashboard (tool #4).

Managing complexity (and chaos) in times of crisis. A field guide for decision makers inspired by the Cynefin framework - Snowden, Dave; Rancati, Alessandro (Publications Office of the European Union, 2021)
This field guide helps to navigate crises using the Cynefin framework as a compass. It proposes a four-stage approach through which we can: - assess the type of crisis and initiate a response; - adapt to the new pace and start building sensing networks to inform decisions; - repurpose existing structures and working methods to generate radical innovation; - transcend the crisis, formalise lessons learnt and increase resilience. The guide stresses the importance of setting and managing boundaries, building informal structures, keeping options open, distributing engagement and keeping an ongoing assessment of the evolving landscape. Action items, real life examples and demonstrations complement the references to the developing theoretical framework.

Uniting Against Corruption - A Playbook on Anti-Corruption Collective Action - UN Global Compact (UN Global Compact, 2021)
The 2021 guide provides an easy-to-follow six-step approach on how to develop, implement, and sustain a CA, with respect to the reader’s local corruption landscape and potential stakeholders. The adaptive framework proposed can be used to address corruption challenges, mitigate possible business risks, and achieve optimal results.

What have we learned about learning? Unpacking the relationship between knowledge and organisational change in development agencies - Yanguas, Pablo (German Development Institute / Deutsches Institut für Entwicklungspolitik (DIE), 2021)
Development cooperation has spent decades wrangling over the merits, evidence, and implications of what we may term “the learning hypothesis”: the idea that increased knowledge by development organisations must logically lead to increased effectiveness in the performance of their development activities. Organisations of all stripes have built research and monitoring and evaluation (M&E) departments, adopted a multitude of knowledge management systems and tools, and tinkered with different ways to structure their organograms to stimulate knowledge sharing and learning. The topic of organisational learning is particularly significant as the global development community grapples with increasingly complex problems and the aspiration of evidence-based policymaking. This paper presents an analytical framework for interrogating “the learning hypothesis”, breaking it down into causal steps: knowledge causes learning, learning causes organisational change, change causes effectiveness. The framework focuses on the first two sub-hypotheses, mapping out the conceptual space around them by outlining potential relationships between different types of knowledge – tacit and explicit, internal and external – and between different types of learning – operational and strategic. This map provides a foundation for three key research questions: What impact has the rising knowledge agenda had on development organisations? Which factors appear to enable or inhibit organisational learning? What is the relationship between operational and strategic learning and organisational change? A review of available evaluations and studies, including two cases from former UK Department for International Development (DFID) and the World Bank, reveals that there is insufficient evidence to support the causal claim that knowledge leads to learning and thereby to organisational change in development agencies. Sources point to tacit learning prevailing while explicit knowledge management systems flounder, and external advocacy agendas appear more compelling than internal research and evaluation products. It is not entirely clear how, or indeed, whether operational and strategic learning intersect, with delivery-level lessons hardly aggregating into structural or policy shifts. Organisational change – even that aimed at enhancing learning – is rarely based on lessons learned from practice. More research is necessary to fully unpack the learning hypothesis, but what limited evidence is available disproves rather than confirms its central claim. This has significant implications for the future of learning in development agencies as advocated by thought leaders, researchers, and reformers. In particular, the latter should consider an evidence-based reassessment of the function and value-for-money of research and M&E in development practice, and a more critical examination of the politics of external advocacy efforts around innovative aid approaches like thinking and working politically, adaptive management, or results-based management.
Version Number: 1.0

Instituciones que aprenden - Informe para la XXVII Cumbre Iberoamericana de Jefes y Jefas de Estado y de Gobierno - Oliván Cortés, Raúl (XXVII Cumbre Iberoamericana, 2020)
La pandemia de la Covid-19 ha brotado súbitamente en un momento de transición para las instituciones y organizaciones sociales de medio mundo. Cuando aún no se había superado la crisis de confianza de la última década entre los ciudadanos y los gobiernos, en un momento de gran impulso de las iniciativas de gobierno abierto, planes de innovación y transformación digital, para hacer más democráticas y eficientes las administraciones a través de programas políticos encaminados a configurar un nuevo contrato social, verde y digital en el marco de la Agenda 2030; el coronavirus ha provocado un shock histórico alterando el decurso del siglo XXI y exigiendo una aceleración de todos los procesos en marcha. La complejidad y dimensión de la pandemia ha puesto de manifiesto la necesidad de arquitecturas institucionales más flexibles, ágiles y resilientes, que sean capaces de incorporar toda la energía civil para aprender de su talento y creatividad, dándole mayor protagonismo a la ciudadanía (activistas, emprendedores, tejido asociativo, academia, makers…) no solo en la toma de decisiones sino también en el diseño e implementación de las estrategias. Conforme se centrifugaba a millones de empleados a teletrabajar desde sus casas, se hacía más poderosa la idea de pensar las organizaciones como flujos más allá de los organigramas estáticos de lugares y personas. Transformar las jerarquías en redes, concebirlas como cuerpos sociales dinámicos, no solo amplía su radio de acción y su conectividad exterior, también reactiva sus fortalezas internas, aflora los liderazgos ocultos, multiplica el valor social producido y maximiza el uso eficiente de los recursos en una época de limitaciones. Los laboratorios de innovación pública, social o ciudadana, o laboratorios de gobierno, junto a otras formas de innovación abierta y diseño social, se reivindican como proyectos inspiradores de un cambio de paradigma: de las instituciones que ordenan a las instituciones que aprenden. Pensar las organizaciones bajo el prisma de la ciencia de redes y la ética de los rizomas –nodos, enlaces, hubs, comunidades…– nos debería permitir una aproximación a la compleja y escurridiza tarea de configurar los ecosistemas de innovación y creatividad en el ámbito de lo público y lo social. Este informe propone un modelo denominado Hexágono de la Innovación Pública (HIP) que promueve un cambio sistémico a través de seis vectores (OPEN_ abierto, TRANS_ transversal, FAST_ ágil, PROTO_ modelado, CO_ colaborativo y TEC_ tecnológico) basados en las propiedades de las redes y en el análisis de 105 metodologías que usan las agencias más innovadoras del mundo. Se incluyen una herramienta de autodiagnóstico y el HIP-SIM, una primera aproximación a un software abierto para visualizar, modelar y simular la creación de ecosistemas innovadores con el que queremos propiciar una comunidad y un debate internacional.

Instituições que aprendem - Relatório para a XXVII Cimeira Ibero-Americana de Chefes de Estado e de Governo - Oliván Cortés, Raúl (XXVII Cimeira Ibero-Americana, 2020)
A pandemia da Covid-19 surgiu de súbito num momento de transição para as instituições e organizações sociais de meio mundo. Antes ainda de se ter ultrapassado a crise de confiança da última década entre os cidadãos e os governos, num momento de grande impulso das iniciativas de governo aberto e de planos de inovação e transformação digital para tornar mais democráticas e eficientes as administrações através de programas políticos encaminhados para configurar um novo contrato social, verde e digital no quadro da Agenda 2030, o coronavírus provocou um choque histórico alterando o curso do século XXI e exigindo uma aceleração de todos os processos em andamento. A complexidade e dimensão da pandemia revelou a necessidade de arquiteturas institucionais mais flexíveis, ágeis e resilientes, capazes de incorporar toda a energia civil para aprenderem com o seu talento e criatividade, dando maior protagonismo à cidadania (ativistas, empreendedores, tecido associativo, academia, makers…) não só na tomada de decisões mas também na configuração e implementação das estratégias. À medida que se afastava do centro a milhões de empregados para teletrabalhar a partir de casa, tornou-se mais poderosa a ideia de pensar nas organizações como fluxos que excedem os organigramas estáticos de lugares e pessoas. Transformar as hierarquias em redes, concebê-las como corpos sociais dinâmicos, não só alarga o seu raio de ação e conectividade externa, mas também reativa as suas forças internas, aflora as lideranças ocultas, multiplica o valor social produzido e maximiza o uso eficiente dos recursos numa época de limitações. Os laboratórios de inovação pública, social ou cidadã, ou laboratórios de governo, a par de outras formas de inovação aberta e configuração social, reivindicam-se como projetos inspiradores de uma mudança de paradigma: de instituições que ordenam para instituições que aprendem. Pensar nas organizações sob a ótica da ciência das redes e da ética dos rizomas – nós, ligações, hubs, comunidades… – deverá permitir-nos uma aproximação à complexa e subtil tarefa de configurar os ecossistemas de inovação e criatividade no âmbito público e social. Este relatório propõe um modelo denominado Hexágono da Inovação Pública (HIP) que promove uma transformação sistémica através de seis vetores (OPEN_ aberto, TRANS_ transversal, FAST_ rápido, PROTO_ modelado, CO_ colaborativo e TEC_ tecnológico) baseados nas propriedades das redes e na análise de 105 metodologias usadas pelas agências mais inovadoras do mundo. Incluem-se uma ferramenta de autodiagnóstico e o HIP-SIM, uma primeira aproximação a um software aberto para visualizar, modelar e simular a criação de ecossistemas inovadores com o qual queremos favorecer a comunidade e o debate internacional.

Diseño transformacional de proyectos - Kehrer, Daniel (GIZ, 2020)
There are many definitions of the term ‘transformation’ or ‘transformational change’. The first section of the report develops a basic understanding of transformations or transitions (used synonymously) viewed from various perspectives. In this, transformations are defined as processes that use disruptive innovations to change systems into fundamentally new systems that subsequently form the new mainstream. Section two describes existing approaches to environmental and climate finance in international cooperation and discusses them in light of the proposed definition. All of the approaches have the potential to be further refined and in that process often to increase the precision of what is understood by each type of transformation. The definitions already state which criteria and indicators are referred to and how relevant they are. There is wide diversity in the type of criteria and indicators used by the various organisations, and how they are classified, but certain common features can be identified and with the aid of the literature on transformations they can be combined to form a comprehensive framework. With this in mind, the derivation of quality criteria for transformative interventions is explained in section 3.1. Transformational change at one and the same time calls for big decisions and innumerable projects in a particular field of transformation; the projects cannot be planned on the drawing board but still should be coordinated with each other. Section 3.2 offers guidance on this. Section 3.3 argues in favour of focusing more closely on the ‘process promise’ and employing a more iterative and more adaptable commissioning procedure. Finally, section 3.4 introduces two types of indicators under the various criteria: design indicators, which measure the quality of interventions that are aimed at influencing transformations (process orientation), and outcome indicators, which measure the process and/or progress of a transformation itself.

Transformative project design - Kehrer, Daniel (GIZ, 2020)
There are many definitions of the term ‘transformation’ or ‘transformational change’. The first section of the report develops a basic understanding of transformations or transitions (used synonymously) viewed from various perspectives. In this, transformations are defined as processes that use disruptive innovations to change systems into fundamentally new systems that subsequently form the new mainstream. Section two describes existing approaches to environmental and climate finance in international cooperation and discusses them in light of the proposed definition. All of the approaches have the potential to be further refined and in that process often to increase the precision of what is understood by each type of transformation. The definitions already state which criteria and indicators are referred to and how relevant they are. There is wide diversity in the type of criteria and indicators used by the various organisations, and how they are classified, but certain common features can be identified and with the aid of the literature on transformations they can be combined to form a comprehensive framework. With this in mind, the derivation of quality criteria for transformative interventions is explained in section 3.1. Transformational change at one and the same time calls for big decisions and innumerable projects in a particular field of transformation; the projects cannot be planned on the drawing board but still should be coordinated with each other. Section 3.2 offers guidance on this. Section 3.3 argues in favour of focusing more closely on the ‘process promise’ and employing a more iterative and more adaptable commissioning procedure. Finally, section 3.4 introduces two types of indicators under the various criteria: design indicators, which measure the quality of interventions that are aimed at influencing transformations (process orientation), and outcome indicators, which measure the process and/or progress of a transformation itself.

Guidelines for Country Strategic Frameworks Programmes and Projects - Ministry of Foreign Affairs of Denmark (Government of Denmark, 2020)

Linking constituent engagement and adaptive management - Buell, Stephanie; Campbell, Megan; Pett, Jamie (Overseas Development Institute, 2020)
Constituent engagement is the two-way process of involving constituents in the design, delivery, monitoring and evaluation of programmes. Constituent engagement and adaptive management together can be a powerful combination; high-quality constituent engagement can reinforce effective adaptive management, and vice versa. By highlighting stories from leading practitioners and their organisations, this paper explores how programmes ensure that constituent engagement informs meaningful adaptation. Key messages Constituent engagement and adaptive management are both important tools for implementing responsive and effective development programmes. Together, they can be a powerful combination: input from constituent engagement can be a key source of information and evidence that meaningfully informs programme design and adaptation, and closing the feedback loop in this way increases the quality of future engagement. Both adaptive management and community engagement principles recognise that, for a programme to be effective, it must be responsive to the people meant to ultimately benefit from it. Beyond providing a key source of information for potential programme adaptations, constituent engagement efforts also help build trust with stakeholders, align expectations and promote accountability. This paper explores five key elements for ensuring that constituent engagement and adaptive management are effectively linked within a programme: strong internal systems and external channels; skilled staff that value engagement and adaptation; decision-maker champions; clear points for reflection and action; and a meaningful role for constituents.

Adaptive management: A practical guide to mitigating uncertainty and advancing evidence-based programming - Byom, K.; Ingram, M.; Oakley, A.; Serpe, L. (PACT, 2020)
Pact’s Adaptive Management Guide provides practical guidance to development practitioners globally on the mindsets, behaviors, resources, and processes that underpin an effective adaptive management system. It presents an approach to managing adaptively that is rooted in complexity analysis and program theory. It draws on Pact’s global experiences and work on topics as diverse as health, livelihoods, markets, governance, capacity development, women and youth, and more. This document begins with an introduction to adaptive management, then walks through successive steps to determine how much adaptation a project requires and how to design an appropriate system. The second half of this guidebook contains a toolkit of examples and templates that projects can tailor to their needs.

Using middle-level theory to improve programme and evaluation design - Cartwright, N (CEDIL, 2020)
What can middle-level theory do? Middle-level theory (MLT) has several uses in development planning and evaluation. It helps predict whether a programme can be expected to work in a new setting. It offers insights into what design features are needed for success. It provides invaluable information for monitoring to see if the programme is on track and to fix problems that arise. It reveals the causal processes and related assumptions to be tested in an evaluation and helps identify evaluation questions. It helps in interpreting evaluation findings, assessing their relevance and locating a description of them that is useful for programme design and evaluation in other settings

Inspiring Radically Better Futures - Evidence and Hope for Impact at Scale in a Time of Crisis - Guijt, Irene; Mayne, Ruth (Oxfam, 2020)
The world faces converging crises of health, climate, gender and racial injustice and extreme economic inequality. The calls are mounting to ‘build back better’ to create more inclusive, caring and environmentally sustainable futures. But what evidence exists that this is possible? The Inspiring Better Futures case study series investigates whether radical change at scale is possible and how it was achieved. This paper synthesises 18 cases which show that people are already successfully building better futures, benefitting millions of people, even against the odds in some of the world’s toughest contexts in lower-income countries. Together they offer hope that transformative change and radically better futures after the pandemic are within reach.

Building Better Systems - An introduction to System Innovation - Winhall, Jennie; Leadbeater, Charles (The Rockwool Foundation, 2020)
This paper lays out a series of steps people can take to create the new systems we need to meet shared, public challenges. Systems are ubiquitous and powerful. We rely on them to support our daily lives: every time we turn on a tap, flick a switch for electricity, drop our child at school, jump on a bus or visit a doctor we rely on a wider system. There is a widespread sense, among decision makers and citizens that in the coming decades society will need not just new products, software and services, but new systems for living sustainably in a socially inclusive society. The need for better, different systems will be heightened by the impact and lessons of the Covid-19 pandemic. Systems are productive precisely because they are more than standalone products. A system pulls together all the different ingredients needed to meet a need or to produce an outcome: the shipping container is a product, containerisation is a system; a contactless payment card is a product which only works as part of a payments system; an operation in a hospital can only take place because it is part of a wider health system. To understand how a system works it has to be seen as a whole, from the macro policy frameworks of social security systems right down to how a citizen goes about finding a job. Many of the systems we rely on for care and work, energy and transport, education and health are under pressure to change. Society faces both deeply entrenched and growing challenges that are outpacing the systems we have. We also have opportunities to create new, alternative systems as new knowledge, values and technologies emerge, from artificial intelligence and bitcoin, to circular and renewable systems of production. Rising to the challenge of fixing an existing system and exploring the possibility of creating a new system are different undertakings. The first is about optimising what exists, the second is about creating something different and better. We want this project to yield practical insights for those who want to respond to the systemic challenges of today by stepping into the possibilities of the future. Acting to change systems depends on new ways of seeing both challenge and opportunity: why systems come under strain and what unlocks the potential for alternatives. It depends on better understanding how new systems form, and what and who is part of initiating and driving the transition to them. In putting together this paper and the ones that will follow from it we want to clarify how to assess the need for, invest in and act on the process of deliberate system change.

Rapid Action Learning for Sanitation and Hygiene Programming - Chambers, Robert; Myers, Jamie; Vernon, Naomi (IDS, 2020)
There is a glaring gap and compelling need for approaches and methods that realign to new rigour through timeliness, cost-effectiveness, relevance and being actionable. Over the past few years, the Sanitation Learning Hub, in collaboration with the Government of India, Praxis, WSSCC and WaterAid India, have been developing Rapid Action Learning approaches. Multiple approaches have been trialled, with flexible formats, but the essential criteria is that learning is timely, relevant and actionable. These learning approaches are the focus of the latest edition of the Frontiers of Sanitation series. This Frontiers explains the advantages and disadvantages of the approaches trialled and sets out a challenge to those working in the water, sanitation and hygiene sector to: Reflect on what, for you, constitutes rigour. Adopt and adapt approaches to fit your context and needs. Develop your own approaches. Record your experiences and lessons learnt. Take the time to share your experiences with us

Participación Digital y mejora democrática - un camino sin regreso - Palumbo, Gabriel (Luminate, 2020)
Tal vez como nunca en la historia, la democracia está siendo desafiada. A los viejos problemas, muchos no resueltos, se les agregan otros, surgidos de la incorporación de tecnología, de la emergencia de nuevos actores sociales y de una conformación de la subjetividad política cada vez más compleja e impredecible. Esta investigación pretende reactualizar algunos de los debates clásicos sobre la democracia y, al mismo tiempo, sentar una base empírica que permita pensar de qué modo la tecnología puede ayudar a mejorar la calidad democrática. Según nuestro estudio, la participación digital está promoviendo modificaciones importantes en la relación entre la ciudadanía y los decisores políticos, ampliando los espacios de proximidad. Esto sucede, fundamentalmente, porque baja las barreras de acceso y de incentivos haciendo más fácil, más efectiva y más mensurable la participación ciudadana. La participación digital aumenta el períodos electorales y fomenta el asociacionismo cívico como posibilidad cierta y concreta de presionar sobre los decisores políticos. Lo digital habilita, además, participación offline y genera espacios de pedagogía cívica. La capacidad de implantación de temas en el debate público que se puede generar a través de ella mejora la conversación pública, impactando a su vez la calidad de la democracia. Desde Luminate entendemos que favorecer los espacios de rendición de cuentas desde la ciudadanía hacia el poder político es una apuesta al robustecimiento de la experiencia democrática y que las formas digitales están abriendo posibilidades que otras opciones no pudieron lograr. Esperamos que esta investigación sea un aporte para poder seguir reflexionando y avanzando en esta dirección.

Navigating adaptive approaches for development programmes - Pett, Jamie (ODI, 2020)
This working paper compares six of the most prominent adaptive approaches to emerge over the past two decades. Three come from the world of innovation, largely in the private sector (agile, lean startup and human-centred design), and three from the global development sector (thinking and working politically, forms of adaptive management and problem-driven iterative adaptation). While all of these approaches are valuable when used in the right context, practitioners may be perplexed by the multiplicity of methods and jargon. This paper aims to address some of this confusion by mapping where these approaches have come from and showing how they can be applied across the adaptive programme cycle. Armed with this knowledge, practitioners might experiment with different combinations and sequences of adaptive approaches according to the kind of problem and context faced. In turn, this may help us move beyond a siloed view of approaches linked to innovation, adaptive management or more politically smart ways of working. Key messages: • Adaptive approaches have emerged in several sectors, including software development, product and service design, technology startups and international development. • Adaptive approaches can help practitioners counteract misplaced certainty. By talking to potential users, understanding institutions, interests and ideas and investigating the root causes of a problem, practitioners applying these approaches can illuminate the underlying nature of the problem and context. • Rather than building a whole solution straight away, these approaches commonly encourage practitioners to start small and use structured cycles of testing and learning. There is scope to further consider how different approaches can be better brought together and combined. • Adaptive approaches in development provide a wider range of options for what to create and facilitate – not only products or services, but also forms of collective action. There are also alternative ways to think about scale – considering how others might take up an idea and looking for leverage, rather than quantity.

SystemCraft - a primer: How to Tackle our Toughest Problems - Simpson, Kate; Randall, Ian (Wasafiri, 2020)
Systemcraft is our applied framework to help leaders and organisations get started and keep going when faced with complex problems. It is built on our practical experience. It draws on a broad body of research, action and theory from the worlds of complexity thinking, systems theory, adaptive management, leadership development, social movements, development theory and beyond. Systemcraft has been designed to make systems thinking something any leader can apply when they find themselves faced with a complex problem and asking, ‘So what do I do next?’

Big Data to Data Science - Moving from “What” to “How” in the MERL Tech Space - Bertermann, Kecia; Robinson, Alexandra; Bamberger, Michael; Higdon, Grace Lyn; Raftre, Linda (MERL Tech, 2020)
This paper probes trends in the use of big data by a community of early adopters working in monitoring, evaluation, research, and learning (MERL) in the development and humanitarian sectors. Qualitative analysis was conducted on data from MERL Tech conference records and key informant interviews. Findings indicate that MERL practitioners are in a fragmented, experimental phase, with use and application of big data varying widely, accompanied by shifting terminologies. We take an in-depth look at barriers to and enablers of use of big data within MERL, as well as benefits and drawbacks. Concerns about bias, privacy, and the potential for big data to magnify existing inequalities arose frequently. The research surfaced a need for more systematic and broader sharing of big data use cases and case studies in the development sector.

Emerging Technologies and Approaches in Monitoring, Evaluation, Research, and Learning for International Development Programs - Bruce, Kerry; Gandhi, Valentine J; Vandelanotte, Joris (MERL Tech, 2020)
Emerging technology is making monitoring, evaluation, research and learning (MERL) more precise and enriching data. However, this evolution is so rapid that it can be difficult to stay informed about the field overall. This paper presents examples of emerging technology that are most often used in MERL for development programs, describes the pros and cons of their use, and discusses technology and ethics concerns that practitioners should keep in mind. The paper covers new types of data sources (application data, sensor data, and drones), new ways to store data (distributed ledger technology and the cloud), and new ways to analyze data (text analytics and supervised and unsupervised learning).

Evaluating CLARISSA: Innovation Driven by a Participatory Learning Agenda - Apgar, Marina; Snijder, Mieke; Kakri, Shanta; Macleod, Shona; Paul, Sukanta; Sambo, Anna; Ton, Giel (Institute of Development Studies, 2020)
Children end up in child labour as a result of many, often unknown or hidden, interactions between multiple actors and multiple factors within households, communities, and labour systems, leading to unpredictable outcomes for children and other sector stakeholders and sometimes resulting in the worst forms of child labour (WFCL). It is a complex problem, and interventions aimed at tackling it are also, inevitably, complex and challenging. The way they influence change is non-linear, causality is uncertain, and unintended consequences may result. Programmes such as the Child Labour: Action-Research-Innovation in South and South-Eastern Asia (CLARISSA) that are engaging with such intractable challenges and aim to reach the most left behind (children in WFCL) are operating in conditions of complexity. This complexity poses significant challenges to the way programmes are designed, planned, implemented, and evaluated, and requires a move away from linear and predetermined models. In this Working Paper, we share our experience and early learning about how to design and implement monitoring, evaluation and learning that intentionally embraces the challenge of complexity.
Accepted: 2020-06-25T14:55:41Z

Not Always Movements. Multiple Approaches to Advance Large-Scale Social Change - Stachowiak, Sarah; Gienapp, Anne; Kalra, Nikki (ORS Impact, 2020)

What We Know About Traditional MERL Tech - Insights from a Scoping Review - Tilton, Zach; Harnar, Michael; Raftree, Linda; Perrin, Paul; Bruening, Gretchen; Banerji, Soham; Gordley, John; Foster, Hanna; Behr, Michele (Western Michigan University, 2020)
This paper explores the peer-reviewed evidence base of “traditional” technology-enabled monitoring, evaluation, research, and learning (MERL Tech) in international development assistance from 2015 to 2019. The authors conducted a scoping review that searched seven databases, screened 3,054 reference titles and abstracts, coded 886 abstracts, and extracted and analyzed conclusions and recommendations from the full texts of 256 studies. The findings reveal the most frequently reported technologies, MERL activities, and the sub-sectors, and the geographies where those tech-enabled activities occur. Gaps in the evidence for specific technologies, MERL activities, and sectors are mapped. The data reveals which technologies are trusted more than others and reported barriers to effective MERL Tech implementation and areas that researchers suggest for further investigation. The results suggest that the evidence from peer-reviewed studies is not proportional to estimated MERL Tech activity, significant publication bias exists, and further knowledge synthesis of unindexed grey literature is needed to provide a more comprehensive and possibly accurate description of MERL Tech practice.

Uncertainty and COVID-19: A turning point for Monitoring Evaluation, Research and Learning? - A discussion note for aid actors, policymakers and practitioners - Tyrrel, Lavinia; Roche, Chris; Jackson, Elisabeth (Abt Associates, 2020)
The COVID-19 pandemic has significantly shifted the context in which aid and development is being delivered. The global scale of the pandemic and the speed at which it is spreading mean that the ‘normal’ economic, ideological and organisational influences which shape (if not determine) aid delivery are in flux. This means that – for a relatively short-period – there is scope for aid actors to work collectively to embed more locally-led, politically-informed and adaptive forms of MERL in aid and development practice. These forms of Monitoring Evaluation Research and Learning (MERL) are not only well-suited to the current global pandemic. They also offer ways for aid program decision makers and practitioners to make sense of the complex and uncertain contexts in which much development work takes place. Applying locally-led, politically-informed and adaptive forms of MERL in the COVID-19 context and beyond requires a shift in mindset and approaches. Situations of complexity, in which it is difficult to predict the relationships between cause and effect, do not lend themselves to linear approaches and fixed indicators. Instead, they require ‘navigation by judgement’, ongoing learning and adaptation and greater privileging of local knowledge, and of the perspectives of those who are often excluded. Rather than being focused on upwards accountability, simple numbers and good news stories, the core function of MERL in this context is to support a better understanding – in real-time – of the changing operating context, to generate learning about the immediate impact of policy and program responses and their longer-term effects, and to inform decision making by front line staff. Whether the opportunities afforded by this ‘critical juncture’ are realised will depend on the degree to which those in the aid and development sector use this opportunity to promote a shift in the deep incentive structures within which development agencies are embedded. On the one hand, the pandemic underscores the limits of the linear understandings of change which underpin many orthodox approaches to planning, design and associated MERL. On the other hand, there is a vested interest in the status quo amongst many organisations, consultants, researchers and MERL practitioners. This is because approaches which promote locally-led development inevitably require those in power to relinquish control. While a range of factors make this shift difficult, there is more scope to change internal ways of working in development agencies than is commonly acknowledged. There is no time like the present to advocate for a ‘new normal’ for MERL.

LEARN End of Contract Report - Learning Lab (USAID, 2020)
In September of 2014, USAID’s Office of Learning, Evaluation & Research (LER) awarded the Learning and Knowledge Management (LEARN) contract to Dexis Consulting Group and subcontractor RTI International.1 This document—the End of Contract Report—captures five and half years of results and reflections for our stakeholders. Our intention is to share the good and the bad, and while this report would not be considered a “tell all,” we think we have a story worth sharing, particularly to USAID CORs and AORs, activity managers, and other implementers of institutional support contracts.2 LEARN’s primary purpose was to support organizational change at USAID. More specifically, the contract was focused on helping USAID staff integrate collaborating, learning, and adapting (CLA) approaches into the design, implementation, monitoring, and evaluation of programs (what is known at USAID as the Program Cycle). It was clear that most USAID staff, whether they realized it or not, were already integrating CLA into their work to some extent. The focus of our efforts, therefore, was to make those practices more systematic, intentional, resourced, and ultimately more widespread throughout the Agency, which would have a ripple effect on implementing partners and even other stakeholders, such as host country governments. This was based on the theory—later borne out by evidence—that by becoming a better learning organization, USAID could be a more effective development organization. And that theory brought the USAID CLA team within USAID’s Bureau for Policy, Planning & Learning (PPL) and LEARN contractors together, driven by a shared purpose of improving how USAID does business. LEARN was designed based on this belief and, as you might expect from a learning-oriented contract, began with more questions than answers. Primary among those questions was: could an institutional support contract do more than carry out requested services—could it actually accelerate positive organizational change at USAID? And if so, how and under what conditions?

How Do Donor-led Empowerment and Accountability Activities Take Scale into Account? Evidence from DFID Programmes in Contexts of Fragility, Conflict and Violence - Anderson, Colin; Fox, Jonathan; Gaventa, John (IDS, 2020)
Development donors invest significantly in governance reform, including in contexts characterised by conflict and fragility. However, there is relatively little comparative study of their change strategies, and little understanding of what works and why. This paper explores the strategies of six recent DFID-funded programmes in Mozambique, Myanmar, and Pakistan with empowerment and accountability aims. Document review and field interviews are used to analyse the application of multi-scalar or multi-level change strategies, since such approaches are hypothesised to potentially generate more leverage for public accountability reforms. Analysis suggests that these strategies can strengthen citizen ability to navigate governance systems to resolve problems and claim accountability, and can bolster pro-accountability coalitions’ internal solidarity and external legitimacy. Multi-level strategies also appear associated with establishing more significant pressure for reform than exclusively local or national approaches. Yet conventional project reporting focuses on counting activities and outputs rather than analysing the dynamic, interactive processes at work in these strategies, and few evaluations are publicly accessible. To fully understand what kinds of action strengthen citizen demands for accountability requires a more transparent and rigorous approach to learning from donor-led governance interventions.

Monitoring for problem-solving, adaptive management, reporting and learning - Dominique Morel; Dzino-Silajdzic, Velida; Hagens, Clara (Catholic Relief Services, 2020)
Internal and external stakeholders have different information needs over a project’s life, for purposes that include adaptive management, accountability, compliance, reporting and learning. A project’s monitoring, evaluation, accountability and learning, or MEAL, system should provide the information needed by these stakeholders at the level of statistical reliability, detail and timing appropriate to inform data use. In emergency contexts where the situation is still fluid, ‘informal monitoring’ has proved helpful to staff’s ongoing assessment of the broader environment in order to identify changes in the situation, in other actors’ responses, and in priority unmet needs that would require corresponding changes in the response.2 The same distinction between informal monitoring of possible changes in the project’s operating context—whether identified as project assumptions and risk factors or not—and formal monitoring of the activities included in the response and project indicators, is relevant for development contexts too. Informal monitoring: Ongoing assessment of changes in operating context Formal monitoring: Tracking progress against project activities and indicators Within formal monitoring, it is useful to further differentiate between light monitoring and rigorous monitoring: - Light monitoring aims to provide timely feedback on new activities (or new locations or target groups) or aspects of the project’s theory of change (activity-to-output or output-to-IR change) logic that staff are less confident about, to check for early signs that progress is being made and that assumptions are holding true while there is still ample time to make adjustments if necessary.3 - Rigorous monitoring aims to collect representative data for evidence-based project management, reporting and learning, not just at midterm but throughout project implementation.

MERL Tech State of the Field - The evolution of MERL Tech - Raftree, Linda (MERL Tech, 2020)

Adaptive leadership in the coronavirus response - Ramalingam, Ben; Wild, Leni; Ferrari, Matt (ODI, 2020)
The coronavirus pandemic poses unprecedented challenges to science, policy and the interface between the two. How – and how quickly – policy-makers, practitioners and researchers react to this emerging and complex crisis is making a profound difference to people’s lives and livelihoods (WHO, 2020). But how can we ensure effective collective decision-making on the basis of emerging evidence, changing trends and shifting scientific understanding, all in the face of considerable uncertainty? Recent experience highlights the need for adaptive leadership in national and global responses to the outbreak. This briefing paper sets out key principles for what this might look like, and proposes a roadmap for policy-makers, practitioners and researchers to move towards such an approach as they tackle the unfolding crisis. Key messages Tackling the coronavirus outbreak requires adaptation at operational and leadership levels. Operationally, there is scope to strengthen evidence-based adaptive management practices, to adjust the mix and type of interventions being implemented and learn as we go so as to achieve shared goals. This requires adaptive leadership capacities, being open and transparent about learning, using collective decision-making processes and building trust with communities and individuals.

Adaptation in practice: lessons from teenage pregnancy programmes in Sierra Leone - Castillejo, Clare; Buell, Stephanie (ODI, 2020)
A discussion of initial learning emerging from the SLRC ’Adaptive approaches to reducing teenage pregnancy in Sierra Leone’ action research project.

Politically informed approaches to gender in fragile and conflict-affected settings - Castillejo, Clare; Domingo, Pilar; George, Rachel; O’Connell, Shannon (ODI, 2020)
This report summarises the discussions at a meeting held in September 2019 of a group of global development research and policy experts and practitioners, convened by ODI and the Westminster Foundation for Democracy, to share experiences and knowledge, reflect on what we already know about working politically on gender in fragile and conflict-affected settings (FCAS), and identify what further evidence would be useful. Key messages: - It is important to identify, document and learn from politically informed and adaptive ways of working in practice on gender in FCAS. - Different analytical and monitoring, evaluation and learning tools are required for politically informed programming on gender. These should be embedded within programme teams and processes, and be both gender-responsive and responsive to the political economy context. - There is a need for more politically smart use of quantitative and qualitative data in order to identify plausible entry points and ways of working on gender in FCAS. This should include increasing the capacity of programme staff to use data to inform, adapt and correct programmes. - Staff promoting politically informed work on gender equality are often isolated, with little opportunity to share experiences or learn from others. Platforms should therefore be created to share experience and knowledge, and to bring together donors and implementers involved in this work.

The Magenta Book - Central Government guidance on evaluation - HM Treasury (UK Government, 2020)
HM Treasury guidance on what to consider when designing an evaluation.

The Magenta Book - Suplementary guide: Handling Complexity in Policy Evaluation - HM Treasury (UK Government, 2020)
HM Treasury guidance on what to consider when designing an evaluation.

Human trafficking in South Asia: Assessing the effectiveness of interventions - Jensen, Charity; Oosterhoff, P.; Pocock, N. (FCDO, 2020)
This Rapid Evidence Assessment (REA) examines current evidence on the effectiveness of interventions to combat human trafficking in four South Asian countries (Bangladesh, India, Nepal, and Pakistan). This REA is being carried out as part of a wider assignment for the UK Department for International Development (DFID), with the overall objective of synthesising evidence on the effectiveness of interventions that tackle modern slavery in South Asia. Two REAs were conducted on different types of modern slavery, one on human trafficking for labour and sexual exploitation, and another on child labour (Idris et al., 2020). The research question for this REA is: ‘What has been the effect of interventions to combat and/or reduce sexual and labour exploitation in Bangladesh, Pakistan, India, and Nepal?’ Trafficking in persons is a form of ‘modern slavery’, which is an umbrella term for the variety of situations in which someone is forcibly controlled by an individual or group for the purpose of exploitation. The Global Slavery Index estimated that, on any given day in 2016, 40.3 million people were victims of modern slavery, including 24.9 million people in forced labour and 15.4 million people in forced marriage. Of the estimated almost 25 million people in modern slavery in Asia, 66% were exploited for labour (Global Slavery Index, 2018).

Mainsreaming gender in an adaptive, politically smart governance programme - Lessons from Institutions for Inclusive Development in Tanzania - Laws, Ed (ODI, 2020)
• This paper looks at the experience of gender mainstreaming in the Institutions for Inclusive Development (I4ID) programme – an adaptive, politically smart governance programme in Tanzania. • When development programmes try to engage with political stakeholders and align with the priorities of wider coalitions there is a danger that gender equality is de-prioritised. • It is important that formal political economy analysis, as well as other data collection, analysis and consultation exercises, are gender-sensitive. Teams should also look for ways to make gendered political and power analysis part of the everyday routine practice of staff. • Working politically and adaptively to advance gender objectives calls for staff with a specific skillset, as well as links to appropriate networks and political stakeholders. It also implies establishing checks and incentives to hold staff and partners accountable for gender objectives, and strong and consistent messaging from team leaders.

Rethinking value for money for adaptive, politically smart programmes - Lessons from Institutions for Inclusive Development in Tanzania - Laws, Ed (ODI, 2020)
- This short paper draws out lessons for working effectively with and through partners, based on the experience of the Institutions for Inclusive Development (I4ID) programme – an adaptive, politically smart governance programme in Tanzania. • Cultivating effective partnerships can be a key part of delivering locally legitimate projects that have the potential to create sustainable change. Adaptive and politically informed ways of working create specific opportunities and challenges for doing this well. • Flexible and adaptive programmes are deliberately designed to experiment and to make small investments in different areas, to see what will work. While this is often important for making headway on complex challenges, it can also leave partners exposed and can undermine trust. • It can also be challenging to balance the need to meet accountability commitments to donors while allowing local partners to take the lead in pursuing their own objectives. • Co-creating plans, priorities and activities with partners has the potential to resolve some of these tensions. But the time and patience required to do this successfully should not be underestimated, and can be difficult to maintain in the face of pressure from donors to see results within a confined timeframe.

Working effectively through partnerships - Lessons from Institutions for Inclusive Development in Tanzania - Laws, Ed (ODI, 2020)
• This paper looks critically at the approach to value for money (VfM) in the Institutions for Inclusive Development (I4ID) programme – an adaptive, politically smart governance programme in Tanzania. • Adaptive, politically smart programmes like I4ID aim to deliver VfM by learning about what will work in complex environments, and quickly incorporating those lessons into delivery. When functioning properly, they can rapidly wind down activities as new information emerges and divert funding to more effective alternatives. • This means that adaptive programmes will achieve their potential to deliver strong VfM when their processes are good – when appraisal of experimental efforts is timely, consistent, knowledgeable and politically astute. As these programmes mature in their implementation phase, VfM evaluation should be focused on checking for a culture of adaptation and learning supported by strong adaptive processes. • While economy is important for adaptive programmes, it is also important that keeping costs low does not deprive teams of the resources, staff, and management and administration time they need to gather information, experiment, learn and adapt.

Evidence base for Collaborating, Learning, and Adapting: summary of the literature review (2020 update) - Learning Lab (USAID, 2020)
The LEARN contract and the United States Agency for International Development/Bureau of Policy, Planning, and Learning (USAID/PPL) are managing an area of work known as the Evidence Base for Collaborating, Learning, and Adapting (EB4CLA). The purpose of this work is to answer the following key learning questions: ● Does an intentional, systematic, and resourced approach to collaborating, learning, and adapting (CLA) contribute to organizational effectiveness and/or development outcomes? ● If so, how? And under what conditions? ● How do we know? How do we measure any contribution that CLA makes to development results? As we began this work, we identified the need to conduct a literature review looking at these questions to understand what is known, what remains unknown, and how others have tried to answer these questions to date. We were primarily interested in answering these questions: ● What evidence is there, if any, that collaborating, learning, and/or adapting contributes to organizational effectiveness, development outcomes, or both? What are the strongest pieces of evidence? ● Does the literature identify any factors critical to CLA that are not currently included in the CLA framework? ● Who else is working on measuring the impact of collaborating, learning and adapting? ● What methods and measures did researchers use to study the effects of collaborating, learning, and adapting? ● Where are there gaps in the research relevant to collaborating, learning, and adapting? ● When taken together, what practical guidance does the evidence for collaborating, learning, and adapting offer to practitioners and policy makers to improve organizational effectiveness and development outcomes?

What difference does CLA make to development? Key findings from a recent literature review (2020 update) - Learning Lab (USAID, 2020)
USAID’s Bureau for Policy, Planning and Learning and its LEARN support contract are working to integrate systematic, intentional and resourced collaborating, learning and adapting (CLA) throughout program planning and implementation to achieve more effective development programs. As part of this effort, USAID is exploring several approaches to understand whether and how strategic collaboration, continuous learning and adaptive management make a difference to organizational effectiveness and development outcomes. To begin this work, we have undertaken a foundational literature review of academic and gray literature to answer our key learning questions: • Does an intentional, systematic and resourced approach to collaborating, learning and adapting contribute to organizational effectiveness? To development outcomes? • If so, how? And under what circumstances? • How do we measure the contribution? The 13 key findings, mapped to the CLA Framework below, are described in greater detail in the following pages.

Governance Diaries: An Approach to Governance Research from the Ground Up - Loureiro, Miguel; Joshi, Anuradha; Barnes, Katrina; Chaimite, Egídio (IDS, 2020)
Research on empowerment and accountability tends to focus on collective action and its potential for empowering citizens undertaking the action and on achieving state accountability. In fragile, conflict and violence-affected settings (FCVAS) collective action is rare and risky. So how do citizens, particularly the chronically poor and most marginalised, interact and make claims on the different public authorities that exist in these settings, and how do these interactions contribute to citizens’ sense of empowerment and accountability? Given the current agenda of ‘leave no one behind’, an understanding of how such populations interact with public authorities to meet their governance needs can help identify the constraints to achieving development for all in these challenging settings. We developed ‘governance diaries,’ a cross between a panel survey and multi-sited ethnographies, as an iterative approach to capture their experiences around governance issues over time. We explain here how this approach works, and the challenges and opportunities it offers for research.

Actually Navigating by Judgment: Towards a new paradigm of donor accountability where the current system doesn’t work - Honig, Dan (Center for Global Development, 2020)
This working paper explores how donors can move towards greater Navigation by Judgment, highlighting the actions people inside and outside aid agencies can work to make change—encouraging more Navigation by Judgment on the margin, starting today.

Managing Better: What All of Us Can Do to Encourage Aid Success - Honig, Dan (Center for Global Development, 2020)
Management by way of top-down controls and targets sometimes gets in the way of aid donors’ aims, undermining project success. These unhelpful controls often stem from a need to account for performance; legislatures or executive boards induce agencies to exercise tight process controls and orient projects towards what is measurable and reportable.

Transforming our work: Getting ready for transformational projects - Kehrer, Daniel; Flossmann-Kraus, Ursula; Ronco Alarcon, Sabrina Valeria; Albers, Vivien; Aschmann, Gwendolin (GIZ, 2020)

A New Policy Narrative for Pastoralism? Pastoralists as Reliability Professionals and Pastoralist Systems as Infrastructure - Roe, Emery (ESRC STEPS Centre, 2020)
This paper proposes that pastoralist systems are better treated, in aggregate, as a global critical infrastructure. The policy and management implications that follow are significant and differ importantly from current pastoralist policies and recommendations. A multi-typology framework is presented, identifying the conditions under which pastoralists can be considered real-time reliability professionals in systems with mandates preventing or otherwise avoiding key events from happening. The framework leads to a different policy-relevant counternarrative to pastoralism as understood today. Some features of the counternarrative are already known or have been researched. The paper’s aim is to provoke further work (including case research and interactions with decisionmakers) on how robust the counternarrative is as a policy narrative for recasting today’s pastoralist policy and management interventions.
Accepted: 2020-01-14T10:22:59Z

Embracing complexity - Towards a shared understanding of funding systems change - Ashoka (Ashoka, 2020)

Contribution Rubrics: A simple way to assess influence - Aston, Tom (Independent Consultant, 2020)
This document explores how organisations can measure the level of influence that they had over an outcome. It summarises influence in terms of the significance of the outcome, the level of contribution and the strength of evidence.

Adaptive Management in SDC: Challenges and Opportunities - Prieto Martin, Pedro; Apgar, Marina; Hernandez, Kevin (IDS, 2020)
Adaptive management (AM) is a programme management approach that helps international development organisations to become more learning-oriented and more effective in addressing complex development challenges. AM practices have been applied for decades within other sectors as varied as logistics, manufacturing, product design, military strategy, software development and lean enterprise. At its core, AM is not much more than common sense, as it essentially recognises that the solutions to complex and dynamic problems cannot be identified at the outset of a programme but need to emerge throughout the process of implementation as a result of systematic and intentional monitoring and learning. The generic AM process typically involves an iterative cycle of design, implementation, reflection and adaptation activities, supported both by system monitoring and stakeholder involvement to obtain a better understanding of the evolving system and improve how the intervention is managed. A favourable context for AM in development. During recent decades, the international development sector has aimed to increase its results and impact orientation. As a result, a growing number of development organisations and governments have become increasingly aware of the limitations of traditional ‘linear and prescriptive’ programming approaches. They are now recognising the need to handle complexity better, and have begun to adapt their policies and practices to facilitate adaptive approaches. The World Bank, for example, now acknowledges that aid agencies need to increase flexibility of implementation, tolerate greater risk and ambiguity, devolve power from aid providers to aid partners, and avoid simplistic linear schemes for measuring results. Multilateral and bilateral organisations such as the World Bank, the United Kingdom’s (UK) Department for International Development (DFID) and the United States Agency for International Development (USAID) are currently experimenting with adaptive approaches. A multitude of adaptive approaches and communities of practice have emerged that aim to improve the effectiveness of aid, including Collaborating, Learning, and Adapting, Thinking and Working Politically, Doing Development Differently, Market Systems Development, Conflict-Sensitive Programme Management, and Science of Delivery. Since generic AM approaches have existed for decades in other sectors, AM has the potential to act as a neutral ‘bridge language’ that facilitates exchange and learning among the different communities and donors. This report is the result of a learning partnership between the Swiss Agency for Development and Cooperation (SDC) and the Institute of Development Studies (IDS). It assesses the relevance of AM to SDC, how it relates to working practices across SDC, and the key challenges and opportunities for SDC. Its process of elaboration involved a literature review on AM, an exploration of AM approaches from several bilateral donors, a series of 6 interviews with SDC staff and partners working in different countries and thematic domains, and a learning workshop at SDC headquarters (HQ), where staff from several SDC divisions reflected on AM and on how to advance the organisation’s capacity for adaptive programming and learning.

Contribution analysis for adaptive management - Apgar, Marina; Hernandez, Kevin; Ton, Giel (IDS/GLAM, 2020)
This briefing note shares practical learning on the use of contribution analysis for adaptive management (CA4AM). It examines how the approach enables programmes to work with theories of change in a practical, reflexive way, and how, combined with assessing evidence of a programme’s contribution to change, its findings can inform programme adaptation. It also examines both how and to what extent CA enables AM through the experiences of four large complex programmes all working towards systems-level change and employing a structured process of reflection on theories of change. Key messages CA4AM can enable programmes to work with theories of change in a practical, reflexive way. It is particularly useful for programmes operating in conditions of complexity, when it is difficult to discern attribution and when systems-level change is the goal. A range of enabling factors help CA4AM to be used most effectively, including contractual flexibility; embedded monitoring, evaluation and learning (MEL); and supportive leadership.

Outcome mapping: learning brief - Buell, Stephanie; Malallah, Haneen; Mason, Paige (ODI/GLAM, 2020)
Adaptive programmes recognise that certain changes, particularly in behaviours, are complex, non-linear and difficult to measure. This briefing note explores the use of outcome mapping (OM) as a monitoring, evaluation and learning (MEL) approach to track behavioural change and inform adaptation for two programmes: the Pathways to Resilience in Semi-arid Economies (PRISE) research consortium and the Accountability in Tanzania programme (AcT I and AcT II). It discusses the implementation of OM, the ways in which it has enabled adaptation and enabling contexts in order to identify key considerations for MEL specialists and programme managers as they determine whether OM may be the right fit, and how best to use the approach. Key messages OM has a number of different benefits as a MEL approach, including unpacking different uses of information at different levels of programme implementation; helping to develop a common language around progress markers; and going beyond monitoring to inform adaptation throughout implementation. These benefits are important aspects of monitoring, evaluation and learning for adaptive management (MEL4AM), as they provide richer evidence for decision-making at a frequency that could mean real-time learning and change. OM works best when it is embedded throughout the organisation, and as part of programme and organisational culture, rather than tasked to a MEL unit or individual.

Using the Business Model Canvas for Social Enterprise Design - Burkett, Ingrid (The Yunus Centre, Griffith University, 2020)
What should come first? Your business plan or your business model? I strongly believe it’s option B — your business model. But so many social enterprises I’ve seen, get it backwards which means they struggle to articulate how they create, deliver and capture value. This is what first inspired me to create the Business Model Canvas for Social Enterprise Design in 2013. It outlined how I had used, tested and learnt about the Business Model Canvas, created by Alex Osterwalder and Yves Pigneur, adapted for designing and growing social enterprises. Since then I’ve applied the Canvas in a range of contexts and learnt more about some of the nuances of using it in relation to social and impact enterprises. Note, I now tend to use the nomenclature of ‘impact enterprises’ rather than just social enterprises, but for ease of continuation from the first edition I’ve stuck with the terminology. It should however be recognised that the ‘impact’ referenced in the document could refer to social, environmental, cultural or economic outcomes that impact people, communities or societies. How to use the Canvas The Business Model Canvas for Social Enterprise Design and accompanying booklet is a companion to your strategy and planning. It’s a tool for you to get clear on how you’re creating revenue, how you’re creating impact, for who, and how both sides interact. You can use it at any stage of an enterprise’s lifecycle — before or at start-up, for testing a ‘minimum viable product’ model, for growth and development, and when pivots are required to build viability or sustainability. Completing a business model is not a desk activity, it is an immersive, practical, collective and engaged exercise that needs to be repeated over time. I like to use large forms of the BMC, in group settings and I like to display the canvas following a workshop so that team members and stakeholders of the social enterprise can engage, understand and continue to co-design the business model. Impact enterprises are extraordinary entities. If we can design their business models to reflect the best blends of commerce and impact we have the opportunity to truly transform people, places and planetary futures. What better way is there to do business?

Learning From Failure 2020 - What CARE’s evaluations tell us about how to improve our work - CARE (CARE, 2020)
“Ever failed. No matter. Try again. Fail again. Fail better.” –Samuel Beckett Here’s my favorite part of that quote: the ultimate goal is not a lack of failure; it’s better failures. That’s good news for CARE, because we just published round two of our Learning From Failure initiative, and…I know this will surprise everyone…we haven’t stopped failures yet. We do have some hopeful signs that we’re failing better; or at least, that we’re improving on some concrete weaknesses we identified in the first round. It’s an interesting process to launch the second phase of learning from failure. The first round, we didn’t know what we were going to find. We spent as much time talking about how it was the first-ever report of its kind as we did about the actual failures. Our case study admitted, “It's still very early to see specific development impacts.” Round two isn’t quite the same. It’s not new anymore, so there’s less excitement at having invented something. We’re not discovering data and themes for the first time. In a lot of ways, the stakes are higher. Round two of learning from failure becomes an exercise in continuous performance improvement, rather than a journey of discovery. If we don’t see improvements, we don’t have the excuse that it’s too early to tell. It also takes a sustained commitment. Launching an exploratory exercise at a small scale is easy, especially when no one quite knows what the answers will be. Pulling together a few pieces of content over a few months is pretty straightforward. It takes some staying power—and real support from leadership—to keep up the work over time, especially in the middle of a pandemic. That’s even more true once we’ve seen one round of results and had a chance to understand the work that it takes to improve.

Listen carefully. Tread lightly. Adapt quickly. Approaching Adaptive Management: Examples from our Somalia Education Programming - CARE (CARE, 2020)
Adaptive management approaches potentially offer us opportunities to deliver high quality results in circumstances where change is complex, including in fragile, unstable or conflict affected places. However, building adaptive programming continues to be a challenge for the sector. For CARE, our Department for International Development -UK Aid funded Girls’ Education Challenge (GEC) programming has provided useful learning on how to operationalise adaptive approaches. In this paper we expand on our learning from this project and offer some recommendations for how to create more opportunities for truly adaptive programming in the future. In particular: • Projects that are designed to adapt need budget structures, results frameworks and governance that enable the process of adaptation. In our GEC projects the approaches employed by DFID, including the introduction of Review and Adaptation meetings have served to support meaningful adaptation. • Adaptive projects require both strong participatory elements and flexible governance and accountability structures. Whilst rigorous and comprehensive Monitoring, Evaluation and Learning (MEL) systems are important without these other elements appropriate adaptation can be hindered. • Adaptive Management requires resources. Where the expected change is complex, adaptation is frequently necessary to ensure we are responding to context and evidence. This should be adequately resourced if we are to expect results. In an environment where many INGOs work consistently within complex environments, the sector also needs more opportunities to trial these approaches and could benefit from more funding streams available which include the kinds of approaches used by DFID in current GEC programming

Exit for Sustainability Checklists - Cekan, Jindra (Valuing Voices, 2020)
These Checklists for Exit along the Sustainability Project Cycle are based on lessons from ex-post evaluations that Valuing Voices has done or researched. Ex-post project completion evaluations are rare and even more so those that consulted partners and participants in the field about sustained impacts. Over $3.5 trillion has been spent on public foreign aid projects in the past 70 years (OECD 2019) yet we have evaluated less than 1% of these projects for sustainability. Our Valuing Voices ex-post research of 39 organizations’ evaluations of sustainability shows that most project results decrease (20-90%) as early as two years ex-post in addition to An Asian Development Bank study of post-completion sustainability found that “some early evidence suggests that as many as 40% of all new activities are not sustained beyond the first few years after disbursement of external funding,” Most project exits are in the last quarter and sustainability handover assumptions are not validated expost. Learning from what was sustained helps us know how to exit for sustainability from the very onset of the project (green slices) as compared to the typical project cycle (orange), above. We encourage those tasked with funding, designing, implementing, monitoring & evaluating projects to use these longer checklists and view the full recording shared with participants. A partial PowerPoint can also be found on „Sustaining All of our Hard Work“ presentation for the Vienna Evaluation Network (10/20). These checklists are aimed at donors/designers and implementers of foreign aid projects outcomes and impacts and can be adapted by local NGOs, national governments, private sector, academics, to create exit plans. Local participation in creating these and feedback on how well exit is going will help them sustain results.

Shifting Relationships with Grantees: Advancing Trust-Based Philanthropy through Emergent Learning - Gaspari, Brittney (The Emergent Learning Community Project, 2020)

Beneficiary Feedback Mechanisms - Hayman, Rachel (INTRAC, 2020)
There is a growing emphasis among NGOs and donors on ensuring that the voices of beneficiaries are heard. A range of beneficiary feedback mechanisms (BFMs) exist which enable beneficiary perspectives and suggestions to be gathered and used within an M&E system. BFMs are tools designed to enable a continuous cycle of interaction between those receiving and those delivering aid-funded interventions.

The Hidden Life of Theories of Change - Ho, Wenny; Tamas, Peter; van Wessel, Margit (Hivos, 2020)
Theory of Change is thought to be very useful for learning and adaptive management of complex interventions such as advocacy. Nevertheless, the use of Theory of Change is also under critique. One common criticism is that Theory of Change is often used as a framework that fixes agreements rather than as a living, guiding tool that helps reflection and adaptation. However, while such criticism stresses forms of control, little research has looked at the way Theory of Change and advocacy practice relate. This is a pertinent issue considering that formally agreed Theories of Change and realities on the ground can be very different. This raises questions: Do advocates work in ways different from what Theory of Change states, and if so, how, and why? How does the way they strategize relate to formal Theories of Change? With what implications? In this brief, we explore these more hidden aspects of the life of Theories of Change.

AdaptScan - Improving your Team's Adaptive Management - Mercy Corps (Mercy Corps, 2020)

A Pragmatic Approach to Assessing System Change: How to put it into practice - Miehlbradt, Alexandra; Shah, Rachel; Posthumus, Hans; Kessler, Adam (DCED, 2020)
Planning for and assessing system change is a strategic management issue. It is critical for everything from developing a strategy and designing interventions, to adapting strategy, improving implementation and reporting impact. But many programmes get stuck when it comes to assessing system change. The private sector development field has struggled to agree on an approach that programmes can implement and stakeholders can understand. However some mature programmes are starting to assess system change more effectively. Building on these emerging practices, this paper outlines a process that programmes can use to assess system changes regularly and practically. Two complementary papers: Overview and How to put it into practice The Overview summarises the approach and How to put it into practice provides more detailed implementation guidance, worked examples, and useful tips. The Overview explores how to: develop a system change strategy and intervention plans that lay the groundwork for system change assessment, including how to set system boundaries and how to identify the system changes a programme aims to catalyse assess system changes using both: - an intervention lens focused on changes introduced by specific interventions - a helicopter lens that provides a whole system view By analysing findings from both lenses, programmes can improve their strategy and report on their contribution to system change. How to put it into practice uses two case examples for illustration throughout the paper - PRISMA’s work in the maize system in East Java and Indonesia and S4J’s work in the Vocational Education and Training (VET) system in Albania. It targets practitioners responsible for facilitating and/or assessing system change. The paper explains how to: articulate the system changes that a programme aims to catalyse assess those changes use the results to inform decision making and reporting The approach described in the paper builds on the practices outlined in the DCED Results Measurement Standard. The guidance provided has been designed to be useful to programmes that aim to catalyse system changes whether or not they apply the DCED Standard.

A Theory of Change of citizen participation: an update - Peña-López, Ismael (ICTlogy, 2020)
when it was reduced to a subsidiary internal service lacking all kind of political attributions. The work done in those years had been formidable, but too many things had passed since, especially the 15M Spanish Indignados Movement, the raise of technopolitics… and the raise of populism and fascism all across Europe. We urgently needed a theoretical framework in which to substantiate our political strategy, so I came up with a Theory of Change of citizen participation (see Figure 1) which defined four expected impacts of our political action: 1. Efficiency, efficacy and legitimacy of public decisions improves. 2. Populism has decreased in institutions and the public sphere. 3. Citizens understand the complexity of public decision-making. 4. Citizen participation and political engagement clearly shifts towards a technopolitical paradigm. These impacts were expected to be achieved after some outcomes resulting from some outputs grouped in five programmes: 1. Programme of citizen participation. 2. Programme of internal participation. 3. Programme of collaboration. 4. Programme of intermediaries, facilitators and infomediaries. 5. Programme of e-participation, e-voting and technopolitics. 20 months after, the Theory of Change of Citizen Participation has worked quite well. But it does have some limitations, especially at the operational level-which is what the whole thing was about, to help in putting some order in our daily work.

Measuring Governance, Advocacy and Power: A Database of Existing Indicators, Tools and Indices - Poirrier, Caroline; Tolmie, Courtney (R4D, 2020)
Measuring Governance, Advocacy, and Power is an excel sheet that brings together existing indicators, tools, and indices that may be useful to practitioners responsible for the measurement of outcomes in the field of governance, advocacy, and power in an easily accessible and filterable format.

A Pragmatic Approach to Assessing System Change - Posthumus, Hans; Shah, Rachel; Miehlbradt, Alexandra; Kessler, Adam (DCED, 2020)
Planning for and assessing system change is a strategic management issue. It is critical for everything from developing a strategy and designing interventions, to adapting strategy, improving implementation and reporting impact. But many programmes get stuck when it comes to assessing system change. The private sector development field has struggled to agree on an approach that programmes can implement and stakeholders can understand. However some mature programmes are starting to assess system change more effectively. Building on these emerging practices, this paper outlines a process that programmes can use to assess system changes regularly and practically. Two complementary papers: Overview and How to put it into practice The Overview summarises the approach and How to put it into practice provides more detailed implementation guidance, worked examples, and useful tips. The Overview explores how to: develop a system change strategy and intervention plans that lay the groundwork for system change assessment, including how to set system boundaries and how to identify the system changes a programme aims to catalyse assess system changes using both: - an intervention lens focused on changes introduced by specific interventions - a helicopter lens that provides a whole system view By analysing findings from both lenses, programmes can improve their strategy and report on their contribution to system change. How to put it into practice uses two case examples for illustration throughout the paper - PRISMA’s work in the maize system in East Java and Indonesia and S4J’s work in the Vocational Education and Training (VET) system in Albania. It targets practitioners responsible for facilitating and/or assessing system change. The paper explains how to: articulate the system changes that a programme aims to catalyse assess those changes use the results to inform decision making and reporting The approach described in the paper builds on the practices outlined in the DCED Results Measurement Standard. The guidance provided has been designed to be useful to programmes that aim to catalyse system changes whether or not they apply the DCED Standard.

Learning and Accountability - Simister, Nigel; Scholz, Vera (INTRAC, 2020)
Most people agree that monitoring and evaluation (M&E) should be used for both learning and accountability. However, there is no consensus about which one is more important. The debate matters as there is sometimes tension between the two purposes. In the past there has often been a disconnect between M&E and learning. Many M&E systems are primarily designed to enable accountability to donors.

Chaos Report 2020 - Standish Group (Standish Group, 2020)

Guide for Adopting Remote Monitoring Approaches During COVID-19 - USAID (USAID, 2020)
USAID maintains staff and operations in more than 80 countries around the world, all of which the COVID-19 pandemic will disrupt. USAID remains committed to protecting the health and safety of our staff, while continuing appropriate oversight of our programs and ensuring the accountable and effective use of U.S. taxpayer funds. In the current operating environment, USAID and implementing partners face new challenges in implementing activities, monitoring progress, collecting data, and tracking indicators. As we adapt our approaches, we will work with implementing partners to find innovative, responsible, and safe ways to monitor and evaluate programming. Digital tools can support novel approaches to remote monitoring. Responsible use of digital tools also supports operating unit alignment with the Digital Data Collection mandate in the Agency’s new Digital Strategy. This guide provides information for Agency staff and implementing partners on remote monitoring techniques and when they can be employed. We encourage use of this guide to identify and pursue appropriate remote-monitoring approaches for your needs. CORs/AORs and Activity Managers should work with implementing partners to document updated approaches in each agreement’s plans for monitoring, evaluation, and learning (MEL), and should upload these amended Activity MEL Plans into the Agency Secure Image and Storage Tracking System (ASIST) as soon as possible. This can be done in collaboration with Mission and Washington Bureau M&E Specialists, as appropriate

Applying Rigorous Impact Evaluation in GIZ Governance Programmes: Results of a GIZ Initiative on Impacts in Governance - von Schiller, Armin (GIZ GmbH, 2020)
Pressure is mounting on international development cooperation agencies to prove the impact of their work. Private and public commissioners as well as the general public are increasingly asking for robust evidence of impact. In this context, rigorous impact evaluation (RIE) methods are increasingly receiving attention within the broader German development system and in GIZ. Compared to other implementing agencies such as DFID or USAid, the Deutsche Gesellschaft für Internationale Zusammenarbeit (GIZ) GmbH has so far relatively little experience in systematically applying rigorous methods of impact evaluation. This is particularly true in the governance sector. In order to gain more experience and to understand which methods and formats are best suited for GIZ governance programmes, the Governance and Conflict division and the Africa department launched the ‘Impact Initiative Africa’ in 2016, a cooperative effort with several programmes in Africa. The Initiative set out to apply the experiences from GIZ governance programmes to design and conduct RIEs, and to use the results to steer programme implementation. Initially, the Initiative included three countries: Benin (Programme for Decentralisation and Local Development), Malawi (Support to Public Financial and Economic Management) and Mozambique (Good Financial Governance in Mozambique). During its implementation, the Initiative also benefitted from the experience of two additional governance programmes which had already undertaken RIEs, namely Peru (Citizen-oriented State Reform Programme) and Pakistan (Support to Local Governance Programme II). This report summarizes the insights gained from these experiences and discusses opportunities and limitations regarding the use and usability of RIEs in GIZ governance programmes as well as proposals on how to organise RIEs to maximise learning potential and benefits for the specific programmes and the GIZ Governance sector at large.

Rigorous Impact Evaluation in GIZ Governance Programmes - von Schiller, Armin (Bonn, 2020)
Why should I integrate an impact assessment in my programme? How will the programme benefit from it? Are the benefits worth the effort and expenses? How do I design and implement it in detail? Who is addressable for support? What should I pay attention to in order to get the most out of it? This note is meant to answer these questions. It addresses leaders and project staff of governance programmes who are interested in using this tool within their specific governance programme or project. This note provides a guideline and good practice recommendations on how to design and conduct an impact assessment and on how to fully utilise the benefits of the results for the programme and for communication with commissioners, partners and other donors. Additionally, this note will point to indirect benefits that can arise and that should not be ignored. Results of impact assessments are highly relevant for the GIZ as an institution. However, in this note we stress the benefits for the programme or project itself. In particular this note addresses the following aspects: • What are rigorous impact assessments and why should GIZ Governance programmes conduct them more often within their programmes? • Which phases does an impact assessment include? How do I set one up and which aspects deserve special attention in each phase to maximise the benefits for my programme? • What are the benefits I can expect from implementing and impact assessment? • Whom to ask at headquarters in case I need support? This note complements the policy brief “Strategic use of Rigorous Impact Evaluation” and the corporate strategic review on “Rigorous Impact Evaluation” written by the GIZ evaluation unit which focuses on the strategic use of rigorous impact evaluations (RIE) at GIZ. Based on the review findings, the policy brief presents recommendations for strategic planning and implementation of purpose-sensitive RIE using a number of key levers. By adopting central coordination and needs-based support mechanisms, the evaluation unit intends to promote the strategic use of RIE for evidence-based learning and decision-making within the organisation.

USAID Wildlife Asia as a case study in adaptive rigour - Ziegler, Jessica (ODI/GLAM, 2020)
This briefing note looks at how the United States Agency for International Development (USAID) Wildlife Asia programme has operationalised the concepts of adaptive rigour and adaptive management as part of its approach to collaborating, learning and adapting. As described by the Global Learning for Adaptive Management (GLAM) initiative, adaptive rigour is about ensuring that the data, information, methods, processes and systems that underpin adaptive management are robust, systematic and high‑quality. Key messages When faced with programmatic complexity, it is important to take an adaptive approach driven by continuous and iterative monitoring, evaluation and learning (MEL). USAID Wildlife Asia, which works to advance regional action towards ending illegal wildlife trafficking in Southeast Asia and China, has embraced this approach as a way of operationalising adaptive rigour. Throughout programming, MEL approaches should reflect the characteristics of adaptive rigour: comprehensiveness, usefulness, practicality, timeliness and support. Utilising performance monitoring and research in order to test and revise technical approaches and employing mixed methods to collect both qualitative and quantitative data, as well as looking for relevant lessons generated by others, can ensure access to the most useful information for decision-making throughout implementation. For adaptive management, it is not enough to monitor, evaluate and learn; it is also essential to pause and reflect in order to analyse and process evidence gained through MEL with colleagues and stakeholders to reach the right conclusions and make good decisions.

Adapting data collection and utilisation to a Covid-19 reality: Monitoring, evaluation and learning approaches for adaptive management - Ziegler, Jessica; Mason, Paige (ODI/GLAM, 2020)
This briefing note focuses on the remote collection and use of data for adaptive management during the Covid-19 pandemic, setting out key considerations to help practitioners think through a transition from more ‘traditional’ monitoring, evaluation and learning (MEL) to MEL for adaptive management (MEL4AM) that reflects the unique data collection challenges presented by Covid-19. The brief provides an overview of some key considerations in remote data collection, when this is required, and identifies other sources that address these issues in more detail. It concludes with a discussion of how to bring the information resulting from remote monitoring into decision-making to enable adaptive management. Key messages - When planning for remote data collection during the Covid-19 pandemic, first determine what information is still necessary, because data needs may have changed, e.g. if programming has pivoted or needs to pivot due to Covid-19. Then identify how the programme’s information needs align with existing data sources and what gaps remain, which will guide the need for remote data collection. - Also consider what data is ‘good enough’ for current decision-making needs in order to provide sufficient information to the right people at the right time to an acceptable standard of rigour. - There may be pragmatic reasons to reduce the number or scope of MEL activities, such as logistical constraints or ethical considerations introduced by the pandemic. - MEL activities should be accompanied by frequent feedback loops and pause points to reflect on emergent needs and challenges, information needs that have been met, and contextual changes that may affect MEL. - Be clear with decision-makers about the assumptions and gaps in the data, including proxies used and their limitations, sampling changes, and how these changes and assumptions may affect the decisions/options being discussed.

Evaluability Checklist for Post Project Evaluation - Zivetz, Laurie; Cekan, Jindra (Valuing Voices, 2020)
Considerations for planning a post project evaluation during the project, at the end, or after it has closed.

Supporting adaptive management. Monitoring and evaluation tools and approaches - Pasanen, Tiina; Barnett, Inka (ODI, 2019)
Key messages • This working paper introduces a set of monitoring and evaluation (M&E) tools and approaches, discussing their potential usefulness in supporting adaptive management in development and humanitarian programmes. It emphasises adaptive programmes characterised by complex aspects, such as: (1) they are innovative; (2) they have uncertain pathways for change; or (3) they operate in uncertain or unstable environments. • The majority of these tools have been used in international development for many years. However, adaptive management brings additional challenges for monitoring and evaluating programmes, as they require intentional M&E design from the start that is oriented towards both learning and accountability. • All of the tools and approaches introduced in this paper have potential to support learning and adaptation, although in various ways and at different stages of a programme. Some tools can support strategic planning and diagnosing throughout a programme – especially during design and inception – while others can help analyse causal relationships at specific points in a programme. It is important to tailor the approach used for its intended purpose. However, whether learning and adaptation happens depends also on factors other than the choice of M&E methods. • For some of these approaches a considerable body of evidence already exists but, for many, more practical examples and systematic analysis is needed. In addition to building the evidence base concerned with which approaches are suitable for different types of adaptive programmes and why, it is also important to improve understanding of the enabling environmental conditions necessary for the tools and approaches outlined here to facilitate and strengthen evaluative thinking, evidence-informed decision-making and ongoing programme iteration.

Towards evidence-informed adaptive management - Hernandez, Kevin; Ramalingam, Ben; Wild, Leni (ODI/GLAM, 2019)
Development and humanitarian organisations seeking to be adaptive have emphasised the need to be transparent about complexity and uncertainty; to be honest about their inability to control what happens; and to design programmes that change over time to become more appropriate and relevant. At their heart, adaptive management approaches emphasise the ability to lean, 'unlearn' and adapt programming accordingly. The cornerstone of effective learning is the creation, gathering, accumulation, interpretation and use of data and evidence. This working paper provides development professionals with tools, strategies and ideas to help them use evidence for adaptive management in practical and evidence-informed ways. Key messages - Evidence is central to effective and rigorous adaptive management. However, despite this central importance, exactly how evidence has been used to inform decisions for adapting development and humanitarian programmes in the past remains unobservable to many. - There is a need to strengthen and document evidence-informed adaptive management. This working paper proposes a roadmap to do this. - Those seeking to use evidence for adaptive management will need to manage trade-offs between ensuring a rigorous, documented (and auditable) trail of evidence-informed actions, being pragmatic about the time and resources allocated to documentation and recognising that it may be necessary to proceed without rigorous evidence when it is unavailable.

Livelihoods in the Za'atari Camp: Impact evaluation of Oxfam’s Cash for Work activities in the Za’atari camp (Jordan) - Lombardini, Simone; Mager, Franziska (http://hdl.handle.net/10546/620883, 2019)

Stakeholder and Social Network Analysis - Guidance Note - IRC (International Rescue Committee, 2019)
Stakeholder Analysis is used to identify the actors and relationships that influence project outcomes. This guidance, from IRC, can help you determine how to work and who to partner, coordinate or engage with in order to best achieve the outcome. It builds on existing stakeholder information and typically will include a participatory internal meeting or workshop.

Theory of change in ten steps - Noble, James (NPC, 2019)

Citizen engagement: emerging digital technologies create new risks and value - Peixoto, Tiago; Steinberg, Tom (World Bank, 2019)
The recent rapid evolution of digital technologies has been changing behaviors and expectations in countries around the world. These shifts make it the right time to pose the key question this paper explores: Will digital technologies, both those that are already widespread and those that are still emerging, have substantial impacts on the way citizens engage and the ways through which power is sought, used, or contested? The authors address this question both to mitigate some of the World Bank’s operational risks, and to initiate a conversation with peers about how those risks might require policy shifts. The overall framing question also is being explored in case theapproaches to citizen engagement advocated by the World Bank are changing and may require different advice for client countries. Despite the lower technology penetration levels in developing countries, their more malleable governance contexts may be more influenced by the effects of emerging technologies than older states with greater rigidity. Digitally influencedcitizen engagement is, in short, one of those “leapfrog” areas in which developing nations may exploit technologies before the wealthier parts of the world. But countries can leapfrog to worse futures, not just better ones. This paper explores what technology might mean for engagement, makespredictions, and offers measures for governments to consider.

LAC MEL Specialists Use Peer Network to Improve Quality and Use of Evidence - Anderson, Todd M; Prevatt, Amy (USAID, 2019)
Based on consultations and stocktakings with LAC Missions, the LAC Bureau identified Mission Monitoring, Evaluation and Learning (MEL) Specialists as influential decision makers regarding data and evidence use. The LAC Bureau aims to improve the quality and use of evidence for decision making, and ultimately to improve development outcomes.

Failing Forward: How CARE is focusing on what goes wrong to improve impact - Janoch, Emily (USAID LEARN, 2019)
CARE's Failing Forward initiative is sparking opportunities to showcase the ideas that don't work so we can spend more time implementing the ones that do. It's changing the conversation inside the organization, and leading to changes in the way we design and implement programs. It's also allowing us to make connections across a global portfolio of more than 900 projects, and with new and different partners to learn from their experience.

The COM-B Theory of Change Mode - v.5 - Mayne, John (Palladium, 2019)
ResearchGate is a network dedicated to science and research. Connect, collaborate and discover scientific publications, jobs and conferences. All for free.

Adaptive Programming in Fragile, Conflict and Violence-Affected Settings, What Works and Under What Conditions?: The Case of Institutions for Inclusive Development, Tanzania - Green, Duncan; Guijt, Irene (Itad, Oxfam and IDS, 2019)
Adaptive Management involves a dynamic interaction between three elements: delivery, programming and governance. This case study focuses on a large DfID governance project, the Institutions for Inclusive Development (I4ID), a five-year initiative in Tanzania. The study forms part of a research project to examine whether and how adaptive approaches can strengthen aid projects promoting empowerment and accountability in fragile, conflict and violence-affected settings (FCVAS). The research examines some of the assertions around the adaptive management approach and explores if and how adaptive approaches, including rapid learning and planning responses (fast feedback loops and agile programming) are particularly relevant and useful for citizen empowerment and government accountability (E&A) in FCVAS.
Keywords: A4EA; Adaptive Development; Economy; Fishery

Supporting the policy environment for economic development (SPEED+) - Strategic Review Report - MSI (USAID, 2019)
Donor-funded policy reform has a long history in Mozambique, with United States Agency for International Development (USAID) efforts dating to the mid-1990s. Many laws and regulations adopted during the past quarter century are the consequence of these efforts. And yet, even with several years of robust economic growth, Mozambique has not experienced the broad economic transformation that policy reforms can trigger: per capita income in 2017 was $519 and more than 80 percent of the country lived on less than two dollars per day. The country ranks 180 out of 189 on the Human Development Index and 135 out of 180 on the World Bank’s Doing Business Index. Supporting the Policy Environment for Economic Development (SPEED+), a $37 million, four-year, USAID-funded program, represents the latest iteration of USAID’s policy reform efforts. SPEED+ began in August 2016 and is scheduled to run through August 2020. As the program passed the midway point, USAID commissioned Management Systems International (MSI) to conduct a strategic review (SR) of the SPEED+ program, with the objectives of advising on: • how to maximize the impact of SPEED+ during its final 18 months; • what type of follow-on program USAID should consider; and • what broader lessons can be drawn from SPEED+’s experience in Mozambique.

Top Tips: How to design and manage adaptive programmes - Valters, Craig; Wild, Leni (Overseas Development Institute, 2019)
Within DFID, there is now a commitment to more flexible and adaptive programming. This recognises that: • DFID works in contexts that continuously evolve and change, sometimes in unpredictable ways. To respond to this, the agency needs to remain flexible – to expect change and have a good understanding of context, with resources that can be adjusted and scope to change direction if needed. All DFID programmes should be able to do this. • Some DFID programmes aim to support change in complex systems, behaviours and incentives. Efforts to address women’s empowerment, improve sanitation or build more sustainable health systems, for instance, all require engagement with the way in which complex systems operate and the people and behaviours within them. Trying to deliver reforms in these circumstances is challenging because the pathway to reform itself will be unclear: as a reform is rolled out, the system itself will react and respond. These types of programmes therefore need to build in from the start deliberate processes of learning and testing, to allow for adaptations as more information is gathered for what works over time. These top tips are concerned with programmes that aim to be flexible and adaptive – which work in dynamic contexts and are trying to address complex problems. While there is growing commitment to these approaches, feedback suggests staff still have questions about how to do this well. This document highlights some of the commonly reported issues related to adaptive programming and a set of tips, strategies and examples to help in addressing them. It is aimed at programme managers and advisors who may be Senior Responsible Owners (SROs) or those managing and supporting adaptive programming in a range of ways. We have collated these lessons from discussions with country offices and SROs, feedback from surveys and the wider evidence. We have looked specifically at the adaptive programmes we can find in the DFID portfolio, but this is not an exhaustive list. It should be noted that there is as yet no wide variety of case law to review, but there is a growing set of examples within DFID that can provide continuous learning for the organisation.

Beyond Tweets and Screams: Action for Empowerment and Accountability in Nigeria – The Case of the #BBOG Movement - Aina, Tade Akin; Atela, Martin; Ojebode, Ayo; Dayil, Plangsat; Aremu, Fatai (IDS, 2019)
This paper explores the nature, role and dynamics of new forms of social and political action as pathways to empowerment and accountability in fragile conflict- and violence-affected settings in Africa. Through an in-depth analysis of the case of the Bring Back Our Girls (#BBOG) movement in Nigeria and a multi-methods approach, the paper provides new knowledge that addresses evidence gaps in the following areas: (1) the multiple ways through which social and political action play out in fragile, conflict- and violence-affected settings; (2) whether the conditions in which new forms of social and political action applicable to fragile, conflict- and violence-affected settings – the settings most dominant in African countries – emerge as currently projected in the literature; and (3) whether these social and political actions necessarily produce accountability and empowerment in fragile, conflict- and violence-affected settings. Although often expressed as contentious and/or unruly politics, experiences from the BBOG movement suggest that the new forms of social and political action possess a wide range of implications for citizen action and governance, including leading to multiple forms of empowerment in fragile settings.

Exploring the new world: practical insights for funding, commissioning and managing in complexity - Lowe, Toby; Plimmer, Dawn (Collaborate & Newcastle University, 2019)
The world is complex. If we want to contribute to creating positive social outcomes, we must learn to embrace this complexity. This is the New World that funders and commissioners are discovering: • People are complex: everyone’s life is different, everyone’s strengths and needs are different. • The issues we care about are complex: issues – like homelessness – are tangled and interdependent. • The systems that respond to these issues are complex: the range of people and organisations involved in creating ‘outcomes’ in the world are beyond the management control of any person or organisation. Building on the findings from our previous report, A Whole New World, we have spent the last 12 months working with a growing movement of funders, public sector commissioners, and organisations working on the ground to begin to explore this New World: to find examples of practice that will help people to navigate it effectively. This report explores the key features of their response: working in a way that is human, prioritises learning and takes a systems approach.

The Case for an Adaptive Approach to Empowerment and Accountability Programming in Fragile Settings: Synthesis report - Christie, Angela; Green, Duncan (Itad, Oxfam and IDS, 2019)
Fragile, conflict and violence-affected settings (FCVAS) are messy and ambiguous contexts in which to plan and implement development initiatives. To work there, external actors are increasingly adopting an adaptive approach to empowerment and accountability (E&A) programming, whatever the setting. This means using a compass rather than map, where real-time political economy analysis (PEA) in relation to context and programme monitoring and evidence-informed learning in relation to intervention are used in combination and in shorter-than-usual planning cycles to maintain and adapt strategic direction. This paper brings together three case studies of large Department for International Development (DFID) governance projects in Myanmar, Nigeria and Tanzania.
Keywords: A4EA; Adaptive Development; Economy; Fishery

Implementing Adaptive Approaches in Real World Scenarios: A Nigeria Case Study, with Lessons for Theory and Practice - Bridges, Kate; Woolcock, Michael (The World Bank, 2019)
How does adaptive implementation work in practice? Drawing on extensive interviews and observations, this paper contrasts the ways in which an adaptive component of a major health care project was implemented in three program and three matched comparison states in Nigeria. The paper examines the bases on which claims and counterclaims about the effectiveness of these approaches were made by different actors, concluding that resolution requires any such claims to be grounded in a fit-for-purpose theory of change and evaluation strategy. The principles of adaptive development may be gaining broad acceptance, but a complex array of skills, expectations, political support, empirical measures, and administrative structures needs to be deftly integrated if demonstrably positive operational results are to be obtained, especially when undertaken within institutional systems, administrative logics, and political imperatives that are predisposed to serve rather different purposes.

Knowledge Retention and Transfer (KRT) - Model and Matsurity Matrix - USAID (USAID LEARN, 2019)
The Maturity Matrix is not a standalone tool. It is one of the three major components of the KRT Model. The Maturity Matrix is meant to be used in conjunction with the KRT Toolkit and Implementation Plan.

Managing staff transistions through CLA: preserving institutional memory as staff come and go - USAID (USAID LEARN, 2019)
This document is relevant for any position or hiring mechanism. While this document does not explicitly address what happens when someone arrives into a newly-created position, which has its own set of challenges, many of the principles, actions, and resources can be applied in that context. Section 1 offers guidance for how to set up and implement systems at Mission/OU or office to ensure that all staff help preserve institutional memory and enable continuity of relationships. This section is for any staff in a Mission/OU working to build systems that improve handovers and knowledge capture and sharing across the Mission or OU. If a Mission or OU is just beginning to develop a system to manage staff transitions, you may want to start by developing a system within an office or team and then expand it out to other offices or to the Mission/OU writ large. Section 2 includes key principles and actions individuals could consider following when arriving or departing a position, regardless of hiring mechanism and position. This includes staff new to the Mission/OU and staff who are currently in the Mission/OU and are moving into a new position. Section 3 provides a select number of resources to help you take a systematic and comprehensive approach to manage staff transitions as effectively and efficiently as possible at the individual or organizational level.

Broad Agency Announcement for Locally Led Development Innovation - USAID (USAID, 2019)
This BAA allows USAID Operating Units (OUs) to co-create, co-design, co-invest, and collaborate in the research, development, piloting, testing, and scaling of innovative, practical, and cost-effective interventions to catalyze locally led development. The BAA aligns with a number of Agency priorities and policies, including the Journey to Self-Reliance, resilience, procurement innovation, and expanding and diversifying the partner base - as well as the New Partnerships Initiative and Acquisition and Assistance Strategy. E3/LS has put up the BAA’s first Addendum, "New Directions in Advancing Locally Led Development," which seeks Expressions of Interest on the following: - Engaging the Local Private Sector - Conflict, Post-Conflict, and Non-Permissive Environments - Effective Partnerships - The Changing Role of the Donor - Local Giving, Philanthropy, and Other Private Resources

The Limits of Accounting-Based Accountability in Education (and Far Beyond): Why More Accounting Will Rarely Solve Accountability Problems - Honig, Dan; Pritchett, Lant (Center for Global Development, 2019)
Accountability is rightly at the center of the conversation regarding how to improve governance systems, particularly health and education systems. But efforts to address accountability deficits often focus primarily on improving what can be counted and verified—what we term “accountingbased accountability.” We argue that introducing greater accounting-based accountability will only very rarely be the appropriate solution for addressing accountability problems. We illustrate this by exploring the role of Accountability ICT in (not) improving education system performance. Strengthening “real” accountability is not the same as improving data systems for observation and verification, and often attempts at the latter undermine the former. The development discourse’s frequent semantic misunderstanding of the term “accountability” has pernicious effects on system reform efforts and ultimately global welfare.

Monitoring and learning for country-level portfolio decision-making and adaptation - Buffardi, Anne L; Mason, Paige; Hutchings, Claire; Sharp, Samuel (ODI/GLAM, 2019)
Most measurement and adaptive management approaches were developed for and from individual projects. This briefing aims to guide measurement and management of country-level portfolios of work. It identifies potential purposes portfolio-level analyses can fulfil, types of adaptation, and the relative role of monitoring, learning and evaluation (MEL). Drawing on reviews of practice from the United States Agency for International Development (USAID) and the United Kingdom Department for International Development (DFID), guidance notes, and experiences of members of the Global Learning for Adaptive Management (GLAM) initiative, it offers recommendations and considerations that are particularly relevant for this level of analysis and management. Key messages - Portfolio-level analyses can serve eight potential purposes, each of which answers different questions, involves adaptation at different times and levels, and requires different types of evidence. Identifying the purpose(s) and how the component parts relate to each other should guide the development of monitoring and learning systems. - Portfolios are oriented more towards breadth than depth, involve more people with different perspectives, and draw on multiple sources of evidence with potentially greater variation in quality. - In practice, four activities appear to be applied most frequently at a portfolio level: alignment of indicators and aggregation of monitoring data; synthesis of multiple sources and types of information to provide a summary of outputs, outcomes, common observations and trends; periodic review and reflection sessions; and strategic planning, design or refresh of the portfolio strategy. - The extent to which evidence-informed portfolio management is facilitating learning and adaptation has not been well documented to date, and we suggest potential indicators to do so.

Barrio Mio and Katye: PCI’s neighbourhood approach in cities - Campbell, Leah (ALNAP/ODI, 2019)

Help! I’m hiring new staff and I want them to work adaptively. A guide to hiring adaptive employees. - Salib, Monalisa (USAID LEARN, 2019)
If you are involved in hiring, this tool can help you increase your chances of selecting staff members skilled in adaptive management. It will help answer the questions: Which competencies should I recruit for in order to hire more adaptive employees? Which desired qualifications should I incorporate into position descriptions to attract adaptive employees? Which interview questions should I ask to screen for adaptive employee competencies? You should use this tool when you decide to hire a new individual - whether that be a direct hire, contractor, or consultant - or when you are changing an individual’s current position description or scope of work. Specifically, this tool will help you determine which qualifications are most important for the position and offers interview questions you can use with candidates. — You’re in luck! We just released a new guide for hiring adaptive employees. This visual and user-friendly tool will help increase your chances of selecting staff skilled in adaptive management. Here are the details: What do you mean by “work adaptively” or an “adaptive employee”? An adaptive employee is an individual who, in collaboration with relevant stakeholders, systematically acquires and uses knowledge to make decisions and adjustments in their work in order to achieve greater impact. Why focus on adaptive employees? Why does it matter? With industries, technologies, and organizations in a near constant state of flux, leaders are recognizing the importance of adaptability as a critical capacity. Because of this, it’s not surprising that a recent LinkedIn report found that adaptability was the most important soft skill hiring managers were screening for. In addition, evidence shows that teams that apply more data-driven and adaptive leadership practices perform better than those that focus less on these practices. It makes sense, then, that managers want to recruit more team members who are skilled in adaptive management - it helps achieve results. Who should use this tool? Anyone involved in hiring, responsible for developing or approving position descriptions, participating in interviews, and/or approving new hires. When should I use this tool? Use it as soon as you decide to hire a new staff person or when you’re adjusting existing position descriptions. What does the tool help me do? It will help you think through: Which competencies should I recruit for in order to hire more adaptive employees? Which desired qualifications should I incorporate into position descriptions to attract adaptive employees? (You can even copy and paste qualification language from the tool into scopes of work!) Which interview questions should I ask to screen for adaptive employee competencies? (You can copy and paste interview questions into your screening, interview, and reference check protocols!) Has the tool been tested? Yes, staff within USAID’s Global Development Lab and on the USAID LEARN contract tested the tool, providing feedback on the minimum viable product and subsequent versions. Testers confirmed finding the tool useful, and gave it a 9/10 score when asked if they would recommend it to colleagues. Some specific feedback from testers included: "The tool prompts deeper thought on what you might need and want in a job candidate" and "All [of the interview questions I used from the tool] worked well." What about the existing staff? How can they gain greater skills in adaptive management? Existing USAID staff and implementing partners can access online training in collaborating, learning, and adapting (CLA) and find resources in the CLA Toolkit. USAID staff also have access to an internal CLA community of practice and five-day, in-person CLA training.

A Guide to Digital Feedback Loops. An approach to strengthening program outcomes through data for decision making - Whittle, Dennis; Campbell, Megan (USAID, 2019)
Feedback is information about what happens as a result of what you do. Using that information to adapt what you do or how you do it creates what is called a feedback loop. A digital feedback loop uses digital technology at some stage of the feedback loop. Digital feedback loops help USAID missions improve the effectiveness and efficiency of their activities and can support partner countries on their journey to self-reliance through increased information sharing and improved government and civil society capacity. USAID missions and their partners have used digital feedback loops to improve HIV treatment targeting in Zimbabwe, engage three million young people in Nigeria in policy, and reduce field reporting times from one month to one day. Digital feedback loops provide access to information faster and more cheaply than ever before. As part of an adaptive management approach, digital feedback loops increase our ability to act on opportunities and respond to challenges. That in turn can improve outcomes and reduce the risk of waste and failure. This guide outlines the case for integrating digital feedback loops into your work, and provides tools, resources and guiding questions to support you as you get started. The guide includes examples of how USAID missions are using digital feedback loops to improve their programs in various sectors and provides worksheets to help you apply digital feedback loops to your own context.

How Does the World Bank Build Citizen Engagement Commitments into Project Design? Results from Pilot Assessments in Mozambique, Myanmar, Nigeria, and Pakistan - Nadelman, Rachel; Le, Ha; Sah, Anjali (IDS, 2019)
How and to what degree is the World Bank putting its new institutional citizen engagement (CE) commitments into practice? This question guides an independent assessment that the Accountability Research Center (ARC) at American University has undertaken as part of the Institute of Development Studies-led Action for Empowerment and Accountability (A4EA) research programme’s investigation into how external actors can best support local processes of and conditions for empowerment and accountability. This report investigates the World Bank’s incorporation of CE into project design, the critical early stage of donor engagement. To accomplish this, ARC reviewed the World Bank’s fiscal year 2015–17 investment project portfolios for four A4EA priority countries, Mozambique, Myanmar, Nigeria, and Pakistan, which covers 57 projects that range from US$19 million to U$600 million. The analysis determines whether projects commit to seeking a strategic approach to CE, which involves combining multiple tactics so that the whole could be greater than the sum of the parts. This assessment of CE commitments is intended to help to inform possible national, civil society organisation strategies to monitor whether and how these commitments are actually implemented in practice.

A study of the sustained uptake developmental evaluation: How does developmental evaluation work in the USAID context, what factors help and hinder its success, and what is its value to stakeholders? - Baylor, Rebecca; Fatehi, Y. K.; Esper, H. (USAID, 2019)

DFID’s partnerships with civil society organisations - A performance review - ICAI (Independent Commission for Aid Impact (ICAI), 2019)
DFID values civil society organisations (CSOs), but its funding and partnership practices do not fully support the long-term health of the civil society sector.

Making adaptive rigour work - Principles and practices for strenghening monitoring, evaluation and learning for adaptive management - Ramalingam, Ben; Wild, Leni; Buffardi, Anne L (ODI/GLAM, 2019)
Adaptive programmes can be accountable, rigorous and high quality in how they use evidence by taking an ’adaptive rigour’ approach. Core development and humanitarian challenges are complex, and require processes of testing, learning and iteration to find solutions – adaptive management offers one approach for this. Yet large bureaucracies and development organisations can have low tolerance for experimentation and learning, and adaptive management can be viewed as an excuse for ‘making things up as you go along’. This briefing from the Global Learning for Adaptive Management (GLAM) initiative argues that adaptive programmes can be accountable, rigorous and high quality in how they use evidence – but this requires rethinking some key assumptions about how they are practised. The paper sets out three key elements of an ‘adaptive rigour’ approach: - Strengthening the quality of monitoring, evaluation and learning data and systems. - Ensuring appropriate investment in monitoring, evaluation and learning across the programme cycle. - Strengthening capacities and incentives to ensure the effective use of evidence and learning as part of decision-making, leading ultimately to improved effectiveness.

Learning from Results-Based Management evaluations and reviews - Vähämäki, Janet; Verger, Chantal (OECD, 2019)
What have we learned from implementing results-based management in development co-operation organisations? What progress and benefits can be seen? What are the main challenges and unintended consequences? Are there good practices to address these challenges? To respond to these questions this paper reviews and analyses the findings from various evaluations and reviews of results-based management systems conducted by members of the Development Assistance Committee (DAC), the OECD/DAC Results Community Secretariat and other bodies in the past four years (2015-2018). It also draws on emerging lessons from new methods for managing development co-operation results. This analytical work aims to: identify recent trends in results-based management, explore challenges faced by providers when developing their results approaches and systems, select good practices in responding to these challenges that can be useful for the OECD/DACResults Community, considering new approaches, new technologies and evolving contexts. This body of evidence will inform the development of a core set of generic guiding principles for results-based management in development co-operation.

Knowing When to Adapt - A Decision Tree - USAID (USAID LEARN, 2019)
This job aid will help you determine how best to resolve a challenge/issue in programming. It is intended for use by USAID, partners, and others working in international development.

Bringing adaptive management to life: Insights from practice - Arora, Anmol; Gogoi, Elizabeth; Joy, Divya; Kumar, Pankaj; Luthra, Rajni; Pal, Uma; Pervaiz, Arif (Oxford Policy Management, 2019)
The increasing popularity of adaptive programmes in the development sector in the past decade is partly a response to the complex and interconnected challenges facing many developing countries, including the impacts of climate change. The pathway to adapting to climate change is unknown, and there are many deep-rooted institutional, political, economic and social barriers. Adaptive programmes provide the flexibility to allow those delivering the technical assistance to support governments to experiment with different entry points and to adapt or change course when some are not successful. The Action on Climate Today (ACT) programme is a five-year adaptive programme that aims to strengthen systems of planning and delivery for adaptation to climate change in South Asia. As the programme draws to a close, this Learning Paper reflects on some of the major lessons learnt on the challenges and opportunities of using an adaptive programme management approach to support governments to adapt to climate change. The authors are members of the ACT implementation team, representing the management, operational and technical teams, from across different locations. The paper therefore reflects their first-hand account of the reality of implementing adaptive programmes. The paper sets out a framework for the essential ingredients of an adaptive programme management approach. These include a set of essential core principles: an evolving theory of change; a locally led and politically savvy delivery approach; experimentation and learning; and stakeholder alignment. It also includes two sets of essential resources required: management flexibility and adequate financial resources. The paper then provides learning from ACT on how to operationalise these core principles and mobilise the necessary resources. It gives detailed examples of challenges faced by ACT with regard to each component of the framework, and how the programme has overcome them, as well as retrospective reflections of additional actions the programme could have taken. A set of case studies unpacks how adaptive programme management has been operationalised in practice in particular locations. The authors are confident that the impact ACT has had owes in large part to it being an adaptive programme. The programme’s flexibility, decentralised structure and strong emphasis on learning have meant the team has been able to respond to opportunities as they have emerged and to invest time and resources where there was most potential for impact. However, this paper is also honest about the challenges ACT has faced as an adaptively managed programme. The paper therefore concludes with a set of lessons learnt from ACT for the benefit of others designing and delivering adaptive programmes on climate change and other issues.

How DFID can better manage complexity in development programming - Sharp, Samuel; Valters, Craig; Whitty, Brendan (ODI, 2019)
The United Kingdom’s (UK) Department for International Development (DFID) is an ambitious government department that is committed to reducing poverty and conflict overseas. Many of the issues on which DFID works are complex; whether focused on climate change, gender equality, health or other priorities, simple solutions rarely exist. And to tackle these complex challenges, DFID staff must interact with unpredictable systems of political, organisational and individual behaviours and incentives. There is a risk that complex problems spur more complicated programmes; that the complexity of development challenges is addressed through designing programmes with too many projects and implementers. While there may be valid reasons for this, too many of these complicated programmes will overburden staff. This briefing note is the outcome of an ongoing process within DFID to confront these issues and answer the question: how can DFID design and manage programmes to address complex development challenges without creating too much staff workload? Key messages DFID deals with complex problems, which require flexible systems to support testing, learning and adaptation. • Complex problems do not necessarily require workload-heavy delivery structures, but simpler approaches depend on delivery partners’ experience and competence. • However, programmes that tackle complex problems do require more hands-on engagement and face more workload from inflexible compliance requirements. We suggest that DFID and similar agencies: • Pay closer attention to delivery options in programme design, making use of existing options where possible and, where not, fostering local organisations through long-term investments. • Encourage programme designers to articulate how ambition matches resources and consider ‘good enough’ design. • Reduce the burden of compliance by cultivating partner autonomy, reassessing results and valuefor-money requirements and promoting more flexible contracting and procurement. • Plan and prioritise management resources across a portfolio of programmes to make sure they can be focused in the right areas, where the complexity of the problem requires greater engagement.

Short Course - Adaptive Management, an overview - Bazaz Smith, Komal (American University, 2019)

No Royal Road: Finding and Following the Natural Pathways in Advocacy Evaluation - Coe, Jim; Schlangen, Rhonda (Center for Evaluation Innovation, 2019)
As efforts at driving change become more diffuse, involve more actors, and have more transformational goals, we need a radically different approach to thinking about and assessing effective advocacy. Clear answers, certain judgments, and simple tools won't result in good representations of reality. This brief proposes ideas for adjusting how we approach advocacy monitoring, evaluation, and learning.

Monitoring, Evaluating and Learning for Complex Programs in Complex Contexts: Three Facility Case Studies - Davda, Tara; Tyrrel, Lavinia (Abt Associates, 2019)

AID Programming Guide - DFAT (Australian Government, 2019)

Peacebuilding design, monitoring, and evaluation: A Training Package for participants and trainers at intermediate to advanced levels - Ernstorfer, Anita; Barnard-Webster, Kiely (Carnegie Corporation, 2019)
This training package includes 7 Training Modules and a set of Annexes (Annexes A-O). The Training Modules build on each other and should ideally be used in a sequenced way in a training setting. However, for groups with specific training needs around particular areas, modules can also be used individually, but need to be tailored by the trainers and facilitators to meet the needs of specific audiences. The annexes provide worksheets and hand-outs that can be used as resources during the training for specific modules and exercises.

Designing Monitoring and Evaluation Approaches for Learning An FOS How-To Guide - FOS (FOS, 2019)

Local Works Guidance - Round 4 - USAID (USAID, 2019)

Integrating Big Data and Thick Data to Transform Public Services Delivery - Ang, Yuen Yuen (IBM Center for The Business of Government, 2019)
This report explains how public organizations can combine big data and thick data to transform public services delivery—a strategy called mixed analytics. Governments can greatly enhance the value of big data by combining it with “thick” data—rich qualitative information about users, such as their values, goals, and consumption behavior, obtained by observing or interacting with them in their daily lives. Big data holds great promise for improving public services delivery and innovation in government, but they are not a panacea . Having lots of data can be overwhelming or have little utility if the data are “thin”—that is, they lack meaning for users or fail to capture issues that matter most . By yielding insights into what citizens really care about and how they consume services, thick data can inform both the collection and analysis of big data. This report introduces the concept of “mixed analytics,” integrating big data and thick data to transform government decision making, public services delivery, and communication. The report presents three case studies of organizations that employ mixed analytics at the international, federal, and city level, respectively. Together, this research offers a set of transferable lessons for agencies at all levels of government: Lesson 1: Big data is a means to an end, rather than an end. Lesson 2: Thick data can identify unexpected problems or previously unexpressed needs. Lesson 3: Thick data can inform the analysis of big data. Lesson 4: Mixed analytics can offer both scale and depth. Lesson 5: Applying technology is a social activity, not an isolated technical task. Lesson 6: The best solutions are not always high-tech. The report concludes with five actionable recommendations for public managers. Make data and technology relevant to the people who use it. Leverage thick data at appropriate stages of the problem-solving process. Build an interdisciplinary team of quantitative and qualitative experts who work closely with stakeholders. Combine big and thick data to improve communication Improve government agencies’ knowledge of mixed research methods. This report builds on multiple past IBM Center reports about how government can leverage data and analytics to improve decisions, including Data-Driven Government: The Role of Chief Data Officers, by Jane Wiseman; Ten Actions to Implement Big Data Initiatives: A Study of 65 Cities, by Alfred Ho and Bo McCall, and Realizing The Promise of Big Data, by Kevin DeSouza. The report also complements several chapters that assess the past and potential use of data across the public sector in our recent book, Government For The Future: Reflection and Vision For Tomorrow’s Leaders. At a time when the US and other governments continue to enhance their use of data as a strategic asset for transformation, we hope that Professor Ang’s report provides useful insights for government managers and stakeholders.

Developmental Evaluation: How Barriers & Enablers Emerge Over Time - Baylor, Rebecca; Fatehi, Y. K.; Esper, H. (USAID, 2019)

Inclusive Social Accountability - Counterpart International (Counterpart International, 2019)
Counterpart International’s Inclusive Social Accountability (ISA) developmental framework integrates elements of social inclusion and community accountability into one comprehensive approach.

Pocket Guide to Systems Wayfinding - Davidson, Seanna (The Systems School, 2019)

DFID Smart Rules: Better Programme Delivery - version X (2019.01) - DFID (DFID, 2019)

Breaking the Mould: Alternative approaches to monitoring and evaluation - Dillon, Neil (ALNAP, 2019)
This paper looks at a range of M&E innovations that are designed specifically to provide input to ongoing iterative decision-making and learning at the project level. It identifies three key areas for potential innovation: 1) timing of M&E data provision; 2) flexibility of M&E frameworks to evolve with programme change; and 3) approaches to integrate diverse perspectives on project implementation in a meaningful way. It then looks at a collection of approaches currently being used in each of these three areas through a series of ‘practice examples’, considering the key lessons learned. Finally, the paper discusses the major opportunities and challenges for applying and scaling up the use of these approaches inside the humanitarian sector.

Back to the Drawing Board: How to improve monitoring of outcomes - Dillon, Neil; Sundberg, Amelie (ODI/ALNAP, 2019)

Empowerment and Accountability in difficult settings: What are we learning? - Gaventa, John; Oswald, Katy (IDS, 2019)
Empowerment and Accountability in Difficult Settings: What Are We Learning? Key Messages Emerging from the Action for Empowerment and Accountability Programme Empowerment and accountability have long been part of the international development vocabulary and a core part of governance, social development and civil society programmes. Yet, much of what has been learnt about these approaches has been drawn from studies in somewhat stable, open and middle-income places around the world. Less is known about how empowerment and accountability are achieved through social and political action in more difficult settings – those faced by institutional fragility, conflict, violence, and closing civic space. This document highlights key messages emerging from the work of the Action for Empowerment and Accountability Research Programme (A4EA), and the implications for how donors, policy makers and practitioners support strategies for empowerment and accountability in fragile, conflict and violence affected settings (FCVAS). Our eight key messages have strong implications for the theories of change used for effective programming in the field.

Adaptive Management Practice in the UK Government - The MUVA case - HLS (Human Learning Systems, 2019)
Case study about the MUVA programme in Mozambique. (Maybe it "misses the point of AP which is not learning for learning. Is learning for impact. The word impact doesn’t even come up once!") Adaptive Management programming within the Foreign & Commonwealth Development Office demonstrates that the UK Government has examples of optimising for learning within its existing management practice. However, currently, the adaptive management practices are unhelpfully framed by an approach which limits both their effectiveness and scope. The programmes use learning as their metastrategy. They succeed in connecting practicebased learning with strategic learning, and make a shift towards reframing accountability as accountability for learning. This learning strategy is enabled by funding and contract management arrangements which prioritise learning. Both at macro-level of programme management and micro level of de-risking experimentation and enabling necessary failure by decoupling people’s job security from potential failure.

Understanding Innovation Ecosystems: A Framework for Joint Analysis and Action. Cambridge: MIT D-Lab - Hoffecker, Elizabeth (MIT D-Lab, 2019)
Place-based innovation ecosystems play a crucial role in driving local and regional economic development. This role has been documented and understood for over 40 years in industrialized economies but is only starting to be appreciated in the context of emerging and developing economies. However, the past several years have seen an intensification of interest in innovation ecosystems among global development actors as well as practitioners and policymakers working across the Global South. Whether at the municipal level, in places such as Medellín, Colombia and Guadalajara, México, or the national level in Rwanda, South Africa, or India, we see governments, multi-laterals, donors, and civil society actors embarking on initiatives to strengthen local innovation ecosystems. The International Development Innovation Alliance, for example, which includes many of the largest public and private global development agencies, has created a set of recommendations for why and how actors investing in economic development should support the strengthening of innovation ecosystems. Within the past year, MIT D-Lab has been invited to play a role in some of these ecosystem-strengthening efforts. In contexts ranging from Oaxaca, México to Accra, Ghana, we have been asked to convene ecosystem actors and stakeholders in order to facilitate joint ecosystem strengthening work. In preparing for these engagements, we have researched the state of the field regarding both innovation and entrepreneurial ecosystems as well as existing ecosystem frameworks, models, and tools. In doing so, it has become clear that much of the current thinking and practice related to these concepts is drawn from research on innovation processes and entrepreneurial clusters in highly developed economies — places like Silicon Valley and Kendall Square, Boston. Much less is known about innovation ecosystems in less-developed contexts in terms of how they can be characterized, how they function, and — most importantly — how they can be strengthened. To that end, the Local Innovation Group at D-Lab has been conducting multi-year research on local innovation ecosystems in the types of contexts where D-Lab and our partners engage. This involves learning about diverse processes of ecosystem development through primary and secondary case study research. Based on this research, we have developed a framework for understanding local innovation ecosystems, which we share in this publication. We have found the framework, and accompanying visual model, to be a useful tool for orienting and organizing conversations among ecosystem actors on how particular innovation ecosystems are functioning, what their strengths and weaknesses are, and where opportunities for further development might lie. This document shares this framework and how we have used it over the past year to catalyze ecosystem-strengthening efforts. We start by clarifying the concept of a “local innovation ecosystem” and presenting the core ideas informing the visual model. We then describe the model and each of its individual components. We follow with guidance from our research on best practices for conducting ecosystem strengthening work, and share three examples of how we have used the model to facilitate ecosystem-strengthening conversations in distinct local contexts.

Monitoring and Result Measurement for Adaptive Programming - How to Use Data to Manage a MSD Program: Lessons from PRISMA - Khan, Khaled; Seely, Kevin; Ridwan, Mustika; Mulya, Bodhiya (PRISMA, 2019)
Using monitoring data to improve interventions is harder than it seems. Decision-makers are often busy implementing activities, unclear about their roles in data collection and analysis, and uncertain what data matters most or when. PRISMA, an AUD77 million agricultural Market Systems Development (MSD) programme funded by DFAT Australia, has encountered these challenges. With the programme completing its first five year phase, this case study shares ten key lessons divided into three sections: shaping the culture, developing systems, and top management decisions. These lessons aim to help program, sector and intervention managers make better use of monitoring data to improve interventions.

Joint Programming in Conflict-Affected and Fragile States - Koenig, Sibylle; Brusset, Emery (Joint Programming - EU, 2019)
On the Joint Programming Practices from the EU in fragile settings

DAC Peer Review of Switzerland - 2019 - OECD (OECD, 2019)

Planning & Navigating Social Change - Tools for Pacific voyagers - Orr, Douglas Epeli; Cavatore, Maria; Aston, Tom (Oxfam, 2019)
Those working to advance social change in the Pacific must understand, work with and respond to the complex and changing relationships and dynamics of power that exist within such networks, and situate their work in the context of decolonisation and self-determination. But the approaches typically used by NGOs to plan our projects and programs have failed to give due attention to these dynamics. The result is that our projects fall short of achieving their promise, despite being technically sound and logical. In this guide, we aim to provide a process and tools that prioritise and draw out the rich, often implicit, knowledge that Pacific Islanders have about our contexts to plan and manage social change initiatives in a manner that values and responds to this ocean of relationships. Drawing from these relationship-based and voyaging traditions, we emphasise the need to: • frame change in terms of change in people; e.g. change in the situation of groups of people and change in terms of the actions of people; • identify a course towards a particular ‘change destination’; • identify the signs to read that will help you determine where you are on that course; • be constantly attentive to the context as you make the journey, and • be ready to adjust your course in response to changes in your context. To support this way of working, we have selected a number of tools from approaches that we believe fit. In particular, we have drawn from the following approaches: • Thinking and Working Politically and other ‘politically aware’ approaches that recognise that the success of a development initiative will be determined by the interplay of relationships of power within a particular context, not just by having the right technical solution; • Outcome mapping because of its focus on people and relationships, and on identifying and supporting needed changes in the behaviour of specific actors in relation to others. This is key, because it’s individuals who deliver services, individuals who decide whether to allocate budget and resources to address a problem, and individuals who make the choice to believe whether a service is relevant to solving their issues; • Adaptive management because of its recognition that development initiatives take place within complex and changing systems and need to adjust to these and that this requires ongoing learning and adaptation.

Monitoring And Evaluation Tools and Approaches to Support Adaptive Management - Pasanen, Tiina (GLAM, 2019)

The Rapid Evaluation, Action, and Learning (REAL) Approach: A toolkit to measure and refine changes and interventions in health campaigns - PATH (PATH, 2019)
Evaluative Tools for Improving Campaign Effectiveness Public health campaigns aim to control disease or deliver essential health services and products through time-limited and periodic channels. Many countries use campaigns to augment or replace routine service delivery, to target certain populations, or to accelerate progress towards coverage targets. The Root Cause Analysis and Rapid Evaluation, Action, and Learning toolkits present a systematic but flexible approach to identifying the root causes of campaign bottlenecks and then designing, testing, and refining solutions to optimize potential impact. These toolkits respond to a growing need for “fit-for-purpose” rapid-testing, adaptive learning approaches to evaluation and the need for a culture shift toward iterative adaptation and improvement that integrates measurement and evidence-informed decision-making into daily practice. The comprehensive package of toolkits, in addition to the individual Root Cause Analysis toolkit and Rapid Evaluation, Action, and Learning toolkit are available below.

What is Uncertainty and Why Does it Matter? - Scoones, Ian (STEPS centre, 2019)
Uncertainty defines our times. Whether it is in relation to climate change, disease outbreaks, financial volatility, natural disasters or political settlements, every media headline seems to assert that things are uncertain, and increasingly so. Uncertainty, where we do not know the probabilities of either likelihoods or outcomes, is different to risk, the implications of which are explored in this paper through five different ways of thinking about uncertainty, derived from highly diverse literatures encompassing societal, political, cultural, practice and individual perspectives. The paper continues by examining how these perspectives relate to four domains: finance and banking; critical infrastructures; disease outbreaks and climate change; natural hazards and disasters. Reflecting on these experiences, the paper argues that embracing uncertainty raises some fundamental challenges. It means questioning simple, linear perspectives on modernity and progress. It means rethinking expertise and including diverse knowledges in deliberations about the future. It means understanding how uncertainties emerge in social, political and economic contexts, and how uncertainties affect different people, depending on class, gender, race, age and other dimensions of social difference. And, if uncertainty is not reducible to probabilistic risk, it means a radically different approach to governance; one that rejects control-oriented, technocratic approaches in favour of more tentative, adaptive, hopeful and caring responses. The paper concludes by asking whether we can learn from those who live with and from uncertainty – including pastoralists in marginal settings – as part of a wider conversation about embracing uncertainties to meet the challenges of our turbulent world.

Managing and Adapting a development program: Lessons from PRISMA - Seely, Kevin (PRISMA, 2019)
This case study is part of the AIP-Rural Learning Series. Funders and implementers of international development programs largely agree that adaptive management is industry best practice. Most development experts also broadly agree on what ‘adaptive management’ means. In this case study, we use a common definition of ‘adaptive management’, including the following features: Flexibility. Implementers create, modify and drop interventions when circumstances change or new information emerges, in order to tailor strategies to context and maximise impact. Purposeful experimentation. Implementers test different activities at the same time, monitoring them to learn what will achieve the desired impact. Since 2015, the number of publications calling for and praising adaptive management in development programs has grown fast. These publications give us an idea of the state of adaptive management in development programs. They reveal progress in some areas, such as tailoring interventions to local context and using evidence for decision-making. Yet they also reveal multiple, ongoing, real-world constraints to adaptive management. This case study explores how one development program, PRISMA, has avoided and overcome some of these constraints, whilst continuing to battle others. Four major constraints revealed in the adaptive management literature are discussed in this case study: Programs are designed in ways that make it hard to adapt interventions, target regions or sectors based on new learning or changing circumstances. Programs struggle to create an organisational culture that encourages learning and adapting. When staffing programs, recruiters prioritise sectoral expertise and length of experience over adaptive managerial competence. Program managers face pressures to spend their budgets predictably and before their program ends. Learning and testing take time and cost little, so managers feel pressure to deprioritise them. The first of these constraints is explored in the next section, ‘Design’, which looks at how the selection of target sectors and performance targets affects PRISMA implementers’ ability to manage adaptively. The third section, ‘Organisational Culture’, shares lessons on encouraging staff to test and improve interventions. Section 4, ‘Staffing’, looks at hiring adaptive managers, and freeing up their time to improve interventions. The final section summarises this case study’s key lessons. Throughout this paper, findings are based on 16 in-depth interviews with PRISMA’s junior, mid-level and senior staff, and its funders. To contextualise these findings, the authors reviewed the ‘adaptive management’ literature, and drew on their experience with development programs worldwide.

Doing adaptive management at Sida - Tim Ruffer; Helen Bailey; Stefan Dahlgren; Patrick Spaven; Mark Winters (SIDA, 2019)
Lessons from the market systems development approach The evaluation focuses on Sida’s management of MSD projects. The projects are applying the MSD approach with an aim to: - contribute to improved MSD programming by Sida through better management practices across the project cycle - generate recommendations on how Sida can create conducive conditions for systems approaches and adaptive programming more generally. The brief assesses Sida’s organisational capacity for adaptive management in three dimensions: - leadership and culture - staff capacities - skills, and systems and procedures

Was bedeutet “Staying Engaged” im Kontext der aktuellen Krise in Nicaragua? Überlegungen und Strategien der ALAK (Internal memo) - SDC (SDC, 2018)
«Mitigation» (im Sinne von Notlinderung) und «Transformation» sind die zwei Schlagworte, an welchen sich die Neuausrichtung des Portfolios in Nicaragua über die nächsten Monate orientieren wird. Ziel ist es, dass die Schweiz ihr über die letzten Jahrzehnte akquiriertes politisches Kapital und ihre Expertise nutzt, um zu einer Linderung der Auswirkung der Krise für die Bevölkerung beizutragen und mithilft die Grundlagen für eine Transformation der politischen Kultur und eine langfristig angelegte Demokratisierung zu schaffen.

Doing Development Differently in the Global South - Workshop Report - CARE (CARE and British Council, 2018)

Seeking balanced ownership in changing development cooperation relationships - Keijzer, Niels; Klingebiel, Stephan; Örnemark, Charlotte; Scholtes, Fabian (Expert Group for Aid Studies, 2018)
According to the Paris Declaration and the 2030 Agenda, ownership is a prerequisite for effective development cooperation. How can the principle of ownership be promoted in today’s complex development cooperation, in which the numbers of actors have increased? This is the subject of the Expert Group for Aid Studies report Seeking balanced ownership in changing development cooperation relationships. The report contains two country case studies – on Liberia and Rwanda – as well as interviews with the Swedish International Development Cooperation Agency (Sida) and the Ministry for Foreign Affairs, and studies on documents concerning ownership in a Swedish development cooperation context. The authors note that ownership remains relevant as a guiding principle in international development cooperation. The trend from country-to-country cooperation to various forms of support through UN bodies and specialised global funds has created new conditions and competing interests for ownership. An understanding of ownership and how it can be promoted in the complex reality of today needs to be thoroughly re-examined. The study recommends the following: 1. Starting a discussion within the OECD-DAC on effective development cooperation that has global ownership right up to the end of the process. 2. Establishing an international code of conduct in accordance with agreed ownership principles concerning development cooperation funding to and via UN bodies and global funds. 3. Sweden should formulate an explicit policy and approach for how to promote ownership in development cooperation that includes numerous partners.

Modern Slavery Prevention and Responses in South Asia: An Evidence Map - Oosterhoff, Pauline; Yunus, Raudah; Jensen, Charity; Somerwell, Francis; Pocock, Nicola (DFID, 2018)
The Asia Pacific region has the highest numbers of both slavery and child labour victims in the world. The International Labour Organization (ILO) estimates that 16.5 million and 8.4 million persons were in situations of forced labour and forced marriage respectively in Asia Pacific, among 40.3 million in modern slavery globally (ILO & Walk Free Foundation 2017). Although there is a growing body of research and evaluations on specific sub-sectors and interest in the worst forms of labour exploitation, there has not yet been a systematic scoping or synthesis of studies that would help policymakers understand ‘what works’ to reduce the prevalence of “Modern Slavery”2(MS) in the countries of interest to DFID (India, Nepal, Bangladesh, Pakistan). India alone is estimated to have the largest number of people in modern slavery in the world with nearly 8 million persons living in modern slavery, followed by 3.2 million, 592,000 and 171,000 in Pakistan, Bangladesh and Nepal respectively (Walk Free Foundation 2018). While systematic reviews on interventions to reduce prevalence of trafficking, forced or bonded labour or slavery exist, these have not focused on the target countries of interest to DFID. Previous systematic reviews have focussed on particular types of MS, such as cross border sexual exploitation globally (van der Laan et al. 2011), labour exploitation in Europe (Cockbain et al. 2018) or community based interventions for safer migration programming in low and middle income countries (LMICs) (Zimmerman et al. 2016). Several systematic reviews focus specifically on health needs and post-trafficking care or interventions (Ottisova et al. 2016; Muraya & Fry 2016; Hemmings et al. 2016; Dell et al. 2017), and research methods and tools used in trafficking in persons (TIP) and health research (Cannon et al. 2018; Doherty et al. 2016). A recent global review and evidence map of MS interventions found some interventions in the target countries, but the outcomes were not specified (Bryant & Joudo 2018). In this Evidence Map, we address this gap in evidence for India, Nepal, Bangladesh and Pakistan specifically, scoping the range of Modern Slavery interventions and outcomes for specific target populations (survivors, employers, landlords, services providers, criminal justice officials) and at different levels (individual, community, state). This map is foremost targeted to DFID and its partners in India, Nepal, Bangladesh and Pakistan to inform evidence-based policy making. We hope that it is also useful to academics and practitioners working to address modern slavery, or in intervention areas or locations described.

Evaluation of the market systems development approach: Lessons for expanded use and adaptive management at Sida Volume I: Evaluation Report - Ruffer, Tim; Bailey, Helen; Dahlgren, Stefan; Spaven, Patrick; Winters, Mark (Sida, 2018)
This report presents the findings of an evaluation of Sida’s management of the market systems development (MSD) approach. It aims to inform thinking on how Sida can best manage its growing portfolio of MSD programs. Beyond this, it provides insights relevant to Sida’s wider support to complex and adaptive programs. The evaluation identified several factors that affect Sida’s ability to ensure that conducive conditions are in place for effective MSD programs and good development programming more generally. Sida’s relatively flexible framework of rules, guidelines and systems for project management provide the space needed for staff to innovate and manage adaptively. But for this to happen consistently and effectively, Sida needs to invest more deliberately in building the capacity of its staff in relevant areas. In addition, leadership and incentives are key to shaping a culture of active experimentation and learning. This needs to be supported with clearer guidance for those involved in the design and appraisal of MSD projects; and strengthened oversight of project performance, including through adjustments to Sida’s contracts and funding agreements.

Evaluation of the market systems development approach: Lessons for expanded use and adaptive management at Sida Volume II: Case studies - Ruffer, Tim; Bailey, Helen; Dahlgren, Stefan; Spaven, Patrick; Winters, Mark (Sida, 2018)
This report presents the findings of an evaluation of Sida’s management of the market systems development (MSD) approach. It aims to inform thinking on how Sida can best manage its growing portfolio of MSD programs. Beyond this, it provides insights relevant to Sida’s wider support to complex and adaptive programs. The evaluation identified several factors that affect Sida’s ability to ensure that conducive conditions are in place for effective MSD programs and good development programming more generally. Sida’s relatively flexible framework of rules, guidelines and systems for project management provide the space needed for staff to innovate and manage adaptively. But for this to happen consistently and effectively, Sida needs to invest more deliberately in building the capacity of its staff in relevant areas. In addition, leadership and incentives are key to shaping a culture of active experimentation and learning. This needs to be supported with clearer guidance for those involved in the design and appraisal of MSD projects; and strengthened oversight of project performance, including through adjustments to Sida’s contracts and funding agreements.

Volume 3: Participatory Monitoring, Evaluation, and Learning in Complex Adaptive Environments - SACE (SACE Program, 2018)
This volume is the last in a series of papers about systems approaches in complex environments, which includes the use of the collective impact model to address large-scale social problems, and the application of participant-driven MEL techniques across 17 networks of civil society organizations. It is based on the experiences of Root Change and Chemonics, two development partners working on a USAID civic engagement project in Nigeria (2013-2018), as well as hundreds of civil society organization partners. This is the final paper in the series and aims to highlight how the adaptation of participatory monitoring, evaluation, and learning techniques (e.g., most significant change, outcome mapping, and outcome harvesting) evolved and ultimately empowered cluster members. The first paper in the series presented a brief introduction to systems approaches in advocacy settings, the SACE theory of change, and the scope of Root Change’s work as technical lead on capacity building and measurement. The second volume aims to address the innovative use of the advocacy strategy matrix, adapted from work by the Center for Evaluation Innovation, for collective impact and the Collective Impact Model, an approach that engages multiple players in working together to solve complex social problems.

Acquisition and Assistance Strategy - USAID (USAID, 2018)
The U.S. Agency for International Development (USAID) of tomorrow is one in which the Agency enables partner countries to plan, resource, and manage their own development through strengthened capacity and commitment — this is the essence of the Journey to Self-Reliance. In Fiscal Year (FY) 2018, the Agency obligated over 80 percent of its programmatic funding — a total of $17 billion — through acquisition and assistance (A&A) mechanisms. Partnering and procurement are central to how we do our work. Therefore, we must think holistically about with whom we work and how we work, if we are to achieve our ambitious goals for the USAID of tomorrow. The purpose of this document is to provide a transparent strategy that guides changes to Agency policy and practice for both our staff and implementing partners in how we approach our core A&A work. This strategy, which builds on previous and current reform efforts, outlines the shifts we will make to embrace a self-reliance model for A&A and move concretely toward the goal of ending the need for foreign assistance. GUIDING PRINCIPLES OF PARTNERING AT USAID USAID’s partnering approaches have evolved significantly over time as we have tested new and innovative ways of working. The principles outlined below support the Journey to Self-Reliance by empowering our staff and partners to produce results-driven solutions responsive to partner country needs and priorities — building capacity and commitment based on creative and entrepreneurial approaches that have already begun to develop. The Agency must move beyond our traditional approaches to embrace greater collaboration, co-design, and co-financing to improve upon our models of partnering.

Adaptive Programming in Fragile, Conflict and Violence-Affected Settings, What Works and Under What Conditions?: The Case of PERL, Nigeria - Punton, Melanie; Burge, Richard (Itad, Oxfam and IDS, 2018)
This paper examines adaptive approaches to aid programming in Nigeria. Through field research and desk reviews, we have investigated some of the assertions around the ‘adaptive management and programming’ approach, which has arisen in recent years as a response to critiques of overly rigid, pre-designed, blueprint and linear project plans. This is the second of three case studies in a series which explore if and how adaptive approaches, including rapid learning and planning responses, are particularly relevant and useful for promoting empowerment and accountability in fragile, conflict and violence-affected settings (FCVAS). This case study focuses on PERL (Partnership to Engage, Reform and Learn) in Nigeria, a five-year governance programme (2016-21) with a total budget of £100 million. It aims to promote better service delivery through bringing together government and citizens groups to collectively address governance challenges. PERL is viewed by DFID as the final stage of a 20-year investment, building on learning, experience and partnerships from 15 years of DFID-funded governance programming in Nigeria. It was designed to incorporate learning and adaptation through a ‘living’ theory of change, continuous political economy analysis at different levels, adaptive implementation by location-based delivery teams who are encouraged to be flexible and let partners take the lead, regular learning and reflection, and adaptive resourcing, HR and administrative systems. The case study draws on a conceptual framework (the ‘adaptive triangle’) that looks at three types of adaptation – adaptive management, adaptive programming and adaptive delivery – and the interconnections and tensions between them.
Keywords: A4EA; Adaptive Development; Economy; Fishery

Parliament and the people - Rumbul, Rebecca; Moulder, Gemma; Parsons, Alex (2018)
How digital technologies are shaping democratic information flow in Sub-Saharan Africa

Thinking and working with political settlements. The case of Tanzania - Kelsall, Tim (ODI, 2018)
The institutions and power relations that underpin political stability are crucial shapers of development outcomes, to which aid policy ought to adapt. • This paper trials a new political settlements concept, based on the idea that settlements can be distinguished according to the breadth and depth of their social foundation, and the degree to which they concentrate power. • Since independence, Tanzania has oscillated between broad-dispersed and narrow-concentrated political settlements and is currently moving in the direction of a more narrow and concentrated settlement than ever before. • The opportunities for international donors to change this direction of travel appear limited. Adapting to it implies providing financial and technical support to central government, while trying to strengthen the policy role of non-state actors in non-confrontational ways. • In recent years, Political Settlements Analysis (PSA) has become increasingly influential in academic and policy circles • Despite its intuitive appeal, it seems difficult to use in practice • PSA has a natural affinity with Adaptive Development, Thinking and Working Politically, and Doing Development Differently approaches • By answering the simple diagnostic questions supplied here, development partners can identify the types of political settlements in which they work, and draw some broad operational implications

Guidance: Rigor and Reproducibility in Grant Applications | grants.nih.gov - NIH (National Institutes of Health, 2018)

USAID ADS 201 - Program Cycle Operational Policy (Update 10/29/2018) - USAID (USAID, 2018)

Caja de Herramientas PDIA - Un enfoque “hazlo tú mismo” para resolver problemas complejos - Samji, Salimah; Andrews, Matt; Pritchett, Lant; Woolcock, MIchael (Center for International Development at Harvard University, 2018)
(Spanish version of he PDIA toolkit) The PDIAtoolkit is designed to guide you through the process of solving complex problems which requires working in teams. We call it a Do-it-Yourself (DIY) kit, where the ‘you’ is a committed team of 4-6 people mobilized to work together to solve a complex problem that cannot be solved by one person.

PDIA Toolkit - A DIY Approach to Solving Complex Problems - Samji, Salimah; Andrews, Matt; Pritchett, Lant; Woolcock, MIchael (Center for International Development at Harvard University, 2018)
The PDIAtoolkit is designed to guide you through the process of solving complex problems which requires working in teams. We call it a Do-it-Yourself (DIY) kit, where the ‘you’ is a committed team of 4-6 people mobilized to work together to solve a complex problem that cannot be solved by one person.

Learning from adaptive programmes - 10 lessons and 10 case studies - Valters, Craig (Overseas Development Institute, 2018)
Internal DFID document from the DevAdapt Programme. Based on another previous document (also internal).

A guide to agile communication - Defra digital - DEFRA Digital (Departmetn for Environment, Food & Rural Affairs (UK), 2018)
This guide is intended for: people planning communications activity on behalf of agile teams people doing the communications activity (especially blogging, presenting, or filmmaking) people who manage the people described above, who want to understand what they're doing and why The government service standard encourages teams to work in the open as much as possible, echoing item 10 in the government design principles, “make things open, it makes them better”. This guide was written to help teams do just that.

Learning to make a difference: Christian Aid Ireland’s adaptive programme management in governance, gender, peace building and human rights - Booth, David; Balfe, Karol; Gallagher, Róisín; Kilcullen, Gráinne; O'Boyle, Sarah; Tiernan, Alix (ODI, 2018)
This paper assesses the relevance of adaptive or trial-and-error approaches to the field of governance, peace building and human rights. Tackling the problems of poverty, vulnerability and exclusion that persist in parts of the world that continue to be affected by violence or political insecurity is difficult for several reasons. For one, because of the complexity of the prevailing social, economic and political systems, solutions to chronic problems are far from obvious. One response to this aspect of the challenge is adaptive programme design and management. This paper is the product of a multi-year collaboration between ODI and the core team of Christian Aid Ireland to assess the relevance of adaptive or trial-and-error approaches to the field of governance, peace building and human rights. It explains the basis on which Christian Aid Ireland’s current five-year programme has become committed to an adaptive approach. It then describes and seeks to draw lessons from the programme’s first year of experience, considering the possible implications for implementation over the coming years. The authors find that to get full benefits from the move to adaptive management, the new ways of working and their underlying principles will need to become more embedded in organisational practices and cultures.

Achieving value for money through procurement. Part 2: DFID’s approach to value for money through tendering and contract management - A performance review - ICAI (Independent Commission for Aid Impact (ICAI), 2018)
Summary In 2016-17, the Department for International Development (DFID) spent £1.4 billion, or 14% of its budget, through commercial suppliers on contracts ranging from school construction to family planning services and the delivery of humanitarian aid. The quality of its procurement and contract management – how it engages and manages commercial firms to support the delivery of aid programmes on time, to budget and at the appropriate quality – is a key driver of value for money for UK aid. It is also a subject of considerable Parliamentary and public interest. In recent years, DFID has implemented a range of initiatives to strengthen its procurement practices and embed commercial capability across the department – including its 2017 Supplier Review, undertaken to address concerns about excessive profit-making by DFID suppliers. This review on achieving value for money through procurement (part 2) was published in September 2018. We awarded a green-amber score and made three recommendations to government. The follow-up to this review was published in July 2020, and rated the government’s response as ‘inadequate’, as a result of DFID’s failure to put in place a formal contract management regime, despite the risks this entails for programme results. We will return to this in the next follow-up in summer 2021. Review Our review found an appropriate overall approach to procurement with good performance in most areas of tendering, but significant weaknesses in contract management. We therefore awarded a green-amber score and made three recommendations to government. Findings Since 2015, DFID has progressed towards a more mature procurement approach. DFID’s tender process follows current EU legislation and UK government guidelines and cross-government peer reviews confirm improvements in DFID’s procurement approach. The Supplier Review lent momentum to the reforms but risks having unintended consequences. Poor consultation with suppliers has heightened the risk of unintended consequences. DFID is reviewing its business processes to protect aid recipients from sexual abuse and exploitation. DFID does not always choose the most appropriate procurement process but a new strategic sourcing process has resulted in stronger procurement planning. DFID has built up its commercial capability, but this will need to be an ongoing process. The contract management function in DFID is poorly defined and contracts are frequently amended or extended beyond their advertised length and values. Inception phases are often too short for adequate preparation and planning. Progress on flexible and adaptive programming will require more innovative approaches to tendering and contract management. Recommendations Before the next major revision of its supplier code and contracting terms, or future changes that may materially affect suppliers, DFID should conduct an effective consultation process with its supplier market, to ensure informed decisions and minimise the risks of unintended consequences. DFID should accelerate its timetable for acquiring a suitable management information system for procurement, to ensure that its commercial decisions are informed by data. DFID should instigate a formal contract management regime, underpinned by appropriate training and guidance and supported by a senior official responsible for contract management across the department. The new regime should include appropriate adaptive contract management techniques, to ensure that supplier accountability is balanced with the need for innovation and adaptive management in pursuit of development results.

Integrating Food Security and Wild Caught Fisheries Management in USAID Programming. Illustrative situation Model and Results Chain - Swaminathan, Vinaya; Best, Barbara; Ajroud, Brittany (USAID, 2018)
Integrating Food Security and Wild Caught Fisheries Management in USAID Programming is an illustrative case example that demonstrates how two adaptive management programming tools—situation models and results chains—can help USAID staff and its partners to better design and implement activities that integrate food security, nutrition, and sustainable management of wild fisheries.

Building a global learning alliance on adaptive management - Wild, Leni; Ramalingam, Ben (ODI, 2018)

Adaptive Peacebuilding Programming Through Effective Feedback Loops: Promising Practices - Barnard-Webster, Kiely; Jean, Isabella (CDA, 2018)
This case study report was developed by CDA’s part of Peacebuilding Evaluation Consortium collaborative research on the use of feedback loops in support of adaptive programming. Feedback loops are just one element of adaptive programming and are defined and used in different ways by organizations working on social change, peace and development issues. This document includes key findings from our desk review of reports and grey literature, key informant interviews, observations documented during CDA’s advisory and learning engagements with peacebuilding partners as well as the case study on SFCG’s feedback processes in Burundi. The findings are most relevant to program and M&E staff but have implications for senior management and policymakers in light of growing interest in strengthening adaptive management.

Independent Review of Facilities: review and management response - DFAT (DFAT, 2018)

Guidelines for designing and monitoring social accountability interventions - Franco, Erika Lopez; Shankland, Alex (IDS, 2018)

Opportunities for Using Complexity-Aware Approaches to Theory of Change - Goodier, Sarah; Apgar, Marina (IDS, 2018)
The purpose of this briefing note is to review opportunities for using complexity aware approaches to Theory of Change (ToC) to inform the SDC approach. It provides an overview of complexity-aware approaches and then focuses on demonstrating how complexity thinking can support programming by building on the frameworks currently being used in the project/programme cycle management (PCM) processes. It is aimed at SDC staff, in particular Programme Officers and staff of partner organisations involved in the management of SDC interventions.

State of the Art on Use of Theory of Change in the Development Sector - Goodier, Sarah; Apgar, Marina; Clark, Louise (IDS, 2018)
The purpose of this briefing note is to add to SDC’s understanding of Theory of Change (ToC), drawing on the literature and practice to sketch out the current state of the art approach. This involves expanding on ToC beyond SDC’s current practice of using Impact Hypotheses (IH) to bridge it to operational practice and use ToC more explicitly in the project/programme cycle management (PCM) processes. Sharing the state of the art on use of ToC in the development sector, this briefing note outlines what a ToC is, what it is used for and why it is needed in the development sector. It discusses ToC as both a process and a product, providing step by step guidance on how to facilitate a ToC process. The differences between a ToC and a logframe are highlighted. Some key criteria for recognising when you have a ‘good’ ToC are also included. This brief is aimed at SDC staff, in particular Programme Officers, and staff of partner organisations involved in the management of SDC interventions.

Using adaptive monitoring, evaluation and learning in programme design - Nzegwu, Femi (INASP, 2018)

Making humanitarian response more flexible: Challenges and questions - Obrecht, Alice; Bourne, S. (ALNAP/ODI, 2018)

Adaptive Programming in Fragile, Conflict and Violence-Affected Settings, What Works and Under What Conditions?: The Case of Pyoe Pin, Myanmar - Christie, Angela; Green, Duncan (Itad, Oxfam and IDS, 2018)
This paper examines adaptive approaches in aid programming in a fragile, conflict and violence-affected setting (FCVAS), namely Myanmar. A combination of desk review and field research has been used to examine some of the assertions around the ‘adaptive management’ approach, which has arisen in recent years as a response to critiques of overly rigid, pre-designed, blue-print and linear project plans. This paper explores if and how adaptive approaches, including rapid learning and planning responses (fast feedback loops and agile programming) are particularly relevant and useful for promoting empowerment and accountability in such ‘messy places’. This case study focuses on Pyoe Pin (‘Young Shoots’), a DFID-funded, British Council managed governance programme, which has been running since 2007.
Keywords: A4EA; Adaptive Development; Economy; Fishery

Assessing the Demand for a Global Evidence Network. Mapping Existing Initiatives and Understanding Network Lessons and Opportunities - Gandolfo, Ari; Taddese, Abeba (Results for all, 2018)

State of the Field Review: Fiscal Transparency and Accountability - Rudiger, Anja (International Budget Partnership, 2018)

USAID Risk Appetite Statement - USAID (USAID, 2018)

How Decent is Decent Work? Using SenseMaker to understand workers’ experiences - Mager, Franziska; Smith, Becca; Gujit, Irene (Oxfam, 2018)

Thinking and working politically in Somalia: A case study on the Somalia Stability Fund - Laws, Ed (TWP Coommunity of Practice and ODI, 2018)

Prioritizing and Selecting Strategic Approaches in USAID Biodiversity Programming - Stem, Caroline; Swaminathan, Vinaya; Flores, Marco (USAID, 2018)
This guide supplements Biodiversity How-To Guide 2: Using Results Chains to Depict Theories of Change in USAID Biodiversity Programming. Step 4, Brainstorming Strategic Approaches, involves generating a suite of potential strategic approaches for a design team to consider. Step 5 introduces the practice of prioritizing and selecting among these draft strategic approaches but does not provide in-depth guidance on the process and criteria for doing so.

Combating Wildlife Trafficking Case Study Compilation: Capacity Building for Enforcement and Prosecution - USAID (2018)

Program Cycle - Discussion Note: Designing Monitoring, Evaluation, and Learning Platforms - USAID (USAID, 2018)
This Discussion Note complements ADS 201.3.1.2 Program Cycle Principles by elaborating on Principle 2: Manage Adaptively through Continuous Learning. This Discussion Note is intended for USAID staff interested in learning about recent and promising practices in adaptive management across the Program Cycle.

Incubating policy for economic transformation: Lessons from Nepal - Booth, David (ODI, 2018)

Global Affairs Canada: an approach to development innovation - GAC (Global Affairs Canada, 2018)
For Global Affairs Canada, development innovation includes new or improved business models, policy practices, approaches, technologies, behavioral insights or ways of delivering products and services that benefit and empower the poorest and most vulnerable people in developing countries.

Digital Principles into practice - HDIF (HDIF, 2018)

How Does Developing Enforcement Capacity Reduce Wildlife Crime? Increasing program effectiveness by understanding common assumptions - Price, Claire; Buff, Jesse; Peabody, Shawn; Present, Tess; Lauck, Liz (USAID, 2018)
"How Does Developing Enforcement Capacity Reduce Wildlife Crime?" summarizes findings from the literature around four key assumptions about capacity building for enforcement and prosecution. Using the experiences and evidence summarized in this brief, program designers and implementing partners should be able to increase the effectiveness and efficiency of their capacity building efforts.

Thinking and Working Politically through Applied Political Economy Analysis. A guide for practitioners - Rocha Menocal, Alina; Cassidy, Marc; Swift, Sarah; Jacobstein, David; Rothblum, Corinne; Tservil, Ilona (USAID, 2018)
Thinking and Working Politically (TWP) through Applied Political Economy Analysis (PEA). A guide for practitioners Have you ever done everything right in a development program — followed every technical best practice — but still missed the mark? When this happens, it often relates to factors in the context beyond any external development actor’s ability to control.

Program Cycle - Discussion Note: Managing Monitoring, Evaluation, and Learning Platforms - USAID (USAID, 2018)
This Discussion Note complements ADS 201.3.1.2 Program Cycle Principles by elaborating on Principle 2: Manage Adaptively through Continuous Learning. This Discussion Note is intended for USAID staff interested in learning about recent and promising practices in adaptive management across the Program Cycle.

Being the Change. 12 ways foundations are transforming themselves to transform their impact - Ridgway, Abigail; Bockstette, Valerie; Seneviratne, Andria; Cain, Miya; Foster, Tracy (FSG, 2018)
Many foundations are adopting new approaches for creating social change—approaches that aim to influence the actions and investments of the public and private sector, as well as address the complex conditions that hold social problems in place.Based on interviews with 114 practitioners representing 50 foundations and 8 philanthropic services organizations, Being the Change explores how foundations are applying their assets, knowledge, skills, networks, and people in new ways in order to create impact at scale and change systems.

Managing to Adapt: Analysing adaptive management for planning, monitoring, evaluation, and learning - Desai, Harsh; Maneo, Gabriele; Pellfolk, Erica; Schlingheider, Annika (Oxfam, 2018)
Adaptive management is at the heart of ‘Doing Development Differently’. It emerges from stakeholders’ calls for development programmes to be more flexible and responsive to their contexts. Whether it becomes a mainstreamed practice depends on how much it is

Thinking big: How to use theory of change for systems change - Abercrombie, Rob; Boswell, Katie; Thomasoo, Rosanna (NPC, 2018)
We think that, applied well, theory of change can support charities and funders to take a systemic approach to their work. This report identifies five common pitfalls that organisations fall into when using theory of change, and walks through five rules of thumb that will help organisations to use the approach to tackle complex problems. We think that, applied well, theory of change can support charities and funders to take a systemic approach to their work. This report identifies five common pitfalls that organisations fall into when using theory of change, and walks through five rules of thumb that will help organisations to use the approach to tackle complex problems.

Thinking and working politically: Reviewing the evidence on the integration of politics into development practice over the past decade - Laws, Ed; Marquette, Heather (TWP Community of Practice, 2018)

OPM’s approach to assessing Value for Money - King, Julian; OPM (Oxford Policy Management, 2018)
This document offers practical guidance for assessing value for money (VfM) in international development programmes. Since 2016, evaluation expert Julian King has been working with OPM to develop and deploy a robust approach to VfM assessment. The approach combines cutting-edge evaluation practice with concepts from economic evaluation to respond to donor requirements for accountability and good resource allocation, as well as to support reflection, learning, and adaptive management.

Politically informed, gender aware programming: Five lessons from practice - Derbyshire, Helen; Gibson, Sam; Hudson, David; Roche, Chris (DLP Program, 2018)

From Silos to Synergy: Learning from politically informed, gender aware programs - Derbyshire, Helen; Siow, Orlanda; Gibson, Sam; Hudson, David; Roche, Chris (DLP Program, 2018)

The Whistler principles to accelerate innovation for development impact - G7 (G7, 2018)
The Whistler principles to accelerate innovation for development impact : 2018 G7 Summit – Canada 2018 G7 Presidency – Charlevoix, Quebec

DFID’s approach to value for money in programme and portfolio management - ICAI (Independent Commission for Aid Impact (ICAI), 2018)
ICAI published this review on DFID’s approach to value for money in February 2018, and as value for money is both a process and an outcome and cuts across all aspects of DFID’s operations, did not score this review. We made five recommendations and published a follow-up to this review in July 2019. All UK government departments are required to achieve value for money in their use of public funds. In recent years, DFID has been working to build value for money considerations further into its management processes and its relationships with implementers and multilateral partners, establishing itself as a global champion on value for money. Review This review was published in February. Though it was not rated, ICAI made five recommendations, and found that the Department for International Development’s approach to value for money was helping to make UK aid spending go further, but improvements are still needed. Findings This review found that DFID has strengthened its processes and systems for ensuring it gains maximum value for each pound spent, has taken swift remedial action to tackle under-performing programmes, and has become a strong global champion on value for money. The review also found that DFID has been diligent in its efforts to cut waste, detect fraud, and improve efficiency, and that this work is improving the return on the UK investment in aid. However, the review found that DFID’s approach was not adequately reporting and capturing results and value for money at the country portfolio level, or how programmes work together to deliver lasting impact, including reducing future dependency on aid. It also found that weakness in the annual review process could undermine DFID’s approach to value for money. It found that targets were frequently revised, and that there could be pressures for optimistic scoring of programmes. Recommendations Based on this review, we made the following five recommendations to help DFID improve its approach to value for money still further: DFID country offices should articulate cross-cutting value for money objectives at the country portfolio level, and should report periodically on progress at that level. Drawing on its experience with introducing adaptive programming, DFID should encourage programmes to experiment with different ways of delivering results more cost-effectively, particularly for more complex programming. DFID should ensure that principles of development effectiveness – such as ensuring partner country leadership, building national capacity and empowering beneficiaries – are more explicit in its value for money approach. Programmes should reflect these principles in their value for money frameworks, and where appropriate incorporate qualitative indicators of progress at that level. DFID should be more explicit about the assumptions underlying the economic case in its business cases, and ensure that these are taken into account in programme monitoring. Delivery plans should specify points in the programme cycle when the economic case should be fully reassessed. Senior responsible owners should also determine whether a reassessment is needed following material changes in the programme, results targets or context. Annual review scores should include an assessment of whether programmes are likely to achieve their intended outcomes in a cost-effective way. DFID should consider introducing further quality assurance into the setting and adjustment of logframe targets.

Program Cycle - Discussion Note: Adaptive Management - USAID (USAID, 2018)
This Discussion Note complements ADS 201.3.1.2 Program Cycle Principles by elaborating on Principle 2: Manage Adaptively through Continuous Learning. This Discussion Note is intended for USAID staff interested in learning about recent and promising practices in adaptive management across the Program Cycle.

Adapting to Learn and Learning to Adapt: Practical Insights from International Development Projects - Brinkerhoff, Derick W.; Frazer, Sarah; McGregor, Lisa (RTI Press, 2018)
Adaptive programming and management principles focused on learning, experimentation, and evidence-based decision making are gaining traction with donor agencies and implementing partners in international development. Adaptation calls for using learning to inform adjustments during project implementation. This requires information gathering methods that promote reflection, learning, and adaption, beyond reporting on pre-specified data. A focus on adaptation changes traditional thinking about program cycle. It both erases the boundaries between design, implementation, and evaluation and reframes thinking to consider the complexity of development problems and nonlinear change pathways.Supportive management structures and processes are crucial for fostering adaptive management. Implementers and donors are experimenting with how procurement, contracting, work planning, and reporting can be modified to foster adaptive programming. Well-designed monitoring, evaluation, and learning systems can go beyond meeting accountability and reporting requirements to produce data and learning for evidence-based decision making and adaptive management. It is important to continue experimenting and learning to integrate adaptive programming and management into the operational policies and practices of donor agencies, country partners, and implementers. We need to devote ongoing effort to build the evidence base for the contributions of adaptive management to achieving international development results.

Doing Accountability Differently: A 'Vertically Integrated' Approach - Amakom, Uzochukwu; Fashola, Temitope; Gay, Charles; Shutt, Cathy (Christian Aid, 2018)

Supporting innovation and the use of technologies in accountability initiatives: lessons from Making All Voices Count - Edwards, Duncan; Hudson, Hannah; Anderson, Colin; McGee, Rosemary; Brock, Karen (IDS, 2018)
Making All Voices Count was an international initative that harnessed the power of innovation and new technologies to support effective, accountable governance. Focusing on six countries in Africa and Asia, the programme was implemented by a consortium of implementing partners, and used funding from four donors to make grants to support new ideas that amplified the voices of citizens, and enabled governments to listen and respond. From the start, Making All Voices Count was also a learning programme. The objective of this learning was not only to bring about change during the programme’s life cycle, but also to leave a legacy that would help to ensure that future governance programmes and initiatives seeking to capitalise on the transformative potential of innovation and technology are more informed, inclusive and impactful. This programme learning report emerged from a wider process of analysing, discussing and synthesising the data and learning from the programme, which wove together evidence-based learning about technology for accountable governance initiatives with experiential learning on how best to support such work. The report highlights five of the lessons learned from Making All Voices Count about how – and how not – to run large, complex programmes that intend to support innovation in governance.

Reflection methods: Tools to make learning more meaningful - Gordijn, Femke; Eernstman, Natalia; Helder, Jan; Brouwer, Herman (Wageningen Centre for Development Innovation, 2018)
This handbook summarises methods that can be used to facilitate the process of reflection on the knowledge and experiences people acquire during a capacity development trajectory or training event. We believe that by explicitly integrating reflection in the learning process the learning will become clearer and better articulated and will contribute more strongly to meaningful change. Therefore we advise facilitators to deliberately include reflective learning sessions in their process design and implementation. This handbook can inspire you to do so and provides many methods which help to facilitate this.

Appropriating technology for accountability: messages from Making All Voices Count - McGee, Rosemary; Edwards, Duncan; Hudson, Hannah; Anderson, Colin; Feruglio, Francesca (IDS, 2018)
Making All Voices Count was a programme designed to solve the ‘grand challenge’ of creating more effective democratic governance and accountability around the world. It used funding from four donors to support the development and spread of innovative ideas for solving governance problems – many of them involving tools and platforms based on mobile phone and web technologies. Between 2013 and 2017, the programme made grants for innovation and scaling projects that aimed to amplify the voices of citizens and enable governments to listen and respond. It also conducted research and issued research grants to explore the roles that technology can play in securing responsive, accountable government and building an evidence base. This synthesis report reviews the Making All Voices Count’s four-and-a-half years of operational experience and learning. It shares 14 key messages on the roles technologies can play in enabling citizen voice and accountable and responsive governance. These messages are presented in four sections: (1) Applying technologies as technical fixes to solve service delivery problems; (2) Applying technologies to broader, systemic governance challenges ; (3) Applying technologies to build the foundations of democratic and accountable governance systems; (4) Applying technologies for the public ‘bad’, when tech can be dangerous for democracy. The tech optimism of the era in which the programme was conceived can now be reappraised from the better-informed vantage point of hindsight. Making All Voices Count’s wealth of diverse and grounded experience and documentation provides an evidence base that should enable a more sober and mature position of tech realism as the field of tech for accountable governance continues to evolve.

An introduction to systems thinking and tools for systems thinking - Allen, Will; Kilvington, Margaret (Learning fro Sustainability, 2018)
Systems thinking is an approach to integration that is based on the belief that the component parts of a system will act differently when isolated from the system’s environment or other parts of the system. THIS PAGE PROVIDES PLENTY OF RESOURCES ON SYSTEM THINKING

Making humanitarian response more flexible: bibliography - ALNAP (2018)

Combining Big Data and Thick Data to Improve Services Delivery - Ang, Yuen Yuen (IBM Center for The Business of Government, 2018)

Evaluating Systems Change Results: an inquiry framework - Cabaj, Mark (Tamarack Institute, 2018)

What's missing? Adding context to the urban response toolbox - Campbell, Leah (ALNAP/ODI, 2018)

2018 Map of the Complexity Sciences - Castellani, Brian (Durham University, 2018)
A map that provides a macroscopic, trans-disciplinary introductions to the complexity sciences.

Strengthening Advocacy and Civic Engagemetn in Nigeria - Chemonics (Chemonics International, 2018)

Follow the rabbit: Field Guide to Systemic Design - CoLab (Government of Alberta, 2018)
This Field Guide is designed to support budding systemic designers to facilitate and lead systemic design projects. It’s designed for systemic designers who have basic familiarity with SD concepts and are looking for some practical tips and tools to put theory into action. While this Guide was designed with Government of Alberta staff in mind, the content applies across a range of issue areas, sectors, and intersections. The Guide goes through a systemic design project from concept to implementation. It takes you through the workshop planning process, and discusses workshop roles and client relations. In the FAQs, you’ll find explanations to some commonly asked questions about systemic design concepts to help you introduce others to SD and bring them along with you. You’ll get the most out of this field guide if you approach it with a systemic design mindset – open, willing to try things, a desire to learn, and a ‘yes, and’ orientation. Think of this field guide as an early stop in your systemic design journey: it’s not the end of your learning – it’s a jumping off point. In the Additional Resources section, you can connect to more in-depth explorations of concepts and practical applications.

Adaptive Development Switchboard - Final Report (Internal Document) - Feedback Labs (LearnAdapt Programme, 2018)

Switchboard Survey Analysis (Internal Document) - Feedback Labs (LearnAdapt Programme, 2018)

FeverBee’s Online Community Strategy Template - FeverBee (FeverBee, 2018)

GIZ's Evaluation Policy: Principles, guidelines and requirements of our evaluation practice - GIZ (GIZ, 2018)

GIZ’s evaluation system - GIZ Evaluation Unit (GIZ GmbH, 2018)
This provides a general description of GIZ’s new evaluation system, which is designed to implement our new policy and to achieve the objectives of evaluation reform. This general description is accompanied by two other documents that provide supplementary details of GIZ’s key evaluation instruments: project evaluations for BMZ business and corporate strategic evaluations. They are geared in particular to GIZ staff members who commission, implement or support evaluations and to evaluators who conduct evaluations on behalf of GIZ as well as repre-sentatives of commissioning parties, clients and cooperation partners. The current versions are very much a ‘work in progress’ in many regards. We are well aware that we will only be able to meet our own high expectations of our evaluations if we continue to develop their quality in dialogue with project staff, our commissioning parties, clients and partners and with evaluation practitioners and experts from the academic, scientific and research community. We look forward to taking on this task and to sharing our experience with all interested parties.

Summer School Course - Adaptive Management - Working Effectively in the Complexity of International Development (weekplan) - Green, Duncan; Guijt, Irene (Oxfam, 2018)

Leaving No One Behind in a Digital World - Hernandez, Kevin; Roberts, Tony (Institute of Development Studies, 2018)

Measuring Up: Evaluating the Impact of P/CVE Programs - Holmer, Georgia; Bauman, Peter (United States Institute of Peace, 2018)
This report considers the various conceptual and practical challenges in measuring the impact and value of programs designed to prevent and counter violent extremism (P/CVE). It examines potential solutions and emphasizes the significance of efforts to assess changes in attitudes, behaviors, and relationships. The report was developed in tandem with “Taking Stock: Analytic Tools for Understanding and Designing P/CVE Programs” and seeks to help advance more rigor and sustainability in P/CVE programming.

Building a Learning Community in Liberia Through Partnerships at iCampus - iCampus (USAID LEARN, 2018)
Accountability Lab Liberia (ALab) and iLab Liberia established iCampus- a shared innovation, co-working and community space for organizations focusing on the intersection of technology, accountability and social change in Liberia. ALab implements DAI’s Learning activities at iCampus, and it is referred to as the Strategic Learning Partner of USAID Liberia Accountability and Voice Initiative (LAVI). LAVI is a five year USAID program with an overarching goal to strengthen multi-stakeholder partnerships to advocate for and monitor policy and accountability reforms. The iCampus CLA case scenario captures joint efforts of Accountability Lab Liberia, iLab Liberia and USAID LAVI, to ensure that learning and methodologies are shared and applied by development actors; address common capacity challenges that particularly hinder civil society from engaging in evidence-based and issues-based advocacy using various approaches, including network-building, creation of a cross-sectoral learning space- through collaboration and learning. In 2017, iCampus conducted a learning mapping survey to understand how organizations are learning in Liberia. The result indicated that frameworks and concepts of organizational learning are not yet well understood or used to guide strategic or programmatic practices, by either Liberian or international organizations. ALab, iLab and USAID LAVI then adopted a set of strategies, that led to the implementation of a CLA, enabling the establishment of the first multi-sectoral learning community in Liberia.

The water of Systems Change - Kania, John; Kramer, Mark; Senge, Peter (FSG, 2018)
Foundations involved in systems change can increase their odds for success by focusing on the least explicit but most powerful conditions for change, while also turning the lens on themselves. The Water of Systems Change aims to clarify what it means to shift these conditions. We offer the “inverted triangle” framework as an actionable model for funders and others interested in creating systems change, particularly those who are working to advance equity. Top Takeaways Systems change is about advancing equity by shifting the conditions that hold a problem in place. To fully embrace systems change, funders should be prepared to see how their own ways of thinking and acting must change as well. Shifts in system conditions are more likely to be sustained when working at three different levels of change: explicit, semi-explicit, and implicit.

The water of Systems Change: Action learning exercise - Kania, John; Kramer, Mark; Senge, Peter (FSG, 2018)
Based on the “inverted triangle” framework presented in The Water of Systems Change, this activity is designed to help individuals think systemically about social change, explore what is happening below the surface on issues they care about, and determine how they and their organizations can pursue large-scale change in a disciplined and holistic manner. The exercise is divided into 3 parts: Part I uses the “inverted pyramid” introduced in The Water of Systems Change to perform an external assessment of opportunities to make progress on the social or environmental issue you are focused on. Part II uses the same framework to consider internal conditions within yourself and your organization that should change in order for you to better support progress on your issue. Part III takes what was developed in Parts I and II and asks “What to do next?”

The landscape of innovation approaches - Leurs, Bas (States of Change, 2018)
An overview of innovation methods and approaches we’ve come across in government innovation practice.

Developing and Using Useful ToCs - Mayne, John (2018)
Developing useful theories of change (ToCs) for anintervention, especially complex interventions, is not straightforward nor something done in a couple of hourswith limited effort. Agreement is needed on a number of basic concepts and terms, key challenges need to be addressed, and there are numerous issues that need to be considered.Butthe effort is worth the investment of resources and time.Theory-based evaluation approaches need good ToCs.Some of the ideas here are discussed in the context of agricultural research for nutrition and health interventions by Mayne and Johnson (2015) (10) Developing and Using Useful ToCs. Available from: https://www.researchgate.net/publication/323868372_Developing_and_Using_Useful_ToCs [accessed Jun 13 2018].

Evaluation Building Blocks: A Guide - McKegg, K.; Oakden, J.; Wehipeihana, N.; King, Julian (Kinnect Group, 2018)
This guide presents the Kinnect Group’s approach to evaluation. Developed through ten years of collaborative work, it has been well tested. While there are many other guides to evaluation and different ways to do it, our approach emphasises the place of evaluative reasoning in evaluation. The guide draws on the work of many evaluation theorists, as well as our practice-based body of knowledge. We hope that using the building blocks as explained in the guide will help you in your endeavour to do credible and useful evaluation. We find this approach works for us. Our clients tell us it gives them meaningful and insightful findings that they can use to take action. Our view to intellectual property is simple. What you read is yours to use. We just ask that you cite the guide when you draw from it. For those of you who engage with this guide and use it in your practice, we look forward to hearing from you. We are particularly interested in feedback where you find aspects of the guide work well for you. In that way we can all continue to build and evolve our practice.

States of Change - Learning programme - Nesta (Nesta, 2018)

Dynamic gridlock: Adaptive Humanitarian Action in the Democratic Republic of Congo - Obrecht, Alice (ODI/ALNAP, 2018)

OECD Development Assistance Committee Peer Review of Canada - OECD (OECD, 2018)

Social Impact Navigator - The practical fuide for organizations targeting better results - Phineo (Phineo, 2018)

Review of Selected DFAT Facilities: Independent Consultant Report to DFAT, Key Findings - Pieper, Lynn (DFAT, 2018)
For the purposes of this report, a facility is defined as an aid delivery mechanism that provides flexible (adaptive and responsive) services managed in an integrated way. Objectives (or endof-facility outcomes) are specified, but the pathways to deliver them are left unspecified. The facility is a highly relevant model for delivering Australian aid effectively. Achievement of the Sustainable Development Goals depends on flexible approaches that work across sectors and that integrate significant cross-cutting themes such as gender equality and social inclusion. Efficient DFAT management of a $4 billion aid program requires a shift to fewer and larger initiatives, to make best use of limited internal management resources while still enabling policy and program choices that can have real impact. Facilities and other flexible delivery mechanisms offer significant potential for better development results. They can: enable collaborative and responsive partnership approaches to gain traction; allow activities to experiment and adapt based on progress, demand and contextual changes; and provide the opportunity for outcome-focused coherence across sectors, enabling the whole to be more than the sum of the parts. Facilities are not new, but they are different today. Facilities have existed since at least the mid-1990s. What is new is that some facilities today are very large; and the flexibility they enable is increasingly and consciously being used to strengthen links between the technical and the political, for more effective development results. This is a positive trend, but it adds enormously to their complexity and visibility, creates new risks, and has resulted in role confusion (especially in early years of implementation) between DFAT and its contractors.

Decision-Making and Data Use Landscaping [at DFID] - Powell, Josh; Orton-Vipond, Sarah; Bhatia, Vinisha; Kilroy, Annie (Development Gateway, 2018)
As DFID aims to harness the Data Revolution, ensuring that data1 drive decision-making, public accountability, and the achievement of the Sustainable Development Goals (SDGs), ensuring that systems, processes, and skills for data are aligned with these objectives is paramount. Across sector policy teams, country offices, and various analytical and technical cadres, different strengths and weaknesses, as well as needs and ambitions exist. To inform a strategic approach to data, as framed in its forthcoming Data Roadmap, DFID collaborated with Development Gateway to perform a Decision and Data Use Landscaping study. This report details lessons learned from approximately 60 interviews across 4 DFID country offices, all sector policy teams, senior managers, and various analytical and technical cadres and offices.

Snapshot of Adaptive Management in Peacebuilding Programs - Queen, Emily Forsyth; Baumgardner-Zuzik, Jessica; Hume, Elizabeth; Greenberg, Melanie (Alliance for Peacebuilding, 2018)
The Alliance for Peacebuilding developed the report Snapshot of Adaptive Management in Peacebuilding Programs with support from Humanity United. This report examines how existing organizational programs are designing and learning from adaptive management in a conflict effected and fragile states. It further provides practical recommendations for applying adaptive management to peacebuilding programs based on synthesis … Continue reading "Snapshot of Adaptive Management in Peacebuilding Programs"
Keywords: KEY

Exploring new territories for evaluation - Sharp, Cathy (Collective Leadership for Scotland, 2018)
Drawing on action research, this paper recasts evaluation as ‘action inquiry’, an embedded evaluative learning practice that can help navigate complexity when enacting collective leadership. It is offered as an invitation to inquiry amongst a reasonably well-informed audience of policy makers and practitioners who work in and for public services. It will particularly interest those who provide research, evaluation and facilitation support, and those seeking to develop a more relational approach to research and evaluation. Action inquiry is a model of practising change together in environments where ‘nothing is clear, and everything keeps changing’ that significantly challenges the prevailing discourse on evaluation. Action inquiry can be wrapped around and enmeshed within initiatives and programmes that work with complexity - anywhere where success will depend on the quality of relationships that can be developed. The paper reviews some important interrelated concepts that underpin the ideals of collective leadership and public service reform and which confront deeply embedded traditional notions of leadership, expertise and participation. These offer important challenges to ideas about how change happens and recognise that relationships are at the heart of practising change. This warrants a re-examination of the high expectations of evidence-based or informed practice. Collective leadership makes new demands of evidence as it rests on help to determine ‘wise actions’ in real-life situations. This confronts the practical reality of how to work together in conditions often expressed as ‘dynamic’ or ‘turbulent’ and the added human complexities of power, emotions and relationships; too often these elements are denied or avoided aspects of a change process. Facilitated action inquiry makes these elements part of the conversations, in the midst of ‘work-as-we-are-doing-it’, to increase areas of choice for individuals and a group as a whole. Within public policy, there is an overwhelming amount of evidence about ‘what works’ yet change seems to be stubborn and slow. The paper explores some of the deep-rooted vestiges of a ‘hierarchy of evidence’ and assumptions about standardisation and generalisability that act as a ‘barrier to transformation’. These include the narrow framing of what counts as evidence and consequent relegation of community perspectives, lived experience and practice-based evidence. The paper acknowledges the clear appetite for different approaches to evaluation, especially those that better reflect deeply held values and avoid creating a culture of ‘gaming’, rooted in fear of failure and loss of funding, at the expense of learning. The need for new forms of developmental evaluative thinking, collaborative inquiry and action research to create embedded learning is well overdue. Action inquiry is a desirable and necessary response to the complex situations and challenges of human services and recognises the essentialness of knowledge co-production. It is a model of co-creation at every stage and endorses the idea that people learn from participation in evaluation and by testing theories of change through action. Action inquiry builds on the idea of inquiry, or a moment-to-moment awareness and quality of attention and draws from several elements of action research practice. It sees inquiry as an intervention in itself, one that furthermore, explicitly seeks to enhance the probability of the success of a programme, focus on learning, the collaborative development of practice-based knowledge and positive relationships. The paper highlights the importance of building inquiry into living systems, the role of facilitation, systemic inquiry, and evaluative thinking. It proposes an expansion of ideas of appreciation as a relational and collaborative practice that is a driver of emergence. Social recognition that acknowledges someone’s social value to the community and implies mutual moral obligations to cooperation and participation is particularly crucial in a work context that requires successful coordination and multiple contributions to achieve results across hierarchies of position, professional rank and sectors. Hence, appreciation goes beyond the idea of positivity to include social recognition, valuing more explicit forms of inquiry, building participants’ aspirations to design new social systems and acting in new ways to embed change. In developing this discussion, the paper contributes to emerging dialogues about the need for a model of ‘5th generation evaluation’. Such a model would be based on the idea that appreciative and challenging inquiry that is contextual, relational and open-minded will create better opportunities for change and development. The paper sets out some ‘provocative propositions’ that can help us to navigate this terrain, perhaps of a fledgling ‘5th generation approach’ to inquiry. Facilitated action inquiry can hold the key to developing both new knowledge and an adaptive, collaborative and improvisational skill-set, able to respond in new ways to systemic and complex issues on the ground. It’s common to hear the expression ‘it’s all about relationships’ and it is clearly time to shift our focus to relationships; not relationships as ‘things’, but as co-created and dynamic relational processes in which we are embedded. In this way we can bring new qualities to our talking to each other about our various and shared visions of a better future.

Exploring new territories for evaluation - Provocative propositions - Sharp, Cathy (2018)

Tips on Learning from Context: Formal and Informal Approaches to Understanding the Local Political Economy - Swift, Sarah (USAID, 2018)
Successfully adapting your programmatic efforts and policy engagement to the context depends on maintaining your understanding of that context. Given this, continuous learning and feedback loops–not only about your programming, but also about the environment you are working within–are important components of USAID’s work. Since our environments are always changing, in ways big and small, this can be intimidating. One can imagine a scenario where learning consumes the limited time and budgets available, especially for small programs. The question becomes: How can we learn enough, at the right times, to limit missed opportunities and warning signs, while avoiding being consumed by data collection and analysis that never influences our work? This document capitalizes on the experiences of USAID staff as well as emerging learning from other donors and researchers to highlight a range of approaches–formal and informal, time-bound and continuous–that can support ongoing learning within the operational tempo of a busy Mission team or Operating Unit.

A Framework for Freshwater Ecosystem Management - UN Enviornment (2018)

Discussion Note: Complexity-Aware Monitoring - USAID (USAID, 2018)
This Discussion Note complements ADS 201 and outlines general principles and promising approaches for monitoring complex aspects of USAID development assistance. Complexity-aware monitoring is a type of complementary monitoring that is useful when results are difficult to predict due to dynamic contexts or unclear cause and-effect relationships. Key principles of the Program Cycle include applying analytic rigor, managing adaptively, and utilizing a range of approaches to achieve results. ADS 201.3.5.5 identifies three types of program monitoring – performance, context, and complementary. All USAID programming incorporates performance monitoring and should include context monitoring. Performance monitoring “is the ongoing and systematic collection of performance indicator data and other quantitative or qualitative information to reveal whether implementation is on track and whether expected results are being achieved.” Context monitoring is “[t]he systematic collection of information about conditions and external factors relevant to the implementation and performance of an operating units (OU’s) strategy, projects, and activities.” As stated in ADS 201.3.5.5, complementary monitoring may be used by missions and Washington OUs to complement performance and context monitoring in situations where results are difficult to predict due to dynamic contexts or unclear cause-andeffect relationships. This Discussion Note provides an explanation of when to use complementary monitoring approaches that are complexity-aware and summarizes the three principles of complexity-aware monitoring.

The Global Wildlife Program - Knowledge Platform - 2016-2018 - WB (The World Bank, 2018)

Case study - Lepra - Adaptive programming in “reaching the unreached” - UK Aid Direct (DFID, 2017)

Mission-Based Monitoring, Evaluation and Learning (MEL) Platforms Assessment Report - Arntson, Laura; Giannoni, Tonya; Peek, Nancy; Saarlas, Kristin (USAID, 2017)
In 2016, the Bureau of Policy, Planning and Learning (PPL) commissioned an internal stocktaking of USAID's mission-based MEL (Monitoring, Evaluation, and Learning) Platforms.

The Role of External Actors in Supporting Social and Political Action towards Empowerment and Accountability with a Focus on Fragile, Conflict- and Violence-Affected Settings - Christie, Angela; Burge, Richard (IDS and ITAD, 2017)
This paper explores the role and experience of external actors, particularly donors, in supporting social and political action in fragile, conflict and violence affected settings. Evidence is distilled from a wide range of synthesised sources to generate relevant findings and questions in relation to what we know and what we don’t. Included among the source material is a 2016 macro-evaluation of DFID’s empowerment and accountability (E&A) programmes which examined over 50 DFID funded projects. Themes which emerge relate to: how external actors need to think about the context and work politically; who are the most appropriate social and political actors to support in E&A; whether a direct or indirect approach to support for E&A achieves more tangible outcomes; whether external actors should move beyond short-term tools and tactics focused on one-sided engagement; and whether programmes should be designed around more strategic, multi-faceted interventions. The paper concludes with identifying a number of gaps in the evidence which are translated into a range of questions which could potentially inform the Action for Empowerment and Accountability (A4EA) research programme.

Independent Evaluation of SDC’s Results-Based Management System with a Focus on Poverty Reduction - Schmidt, Martin; Palenberg, Markus; Vähämäki, Janet (SDC, 2017)
The SDC's results-based management system ensures that the processes and instruments used for the design and implementation of programs and projects contribute to achieving the desired results (outputs, outcomes and impact). The independent evaluation examines the extent to which these processes and tools have fostered the results culture within the organization, and improved competencies for results-based management decisions, learning and communication. In addition to the findings, conclusions and recommendations, the evaluation report includes the management response of SDC’s Directorate.

The five trademarks of agile organizations - Wouter Aghina; Karin Ahlbäck; Aaron De Smet; Clemens Fahrbach; Christopher Handscomb; Gerald Lackey; Michael Lurie; Monica Murarka; Olli Salo; Elizabeth Seem; Jannik Woxholth (McKinsey anc Company, 2017)
Our experience and research demonstrate that successful agile organizations consistently exhibit the five trademarks described in this article. The trademarks include a network of teams within a people-centered culture that operates in rapid learning and fast decision cycles which are enabled by technology, and a common purpose that co-creates value for all stakeholders. These trademarks complement the findings from The McKinsey Global Survey Results: How to create an agile organization.

How INGOs are Doing Development Differently - Algoso, Dave (Care, IRC, Mercy Corps, Oxfam & World Vision, 2017)

Market Systems Development - Operational Guidance Note - DFAT (DFAT, 2017)

The Futures Toolkit: Tools for Futures Thinking and Foresight Across UK Government - HM Government (2017)

Managing Facilities: a stock-take from the first 12 months - Tyrrel, Lavinia; Teskey, Graham; de Lacy, Jacqui (Abt Associates, 2017)

Valuing peace: delivering and demonstrating mediation results - Dilemmas & options for mediators - Wadley, Ian (Centre for Humanitarian Dialogue, 2017)
Essential points for practitioners and donors • Mediation offers a cost-effective and proven method for resolving armed conflict. Between 1985 and 2015, 75 per cent of armed conflicts in the world were resolved through agreement rather than by force. In most cases these processes will have involved third party facilitation or support. • Professional mediators understand the high stakes involved in their work to prevent, mitigate and resolve armed conflict. In addition, they and their financial supporters are increasingly required to demonstrate ‘value-for-money’ to ensure continued funding. • However, traditional monitoring and evaluation (M&E) methods are not well suited to this task, typically imposing artificially linear project models on a dynamic conflict situation, as well as compliance reporting that moves attention away from real value. • Traditional M&E methods tend to focus on documenting the past and generating vast amounts of data, rather than enabling timely adaptation of the project in the present. • Traditional M&E approaches rely heavily on external evaluation consultants. Even in the best of cases this may interfere with the mediation process and impose a heavy time burden on the project team, leading to low acceptance of traditional M&E approaches by mediation practitioners. • In contrast, an ideal M&E approach for mediation should deliver useful insights in even the most dynamic and sensitive mediation environments, impose a light reporting burden, and be readily accepted by mediation teams. It should protect discretion and trust, enable rapid adaptation, and also provide some assurance that donor funds are being well spent.

Bridging Real-Time Data and Adaptive Management: Ten Lessons for Policy Makers and Practitioners - Ben Ramalingam; Inka Barnett; Kevin Hernandez; Panthea Lee; Anna Levy; Carrie Oppenheimer; Dennis Whittle; Craig Valters; Leni Wild (USAID, 2017)
The ongoing digital revolution has seen rapid growth in the deployment of technology enabled real-time data initiatives in international development and humanitarian work, developed with the goal of accelerating decision-making processes and enhancing aid effectiveness and efficiency.

Theories of Change for Promoting Empowerment and Accountability in Fragile and Conflict-Affected Settings - Green, Duncan (IDS, 2017)
This paper explores the current state of thinking among a range of aid actors (multilaterals, bilateral, applied scholars and international non-governmental organisations) on how to promote empowerment and accountability in fragile, conflict and violence affected settings. It seeks to identify trends, gaps and weaknesses in that thinking, and propose research questions and hypotheses to test. Three underlying sources of confusion are identified that are hindering progress on both understanding empowerment and accountability in fragile, conflict and violence affected settings, and taking helpful action to promote it. They are: Theory of endogenous change (e.g. on how empowerment and accountability arise in situ) versus the theory of action of an external intervention; Fragility versus conflict: there is no clear justification for combining these different aspects into a single category; and Empowerment versus accountability: donor analysis and practice has been overwhelmingly weighted towards accountability, exhibiting limited understanding or interest in the nature of power.
Keywords: A4EA; External actors; Participation

Doing Digital Development Differently: Lessons in adaptive management from technology for governance initiatives in Kenya - Prieto Martin, Pedro; Hernandez, Kevin; Faith, Becky; Ramalingam, Ben (IDS, 2017)
Development projects don’t always work as planned. This has long been acknowledged by those in the sector, and has led to several approaches that seek to solve complex development problems through enabling and encouraging greater adaptiveness and learning within projects (e.g. Doing Development Differently and Problem-Driven Iterative Adaptation). Digital development projects experience many of these issues. Using technology for transparency and accountability (Tech4T&A) projects in Kenya as case studies, our research analysed the many different theoretical approaches to learning and adaptation, and then tested how these play out in reality. Firstly, we conducted an extensive review of the literature on the spectrum of approaches to adaptive learning. The findings were used to develop a framework through which to analyse adaptiveness at the different layers of complexity in projects (e.g. software design and development, programme design and management). The second part of the research consisted of interviews and focus group discussions with participants in Tech4T&A projects in Kenya. Respondents helped us identify the main characteristics of adaptiveness in their projects (e.g. who needs to adapt, and how and when) and the challenges and issues that inhibit projects’ abilities to be adaptive. This process also revealed how accountability interplays with adaptiveness, and considered how better collaboration flows can enable adaptiveness. From our literature review and empirical study, we draw several conclusions for increasing adaptiveness in digital development projects. These include simplifying the many adaptive theories that abound, increasing responsiveness to project beneficiaries and stakeholders, and for stakeholders to keep on experimenting, networking and advocating.
Keywords: IMPORTANT

How to create an agile organization - survey - Salo, Olli; Ahlbäck, Karin; Fahrbach, Clemens; Murarka, Monica (McKinsey & Co., 2017)
Eighteen practices for organizational agility The survey asked respondents about a series of specific actions that underlie each of the 18 practices (9 of them stable, and 9 dynamic) of organizational agility; all of the practices are summarized in the table below. To rate respondents’ organizations, we asked how frequently their performance units engaged in each action that supports a given practice.

Real World SAFe – Leapfrogging a successful waterfall company into Scaled Agile - Skarin, Mattias (CRISP, 2017)

An introduction to the Learning Architecture - HLA (Humanitarian Leadership Academy, 2017)
The Learning Architecture is built on a flexible Learning Methodology and provides principles, technologies and behaviours of Scaffolded Social Learning.

Developmental Evaluation in Practice: Tips, Tools, and Templates - USAID (USAID, 2017)

How to Make ‘Cash Plus’ Work: Linking Cash Transfers to Services and Sectors - Roelen, Keetie; Devereux, Stephen; Abdulai, Abdul-Gafaru; Martorano, Bruno; Palermo, Tia; Ragno, Luigi Peter (UNICEF Office of Research, 2017)
The broad-ranging benefits of cash transfers are now widely recognized. However, the evidence base highlights that they often fall short in achieving longer-term and second-order impacts related to nutrition, learning outcomes and morbidity. In recognition of these limitations, several ‘cash plus’ initiatives have been introduced, whereby cash transfers are combined with one or more types of complementary support. This paper aims to identify key factors for successful implementation of these increasingly popular ‘cash plus’ programmes, based on (i) a review of the emerging evidence base of ‘cash plus’ interventions and (ii) an examination of three case studies, namely, Chile Solidario in Chile, IN-SCT in Ethiopia and LEAP in Ghana. The analysis was guided by a conceptual framework proposing a menu of ‘cash plus’ components. The assessment of three case studies indicated that effective implementation of ‘cash plus’ components has indeed contributed to greater impacts of the respective programmes. Such initiatives have thereby addressed some of the non-financial and structural barriers that poor people face and have reinforced the positive effects of cash transfer programmes. In design of such programmes, further attention should be paid to the constraints faced by the most vulnerable and how such constraints can be overcome. We conclude with recommendations regarding the provision of complementary support and cross-sectoral linkages based on lessons learned from the case studies. More research is still needed on the impact of the many variations of ‘cash plus’ programming, including evidence on the comparative roles of individual ‘plus’ components, as well as the knowledge, attitudes and behaviour pathways which influence these impacts.
Series: Innocenti Working Papers Volume: 2017/10

Strengthening the results chain: Synthesis of case studies of results-based management by providers - Zwart, Rosie (OECD, 2017)
This paper presents an analysis of evidence from seven case studies of results-based management by development co-operation providers. Analysis of themes from the case studies reveal six interrelated challenges which providers face in their efforts to manage for the results of development co-operation. The six challenges are: linking results to development goals, ensuring the purpose of results systems is well-defined, weighing up the benefits of aggregating and attributing results from standard indicators, enabling country ownership, using performance information alongside results information, and building and embedding a results culture. Providers continue to adopt a range of approaches to pursue a focus on results and there are many examples of good practice. The case studies suggest that in a complex development landscape some providers are prioritising the use of aggregated results information for domestic accountability, communication, and performance management over their use for learning and policy direction. In this context, the paper argues that in agency-wide results frameworks development co-operation results are often detached (or de-contextualised) from development results and discusses the implications of this, including the use of standard indicators to measure aggregated outputs. The paper uses case study evidence to discuss and suggest practical ways providers can build and maintain a strong results focus which enables analysis, insights and learning, and has achievement of development outcomes as its objective.

Supporting accountability in fragile settings: A review for the Somalia Implementation and Analysis in Action of Accountability Programme - Nixon, Hamish; Buffardi, Anne; Wales, Joseph; Pasanen, Tiina (ODI, 2017)

From Design Thinking to Systems Change - Conway, Rowan; Masters, Jeff; Thorold, Jake (RSA, 2017)
This report takes the Small Business Research Initiative (SBRI), a twophase pre-procurement innovation programme that aims to match social challenges with new ideas, as its primary case study. It suggests augmenting the excellent design thinking deployed through SBRI with a think like a system, act like an entrepreneur lens in order to drive better social outcomes from SBRI-originating innovations. Programmes like SBRI have great potential to drive change and address pressing challenges, but must be guided by a more developed understanding of how change happens. The stakes are too high to not raise our game when it comes to social innovation. Wicked problems can be overcome but will require sophisticated theories of change able to account for the complexity and unpredictability of modern life. We offer think like a system, act like an entrepreneur as a contribution to this effort. This report is based on the following research: •• Desk based review of relevant literature; •• A series of structured interviews with participants in SBRI competitions; and •• An expert round table that brought together policymakers, commissioners, procurement experts and entrepreneurs to consider and develop initial findings.

How to set up and manage an adaptive programme - Lessons from the Action on Climate Today (ACT) Programme - Cooke, Katherine (OPM, 2017)

Equitable Evaluation Framework™ Framing Paper - EEI (Equitable Evaluation Inititiative, 2017)

Governance for Growth in Vanuatu: Review of a decade of thinking and working politically - Hadley, Sierd; Tilley, Helen (ODI, 2017)

Incorporating Geospatial Analysis into USAID Biodiversity Program Design - Higgins, Mark; Salafsky, Nick; Adeney, Marion; Petrova, Silvia (USAID, 2017)
Incorporating Geospatial Analysis into USAID Biodiversity Program Design is a biodiversity case example that describes the use of geospatial data and analysis for program design, including examples and lessons learned.

Thinking and Working Politically: Lessons from FOSTER in Nigeria - Lopez Lucia, Elisa; Buckley, Joanna; Marquette, Heather; McCulloch, Neil (DLP Program, 2017)
The Facility for Oil Sector Transparency and Reform (FOSTER) was a £14 million programme that has helped Nigeria to transform its governance of the oil and gas industry. FOSTER ran from 2011 to mid-2016, and used an explicit ‘thinking and working politically’ (TWP) approach. It was funded by the UK Department for International Development (DFID) and managed by Oxford Policy Management (OPM). This paper seeks to identify the factors that drove – or constrained – FOSTER’s achievements, and asks what this can tell us about TWP, particularly in challenging political and sectoral contexts. FOSTER sought to help strengthen oversight and accountability in Nigeria’s oil sector. It aimed to support reformers within government institutions that ‘supply’ accountability (those governing how oil and gas revenues are collected and managed), and to support civil society organisations, parliament, the media and the Nigeria Extractive Industries Transparency Initiative to ‘demand’ reform. In addition, FOSTER commissioned a series of studies and provided media training to help broaden understanding of the sector and highlight the need for reform. This paper examines the outcomes from five ‘clusters’ of FOSTER interventions. The FOSTER team viewed the first three of these in the list below as successful, whereas the latter two did not deliver on expectations but offer important opportunities for learning: • Support to the Nigerian Extractive Industries Transparency Initiative • Support to demand-side actors to promote transparency and accountability in the oil sector • Support for the passage of the Petroleum Industry Bill • An oil spill-mapping social media project • Support to the Department of Petroleum Resources The analysis drew on 44 semi-structured interviews conducted during a month of fieldwork in Abuja and Lagos; on reviews of the programme’s monitoring, evaluation and learning frameworks; and on newspaper articles and grey literature on Nigeria’s oil sector.2

Case studies of results-based management by providers: Canada - OECD (OECD, 2017)

Case studies of results-based management by providers: World Bank Group - OECD (OECD, 2017)

How-To Note: Developing a Project Logic Model (and its Associated Theory of Change) - USAID (2017)
This How-To Note describes considerations for developing a project logic model, as well as steps for thinking through a more complete theory of change (TOC). A logic model is a graphic or visual depiction that summarizes key elements of a TOC, and it is often used as a facilitation tool during the design process. There are many types of logic models, including but not limited to logical frameworks (logframes), results chains, results frameworks, and local actor-oriented models, among others. The project logic model and its associated TOC are included in the Project Appraisal Document (PAD) that approves a project design (see ADS 201.3.3.13). While this How-To Note focuses on logic models at the project level, logic models are also used at the strategy level (specifically, results frameworks – see Box 1), and often at the activity level. The concepts and steps presented here are generally applicable to the process of developing logic models and TOCs throughout the Program Cycle.

Program Cycle - How-To Note: Strategy-Level Portfolio Review - USAID (USAID PPL, 2017)

The Beginner’s Guide to Political Economy Analysis (PEA) - Whaites, Alan (National School of Government International, 2017)
Over the last two decades aid agencies and academics have been on a journey of lesson learning and adaptation in relation to `politics.’ This journey has been driven by a determination to improve impact in all areas of development, but for some time it was particularly associated with work on public sector reform. Now, however, there is an increasing expectation that Political Economy Analysis (PEA) should be part and parcel of designing and implementing any programme or activity (and a brief history of the meandering journey of development actors on PEA can be found in The Policy Practice’s Briefing Paper 11 – see below). DFID in the UK is fairly typical among large development organisations in running an excellent course on political economy analysis, complete with 200 pages of resources and various online videos and case studies (and this type of course is recommended for those who want to take their exploration of PEA further). Even so, PEA is not just for those who have `done the course and bought the T-shirt,’ it is something that can be absorbed and implemented quickly by everybody. Indeed, the growth of interest in PEA is a reminder that this can look like a complex and daunting field and so this guide aims to offer an entry-point for all those who want to use PEA in their own work. In doing so, this guide borrows from the best materials that are available while also adapting some approaches by incorporating wider ideas on politics and institutions. This guide affirms that there should never be an official `orthodoxy’ for PEA and so the emphasis here is on questions, prompts and ideas to help thinking and practice. There is also an attempt to clarify jargon wherever needed, while recognising that The Policy Practice (TPP) and the Overseas Development Institute (ODI) have produced a more complete glossary of PEA terminology. The note will instead focus on `the essentials’ of PEA as they relate to the following questions:  Why do we do political economy analysis, and what is it?  What kinds of issues and ingredients are often included in a PEA?  How do we make sense of the different varieties of PEA?  What tools are out there to help us conduct a PEA?  What is thinking and working politically?

External validity and policy adaptation. From impact evaluation to policy design - Williams, Martin J. (Blavatnik School of Government, University of Oxford, 2017)
With the growing number of rigorous impact evaluations worldwide, the question of how best to apply this evidence to policymaking processes has arguably become the main challenge for evidence-based policymaking. How can policymakers predict whether a policy will have the same impact in their context as it did elsewhere, and how should this influence the design and implementation of policy? This paper introduces a simple and flexible framework to address these questions of external validity and policy adaptation. I show that failures of external validity arise from an interaction between a policy’s theory of change and a dimension of the context in which it is being implemented, and develop a method of “mechanism mapping” that maps a policy’s theory of change against salient contextual assumptions to identify external validity problems and suggest appropriate policy adaptations. In deciding whether and how to adapt a policy in a new context, I show there is a fundamental informational trade-o↵ between the strength and relevance of evidence on the policy from other contexts and the policymaker’s knowledge of the local context. This trade-o↵ can guide policymakers’ judgments about whether policies should be copied exactly from elsewhere, adapted, or invented anew.

RFI-521-17-000015 - Improving Governance in Haiti Program (IGHI) - USAID (USAID, 2017)

Systems Thinking Toolkit - Parkhurst, Marcie (FSG, 2017)
A complete compilation of our guides to system mapping.

Bridging the Gap: How Real-Time Data Can Contribute to Adaptive Management in International Development - Ben Ramalingam; Inka Barnett; Kevin Hernandez; Panthea Lee; Anna Levy; Carrie Oppenheimer; Dennis Whittle; Craig Valters; Leni Wild (USAID, 2017)
The ongoing digital revolution has seen rapid growth in the deployment of technology enabled real-time data initiatives in international development and humanitarian work, developed with the goal of accelerating decision-making processes and enhancing aid effectiveness and efficiency.

Responding to complexity: A Case Study on the Use of “Developmental Evaluation for Managing Adaptively” - Boisvert, Kayla (University of Massachusetts, 2017)

What works for Social Accountability? Findings from DFID’s Macro Evaluation - Hughes, Claire (Itad, 2017)

A Whole New World - Funding and Commissioning in Complexity - Knight, Annabel Davidson; Lowe, Toby; Brossard, Marion; Wilson, Julie (Collaborate & Newcastle University, 2017)
Collaborate and Newcastle University Business School Publish Research into Complexity-Friendly Funding

Learning, monitoring and evaluating: achieving and measuring change in adaptive programmes - Callaghan, Sarah; Plank, Georgia (DFID-LASER Programme, 2017)

Why isn’t Tech for Accountability Working in Africa? - de Lanerolle, Indra (SAIIA, 2017)
Expanding mobile networks and falling costs could transform communication between African citizens and governments. So far, however, attempts to harness new technologies to improve transparency and accountability in Africa and elsewhere have had disappointing results. What is going wrong? Research suggests that an important reason for this failure is a poor understanding of technologies and limited skills in developing and using them. It seems that civil society organisations (CSOs) and governments often ‘re-invent the flat tyre’: experimenting with new tools without finding out what has been tried (often unsuccessfully) before. They also do not follow best practices in how to source, develop and test technologies to ensure these are ‘fit for purpose’. Decision makers should focus on building an effective innovation ecosystem with better links between technologists and accountability actors in both government and civil society to enable learning from successes – and mistakes.
Keywords: IMPORTANT

Case studies of results-based management by providers: New Zealand - OECD (OECD, 2017)

Case studies of results-based management by providers: Sweden - OECD (OECD, 2017)

Case studies of results-based management by providers: Switzerland - OECD (OECD, 2017)

Case studies of results-based management by providers: The Netherlands - OECD (OECD, 2017)

Case studies of results-based management by providers: United Kingdom - OECD (OECD, 2017)

Combating Wildlife Trafficking Cross-Mission Learning Agenda - Present, Teresa; Peabody, Shawn (USAID, 2017)

Amateur hour: CfC's 'surprising' success in addressing school congestion in the philippines - Sidel, John T. (Asia Foundation, 2017)
The international development community has increasingly embraced the idea that finding durable solutions to complex development problems requires new ways of working that move beyond industry norms. This paper makes an important contribution to the current debate by outlining an innovative monitoring system called Strategy Testing (ST). This is the third paper in the Working Politically in Practice paper series, launched together with the Australian Department of Foreign Affairs and Trade.
Keywords: IMPORTANT

Conservation Enterprises: Using A Theory of Change Approach to Synthesize Lessons from USAID Biodiversity Projects - Boshoven, Judy (USAID, 2017)
In an effort to increase the understanding of conservation enterprises’ activities and outcomes and to improve the effectiveness of biodiversity programming, this brief synthesizes lessons from past USAID-funded efforts to support conservation enterprises

Causal Link Monitoring - Britt, Heather; Hummelbrunner, Richard; Greene, Jacqueline (2017)
Causal Link Monitoring (CLM) 1 integrates design and monitoring to support adaptive management of projects. CLM helps project planners and managers identify the processes that are required to achieve desired results, and then to observe whether those processes take place, and how. Result-producing processes specify the causal links between results in a logic model or results framework—in other words, the processes between results.2 CLM focuses on how specific individuals or organizations use results to achieve other results. In CLM, planners start by creating a logic model to help document predictable, agreed-upon elements of the project. Next, they refine the causal links by describing the processes that will transform results at one point in the causal chain to the next. Planners are often less certain about these resultproducing processes. Finally, the CLM logic model is enhanced with information about two important sources of uncertainty, contextual factors that may influence the project and diverse perspectives on the problem and its solution.

"That's how the light gets in": Making Change in Closing Political Environments - IBP (International Budget Partnership, 2017)
This collection of learning and reflection essays from the International Budget Partnership's 2016 Annual Report illustrate the multidimensional nature of budget work and democratic engagement. Read more.

The Boda Boda Talk Talk Module - Internews (Internews & USAID, 2017)
Keywords: IMPORTANT

Evidence Base for Collaborating, Learning, and Adapting: summary of the literature review (2017 update) - Learning Lab (USAID, 2017)
LEARN and USAID/PPL are managing an area of work known as Evidence Base for Collaborating, Learning, and Adapting (EB4CLA). The purpose of this work is to answer key learning questions:

(Draft) Managing to Adapt - Analysing Adaptive Management for Planning, Monitoring, Evaluation, and Learning (draft) - Schlingheider, Annika; Pellfolk, Erica; Maneo, Gabriele; Desai, Harsh (Oxfam, 2017)
Adaptive management is at the heart of ‘doing development differently’ (Wild et al., 2016). Whether it is here to stay depends on how much it is mainstreamed into existing development programming by donors and implementers alike, especially in planning, monitoring, evaluation, and learning (PMEL) cycles. In this report, we find that mainstreaming adaptive management in PMEL involves three strategies: 1. planning for flexibility; 2. developing locally owned monitoring and evaluation (M&E); and 3. creating an enabling environment for learning. Adopting these strategies contributes to virtuous cycles of PMEL

Re-Perceiving Results: Aid Programs and Change in Fragile Societies - van Veen, Erwin; Rijper, Alies (Knowledge Platform Security & Rule of Law, 2017)
Aid programmes need to be able to adapt their objectives and operations to changes in their political environment, since development processes are subject to political contestation. Change takes time and is often a matter of seizing the opportunity.
Keywords: IMPORTANT

Refining advocacy assessment: reflections from practice - Buffardi, Anne; Hearn, Simon; Tilley, Helen (ODI, 2017)
• This paper revisits how existing advocacy evaluation frameworks classify people and activities, and define and assess outcomes. We discuss how assessments could be more specific and propose bounding the scope of inquiry in one of four ways: strategy specific, outcome oriented, actor-centric or system-wide. • In classifying activities, the same action or event may be situated at different phases of the change pathway – in some cases used as a tactic to influence a policy outcome, and in others an intended outcome itself. • Because advocacy is more relational than other types of more technical development interventions, there will be fewer sources of directly observable data, and the direction of potential bias may be unknown. • In terms of learning, advocacy initiatives are contextually dependent, therefore lessons may be less directly transferable to subsequent phases of an initiative or to other settings. Organisations have bounded repertoires and the transferability of skill sets is limited, so advocates adapt how and with whom they engage more than what they do

Causality and attribution in market systems development - Jenal, Marcus; Liesner, Mollie (BEAM, 2017)
Resources that contain evidence of market systems interventions. Recently updated.

Landscape Analysis of Learing Agendas: USAID/Washington and beyond - Learning Lab (USAID, 2017)
EXECUTIVE SUMMARY U.S. government agencies and development organizations around the world are increasingly recognizing and supporting learning activities— often informed by a learning agenda—as important tools for improving organizational effectiveness and efficiency. A learning agenda comprises a set of broad questions directly related to the work that an agency conducts; when answered, they enable the agency to work more effectively and efficiently, particularly pertaining to evaluation, evidence, and decisionmaking. This report surveys the landscape of learning agendas at USAID and beyond to inform the learning agenda initiative planned by the Office of Learning, Evaluation, and Research (LER) in the Bureau for Policy, Planning and Learning (PPL). Reporting on research conducted over a 6-month period (September 2016 to February 2017), this landscape analysis focuses on six documented, office-, bureau-, or initiative-wide learning agendas being used at USAID/Washington. It also considers 11 new or noteworthy learning initiatives at USAID and five learning efforts at other federal agencies. The research team interviewed 60 staff from 20 USAID offices and bureaus, as well as staff from five other federal agencies. (See Annexes 1 through IV for more information.) The team also convened two focus groups: one with staff in USAID/Washington who had recently returned from serving in Missions, and one with PPL staff who discussed that bureau’s learning activities. The report details the interviewees’ remarks on six topics:  Motivations behind their decisions to pursue a learning agenda, such as expectations of accountability, especially in response to leadership demands  The key benefits emerging from their learning agenda efforts, which have included identifying gaps in knowledge and evidence, and supporting other cultural and organizational change processes  The participatory and consultative strategies they used to engage with stakeholders, including engagement with Mission staff and inclusion of academics and outside experts  The learning activities and products related to their initiatives  Challenges and strategies on resources, dissemination, and utilization, including ways to update their learning agenda to ensure that it remains a “living document”  Recommendations they had for PPL and others considering embarking on a learning agenda initiative

Thinking and Working Politically – Are We Seeing the Emergence of a Second Orthodoxy? - Teskey, Graham (Abt Associates, 2017)
There now is a persuasive volume of evidence that demonstrates that capacity and technical knowledge alone are insufficient to change deeply entrenched political interests and bureaucratic norms. These critiques demonstrate that an understanding of power asymmetries is frequently the critical missing ingredient in project design and implementation. Many eminent thinkers have looked at the difference between success and failure in development, and all point to the primacy of domestic politics. This point has not been lost on development agencies and some have tried to provoke greater attention to the role that politics plays. However, this recognition is yet to pass into the mainstream of development practice. Despite the slow but sure accretion of this knowledge the international community seems to be wedded to doing development traditionally. The purpose of this paper is to consider the extent to which a ‘second orthodoxy’ is slowly emerging which can stand alongside, and in some cases may supplant, the ‘first orthodoxy’ of the traditional project framework. Care has to be taken in making this argument, as there is certainly no consensus within the development community on its importance, and even among advocates, there are slightly differing interpretations of what this second orthodoxy looks like. Further, it is not absolutely clear that the two orthodoxies can co-exist. The paper does not call for any ‘paradigm shift’ in how development practitioners conceptualise programs and projects, nor does it demand that the project framework be scrapped; the former would be impossible and the latter undesirable. Equally readers will find no claims that Doing Development Differently or ‘Thinking and Working Politically’ will always and everywhere guarantee better development outcomes. Rather, the paper seeks to summarise how the two orthodoxies differ, where the second orthodoxy now stands, and propose how it can be taken forward in practical terms.

Project Cycle Hacker's Toolkit - changing the conversation around your projects - UNDP (UNDP, 2017)
The Istanbul Regional Hub partnered with Nesta and 4 country offices (fYR Macedonia, Moldova, Georgia and Kyrgyzstan) to develop a program that can help us bring innovation closer to the center of the organization by embedding it in the key project management business processes - The Project Cycle Hackers Toolkit.

Measuring Efforts to Combat Wildlife Crime - A toolkit for Improving Actiona and Accountability (v.1.3) - USAID (USAID, 2017)

Innovating in Development Learning Event - Challenge brief - Vogel, Isabel (Ideas to Impact / iMC, 2017)

Digital Democracy: The Tools Transforming Political Engagement - Simon, Julie; Bass, Theo; Boelman, Victoria; Mulgan, Geoff (NESTA, 2017)
This paper shares lessons from Nesta’s research into some of the pioneering innovations in digital democracy which are taking place across Europe and beyond

Innovation in development: Sharing learning, improving impact - Vogel, Isabel; Minkley, Gabrielle; Chowdhury, Sajid (Ideas to Impact / IMC Worldwide, 2017)

Putting theory into practice: how DFID is doing development differently - Wild, Leni; Booth, David; Valters, Craig (ODI, 2017)
Keywords: IMPORTANT

ADS 201 Additional Help- Whole-of-Project Evaluation - USAID (USAID, 2017)

USAID ADS 201 - Program Cycle Operational Policy (Update 1/23/2017) - USAID (USAID, 2017)
The Program Cycle is USAID’s operational model for planning, delivering, assessing, and adapting development programming in a given region or country to advance U.S. foreign policy. It encompasses guidance and procedures for: 1) Making strategic decisions at the regional or country level about programmatic areas of focus and associated resources; 2) Designing projects and supportive activities to implement strategic plans; and 3) Learning from performance monitoring, evaluations, and other relevant sources of information to make course corrections as needed and inform future programming.

Learning to Target for Economic Diversification: PDIA in Sri Lanka - Andrews, Matt; Ariyasinghe, Duminda; Batuwanthudawa, Thamari; Darmasiri, Shivanthika; Silva, Nilupul; Harrington, Peter; Jayasinghe, Prasanna; Jayasinghe, Upul; Jayathilake, Gamini; Karunaratne, Jayani; Katugampala, Lalit; Liyanapathirane, Jeewani; et al. (Center for International Development, Harvard University, 2017)

Political Economy Analysis – Guidance for legal technical assistance - Denney, Lisa; Domingo, Pilar (ROLE UK, 2017)
Political economy analysis is about understanding how change in relation to rule of law and international development is embedded within and shaped by political and economic relations that interact and are particular to each context. These political economy dynamics determine the distribution of power and resources within any given society and must be taken into account when attempting to achieve change. Political economy analysis is important in the justice sector because rule of law and dispute resolution processes reflect the rules of the game about who wins and who loses in the allocation of power and resources. Using political economy analysis in legal technical assistance can help to ensure more relevant and effective assistance that sustainably addresses concrete problems. It can also help to avoid the risk that poorly targeted assistance might inadvertently exacerbate existing power structures in a way that was not intended.

Integrating Livelihood and Conservation Goals: A retrospective Analysis of World Bank Projects - Gómez, Andrés; Price, Claire (USAID, 2017)
Using a database of World Bank project evaluations, E3/FAB’s new study called “Integrating Livelihood and Conservation Goals: A Retrospective Analysis of World Bank Projects” aims to contribute to building the evidence base around the integration of biodiversity conservation and livelihood goals.

Evaluation at IDRC - IDRC (IDRC, 2017)

Systems Practice - Omidyar Group (The Omidyar Group, 2017)

Beyond Logframes - Simister, Nigel (Intrac, 2017)
The logical framework, otherwise known as a logframe, is a commonly used planning tool. Initially designed for use within simple projects, it is increasingly being applied to complex programmes and organisations. In these cases the logical framework has limitations. There are several options that can help overcome these limitations. These include expanding the logframe, and using multiple logframes.

DFID Data Disaggregation Action Plan: Better Data for Better Lives - DFID (2017)

DFID Smart Rules: Better Programme Delivery - version VII (2017.04) - DFID (DFID, 2017)
Smart Rules provide the operating framework for the Department for International Development’s (DFID’s) programmes.
Keywords: IMPORTANT; Practice

DFID Smart Rules: Better Programme Delivery - version VIII (2018.04) - DFID (DFID, 2017)
Smart Rules provide the operating framework for the Department for International Development’s (DFID’s) programmes.
Keywords: IMPORTANT; Practice

Adaptive Programming and Business Environment Reform – Lessons for DFID Zimbabwe - Donovan, Elbereth; Manuel, Clare (DFID, 2017)
Keywords: IMPORTANT

Canada's Feminist International Assistance Policy - GAC (Global Affairs Canada, 2017)

One step to a thousand miles: building accountability in Liberia - Gilberds, Heather (Accountability Lab Liberia, 2017)
Lessons for enabling accountability and integrity in Liberia

Voice or chatter? Making ICTs work for transformative engagement: research report summary - Gurumurthy, Anita; Bharthur, Deepti; Chami, Nandini (IDS, 2017)
What are the conditions in democratic governance that make information and communication technology (ICT)-mediated citizen engagement transformative? While substantial scholarship exists on the role of the Internet and digital technologies in triggering moments of political disruption and cascading upheavals, academic interest in the sort of deep change that transforms institutional cultures of democratic governance, occurring in ‘slow time’, has been relatively muted. This study attempts to fill this gap. It is inspired by the idea of participation in everyday democracy and seeks to explore how ICT-mediated citizen engagement can promote democratic governance and amplify citizen voice. The study involved empirical explorations of citizen engagement initiatives in eight sites – two in Asia (India and Philippines), one in Africa (South Africa), three in South America (Brazil, Colombia, Uruguay) and two in Europe (Netherlands and Spain).

The International Development Innovation Alliance - IDIA (R4D, 2017)
Brochure from the International Development Innovation Alliance

We Rise - Movement Building Reimagined - JASS (Just Associates, 2017)
In our virtual toolkit, We Rise: Movement Building Reimagined, we describe in detail four interconnected cycles of movement building – Rising Up, Building Up, Standing Up, and Shaking Up. At the heart of these cycles lies an analysis of how to challenge and transform power. Each cycle features key ideas, relevant tools, and practical tips from our work and activists in our network that movement builders around the world can learn from and adapt in their own work.

Changing humanitarian action? - Knox-Clarke, Paul (ALNAP/ODI, 2017)
We live in times of profound change. This has had a great impact on humanitarian needs, and the approaches taken to meet these needs. Changes in technology, ecology, politics, economics and demographics have shaped, and will continue to shape, humanitarian action. Many humanitarians and observers of humanitarian action have suggested that change initiatives in the sector have been unambitious and unsuccessful. Indeed, many people think that the humanitarian system is unable, or unwilling, to make the changes that are required to remain effective and relevant in a rapidly changing world. Following the World Humanitarian Summit many stakeholders have made commitments to significant changes. But can the system change? And if so, how? As a network which exists to support change and improvement, it seems appropriate that ALNAP meets to consider how change occurs in the humanitarian system, and how it can be done better. Find out more about the 31st Annual Meeting by reading the concept note, or by visiting alnap.org/31am.

An analysis of what CLA looks like in Development Programming - Learning Lab (USAID, 2017)

Toybox's Adaptive Management in Practice - Malcolm, Emily (Toybox, 2017)
At Toybox, we understand ‘Adaptive Management’ as a broad innovative and learning approach that involves the whole organisation, our partners and funders. ‘Adaptive Programming’ of projects is part of this approach which is greatly enhanced by the Adaptive Management of all actors, including communities and children in the projects. Over the last few years Toybox have been piloting Adaptive Programming with our partners across Latin America, Africa and Asia to support children in street situations. This project has been developed within the wider context of an organisation which is focused on innovation and learning, therefore enabling Adaptive Management to become mainstream within the organisation. This paper captures the experiences and learnings of Toybox to our newly implemented Adaptive Programming approach, through the lens of a small-sized UK-based organisation working with local organisations internationally. This paper explains our learnings, inclusive of why we believe Adaptive Management is important to Toybox and how the organisation has embraced this new approach.

How can we be more adaptive? AdaptScan identifies critical areas for improvement - Mercy Corps (Mercy Corps & IRC, 2017)
Keywords: IMPORTANT

Adaptive Leadership in Development course - Moran, Mark; Curth-Bibb, Jodie; Grice, Tim; Glavey, Sarah (University of Queensland, 2017)

Evaluating Humanitarian Innovation - Obrecht, Alice; Warner, Alexandra; Dillon, Neil (ALNAP, 2017)

Citizen participation and the rise of the open source city in Spain - Peña-López, Ismael (MAVC & IT for Change, 2017)
Research on the Information Society, the Digital Divide and Information and Communication Technologies for development

Bridging Real-Time Data and Adaptive Management in International Devepment - Case Study Report - Ramalingam, Ben; Barnett, Inka; Valters, Craig; Oppenheimer, Carrie; Levy, Anna; Hernandez, Kevin (IDS, 2017)

The Truth of the Work: Theories of Change in a changing world - Reeler, Doug; van Blerk, Rubert (The Community Development Resource Association, 2017)

Confronting the Contradiction - An exploration into the dual purpose of accountability and learning in aid evaluation - Reinertsen, Hilde; Bjørkdahl, Kristian; McNeill, Desmond (Expertgruppen för biståndsanalys (EBA), 2017)
Keywords: IMPORTANT

Citizen-generated data and sustainable development - Rono-Bett, Karen; Kenei, Steve (DevInit and DRT, 2017)
The potential of citizen-generated data in the sustainable development agenda

“Cash Plus” programmes for children - Save the Children (Save the Children, 2017)
This paper provides a resource on “Cash Plus” interventions for children. These interventions combine household cash transfers with complementary, context-relevant interventionsn in order to address the multiple drivers of childhood deprivations and generate more powerful impacts for children across Save the Children’s Breakthroughs, in both development and humanitarian contexts.

I know what I know (but how do I know what I don’t?) - Smit, Debbie; de Lanerolle, Indra; Braam, Tamara; Byrne, Deborah; Legong, Gontse (MAVC, 2017)
An important support function of Making All Voices Count South Africa is to design, plan and facilitate community of practice gatherings for sustained learning and sharing across Making All Voices Count grantees. This report aims to capture the content of a one-day Making All Voices Count South African Community of Practice (CoP) Meeting held in November 2016. The South African MAVC CoP has been running for three years and has met between two and four times a year. It is a space for MAVC grantees and others working to foster innovation in the fields of transparency and accountability, to share experiences and knowledge, and collaborate in learning and improving work.

Thinking and working politically in large, multi-sector Facilities: lessons to date - Teskey, Graham; Tyrrel, Lavinia (Abt Associates, 2017)

Monitoring Humanitarian Innovation - Warner, Alexandra; Obrecht, Alice (ALNAP, 2017)

'Leaving No One Behind': Why Ideas Matter - Evans, Alice (2016)

Going vertical: citizen-led reform campaigns in the Philippines - Aceron, Joy; Isaac, Francis (IDS, 2016)
The Philippines has a long history of state–society engagement to introduce reforms in government and politics. Forces from civil society and social movements have interfaced with reform-oriented leaders in government on a range of social accountability initiatives – to make governance more responsive, to introduce policy reforms, and to make government more accountable. Several theoretical propositions on which strategic approaches work best for social accountability initiatives have been put forward – including the idea of vertically integrated civil society monitoring and advocacy. This multi-authored research report uses vertical integration as a framework for examining seven successful civil society social accountability initiatives in the Philippines, looking at what made them successful, and how the gains they realised can be deepened and sustained.

Rethinking systemic change: economic evolution and institutions - Cunningham, Shawn; Jenal, Marcus (BEAM Exchange, 2016)
Resources that contain evidence of market systems interventions. Recently updated.

Rethinking systemic change: economic evolution and institutions. Discussion paper - Cunningham, Shawn; Jenal, Marcus (BEAM Exchange, 2016)
The primary aim of this research project was to find a conceptually sound definition of systemic change. To do so, it was essential to gain a better understanding of how economies change. The central part of the research work, therefore, was an extended literature review on three bodies of knowledge: evolutionary economics new institutional economics complexity theory There is a growing interest in these bodies of knowledge, combined often called New Economic Thinking, and how they affect economic development. Hence, while rethinking systemic change, this work also contributes to answering the broader question of how market systems approaches can contribute to inclusive economic development. The answer, in short, is to shift the focus away from improving transactions at the micro level towards enabling actors to continuously shape an institutional landscape that supports inclusive economic evolution. This discussion paper briefly presents the key messages from the literature review and seven principles drawing from this literature. The principles can be used by market development practitioners, including technical advisers in donor organisations, programme designers and team leaders, to shape programmes and become more in line with how change happens in the economy. A list of selected references is presented at the end. A technical paper contains a much more detailed discussion of the findings and the principles and an extensive list of references. A case study offers a practitioner perspective through the lens of one market systems development programme: DFID-funded Northern Uganda: Transforming the Economy through Climate Smart Agribusiness – Market Development (NUTEC-MD).

Rethinking systemic change: economic evolution and institutions. Technical Paper - Cunningham, Shawn; Jenal, Marcus (BEAM Exchange, 2016)
The primary aim of this research project was to find a conceptually sound definition of systemic change. To do so, it was essential to gain a better understanding of how economies change. The central part of the research work, therefore, was an extended literature review on three bodies of knowledge: evolutionary economics, new institutional economics and complexity theory. There is a growing interest in these bodies of knowledge, combined often called New Economic Thinking, and how they affect economic development. Hence, while rethinking systemic change, this work also contributes to answering the broader question of how market systems approaches can contribute to inclusive economic development. The answer, in short, is to shift the focus away from improving transactions at the micro level towards enabling actors to continuously shape an institutional landscape that supports inclusive economic evolution. This technical paper provides an in-depth review of the fields of evolutionary economics, new institutional economics and complexity and social change. It is argued that economic development is a complex, non-linear and continuous evolutionary process. Both market and non-market institutions matter greatly in shaping economic performance. The paper then explores the consequences of this understanding for market development practice. It discusses how market development practitioners can engage in and shape an intentional change processes. To translate the theory into practice, seven principles are suggested that can be applied to market systems development practice. The paper concludes with a reframed look at systemic change in market systems development. A discussion paper presents the key messages from the literature review and seven principles drawing from this literature, and a case study offers a practitioner perspective through the lens of one market systems development programme: DFID-funded Northern Uganda: Transforming the Economy through Climate Smart Agribusiness – Market Development (NUTEC-MD).

Scaling accountability through vertically integrated civil society policy monitoring and advocacy - Fox, Jonathan (IDS, 2016)
This working paper argues that the growing field of transparency, participation and accountability (TPA) needs a conceptual reboot, to address the limited traction gained so far on the path to accountability. To inform more strategic approaches and to identify the drivers of more sustainable institutional change, fresh analytical work is needed. This paper makes the case for one among several possible strategic approaches by distinguishing between “scaling up” and “taking scale into account”. This proposition grounds an explanation of the vertical integration strategy, which involves multi-level coordination by civil society organisations of policy monitoring and advocacy, grounded in broad pro-accountability constituencies. To spell out how this strategy can empower pro-accountability actors, the paper contrasts varied terms of engagement between state and society, proposing a focus on collaborative coalitions as an alternative to the conventional dichotomy between confrontation and constructive engagement. The paper grounds this discussion by reviewing the rich empirical terrain of existing multi-level approaches, summarizing nine cases – three each in three countries – to demonstrate what can be revealed when TPA initiatives are seen through the lens of scale. It concludes with a set of broad analytical questions for discussion, followed by testable hypotheses proposed to inform future research agendas

Taking scale into account in transparency and accountability initiatives - Fox, Jonathan (IDS, 2016)
We know that gaining access to information and raising citizen voices are not the same as achieving accountability. It is important to look beyond the symptoms of accountability failure, and consider how to tackle the causes. This short research summary discusses different understandings of scale, one important aspect of making transparency and accountability initiatives more strategic. Scale shapes both the causes of accountability failure and the tactics and strategies needed to address it.

Exploring the potential for interactive radio to improve accountability and responsiveness to small-scale farmers in Tanzania - Gilberds, Heather; Handforth, Calum; Leclair, Mark (IDS, 2016)
What are the possibilities of using new digital technologies alongside radio to help ensure that agricultural development projects are farmer-centred, and meet the needs of the rural citizens they intend to serve? This research assesses Farm Radio International’s Listening Post – a model that combines radio and digital technologies with the aim of collecting and aggregating farmer feedback to aid decision-making and adaptive project implementation. The research shows that linking a mobile-based crowd-sourcing tool with radio is effective at ensuring engagement from a large number of farmers, who felt it was a useful way of raising their questions and concerns to NGOs, policy-makers and experts. The model has clear potential to strengthen the chain of relationships between citizens, extension services suppliers, projects and policymakers. It has also demonstrated its potential to collect real-time feedback from farmers that could be used to aid decision-making and improve accountability in agricultural development initiatives, helping to ensure they are more responsive to farmers. However, it also concludes that ‘closing the feedback loop’ – ensuring that farmer’s comments, questions and concerns are responded to – is a challenge for the Listening Post. Sometimes, the interactive radio programmes close the loop by disseminating answers to questions or concerns raised by farmers, or by connecting farmers to extension services – but only when a solid and systematic process had been developed for this to happen. Therefore is critical to identify and incentivise actors who are capable of responding during the design phase of a Listening Post, and to ensure that they are involved in every stage of the process. It also important that project partners who are interested in beneficiary feedback identify the flexible or actionable points in their project frameworks from the outset, rather than collecting data that they are not able to use to adapt their programmes

Rethinking systemic change in practice - perspectives from NUTEC-MD in northern Uganda - Koleros, Andrew (BEAM Exchange, 2016)
Document description This document is currently being finalised and will be published shortly. Please try again soon. Recent research by The BEAM Exchange seeks to understand theoretical perspectives on how market systems approaches can contribute to inclusive economic development through systemic change. It produced three main insights. Economies are evolving systems, building on the mechanisms of variety creation, selection and amplification. Current economic performance, including aspects like the inclusiveness of growth and economic evolution, are shaped by the ability of a society to explore different options for institutional arrangements and adjust them over time. This process of evolution is complex. While some aspects can be designed and managed, others need to be explored through a process of learning and adjustment. This paper responds to these insights by providing a practitioner perspective through the lens of one programme: “Northern Uganda: Transforming the Economy through Climate Smart Agribusiness – Market Development (NUTEC-MD)”, implemented by Palladium Group. It begins with a brief introduction to the programme and key aspects of its design phase. It then provides a summary of the three main research insights, with reflections on their applicability for programme design based on the NUTEC-MD experience. Finally, it provides some conclusions for the wider practitioner community based on this case.

Doing Development Differently: Who we are, What we’re doing, What we’re learning - ODI (ODI, 2016)
In November 2014, the doing development differently community got together in Harvard to discuss what successful development interventions look like. Two years on, our community is broader than aid. It's broader than donors. It's about all organisations delivering change, producing real solutions to real problems that have real impact. It's about building trust, empowering people and promoting sustainability. Over the past two years, the community has been putting these ideas into practice across the world – being honest about what we are learning, including where we are not getting things right. This document aims to be an entry point for anyone interested in doing development differently. It explores the things that could be stopping you including: You are constrained by a disabling environment. You have strict reporting requirements. People aren't sold on the idea. It's easier to do things the way you've always done it. It's difficult to do in practice. If you are facing one, several, or all of these constraints, this document gives you a place to start. Using 43 case studies from practitoners within donors, governments, implementing organisations and NGOs across the globe, we aim to draw out some key lessons learnt, and give some advice for people considering this approach. These case studies are split across five categories: Swimming against the tide. Working in and with government. Feedback loops and data. Organisational change. Diffusion. Finally, if you want to know more, the document concludes with a suggested list of blogs and reports which explain these concepts in more detail.
Keywords: IMPORTANT

Crowdsourcing citizen feedback on district development in Ghana using interactive voice response surveys - Pegus, Ciana-Marie (IDS, 2016)
Keywords: IMPORTANT

The Science in Adaptive Management - Ripley, Matt; Jaccard, Sabine (ILO, 2016)
‘Adaptive management’ is all the rage in international development circles. But to avoid yet another buzzword – we need to learn from the experience of natural resource science.
Keywords: IMPORTANT

Nigeria Public Sector Accountability and Governance Programme - Business Case - SAVI (DFID, 2016)
In May 2016, SAVI transitioned into a successor programme – the ‘Engaged Citizens Pillar’ (ECP) of a wider DFID-funded governance reform programme ‘The Partnership to Engage, Reform and Learn’ (PERL). ECP is managed by the same service provider, Palladium, and the same core management team as SAVI, and many of the SAVI front line staff...

SAVI Approach Paper 11: Managing and Staffing an Adaptive Citizen Engagement Programme - SAVI (DFID, 2016)
SAVI, a DFID funded programme implemented by Palladium, is an adaptive programme putting learning and adaptation at the centre of all decision-making. In relation to management and staffing SAVI has established the following approaches to facilitate this: – An adaptive approach to programme management: Full time strategic technical leadership, that works closely with and complements...

SAVI Approach Paper 12: Managing Programme Finances to Support Adaptive, Locally-led Processes of Citizen Engagement in Governance - SAVI (DFID, 2016)
SAVI, is seeking to support processes of citizen engagement in governance in ways that are effective in influencing reform, and that are able to take on a life of their own without continuing donor support. SAVI is also an adaptive programme, putting learning and adaptation at the centre of all decision-making. Money is used and...

SAVI Approach Paper 13: Measuring Value for Money and using Value for Money Analysis - SAVI (DFID, 2016)
SAVI has established its own framework for assessing Value for Money in annual performance – in relation to expenditure, economy, efficiency, effectiveness and equity. Routine tracking and analysis of expenditure and economy ensure that inputs are supplied and services delivered to partners in line with SAVI’s core values, whilst also meeting DFID requirements and competing...

SAVI Approach Paper 14: Learning, Adaptation and Communications - SAVI (DFID, 2016)
SAVI, a DFID funded programme implemented by Palladium, is an adaptive programme, aiming to put learning and adaptation at the centre of all decision-making. Learning and adaptation takes place in SAVI at three levels: the work of partners; the work of SAVI delivery teams; and the enabling environment of the programme as a whole. Achieving...

SAVI Approach Paper 15: Introduction to PERL: the SAVI Successor Programme - SAVI (DFID, 2016)
In May 2016, SAVI transitioned into a successor programme – the ‘Engaged Citizens Pillar’ (ECP) of a wider DFID-funded governance reform programme ‘The Partnership to Engage, Reform and Learn’ (PERL). ECP is managed by the same service provider, Palladium, and the same core management team as SAVI, and many of the SAVI front line staff...

Forget the money: De-linking technical assistance - Manuel, Clare (DFID-LASER Programme, 2016)

Global Learning for Adaptive Management (GLAM) - DFID (DFID Development Tracker, 2016)
To improve the value for money and impact of development interventions by promoting the use of better monitoring, evaluation and learning strategies that support development programmes to gather and respond to emergent evidence on the effects of interventions, as well as changes in the development context. The Global Learning for Adpative Management programme will make available technical assistance to DFID, USAID and partners to develop this ‘adaptive management’ approach, as well as build an evidence base to understand links between the use of adaptive management and more effective, efficient, relevant and sustainable development interventions.

Shoulder to Shoulder - Hymowitz, Dan (Tony Blair Africa Governance Initiative, 2016)

Ten Frontier Technologies for International Development - Ramalingam, Ben; Hernandez, Kevin; Prieto Martin, Pedro; Faith, Becky (IDS, 2016)
As new technologies and digital business models reshape economies and disrupt incumbencies, interest has surged in the potential of novel frontier technologies to also contribute to positive changes in international development and humanitarian contexts. Widespread adoption of new technologies is acknowledged as centrally important to achieving the United Nations Sustainable Development Goals by 2030. But while frontier technologies can rapidly address large-scale economic, social or political challenges, they can also involve the displacement of existing technologies and carry considerable uncertainty and risk. Although there have been significant wins bringing the benefits of new technologies to poor consumers through examples such as mobile money or off-grid solar energy, there are many other areas where the applications may not yet have been developed into viable market solutions, or where opportunities have not yet been taken up in development practice.

LEARN Monitoring, Evaluation, Research, and Learning Plan - Learning Lab (USAID, 2016)
LEARN and USAID/PPL are managing an area of work known as Evidence Base for Collaborating, Learning, and Adapting (EB4CLA). The purpose of this work is to answer key learning questions:

Designing for Public Services - Nesta (Nesta/IDEO, 2016)

A practical guide to outcome-focused philanthropy - Twersky, Fay; Grange, Lori (Hewlett Foundation, 2016)
Outcome-focused philanthropy is the Hewlett Foundation’s approach to how we practice strategic philanthropy. It guides how we do our work, from start to finish. We’ve been practicing strategic philanthropy for more than a decade—focusing on what problem or opportunity we’re trying to address, whether we’re achieving our outcomes and why (or why not), and how…

Nairobi Outcome Document (2016)

Where next for development effectiveness? Building a renewed consensus - Blampied, Catherine (ODI, 2016)

Contracts for adaptive programming - Bryan, Kevin; Carter, Paddy (ODI, 2016)
Adaptive programming is an approach to development that encourages experimentation, learning & adaptation. This report examines the contractual underpinnings of this approach.

Where next for development effectiveness? Implementing development effectiveness: a reality check - Carter, Paddy (ODI, 2016)

Thinking and Working politically to support developmental leadership and coalitions: The Pacific Leadership Program - Denney, Lisa; McLaren, Rebecca (DLP Program, 2016)

DFID Smart Rules: Better Programme Delivery - version VI (2016.10) - DFID (DFID, 2016)
Smart Rules provide the operating framework for the Department for International Development’s (DFID’s) programmes.
Keywords: IMPORTANT; Practice

Collaborating, Learning, and Adapting (CLA). Framework and Maturity Matrix Overview - Learning Lab (USAID, 2016)

Grow - Scale - Impact: How to help inclusive business achieve scale - Pérez Castro, Andrea A.; Tora, Krisztina (GIZ, 2016)

Collaborating, Learning and Adapting (CLA) - Maturity Matrix Overview - USAID (Policy, Planning and Learning, 2016)
Collaborating, Learning and Adapting (CLA) involves strategic collaboration, continuous learning, and adaptive management. CLA approaches to development include collaborating intentionally with stakeholders to share knowledge and reduce duplication of effort, learning systematically by drawing on evidence from a variety of sources and taking time to reflect on implementation, and applying learning by adapting intentionally. The purpose of the maturity matrix tool is to help USAID Missions think more deliberately about how to plan for and implement CLA approaches that fit the Mission’s context and needs. Through a set of easy-to-use cards, the CLA maturity matrix offers examples of what CLA looks like at different stages of maturity. USAID staff can use the decks of cards to both assess current practice and plan for the future. The maturity matrix consists of two decks of cards corresponding to the dimensions of CLA: ● CLA in the Program Cycle: By using the matrix cards to guide a conversation, teams can explore how well CLA is incorporated into the planning and implementation processes of the Program Cycle. ● Enabling Conditions: The cards encourage discussion of the conditions that can significantly influence how CLA and the Program Cycle are implemented at a Mission, including operating processes, organizational culture, and resourcing. Within these decks, the matrix covers 16 subcomponents of CLA. Each has one key concept card and five matrix cards that describe the component in practice, along a spectrum ranging from Not Yet Present to Institutionalized.

The 5 Rs Framework in the program cycle - USAID (2016)
USAID’s Program Cycle Operational Policy (ADS 201) provides guidance to missions and other operating units on how to implement the Program Cycle. A key principle of the Program Cycle is to “Promote Sustainability through Local Ownership.” The purpose of this Technical Note is to describe the “5Rs Framework”, a practical methodology for supporting sustainability and local ownership in projects and activities through ongoing attention to local actors and local systems. This Note is rooted in USAID’s 2014 Local Systems Framework paper, which establishes that achieving sustained improvement in development results depends on the contributions of multiple and interconnected local actors. That document also states that USAID needs to improve its systems practice if it is to engage local actors and strengthen local systems more effectively and thus realize sustained results more consistently. The 5Rs Framework, also introduced in the Local Systems Framework, is intended as a simple and practical tool to promote good systems practice. The 5Rs Framework highlights five key dimensions of systems: Results, Roles, Relationships, Rules and Resources. Collectively these 5Rs can serve as a lens for assessing local systems and a guide for identifying and monitoring interventions designed to strengthen them. This Technical Note is divided in two parts. The first part provides an introduction to the 5Rs Framework and the systems practice from which it emerges. The second part demonstrates how systems practice can be embedded in the Program Cycle by continuously applying the 5Rs, especially to the design, implementation, and monitoring of USAID projects and their accompanying activities.

Building justice and peace from below? Supporting community dispute resolution in Asia - Valters, Craig (ODI, 2016)

Forward Look - A vision for the World Bank Group in 2030 - World Bank (World Bank, 2016)

Doing Development Differently at the World Bank: updating the plumbing to fit the architecture - Bain, Katherine A.; Booth, David; Wild, Leni (ODI, 2016)
Keywords: IMPORTANT

The political economy of pre-trial detention: Indonesia case study - Domingo, Pilar; Sudaryono, Leopold (ODI, 2016)

Collaborating, Learning, and Adapting. Framework & Key Concepts - Learning Lab (USAID, 2016)

Adaptive management: What it means for CSOs - O’Donnell, Michael (Bond, 2016)
Keywords: IMPORTANT

Monitoring, Evaluation and Learning approaches in an adaptive management context - Tsui, Josephine (EPS PEAKS, ODI, 2016)

Delivering institutional reform at scale: Problem-driven approaches supported by adaptive programming - Manuel, Clare (DFID-LASER Programme, 2016)
LASER synthesis papers aim to help donors and other stakeholders better understand why and how to approach investment climate reform programming differently. The papers reflect emerging best practice and lessons learnt on what works and what does not work in doing development differently. The papers have been peer-reviewed by experts in the field including senior advisers at DFID, World Bank, IFC and the Donor Committee for Enterprise Development (amongst others). Second synthesis paper - Delivering institutional reform at scale: problem-driven approaches supported by adaptive programming February 2016 Drawing on our experience on the ground, this peer-reviewed paper suggests how problem driven approaches, supported by adaptive programme management can be implemented at scale in relation to donor programming aimed at institutional reform and improving state capability.
Keywords: IMPORTANT

Evidence Base for Collaborating, Learning, and Adapting: summary of the literature review (2016.08) - Learning Lab (USAID, 2016)
LEARN and USAID/PPL are managing an area of work known as Evidence Base for Collaborating, Learning, and Adapting (EB4CLA). The purpose of this work is to answer key learning questions:

Adaptive programming in practice: shared lessons from the DFID-funded LASER and SAVI programmes - Derbyshire, Helen; Donovan, Elbereth (DFID-LASER Programme, 2016)
LASER synthesis papers aim to help donors and other stakeholders better understand why and how to approach investment climate reform programming differently. The papers reflect emerging best practice and lessons learnt on what works and what does not work in doing development differently. The papers have been peer-reviewed by experts in the field including senior advisers at DFID, World Bank, IFC and the Donor Committee for Enterprise Development (amongst others).
Keywords: IMPORTANT; Practice

Biodiversity How-To Guide 1: Developing Situation Models in USAID Biodiversity Programming - USAID (USAID, 2016)
The Biodiversity How-To Guide 1: Developing Situation Models in USAID Biodiversity Programming is the first in a series of three guides that provide in-depth guidance on key tools and practices to support design teams as they design and manage biodiversity programs within the Program Cycle and in accordance with the USAID Biodiversity Policy. It focuses on how to develop situation models to map out the biodiversity conservation problem context to be addressed.

Biodiversity How-To Guide 2: Using Results Chains to Depict Theories of Change in USAID Biodiversity Programming - USAID (USAID, 2016)
The Biodiversity How-To Guide 2: Using Results Chains to Depict Theories of Change in USAID Biodiversity Programming is the second in a series of three guides that provide in-depth guidance on key tools and practices to support design teams as they design and manage biodiversity programs within the Program Cycle and in accordance with the USAID Biodiversity Policy. It builds off the situation model guide to help design teams clearly state the expected results and assumptions behind the proposed strategic approaches that make up the program’s theory of change. The situation model provides the foundation to identify necessary results to reduce threats; brainstorm and prioritize strategic approaches; and then develop results chains to clarify assumptions behind selected approaches.

Biodiversity How-To Guide 3: Defining Outcomes & Indicators for Monitoring, Evaluation, and Learning in USAID Biodiversity Programming - USAID (USAID, 2016)
The Biodiversity How-To Guide 3: Defining Outcomes and Indicators for Monitoring, Evaluation, and Learning in USAID Biodiversity Programming is the third in a series of three guides that provide in-depth guidance on key tools and practices to support design teams as they design and manage biodiversity programs within the Program Cycle and in accordance with the USAID Biodiversity Policy. It uses the results chains developed in the second guide and provides help identifying key results for developing outcome statements and indicators, as required by USAID’s Biodiversity Code, updated in the Biodiversity Policy.

If politics is the problem, how can external actors be part of the solution? - Devarajan, Shantayanan; Khemani, Stuti (The World Bank, 2016)
Despite a large body of research and evidence on the policies and institutions needed to generate growth and reduce poverty, many governments fail to adopt these policies or establish the institutions. Research advances since the 1990s have explained this syndrome, which this paper generically calls
Keywords: IMPORTANT

Using beneficiary feedback to improve development programmes: findings from a multi-country pilot - World Vision (World Vision, 2016)
This briefing summarises the findings from Beneficiary Feedback Mechanisms pilots and is intended to inform organisations and their funders about the development and implementation of feedback mechanisms.

The road to adaptive management: knowledge, leadership, culture and rules - Byrne, Karri Goeldner; Sparkman, Timothy; Fowler, Ben (The BEAM Exchange, 2016)
Keywords: IMPORTANT; Practice

Social network analysis handbook: connecting the dots in humanitarian programs - IRC (International Rescue Committee, 2016)
The following handbook is designed to provide a step by step guide to the application of Social Network Analysis for the IRC.
Keywords: IMPORTANT

External Review of the Building Effective and Accessible Markets (BEAM) programme - Snedden, Ewan; Freer, Gordon; Moffatt, Michelle (WYG International, 2016)

Experiential Learning: Fostering lasting behavioural change - Norton, Brittney M. (Mercy Corps, 2016)
The term ‘experiential learning’ is often mistakenly used to describe any learning experience that involves participatory activities. This paper presents an overview of what true experiential learning is and how this form of learning allows for real-world application - thus promoting lasting behavioural change. Furthermore, it explains how experiential learning was adopted as the methodological core of the Prospects psychosocial programme in Liberia, which builds ‘pre-employment skills’ in 3,500 vulnerable youth aged 15-24 years. The goal of the paper is to provide other youth and education development practitioners with a resource that can be used to strengthen youth and behaviour change programmes through the incorporation of the experiential learning theory. In addition, this paper explains its application in the Prospects programme, before concluding with key considerations of such a design. 1

Has Kenya’s ICT revolution triggered more citizen participation? - Salome, Nyambura (Institute of Development Studies, 2016)
Lessons from policy in Ghana, Kenya, South Africa and Tanzania
Keywords: IMPORTANT

Primer: Technology to monitor and share information on rainforests and forest people's rights - The Engine Room (The Engine Room & Rainforest Foundation Norway, 2016)

Learning to Adapt: Building Adaptive Management as a Core Competency in Development Practice - Ramalingam, Ben (Institute of Development Studies, 2016)
Key messages 1. Development policies and programs increasingly operate in situations of high complexity and uncertainty. 2. There are growing efforts across the sector to design, plan and implement more adaptive responses that are more relevant and appropriate in such contexts 3. At the heart of effective adaptive programming is the capability to gather, interpret and use knowledge, information and data in real-time 4. Strengthening this capability requires a positive enabling environment—including tools and methods, structures and processes, leadership and management and culture and mindsets. 5. For adaptive development to succeed, there needs to be greater attention and investment in both knowledge, information and data systems and in appropriate enabling environments
Keywords: IMPORTANT

When theory meets reality: assumptions, feasibility and implications of a complexity-informed approach - Buffardi, Anne (Methods Lab, 2016)
Over the last half century, repeated calls for adaptive learning in development suggests two things: many practitioners are working in complex situations that may benefit from flexible approaches, and such approaches can be difficult to apply in practice. • Complexity thinking can offer useful recommendations on how to take advantage of distributed capacities, joint interpretation of problems and learning through experimentation in complex development programmes. • However, these recommendations rely on underlying assumptions about relationships, power and flexibility that may not hold true in practice, particularly for programmes operating in a risk averse, results-driven environment. • This paper poses guiding questions to assess the fit and feasibility of integrating complexity informed practices into development programmes.

Reforming solid waste management in Phnom Penh - Denney, Lisa (2016)
This paper tracks the efforts of an Asia Foundation team and local stakeholders as they worked to support improvements in the solid waste management sector in Phnom Penh, Cambodia. The team worked in a flexible way with a range of partners, and with particular focus on understanding the incentives and politics affecting service delivery. While reform of the sector remains in progress, steps have been taken to introduce more competition and better public sector management of solid waste collection in the city. This case study lays out the real-time decisions and processes which drove the strategy and implementation of this project, providing useful insights into how politically astute and flexible programs can be successfully implemented. This case has emerged from an action research process, which was led by a researcher from the Overseas Development Institute (ODI) and conducted over the course of almost two years. By capturing and analyzing the experiences of the program team in Phnom Penh, the paper intends to provide practical insights for others in the development community aiming to implement similar kinds of programming.

You Cannot Go it Alone: Learning from Cooperative Relationships in Civil Society Budget Campaigns - Larsen, Jillian (IBP, 2016)
Keywords: IMPORTANT

Rewards and Risks Associated with Community Engagement in Anti-Poaching and Anti-Trafficking - Wilkie, David; Painter, Michael; Jacob, Anila (USAID, 2016)
This Biodiversity Technical Brief summarizes research on the roles communities do and should play in countering wildlife crime, motivations, and benefits for individuals and communities to engage in anti-poaching and anti-trafficking efforts, and risks and mitigation of risks associated with involvement. Peer-reviewed and grey literature, structured interviews, and case studies informed the research, with highlights from six case studies in Peru, Madagascar, Namibia, Philippines, Kenya, and Indonesia.

Monitoring and Measuring Results in Private Sector Development (Swiss Agency for Development and Cooperation SDC, 2016)

Time to let go: remaking humanitarian action for the modern era - Bennett, Christina (ODI, 2016)
As the international humanitarian system faces a crisis of legitimacy, the Humanitarian Policy Group’s landmark report proposes a new model of humanitarian action.

Using Evidence - What works? - Breckon, Jonathan; Dodson, Jane (Alliance for Useful Evidence, 2016)

Reflections on implementing politically informed, searching programs: lessons for aid practitioners and policy makers - Cole, William; Ladner, Debra; Koenig, Mark; Tyrrel, Lavinia (2016)

Overview for Social Sector Leaders: Assessing Innovation Potential for Social Impact - Global Knowledge Initiative (2016)
We aim to provide decision makers with greater insight and confidence into the process of assessing innovation impact potential. Rather than considering the role innovation plays after an investment is made, or based on historic evaluations of how innovation has or has not delivered solutions to a problem, this approach is forward-looking. This customizable toolset assesses the future impact that innovation can deliver in a system to tackle particularly complex problems

Reforming donors in fragile states: using public management theory more strategically - Gulrajani, Nilima; Honig, Dan (Overseas Development Institute, 2016)
This paper identifies ways in which donors can be more effective in fragile and conflict-affected states by exploiting theories and concepts drawn from public management.

The Science of Using Science: Researching the Use of Research Evidence in Decision-Making - Langer, Laurenz; Tripney, Janice; Gough, David (EPPI-Centre, Social Science Research Unit, UCL Institute of Education, University College London, 2016)

Impact evaluation for portfolio programmes on policy influence: Reflections on the Indonesian Poverty Reduction Support Facility - Mackenzie, Jessica; Hearn, Simon (Methods Lab, 2016)
• Donors are increasingly using portfolio-based programmes that embrace ‘good failure’ and adaptive, political programming. • However, measuring the impact of these programmes is challenging, especially for those working on policy influence and building country systems; not only do you need to measure the positive and negative impact of the overall portfolio, but also the different pathways tested. • Programmes, therefore, need a light-touch monitoring and evaluation system that allows it to remain flexible. • Good practice examples of portfolio-based programmes present six strategies to evaluate impact: 1. Develop appropriate logic models 2. Collect observational data throughout implementation 3. Develop stories of change or case studies 4. Understand causal relationships without a counterfactual 5. Purposefully select which activities to study 6. Be explicit about how impact will be valued across the portfolio. • These strategies are only useful if monitoring and evaluation is placed at the centre of programme decision-making.

Adaptive Management in Practice: a case study on the Prospects program - Maclay, Chris (Mercy Corps, 2016)
Recognising that aid and development programming takes place in complex contexts, Mercy Corps is increasingly seeking to understand how best to manage programs which iterate, adapt and respond to the consistently evolving settings in which we work. This brief Practice Paper provides some examples of what adaptive management looks like in practice on the Prospects youth employment program in Liberia. It does not seek to function as a manual or set of guidelines, but simply provides some practical examples and insights into how a youth employment program governed by principles of adaptive management operates.

Built to improve: Leveraging realtime M&E for adaptive youth employment programming - Skelton, John (Mercy Corps, 2016)
Recognising that aid and development programming takes place in complex contexts, Mercy Corps is increasingly seeking to understand how best to manage programs which iterate, adapt and respond to the consistently evolving settings in which we work. This brief Practice Paper provides some examples of what adaptive management looks like in practice on the Prospects youth employment program in Liberia. It does not seek to function as a manual or set of guidelines, but simply provides some practical examples and insights into how a youth employment program governed by principles of adaptive management operates.

Dealing with uncertainty: Reflections on donor preferences for pre-planned project models - Tyrrel, Lavinia; Cole, William (2016)
Keywords: IMPORTANT

Stakeholder Engagement for Biodiversity Conservation Goals: Assessing the Status of the Evidence - USAID (USAID, 2016)
Learning how to communicate and work with different groups builds constituencies to support conservation aims. An analysis by USAID's research partner the American Museum of Natural History evaluated 148 references, selected through a rigorous systematic review process, to identify key lessons for stakeholder engagement. This briefer describes the analysis, which will soon be published in full.

Politically smart support to economic development: DFID experiences - Booth, David (ODI, 2016)

Evaluating the impact of flexible development interventions using a ‘loose’ theory of change: Reflections on the Australia-Mekong NGO Engagement Platform - Davies, Rick (Methods Lab, 2016)
• For some interventions, tight and testable theories of change are not appropriate – for example, in fast moving humanitarian emergencies or participatory development programmes, a more flexible approach is needed. • However, it is still possible to have a flexible project design and to draw conclusions about causal attribution. This middle path involves ‘loose’ theories of change, where activities and outcomes may be known, but the likely causal links between them are not yet clear. • In this approach, data is collected ‘after the event’ and analysed across and within cases, developing testable models for ‘what works’. More data will likely be needed than for projects with a ‘tight’ theory of change, as there is a wider range of relationships between interventions and outcomes to analyse. The theory of change plays an important role in guiding the selection of data types. • While loose theories of change are useful to identify long term impacts, this approach can also support short cycle learning about the effectiveness of specific activities being implemented within a project’s lifespan.

Sometimes it is about the Tech: Choosing Tools in South African and Kenyan Transparency & Accountability Initiatives - de Lanerolle, Indra; Walker, Tom; Kinney, Sasha (IDS and The Engine Room, 2016)
Keywords: IMPORTANT

Using political economy analysis in conflict, security and justice programmes - Denney, Lisa (ODI, 2016)

Leather sector reform in Bangladesh - Harris, Dan (2016)
This paper examines the Asia Foundation’s efforts to support change in Bangladesh’s leather sector. Working closely with local partners, the Asia Foundation team has specifically supported efforts to move tanneries out of a dangerously polluted location to a modern industrial park that will improve compliance with health and environmental protection standards, and potentially lead to growth in the sector. At the time of release, this critical relocation has already begun. This case study lays out the real-time decisions and processes which drove the strategy and implementation of this project, providing useful insights into how politically astute and flexible programs can be successfully implemented. This case has emerged from an action research process, which was led by a researcher from the Overseas Development Institute (ODI) and conducted over the course of almost two years. By capturing and analyzing the experiences of the program team in Bangladesh, the paper intends to provide practical insights for others in the development community aiming to implement similar kinds of programming. This is the seventh paper in the DFAT-TAF Partnership Working Politically in Practice Paper Series released under the Australian Government Department of Foreign Affairs and Trade (DFAT) and The Asia Foundation (TAF) Partnership.

Using adaptive development to support feminist action - O’Neil, Tam (ODI, 2016)
This paper looks at how adaptive development is being applied by gender programmes and argues that gender and adaptive development communities have much to offer each other.

‘Learning journeys’ for adaptive management – Where does it take us? - Ørnemark, Charlotte (Global Partnership for Social Accountability, 2016)
Keywords: IMPORTANT

When and how to develop an impact-oriented monitoring and evaluation system - Peersman, Greet; Rogers, Patricia; Guijt, Irene; Hearn, Simon; Pasanen, Tiina; Buffardi, Anne (2016)
This guidance note focuses on: • what an impact-oriented monitoring and evaluation system entails • why an organisation may want to establish such a system • when integrating an impact-orientation into an monitoring and evaluation system is most useful • what should be considered in developing the monitoring and evaluation system, or in tweaking an existing system, to become more impact-focused. The primary audience for this guidance note is internal and external monitoring and evaluation advisors involved in designing and implementing, and/or assessing monitoring and evaluation systems to include a focus on impact. It will also be useful for senior management of organisations who need to know how best to plan for a sustainable monitoring and evaluation system that supports impact assessment or to adapt an existing system to incorporate an impact perspective.

Is DFID Getting Real About Politics?A stocktake of how DFID has adopted a politically-informed approach (2010-2015) - Piron, Laure-Hélène; Baker, Aislin; Savage, Laura; Wiseman, Katie (DFID, 2016)
1. Background This internal stocktake assesses whether DFID is “getting real about politics” - how it is taking power and politics into account in all its operations. Country Poverty Reduction Diagnostics undertaken by DFID teams identify politics as the most frequent barrier to poverty reduction and growth. The UK 2015 Aid Strategy has committed DFID to spending 50% of Official Development Assistance (ODA) in fragile states. This requires a “patient, long-term approach” to addressing barriers to peace and stability which are fundamentally political, rather than purely financial or technical. The stocktake is based on three DFID offices case studies (Democratic Republic of the Congo, Pakistan and Malawi) as well as extensive internal and external consultations between June and December 2015. It provides illustrations of how DFID is evolving but does not systematically offer evidence of development impacts or non-country work, as this would have required a different methodology. 2. What does it mean to take politics into account? Politically-informed approaches are based on a large body of evidence that confirms the importance of institutions and politics for sustainable development. External assistance needs to support locally-led change. Success depends on timing, context, political processes and local actors. Desirable outcomes are hard to achieve and difficult to predict. Politically-informed approaches improve development effectiveness through:  The ‘what’: political goals, using development assistance to shift how power is distributed in the economy and society. The two main elements are: aiming for long term transformation of institutions; and supporting locally-led change processes more likely to be sustainable and successful: locallyowned (i.e. with local salience) and locally-negotiated.  The ‘how’: politically-smart methods, with greater realism and feasibility. The three main elements are: understanding power and politics in a specific context in order to identify opportunities and barriers for change; influencing and stakeholder management skills; and proactive risk management. To influence DFID operations, a politically-informed approach needs to be iterative, not one-off. The explicit understanding of context, whether formal setpiece studies or more routine analysis, should inform policy and programme decisions, from high level strategic choices, to day-to-day implementation, for both international policy and country support. This is a dynamic process: as the context evolves and lessons are learned about what works, analyses and decisions are updated. These are the principles behind the ‘flexible and adaptive’ agenda.

SPACES MERL: Systems and Complexity White Paper - USAID (2016)
The Strategic Program for Analyzing Complexity and Evaluating Systems (SPACES MERL) project is an activity funded by USAID’s Global Development Lab and the Bureau for Policy, Planning and Learning (PPL). This three-year activity aims to bring a variety of tools and methodologies that decision-makers can use (alone or in combination) to provide comprehensive systems analysis. The activity is being implemented from 2015 to 2018 by a consortium of organizations expert in systems and complexity, including the Global Obesity Prevention Center (GOPC) at Johns Hopkins University (Prime), Global Knowledge Initiative (GKI), LINC and ResilientAfrica Network (RAN). This Systems and Complexity White Paper is a collaborative effort of the SPACES MERL team, designed to frame the international development landscape, with particular reference to USAID-funded activities, for application of systems and complexity approaches to design, monitoring and evaluation. Customized to the systems and complexity layperson with in-depth knowledge of international development practice, the objectives of this white paper are three-fold:  Provide an overview of systems and complexity practice, its current state of application and relevance to international development practice;  Establish a taxonomy of systems and complexity tools, highlighting the fit of those offered by SPACES MERL within the wider landscape; and  Review and provide information on application of SPACES MERL tools, their purpose and construction, required data, and their applicability to specific contexts.

Putting learning at the centre: Adaptive development programming in practice - Valters, Craig; Cummings, Clare; Nixon, Hamish (ODI, 2016)
Adaptive programming suggests, at a minimum, that development actors react and respond to changes in the political and socio-economic operating environment. It emphasises learning and the development practitioner is encouraged to adjust their actions to find workable solutions to problems that they may face. Being prepared to react to change may seem like common sense – and indeed it is. However much development thinking and practice remains stuck in a linear planning model which discourages learning and adaptation, in part because projects are seen as ‘closed, controllable and unchanging systems’ (Mosse, 1998: 5). This paper critically engages with this problem and makes clear why and how learning needs to be at the centre of adaptive development programming. It begins by clarifying why and what kind of learning matters for adaptive programming. The paper then turns its focus to how strategies and approaches applied throughout a programme’s conception, design, management and M&E can enable it to continually learn and adapt.
Keywords: IMPORTANT

USAID Applied Political Economy Analysis - Field Guide - Cammack, Diana (USAID, 2016)
Political Economy Analysis (PEA) is a field-research methodology used to explore not simply how things happen in an aid-recipient country, but why things happen. It results in a written assessment with recommendations for a mission's County Development Cooperation Strategy (CDCS), project or activity design, or course correction during implementation.USAID's Applied PEA is a problem-focused method specially intended to be used by Mission staff to inform the design of aid interventions at any phase of the USAID program cycle and at any level of effort.

Women and power: overcoming barriers to leadership ODI women and power coverand influence (ODI, 2016)
ODI have just wrapped up an excellent two year project on ‘Women and power: overcoming barriers to leadership and influence’ with a final synthesis report that is well worth reading. It’s an intell...

Moving Targets, Widening Nets: monitoring incremental and adaptive change in an Empowerment and Accountability programme. The experience of the State Accountability and Voice Initiative in Nigeria - Derbyshire, Helen; Barr, Julian; Fraser, Steve; Mwamba, Wilf (DFID, 2016)
Keywords: IMPORTANT; Practice

Transforming governance: what role for technologies? - Edwards, Duncan; Brock, Karen; McGee, Rosie (MAVC, 2016)
The technological innovations of the last two decades – cell phones, tablets, open data and social media – mean that governments and citizens can interact like never before. Around the world, in different contexts, citizens have fast-increasing access to information and communications technologies (ICTs) that enable them to monitor government performance and express their views on it in real time. In February 2016, a learning event in Manila, convened by Making All Voices Count, brought together 55 researchers and practitioners from 15 countries. They all work on using new technologies for accountable governance. They shared their diverse experiences, reflected on how they approach transformative governance, and visited Filipino accountable governance initiatives. This report shares some of what they learned.
Keywords: IMPORTANT

What is impact? - Hearn, Simon; Anne, Buffardi (2016)
• Impact is a multi-dimensional concept. Some definitions focus on very precise understandings of impact, while others cast a much broader net. • How impact is defined and used has a significant effect on the design, management and evaluation of development programmes. • Development programmes should hold explicit conversations with different stakeholders about how impact is used and understood, in order to come to a shared understanding. • There are six dimensions of impact that may help development programmes be clearer about what they mean.

Politically smart approaches to donor investment climate programming. A Guidance Note - LASER (DFID-LASER Programme, 2016)
In order to help enhance the effectiveness of donors and development practitioners on the ground, LASER has produced a range of practical guidance and tools. These are primarily intended for the international development community engaged in designing and implementing investment climate programmes, though can also be used more widely by other stakeholders across sectors. General guidance and tools: Politically smart approaches to donor investment climate programming (and annexes) Revised February 2016 - sets out practical advice on how to adopt a politically smart (which requires being both ‘politically informed’ and ‘politically astute’) approach to programming. The annexes include a traffic light matrix for initial high level analysis; a problem diary template with examples; and an example of a flexible theory of change for adaptive, large scale programming
Keywords: Practice

How to design a monitoring and evaluation framework for a policy research project - Pasanen, Tiina; Shaxson, Louise (Methods Lab, 2016)
Policy research projects face a number of challenges: policy processes are complex, involve multiple actors and often feature a significant time-lag between research and what may or may not happen as a result of it. To complicate matters further, the scope and scale of policy research projects are increasingly moving away from single research studies towards multi-component, multi-site and multi-sector endeavours. These factors mean that developing an overarching monitoring and evaluation framework can be challenging. But it is not impossible. The guidance note aims to support the first steps in designing and structuring the M&E framework (that is, what aspects or areas of policy research projects to monitor and evaluate, why, when and how). It builds on an M&E framework for policy research projects developed and tested by the Research and Policy in Development (RAPID) programme of the Overseas Development Institute, and draws on case studies of how it has been used by RAPID and other research teams.

Bridging the Gap: Synthesising Evidence from Secondary Quantitative and Primary Qualitative Data - Roelen, Keetie (IDS, 2016)

Balancing Inclusiveness, Rigour and Feasibility: Insights from Participatory Impact Evaluations in Ghana and Vietnam - van Hemelrijck, Adinda; Guijt, Irene (CDI, 2016)
This paper by Adinda Van Hemelrijck and Irene Guijt explores how impact evaluation can live up to standards broader than statistical rigour in ways that address challenges of complexity and enable stakeholders to engage meaningfully. A Participatory Impact Assessment and Learning Approach (PIALA) was piloted to assess and debate the impacts on rural poverty of two government programmes in Vietnam and Ghana funded by the International Fund for Agricultural Development (IFAD). We discuss the trade-offs between rigour, inclusiveness and feasibility encountered in these two pilots. Trade-offs occur in every impact evaluation aiming for more than reductionist rigour, but the pilots suggest that they can be reduced by building sufficient research and learning capacity.

From principle to practice: Implementing the Principles for Digital Development - Waugaman, Adele (2016)
The Principles for Digital Development (download PDF here) find their roots in the efforts of individuals, development organizations, and donors alike who have called for a more concerted effort by donors and implementing partners to institutionalize lessons learned in the use of information and communication technologies (ICTs) in development projects.
Keywords: IMPORTANT

Making politics work for development: harnessing transparency and citizen engagement - Khemani, Stuti; Ferraz, Claudio; Finan, Frederico S.; Johnson, Stephenson; Louise, Corinne; Abrahams, Scott David; Odugbemi, Adesinaola Michael; Dal Bó, Ernesto; Thapa, Dikshya (The World Bank, 2016)
Too often, government leaders fail to adopt and implement policies that they know are necessary for sustained economic development. They are encumbered by adverse political incentives, which prevent them from selecting good policies, and they run the risk of losing office should they try to do the right thing. Even when technically sound policies are selected by leaders, implementation can run into perverse behavioral norms among public officials and citizens, who seek to extract private benefits from the public sector. Such behavior might be supported by widespread beliefs that corruption is the norm. Even countries with low corruption and strong institutions experience problems of political incentives and behavior that prevent the public sector from solving shared problems. Ideological polarization among citizens and capture by special interests can lead to policy gridlock and the failure of the state to provide public goods, even in advanced economies. Even educated citizens can hold ideological beliefs about the role of public policy that lead them to deny technical evidence contrary to these beliefs. Too often, government leaders fail to adopt and implement policies that they know are necessary for sustained economic development. Political constraints can prevent leaders from following sound technical advice, even when leaders have the best of intentions. Making Politics Work for Development: Harnessing Transparency and Citizen Engagement focuses on two forces—citizen engagement and transparency—that hold the key to solving government failures by shaping how political markets function. In today’s participative world, citizens are not only queueing at voting booths, but are also taking to the streets and using modern communication technology to select, sanction, and pressure the leaders who wield power within government. This political engagement can function in highly nuanced ways even within the same formal institutional context and across the political spectrum, from autocracies to democracies. Political engagement becomes unhealthy when leaders are selected and sanctioned on the basis of their provision of private benefits rather than public goods, giving rise to a range of government failures. The solutions to these failures lie in fostering healthy political engagement within any institutional context, and not in circumventing or suppressing it. Transparency—citizen access to publicly available information about the actions of those in government and the consequences of these actions—can play a crucial role by nourishing political engagement. The report distills policy lessons for governments, international development partners, and civil society on how best to target transparency initiatives so that the provision of public goods becomes the focus of political contestation. Even so, unhealthy political engagement may persist. But to build institutions that are capable of tackling public goods problems, politics needs to be addressed and cannot be side-stepped. Targeted transparency is one way to move in the right direction: it complements everything else policy makers do and holds the potential to make politics work for development rather than against it. " This pathbreaking report places politics at the heart of the development dialogue—exactly where it belongs. It provides constructive ideas for harnessing the forces of transparency and citizen engagement in ways that are suited to diverse institutional contexts so that reform leaders can overcome political constraints to their countries’ development goals. " Asli Demirgüç-Kunt, Director of Research, Development Research Group, The World Bank "This book not only provides an authoritative statement of what we know about how to align political incentives with the interests of society, but it does so with an eye to making change happen even in the face of political opposition. The World Bank will never be the same again. " James Robinson, University Professor, Harris School of Public Policy, University of Chicago " A lesson for us at the World Bank also comes out of this research. We can do more…to work with our clients to diminish political constraints to achieving development goals…To do this we have to overcome the fear of talking about politics, and confront it as part of the challenge of development. That is what we are doing through this report. " Kaushik Basu, Senior Vice President and Chief Economist, The World Bank

From political economy analysis to doing development differently: a learning experience - Booth, David; Harris, Daniel; Wild, Leni (ODI, 2016)
A study of how development efforts can be strengthened by an awareness of political economy, reflecting on the experiences of PoGo policy researchers.
Keywords: IMPORTANT

Using Research Evidence - A practice guide - Breckon, Jonathan (Alliance for Useful Evidence, 2016)

Getting good at disruption in an uncertain world: Insights from Southern NGO leaders - Buckley, Lila; Ward, Halina (International Institute for Environment and Development, 2016)
Keywords: IMPORTANT

Thinking and Working Politically: from theory building to building an evidence base - Dasandi, Niheer; Marquette, Heather; Robinson, Mark (DLP, 2016)
This paper discusses the steps required to build a robust evidence base for 'thinking and working politically' (TWP) in development. It argues that better understanding what works, when and why is an important step in moving TWP into mainstream development programming. The paper reviews the existing evidence base on TWP, building on this and on other literature on public sector reform and 'pockets of effectiveness' to suggest research questions, case study selection criteria, and a four-level analytical framework: 1) political context; 2) sector; 3) organisation; and 4) individual. The framework aims to help build a 'rigorous enough' evidence base to show whether and how TWP happens and whether or not it influences the effectiveness of programme implementation and outcomes. The paper also calls for more focus on gender issues, and on different – and often more fragile – political contexts.
Keywords: IMPORTANT

Everyday Political Analysis - Hudson, David; Marquette, Heather; Waldock, Sam (Developmental Leadership Program, 2016)
This short note introduces a stripped-back political analysis framework designed to help frontline development practitioners make quick but politically-informed decisions. It aims to complement more in-depth political analysis by helping programming staff to develop the 'craft' of political thinking in a way that fits their everyday working practices. Everyday Political Analysis involves two steps: Understanding interests: What makes people tick? Understanding change: What space and capacity do people have to effect change? For each step five questions, accompanied by prompts, aim to help staff to conduct quick political analysis. The EPA framework can be used at any stage of the aid management cycle, and can help users to respond rapidly to unexpected change. We are keen to hear back from people on their experience of using EPA to help us adapt the framework. Was it useful (or not)? Do people tend to use just one or both steps? Are there missing statements or prompts that would improve the analysis? Please email us at info [at] dlprog.org.

Monday morning in Kigali: what do you do when you get off the plane? Practical guidance for PDIA practitioners - LASER (DFID-LASER Programme, 2016)
In order to help enhance the effectiveness of donors and development practitioners on the ground, LASER has produced a range of practical guidance and tools. These are primarily intended for the international development community engaged in designing and implementing investment climate programmes, though can also be used more widely by other stakeholders across sectors. General guidance and tools: Monday morning in Kigali January 2016 - what do you do when you get off the plane? Practical guidance for PDIA practitioners
Keywords: IMPORTANT; Practice

Rethinking Rigor - Increasing Credibility and Use - Lynn, Jewlya; Preskill, Hallie (Spark Policy Institute & FSG, 2016)
Evaluations of complex systems change strategies or adaptive, innovative programs cannot use the traditional “rigor” yardstick to measure quality. This paper proposes a new definition of rigor, one that applies in any setting, but particularly in complex, adaptive ones. Top Takeaways: 4 criteria for the new definition of rigor: - Quality of thinking - Credible and legitimate claims - Cultural context and responsiveness - Quality and value of the learning process Regardless of its other positive attributes, an evaluation of a complex, adaptive program that fails to take into account systems thinking will not be responsive to the needs of that program.

Case study: Embedding reform and exiting: LASER’s application of the hourglass approach to achieve sustainable results - Manuel, Clare (DFID-LASER Programme, 2016)
This case study describes how LASER has gone about enabling systemic change and sustainable uptake of reforms that address complex institutional problems in Kenya, Rwanda, Sierra Leone, Somaliland and Uganda. In each of these countries LASER has designed-in a sustainable approach from the start based on: (i) local ownership and leadership of reforms based on developing country (rather than donor) priorities; (ii) use of country (rather than donor programme) systems; and (iii) an understanding that the role of the donor / development practitioner is to support (rather than buy) reform. The case study uses LASER’s hour glass methodology, illustrated with examples from LASER interventions, to show how these principles guide LASER operations every step of the way.
Keywords: IMPORTANT; Practice

Innovating for pro-poor services: why politics matter - Mason, Nathaniel; Doczi, Julian; Cummings, Clare (2016)
This report addresses how politics matter for innovations. How politically smart approaches can help deliver access to services.

Qualitative comparative analysis: A valuable approach to add to the evaluator’s toolbox? Lessons from recent applications - Schatz, Florian; Welle, Katharina (IDS, 2016)

Summarising portfolio change: results frameworks at organisational level - Simister, Nigel (Intrac, 2016)

Adapting Aid: Lessons from six case studies - Algoso, Dave; Beloe, Jon; Hemberger, Alison; Hill, Philippa; Proud, Emma (MercyCorps & IRC, 2016)
Keywords: IMPORTANT

A Guide to Peer-to-Peer Learning - Andrews, Matt; Manning, Nick (Effective Institutions Platform, 2016)

Doing Iterative and Adaptive Work - Andrews, Matt; Pritchett, Lant; Woolcock, Michael (Center for International Development at Harvard University, 2016)
Many of the challenges in international development are complex in nature. They involve many actors in uncertain contexts and with unclear solutions. Our work has proposed an approach to addressing such challenges, called Problem Driven Iterative Adaptation (PDIA). This paper is the most recent in a series intended to show how one can do PDIA, building on the first paper, "Doing Problem Driven Work.” The current paper addresses a key part of the approach one moves to once a problem has been identified, performing real-time experimental iterations. This is intended as a practical paper that builds on experience and embeds exercises for readers who are actually involved in this kind of work.
Keywords: IMPORTANT

Adaptive management: an overview of the concept and its practical application in the Scottish context - Blackstock, Kirsty; Brown, Katrina; Gimona, Alessandro; Prager, Katrin; Irvine, Justin (The James Hutton Institute, 2016)

Learning for change in accountable governance programming - Brock, Karen; Shutt, Cathy; Ashlin, Alison (IDS, 2016)
Keywords: IMPORTANT

Participation by All - The keys to sustainability of a CRS food security project in Niger - Cekan, Jindra; Kagendo, Rutere; Towns, Alexandra (Valuing Voices, 2016)
This project that ran from 2006-2012 in Niger and was implemented by three NGOs: CRS, Cooperative for Assistance and Relief Everywhere (CARE), and Helen Keller International (HKI) under the direction of United States Agency for International Development (USAID) Office of Food for Peace (FFP) as a multi-year assistance program (MYAP) to support food security activities in the Dosso, Tahoua, and Zinder regions. PROSAN focused on increasing agricultural production and agro-enterprise, improving household health and nutrition status, reinforcing the capacities of health agents, and enhancing community resiliency. Here are the highlights from the report which itself is an excerpt from a longer analysis we did. Also please note one Annex highlights the similarlties/ differences we found to USAID/ FFP’s 4 elements of sustainability: AIM, METHODS, AND RESEARCH QUESTIONS The aim of this sustainability evaluation was to explore perceptions of sustainability from Nigeriens involved in PROSAN, former CRS staff and donors. It focused on evaluating participants’ adherence to project outcomes and their creation of new innovations. It also evaluated partners’ involvement in sustaining project outcomes. This evaluation used qualitative and quantitative methods including community mapping, focus group discussions, beneficiary interviews, and key stakeholder interviews. The evaluation was carried out in six communities in the Dosso region, with more than 500 interviewees, focusing on the following research questions: Sustainability of activities and groups: Are the communities sustaining the activities three to five years after the end of the project? What can we learn from the communities and their post-project implementation partners? Spread and unexpected outcomes: If the project was considered a success in the eyes of the community, how well did it spread? Fostering Sustainability: What are the long-term prospects for continued sustainability? FINDINGS Three years after PROSAN’s conclusion, the project was considered a success by community members, national partners, the implementer (CRS), and donor (USAID) staff. The main findings include: 1. SUSTAINABILITY OF ACTIVITIES AND GROUPS Eighty percent (80%)[*] of all activities were reported to have become self-sustained and community innovations have emerged: On average, households reported moving from being food secure for 3-6 months per year during PROSAN to 8-12 months at the time of this evaluation, which is a remarkable impact [1]. CRS_Niger_PROSAN_Sustainability_Evaluation_pdf [1] Women reported greater income through the increase in sales of food that was produced and processed due to the grain mills [1]. Respondents also reported improved household health, hygiene, and nutrition, with 91% of survey respondents indicating that their health and sense of well being had improved, especially through the efforts of the health posts and clinics that CRS helped build and the government of Niger’s efforts in sustaining them with resources and staff [1]. Community groups/committees have continued and are well-supported by NGO partners: 81% of the committees set up by PROSAN were functioning at the time of this evaluation, with many participants discussing ways to sustain best practices within their communities, and members still receiving regular trainings or updates [1]. Several new and refresher trainings come through national partners, NGOs, and new channels such as radio programs [1]. Some new NGOs and international organizations have built upon PROSAN’s success, for instance, by using land previously managed by PROSAN for a new vegetable gardening training program, building hygiene programs on past health awareness efforts, or extending agricultural credit for further inputs [1]. Twenty percent (20%) of implemented activities were not sustained or have stagnated: While hygiene practices were sustained by households and there was widespread latrine construction, sanitation was poor in the villages, and most latrines had fallen into disrepair [1]. Fewer than 50% of women reported practicing exclusive breastfeeding for children less than six months of age [1]. While almost half of all health committees no longer exist, new health clinics staff have replaced some of the work of the committees with health and agricultural promotion messages now being sent via radio, television, and cell phones [1]. Literacy training and theater groups have completely ceased [1]. With the exception of the Système Communautaire d’Alerte Précoce-Réponses aux Urgences’ (SCAP-RU) SCAP-RU early warning system which has expanded, other resilience activities such as roadwork and caring for the environment are a lesser priority due in part to the lack of food and cash-incentives to continue doing them [1]. 2. SPREAD AND UNEXPECTED OUTCOMES New innovations and ceased activities reflected the project’s legacy: Community innovations have emerged such as collective funds paying for cleaners of the new health center, community-imposed sanctions for births occurring outside of the health centers, and the monitoring of savings from well water sales. National partners have praised the project, with many lamenting its withdrawal. One non-PROSAN village told an Agriculture Ministry staff and potential NGO partner that “No one should bring a program here unless it is like PROSAN.” PROSAN-trained masons, well repair technicians, and village youth have learned land recuperation techniques (zai holes, bunds and demi-lunes) that helped generate income beyond project communities. Project activities that received free inputs have largely stopped being implemented once the incentives were withdrawn such as Food for Training (FFT), Food for Work (FFW), or Cash for Work (CFW) (e.g. literacy, seedlings, latrines, theater etc.); nonetheless the inputs were highly valued and have continued to support agriculture and health (carts, bicycles). 3. FOSTERING SUSTAINABILITY The following areas were identified as potential barriers to sustainability that could be systematically explored in other projects: Although most committees are still functioning, there are no processes in place to engage and train youth and new inhabitants of the villages. While village communities have been maintained, there is an increasing lack of ministry resources (e.g., staff, transportation, and communications) to take the place of NGOs like CRS after a program ends. There is little management of knowledge around project data, which is further exacerbated by staff changes in NGOs, government ministries, and donors. Project data (proposal content, monitoring data, evaluation results, participant lists, partner names, and exit agreements) must be managed ethically, locally and be held online, accessible for future projects to use and for villages to conduct self-evaluations.

Civic Technologies as tools for change: Is balance between online and offline actions a necessary condition for achieving social change through civic innovation? - Civic Innovation Accelerator Fund (Fundación Avina & Omidyar Network, 2016)
Keywords: IMPORTANT

Scaling up - From Vision to Large-Scale Change - A Management Framework for Practitioners - Cooley, Larry; Kohl, Richard; Ved, Rajani R. (MSI, 2016)
This document was originally published in 2006 and re-issued in a substantially revised Second Edition in 2012.

Learning Evaluation Theory: Brinkerhoff’s Success Case Method - Downes, Andrew (Watershed, 2016)
Brinkerhoff’s model isn’t restricted to learning. It can be used to analyze any major business change, such as the purchase of new equipment or implementation of a new process. It’s based on the assumption that any initiative, no matter how successful or unsuccessful, will always include some success and some failure. It seeks to uncover the most impactful successes and failures of an initiative and then tell the stories behind them, backed by evidence. Your organization can use these stories to learn how to be more successful in the future.

Connecting the Dots for Accountability: Civil Society Policy Monitoring and Advocacy Strategies - Fox, Jonathan; Halloran, Brendan (Transparency & Accountability Initiative, 2016)
Keywords: IMPORTANT

Assessing Innovation Potential for Social Impact: Overview for Social Sector Leaders - Global Knowledge Initiative (2016)

Funding Systems Change: Challenges and Opportunities - Junge, Jordan; Pulford, Louise; Spitz, Kelsey; Draimin, Tim; Birney, Anna; Sinha, Rachel (SIX, SIG, Forum for the Future & The Systems Studio, 2016)

Value for Investment. A Practical Evaluation Theory - King, Julian (Kinnect Group, 2016)
I am a public policy consultant from Auckland, New Zealand. My practice specializes in evaluation and economic analysis. In this booklet I set out a model for evaluating value for investment (VFI) in social programs. The model defines VFI as an evaluative question about an economic problem. It uses explicit evaluative reasoning to provide a clear answer to the VFI question. Methods are matched to context, to provide the specific mix of evidence (economic and/or other) needed to support a wellreasoned, well-evidenced evaluative judgment.

Reciprocity in International Volunteer Cooperation - Lough, Benjamin J. (Fredskorpset Norway, 2016)

More than just luck: Innovation in humanitarian action - Obrecht, Alice; Warner, Alexandra T. (Humanitarian Innovation Fund, 2016)
In South Sudan, a new water treatment system that provides a community with more clean water at a lower cost. In Sierra Leone, a poster explaining how to prevent the...

DAC - Mid-term Peer Review of Switzaerland - 2016 - OECD (OECD, 2016)

A Holistic Approach to Evaluating Research - Ofir, Zenda; Schwandt, Thomas; Duggan, Colleen; McLean, Robert (IDRC, 2016)

Winners and Losers in the Global App Economy - Pon, Bryan (2016)

All about behaviour: KAPE, Adaptation and ' Sticky' Institutional Change - Power, Greg (Global Partners Governance, 2016)
This paper describes Global Partners Governance’s (GPG’s) approach to institutional reform and political change. Developed over the last decade of working in some of the most complex and sensitive political environments with politicians and officials in parliaments, political parties, ministries and local government, it describes the KAPE® (knowledge-application-practice-effect) methodology that we adopt to get ‘sticky’ institutional and behavioural change. Contents 1) Two dimensions of ‘adaptive programmes’: Flexible delivery and getting behaviour change 2) Enabling Change: KAPE and The Logic of Institutional Reform (Knowledge-Application-Practice-Effect) - K: Knowledge – Defining the problem and what to do about it - A: Application – Making Systems Work in Practice - P: Practice – Pockets of good practice and establishing ‘the new normal’ - E: Effect – Improved performance and the ‘Ripple Effect’ 3) Measuring Impact: Monitoring and Evolving 4) Conclusion: Behavioural insights, adaptive management and sticky change

Reflections from a Realist Evaluation in Progress: Scaling Ladders and Stitching Theory - Punton, Melanie; Vogel, Isabel; Lloyd, Rob (IDS, 2016)
Realist evaluation provides valuable insights into how and why programmes lead to change, and can generate transferable lessons to help practitioners roll out or scale up an intervention. However, as ...

Engines of Change: What Civic Tech can learn from Social Movements - Purpose; Omidyar Network (Omidyar Network and Purpose, 2016)

Doing things differently: Rethinking monitoring and evaluation to understand change - Saferworld (Saferworld, 2016)
Doing things differently: Rethinking monitoring and evaluation to understand change Learning paper Over the past four years, Saferworld has put in place a way of monitoring, evaluating and learning from our work focused on behaviour and relationship change. This paper outlines the process we have gone through to adapt, embed, and embrace an approach inspired by Outcome Mapping and Outcome Harvesting (OH). Key benefits of our monitoring, evaluation and learning approach are: It is simple, but promotes complex discussion and analysis. It allows conflict- and gender-sensitivity to be built into programmes; it promotes analysis of and adaptation to context. Bringing front-line staff and partners into wider conversations with others substantially increases cross-organisational learning.

State Accountability and Voice Initiative (SAVI) - Approach Paper Series - SAVI (DFID, 2016)
We are piloting a new approach to identifying and supporting our partners. This breaks with convention in many ways in order to improve impact. Our approach is summarized in the Introduction to SAVI’s way of working. We have broken our approach down into key components – depicted in the form of our ‘knowledge tree’ –...
Keywords: IMPORTANT

Dispatch on Switzerland's International Cooperation 2017–2020: Key points in brief - SDC (Federal Department of Foreign Affairs FDFA, 2016)
The “Dispatch on Switzerland’s International Cooperation, 2017–2020”, is a report to the Swiss Parliament outlining the strategic priorities proposed by the Federal Council in this area. It includes framework credits for each of the five policy instruments used by Switzerland to implement its international cooperation strategy. These are implemented by the Swiss Agency for Development and Cooperation (SDC) of the Federal Department of Foreign Affairs (FDFA), the Human Security Division (HSD) of the FDFA’s Directorate of Political Affairs, and the State Secretariat for Economic Affairs (SECO) of the Federal Department of Economic Affairs, Education and Research (EAER).

Towards an Alternative Development Management Paradigm? - Shutt, Cathy (Expertgruppen för biståndsanalys (EBA), 2016)
Demonstrating results has been a concern in international development cooperation ever since it was started and in recent years there has been an increased focus on achieving and reporting on “results”. Despite the fact that everyone involved in development cooperation wants to make a difference there has been a growing criticism from practitioners about the “results agenda” based on a concern that the approaches used are not fit for purpose. In the EBA-report, Cathy Shutt, at the University of Sussex, scrutinizes the recent critical debate about results based management, and the main arguments and motives behind the criticism. She shows that the debate is not only a matter of obsessive measurement and reporting of meaningless numbers for political accountability, but also a matter of problematic assumptions and how we think about development, evidence and learning. In the report, Shutt also explores what could be learned from those who are not just critiquing results based management approaches but also proposing alternatives. Are these new alternatives an answer to the criticism?
Keywords: IMPORTANT

An introduction to multi-stakeholder partnerships - Stibbe, Darian; Prescott, Dave (The Partnering Initiative, 2016)
This briefing document for the GPEDC High Level Meeting in November 2016 offers a definition of multi-stakeholder partnerships (MSPs) and shows how they are being used, where they have come from, and what the different sectors bring to the table. It provides an overview of the challenges the present, what they require from individuals and organisations, and how they get started. The report was written with the support of the PEP Initiative, and funded by the Netherlands Ministry of Foreign Affairs.

Collaborating, Learning, and Adapting Framework & Key Concepts - USAID (USAID, 2016)
Although collaborating, learning, and adapting (CLA) are not new to USAID, they often do not happen regularly or systematically and are not intentionally resourced. The CLA framework above identifies components and subcomponents to help you think more deliberately about what approach to CLA might be best tailored to your organizational or project context. The framework recognizes the diversity of what CLA can look like in various organizations and projects while also giving CLA structure, clarity, and coherence across two key dimensions: CLA in the Program Cycle: how CLA is incorporated throughout Program Cycle processes, including strategy, project, and activity design and implementation; and Enabling Conditions: how an organization’s culture, business processes, and resource allocation support CLA integration. Organizations need both integrated CLA practices appropriate for their context and conducive enabling conditions to become stronger learning organizations capable of managing adaptively. The framework stresses the holistic and integrated nature of the various components of CLA to reinforce the principle that CLA is not a separate workstream—it should be integrated into existing processes to strengthen the discipline of development and improve aid effectiveness.

Multi-project programmes functions, forms and implications for evaluation and learning - Buffardi, Anne; Hearn, Simon (Methods Lab, 2015)
Multi-project programmes can serve different purposes. For instance, they may coordinate multiple implementing entities; standardise management and technical support; compare intervention approaches across different contexts; enhance leverage through joint action; or foster sustainability by building relationships among organisations. • At the same time, multi-project programmes are costly, potentially duplicate other mechanisms that fulfil similar functions, and can dilute focus and create confusion. • To guide decisions on what coordination, evaluation and learning mechanisms are needed, it is helpful to articulate the intended purpose of using a programme model. Identifying the purpose(s) can help staff determine what types of knowledge sharing strategies may be most useful to the programme, and how project and programme theories of change can be sequenced

What does ‘adaptive programming’ mean in the health sector? - Tulloch, Olivia (ODI, 2015)
• There is considerable interest in the concept of adaptive development and what it may look like in different sectors, including health. • Adaptive types of programming from the health sector are relatively advanced; as we work towards the Sustainable Development Goals (SDGs), experiences in health can provide useful lessons for other areas of development. • The international health community may not use the label ‘adaptive development’, but many are already conducting adaptive work. ‘Quality improvement’ is one such tried and tested approach. • Quality improvement is problem-driven, iterative and flexible. The methodology and principles can be used to identify, test and implement changes in any context or part of a health system. • Quality improvement is being used successfully, but there is still limited experience and evidence of how to apply its values and philosophy beyond the project level and embed it within national structures and systems. • Systems thinking and analysis of the political economy environment may help embed complex interventions like quality improvement and sustain their achievements

Learning to Adapt: Exploring Knowledge, Information and Data for Adaptive Programmes and Policies - Workshop Summary Report - fhi360 (IDS, 2015)
Adaptive management is increasingly seen as critical capability for development programmes and policies that are more effective, efficient, relevant and sustainable. There is increasing recognition that such work requires significant changes to the organizational structures, management processes, accountability and performance cultures and indivi

Summary of Indicators for Combating Wildlife Trafficking - Environmental Incentives; Foundations of Success; ICF International (USAID, 2015)
USAID's Measuring Impact conducted a survey and analysis of existing wildlife crime indicators to inform the development of USAID indicators and build the evidence base for better alignment of the Agency’s monitoring efforts with best practices. This report summarizes the search strategy and main results of the survey.

Theory of Change Thinking in Practice - van Es, Marjan; Guijt, Irene; Vogel, Isabel (2015)
Want to know better how your interventions can contribute to change? A Theory of Change (ToC) approach helps in deepening your understanding - and that of your partners - of how you collectively think change happens and what the effect will be of your intervention. Not only does it show what political, social, economic, and/or cultural factors are in play, it also clarifies your assumptions. Once a ToC has been developed, it can be used to continually reflect on it in ways that allow for adaptation and checking of assumptions of your intervention. This user friendly guideline helps you to use a Theory of Change approach. Theories of change are the ideas and beliefs people have – consciously or not – about why and how the world and people change. How people perceive and understand change and the world around them is infused by their underlying beliefs about life, human nature and society. They are deep drivers of people’s behaviour and of the choices they make. Social change processes are complex and characterised by non-linear feedback loops: our own actions interact with those of others and a myriad of influencing factors. This triggers reactions that cannot be foreseen and makes outcomes of change interventions unpredictable. Given these uncertainties, how can we plan strategically and sensibly? How can social change initiatives move forward in emerging change processes in a flexible way, while remaining focused on the goal? In this context of complexity, Hivos has found a theory of change (ToC) approach useful in guiding its strategic thinking and action, as well as its collaborative efforts with others. As it fosters critical questioning of all aspects of change interventions and supports adaptive planning and management in response to diverse and quickly changing contexts. It contributes to the quality and transparency of strategic thinking, and therefore to personal, organisational and social learning. This guide builds on the experiences of Hivos working with a ToC approach. It is a practical guide for Hivos staff in applying a Theory of Change approach, but is also very useful for others working on social change such as social entrepreneurs and innovators. This guide builds on the work of a ToC Learning Group initiated by Hivos and comprising of staff of the Centre for Development Innovation (CDI) of Wageningen University and Research Centre and of experts Iñigo Retolaza Eguren, Isabel Vogel and Irene Guijt. For current thinking and work on the use of Theory of Change thinking in complex change processes, see http://www.theoryofchange.nl

Implementing Innovation: A User's Manual for Open Government Programs - Reboot Design (2015)
This guide draws from our experience around the world helping government reformers achieve real change. It is a practical resource for anyone working to implement an open government initiative.

Why Lean Enterprise Transformation is Hard - McClure, Dan (ThoughtWorks, 2015)

Beyond the toolkit: supporting peace processes in Asia - Denney, Lisa; Barron, Patrick (The Asia Foundation, 2015)

Addressing gender in impact evaluation: What should be considered? - Fletcher, Gillian (Methods Lab, 2015)
Gender and sexuality are intimately entwined; we must not lose sight of the ways in which gender affects non-heterosexual people, transgender people and people who do not identify as either male or female. • Gender and gender-related injustice is a feature of all interventions, whatever the focus, be it agriculture, capacity building, disaster management, education, health, peace building, water, sanitation and hygiene, or other. • Showing an increase in the number of women participants in an intervention is not the same as demonstrating gender impact. An ‘add women and stir’ approach is not good enough. • A good intervention design will identify critical inequalities and conduct a needs assessment that clearly identifies gender-related issues. If this needs assessment feeds directly into the programme theory, it will facilitate assessment of the intervention’s gender-related impact and will be more likely it is to have positive gender-related impact.

Operationalizing the Science of Delivery Agenda to Enhance Development Results - Gonzalez Asis, Maria; Woolcock, Michael (Washington, DC: World Bank, 2015)
The clear development gains achieved in recent decades should not deflect attention from the scale and type of challenges that remain. The strategies largely responsible for these initial gains have been technical reforms promoting economic growth and logistical systems supplying basic inputs. Today, strategies are needed that focus on enhancing the quality of implementation— for example, ensuring learning and not just building schools and enrolling students. This concern now spans numerous domains of professional practice (especially health) and has entered World Bank discussions framed as the “science of delivery.” At the World Bank, the Global Delivery Initiative (GDI) is an operational manifestation and extension of these ideas. To date, the GDI has prepared a number of different case studies across numerous sectors on ways in which innovative teams solve particular problems during project implementation. On the basis of the initial case studies, the authors outline five key principles of how high-quality implementation occurs and invite others to add to this growing storehouse of knowledge. Specifically, task teams are encouraged to develop “live” case studies by and for their staff, documenting how, in real time, implementation challenges are being met. Projects must “learn” more rapidly and systematically how to solve the myriad range of complex implementation challenges they inevitably encounter, since most of these (by definition) cannot be anticipated ex ante. Delivery challenges of this kind will only intensify in the coming years as citizens demand effective responses to ever-more complex—and contentious—policy domains, such as justice, regulation, and taxation.

Adaptive Management Tools: Concept Notes System - Mercy Corps (Mercy Corps, 2015)
Mercy Corps finds adaptive management is supported by four different but interconnected elements: Culture, People & Skills, Tools & Systems, and Enabling Environment. These elements are discussed in the context of one of Mercy Corps’ complex programmes in pastoral Ethiopia.

Webinar: Operationalizing Adaptive Management: What it takes. Why it matters. - Mercy Corps (Mercy Corps, 2015)
Development actors increasingly agree that managing programs adaptively – especially complex interventions – can improve their effectiveness. A growing body of evidence supports this claim. But what does adaptive management look like in practice? What does it require of managers and donors to make happen? How can we reconfigure incentives and success metrics to support adaptation, while remaining compliant?

Assessing the Evidence: The Effectiveness and Impact of Public Governance-Oriented Multi-Stakeholder Initiatives - Brockmyer, Brandon; Fox, Jonathan A. (Transparency & Accountability Initiative, 2015)
Transnational multi-stakeholder initiatives (MSIs) – voluntary partnerships between governments, civil society, and the private sector – are an increasingly prevalent strategy for promoting government responsiveness and accountability to citizens. While most transnational MSIs involve using voluntary standards to encourage socially and environmentally responsible private sector behavior, a handful of these initiatives – the Extractive Industries Transparency Initiative (EITI), the Construction Sector Transparency Initiative (CoST), the Open Government Partnership (OGP), the Global Initiative on Fiscal Transparency (GIFT) and the Open Contracting Partnership (OCP) – focus on information disclosure and participation in the public sector. Unlike private sector MSIs, which attempt to supplement weak government capacity to enforce basic social and environmental standards through partnerships between businesses and civil society, public sector MSIs ultimately seek to bolster public governance. But how exactly are these MSIs supposed to work? And how much has actually been achieved?The purpose of this study is to identify and consolidate the current state of the evidence for public governance-oriented MSI effectiveness and impact. Researchers collected over 300 documents and interviewed more than two-dozen MSI stakeholders about their experiences with five public governance oriented multi-stakeholder initiatives.This report provides a ‘snapshot’ of the evidence related to these five MSIs, and suggests that the process of leveraging transparency and participation through these initiatives for broader accountability gains remains uncertain. The report highlights the ongoing process of defining MSI success and impact, and how these initiatives intersect with other accountability actors and processes in complex ways. The study closes with key recommendations for MSI stakeholders.
Keywords: Accountability; Participation; Program evaluation; Transparency; global governance; multi-stakeholder

Strengthening Accountability Ecosystems: a Discussion Paper - Halloran, Brendan (Transparency & Accountability Initiative, 2015)

Strategy Testing: An innovative approach to monitoring highly flexible aid programs - Ladner, Debra (The Asia Foundation, 2015)
The international development community has increasingly embraced the idea that finding durable solutions to complex development problems requires new ways of working that move beyond industry norms. This paper makes an important contribution to the current debate by outlining an innovative monitoring system called Strategy Testing (ST). This is the third paper in the Working Politically in Practice paper series, launched together with the Australian Department of Foreign Affairs and Trade.

Building a Learning Culture – The case of the Market Development Facility in Fiji - Miehlbradt, Alexandra (The Donor Committee for Enterprise Development, 2015)
Keywords: Case Report

SAVI Approach Paper 1: Core Values - SAVI (London, 2015)
Governance reform is about government and citizens working together in more responsive, inclusive and accountable ways for the benefit of citizens. More responsive, inclusive and accountable attitudes and behaviour on the part of government and non-government stakeholders are the critical factors which lead to meaningful reform processes, and replicate and sustain reforms beyond the lifetime...

SAVI Approach Paper 10: Engaging State Governments - SAVI (DFID, 2015)
SAVI as a programme does not directly work with state governments – but we work in close conjunction with sister programmes who are supporting state governments on governance and sector reforms. SAVI supports non-government and SHoA partners to play their part in promoting more responsive, inclusive and accountable state governance delivering better services for citizens....

SAVI Approach Paper 2: Programme Design - SAVI (DFID, 2015)
SAVI supports citizen engagement in governance through a facilitated partnership approach, in contrast to the usual approach of grants to civil society organisations (CSOs). The overall aim is to facilitate and support working relationships and processes of reform that are home-grown, self-sustaining and, after initial engagement, not dependent on external support. Our way of working...

SAVI Approach Paper 3: Theory of Change - SAVI (DFID, 2015)
SAVI’s theory of change is a simple, practical guide that staff and partners use to plan and to monitor change – as well as to reflect on and enhance their own effectiveness. It sets out broad stages of attitude and behaviour change over time to facilitate effective citizen engagement in governance processes, systems and structures....

SAVI Approach Paper 4: Thinking and Working Politically - Supporting partners and staff through a participatory apporach to political economy analysis - SAVI (DFID, 2015)
Thinking and acting politically is central to the SAVI programme. We support staff and partners to analyse the power relations that shape change in their state, and to use this knowledge to inform their decision-making. This includes decisions made by SAVI state teams relating to the issues and partners they engage with and support, and...

SAVI Approach Paper 5: Defining and Measuring Results - SAVI (DFID, 2015)
SAVI aims to facilitate replicable and sustainable processes of citizen engagement in governance. The programme in each state is locally defined, flexible and adaptive, and results are not predictable in advance. Standardised monitoring tools are not applicable, and consequently we have evolved our tools and frameworks during the programme through processes of learning by doing....

SAVI Approach Paper 6: Engaging Civil Society - SAVI (DFID, 2015)
SAVI state teams provide support to CS groups to become more effective agents of citizens’ voice and public accountability, through a variety of mutually reinforcing interventions. These include: hands-on support to demonstration civil society Advocacy Partnerships (APs) facilitating working partnerships between civil society APs, SHoAs, and the media brokering working relationships between all of these...

SAVI Approach Paper 7: Promoting Gender Equality and Social Inclusion - SAVI (DFID, 2015)
We promote attention to gender equality and social inclusion in all of our engagement with CS groups, the media and SHoAs, and in all of the issues and processes they work on. In all the states we work in, we also support partners to focus on some issues and form some partnerships and networks which...

SAVI Approach Paper 8: Engaging the Media - SAVI (DFID, 2015)
The overall aim of SAVI engagement with the media is for media representation of citizens’ interests to become normal, and play its part in helping state governments to be more responsive, inclusive and accountable to their citizens. Media partners – individual media personnel and selected media houses – are supported to be more effective agents...

SAVI Approach Paper 9: Engaging State Houses of Assembly - SAVI (DFID, 2015)
The overall aim of SAVI engagement with SHoAs is to promote lasting reforms that are not dependent on external funding and which promote an increasingly more responsive and accountable relationship between Nigerian state governments and their citizens. SHoAs are supported to be more effective agents of citizen voice and public accountability, demanding better performance from...

Using Information on Results in Program Management – The case of Samarth-NMDP in Nepal - Stewart, Tim; Joshi, Sanju; Miehlbradt, Alexandra (The Donor Committee for Enterprise Development, 2015)
Keywords: Case Report

Theories of Change: time for a radical approach to learning in development - Valters, Craig (ODI, 2015)
The Theory of Change approach demands a radical shift towards more and better learning in development thinking and practice, creating a productive and much-needed space for critical reflection.

Engineering Complex Scaled Up Innovations in the Humanitarian and Development Sector - McClure, Dan; Gray, Ian (ThoughtWorks, 2015)

Guidelines for writing a case study on implementation (Book Project: Smart Implementation in Governance) - Kirsch, Renate (GIZ GmbH, 2015)

Evaluability assessment for Impact Evaluation: Guidance, checklists and decision support - Peersman, Greet; Guijt, Irene; Pasanen, Tiina (ODI, 2015)
This guidance note focuses on the utility of, and guidance for, evaluability assessment before undertaking an impact evaluation. The primary audience for this guidance note is evaluators conducting an evaluability assessment for impact evaluation. The secondary audience is people commissioning or managing an evaluability assessment for impact evaluation, as well as funders of an impact evaluation. Sections one and two provide an overview of evaluability assessment and how it can be used for impact evaluation. Section three provides guidance for planning to undertake an evaluability assessment for impact evaluation. This is informative for all intended users of the guidance note. Section four includes checklists and decision support for evaluability assessments. The checklist is geared to those conducting the evaluability assessment and can be adapted to suit a particular context or purpose. The decision support provides those conducting an evaluability assessment with evidence-based recommendations for impact evaluation funders and commissioners, about whether, when and how to proceed with the evaluation. Sections five and six provide guidance on what to do after the assessment is concluded, and offer lessons learned from evaluability assessments in practice.

Framework for the Biodiversity Cross-Mission Learning Program - Environmental Incentives; Foundations of Success; ICF International (USAID, 2015)
USAID’s Bureau for Economic Growth, Education, and the Environment/Office of Forestry and Biodiversity (E3/FAB) launched the five-year Measuring Impact program (MI) to help strengthen USAID’s biodiversity programs by building the capacity of the Agency to design, monitor, and evaluate effective programs and by enhancing the evidence base that informs programming decisions. A core programmatic strategy of MI is to “Improve Biodiversity Conservation Approaches” with the life-of-project (LOP) objective that E3/FAB and the Missions engaged with MI have a greater understanding of conditions under which commonly deployed conservation strategic approaches are effective. To accomplish this, MI is working with E3/FAB to design and implement a Biodiversity Cross-Mission Learning Program (Learning Program) to systematically capture and share lessons on theories of change1 (TOCs) for common conservation strategic approaches in the USAID biodiversity portfolio.

Watering the Grassroots: A Strategy for Social Movement Support - Joyce, Mary (Transparency & Accountability Initiative, 2015)
In this third and final think piece exploring the role of grassroots organizations and movements for state accountability (find the previous pieces here and here), the author Mary Joyce, proposes new thinking about how funders and other external organizations can best support social movements. Funding and other support for popular organizations and movements is not straightforward, and there are potential risks and distortions involved. Interested supporters need to clearly identify and articulate these obstacles in their discussions with movement activists before pursuing and funding or other support activities. This think piece outlines several potential avenues of support: small, flexible grants; funding intermediary organizations; providing external consultants or researchers to work with movements; and other options as well. These suggestions provide ideas for funders and other organizations seeking to move beyond projects to supporting robust accountability ecosystems, in which grassroots organizations and movements play a critical role.

A Movements Perspective on Transparency and Accountability - Joyce, Mary; Walker, Tom (Transparency & Accountability Initiative, 2015)
This Think Piece describes social movements pursuing government accountability and the challenges and opportunities to supporting them.

Using Data to Influence Government Decisions. Opportunities and Challenges for User-Centered Design to Improve Governance Data Impact - Reboot (Reboot & OSF, 2015)

Managing the Journey to Scale Up Innovation in the Humanitarian and Development Sector - McClure, Dan; Gray, Ian (ThoughtWorks, 2015)

Managing Complexity: Adaptive Management at Mercy Corps - Mercy Corps (2015)
Managing Complexity: Adaptive management at Mercy Corps

Why is it so hard for non-state actors to be heard? Inside Tanzania's education policies - Katera, Lucas (MAVC, 2015)
Lessons from policy in Ghana, Kenya, South Africa and Tanzania

An Adaptive Capacity Guide Book: assessing, building and evaluating the capacity of communities to adapt in a changing climate - Jacobs, B.; Nelson, R.; Kuruppu, N.; Leith, P. (University of Technology and University of Tasmania, 2015)
A capacity to adapt to change is essential for managing Australia’s natural resources. The individuals, communities and organisations who manage our natural resources all have an innate capacity to adapt to change. Changes in climate, markets and technology have shaped the way we adapt the management of natural resources in urban, rural and coastal landscapes. Some of these changes are predictable and easy to manage. Others are expected, but their timing and magnitude are uncertain. Whatever the future holds, this guide can be used to build our capacity to meet future change with confidence.

Systems change: A guide to what it is and how to do it - Ellen Harries; Rachel Wharton; Rob Abercrombie (NPC, 2015)
Systems change has been attracting the attention of those in the social sector who want to deal with the root causes of problems, but, despite the buzz, much of what is written is abstract in tone. With the support of LankellyChase Foundation we have produced this guide to plug a gap in the systems change literature—providing accessible material and recommendations for action. Systems change has been attracting the attention of a range of progressive charities, funders and practitioners who are interested in dealing with the root causes of social problems. But while there is a buzz about a subject relatively new to the social sector, it is easy to feel frustrated by the literature—much of what is written is abstract in tone and there are few examples of success. We have produced this paper to address this problem and offer accessible material and recommendations for action. This systems change guide: Clarifies what is meant by systems and systems change Describes the main perspectives on systems change Outlines good practice for systems change Identifies what is and is not agreed upon by experts in the field Provides recommendations for charities, funders and the public sector on how to act systemically. We hope this presents a manageable introduction to the systems change field, especially for those new to it, and also guides those interested in acting systemically to improve the lives of people in need. Our conclusion is that although it may not be as novel as some claim, there is a good deal of value in a systems change approach and it offers a welcome reminder of what effective action looks like when it comes to the pursuit of social change.

Mentoring Programmes: Supporting Effective Technology Use in Transparency and Accountability Organisations - Herringshaw, Vanessa; Faith, Becky (Transparency & Accountability Initiative, 2015)
The global movement to hold governments and companies accountable is growing rapidly, and technology can play a vital role. Some actors harness this potential to brilliant effect. But many others waste precious money and staff time on technology that isn’t a good fit for their aims or capacity. Mentorships can be a really effective way …

DFID’s approach to delivering impact - ICAI (Independent Commission for Aid Impact (ICAI), 2015)
UK aid, at its best, makes a real and positive difference to the lives and livelihoods of poor people around the world. Ensuring the best possible performance across a large and multifaceted aid programme is, however, a complex management challenge. This report reviews ICAI’s previous 44 reports and looks at how well DFID ensures positive, long-term, transformative impact across its work.

Changes for Justice Project - Final Report - Chemonics International (USAID, 2015)

Improving the Measurement of Civic Space - Malena, Carmen (Transparency & Accountability Initiative, 2015)
‘Civic space’ – i.e. the freedom and means to speak, access information, associate, organise, and participate in public decision-making – is essential to the healthy functioning and development of any society and a precondition for accountable governance and social justice. When civic space is restricted, human and civil rights are denied, government accountability is jeopardised, …

Investment Climate Reform: Doing it Differently - Manuel, Clare (DFID-LASER Programme, 2015)
LASER synthesis papers aim to help donors and other stakeholders better understand why and how to approach investment climate reform programming differently. The papers reflect emerging best practice and lessons learnt on what works and what does not work in doing development differently. The papers have been peer-reviewed by experts in the field including senior advisers at DFID, World Bank, IFC and the Donor Committee for Enterprise Development (amongst others). First synthesis paper - Investment climate reform: doing it differently June 2015 Drawing on our experience in taking a problem-focused approach to investment climate reform, we discuss in this peer-reviewed paper emerging lessons and reflections on what works, both in recent literature and in practical experience.

Measuring Impact. Making Use of the Portfolio: Organizational Learning at USAID - Peabody, Shawn (USAID, 2015)
This technical analysis explores previous and ongoing social learning efforts, best practices, challenges, and lessons in USAID as a foundation for improving the implementation and design of the Agency’s forestry and biodiversity programs. This analysis is particularly relevant as the Bureau of Economic Growth, Education and the Environment’s Office of Forestry and Biodiversity (E3/FAB) begins to develop a Cross-Mission Learning Program under the Measuring Impact initiative, a five-year activity to promote the adoption of best practices in the USAID program cycle among Missions using biodiversity funds. The findings reported in this document can help inform the overall design and structure of the Learning Program and provide insight into possible challenges and best practices.

Supporting Learning? Exploring the relationship between grantee learning and grantmaking practice in the transparency and accountability sector - Ross, Jenny (INTRAC, 2015)
Learning is fundamental to work on transparency and accountability in complex environments. But how can funding practices best support learning?

The Qualitative Impact Assessment Protocol (QUIP) - Copestake, James; Remnant, Fiona (2015)
This brief presents an overview of the QUIP in three steps: the background to the QUIP and its main aims; the data collection and analysis methodology; and QUIP in the context of other approaches to evaluation. Each section can be read independently, depending on time available.

Mobilizing Accountability: Citizens, Movements and the State - Halloran, Brendan; Flores, Walter (Transparency & Accountability Initiative, 2015)

Scaling: Innovations Missing Middle - McClure, Dan; Gray, Ian (ThoughtWorks, 2015)

Thinking and Working Politically: Professional Development Reading Pack - Booth, David (GSDRC Applied Knowledge services, 2015)

Effective Governance. Strategy for Australia’s aid investments - DFAT (Department of Foreign Affairs and Trade, Australian Government, 2015)

Doing legal empowerment differently: Learning from pro-poor litigation in Bangladesh - O'Neil, Tam; Valters, Craig; Farid, Cynthia (ODI, 2015)
Marginalised groups and their allies can, and do, use the law and justice systems, including public interest litigation, to improve their access to rights, goods and services.  Yet there is no automatic link between legal action and improved outcomes for poor people.  Where some minimum conditions are met – a progressive legal framework, a sympathetic judiciary and legal advocacy organisation – pro-poor litigation is a potential tool in disputes over rights and resources.  But concrete benefits for poor people also requires state action to enforce progressive rulings.  Much depends on whether claimants, legal activists and state reformers cooperate around shared interests, and whether activists can negotiate power and interest structures to motivate government action to implement rulings.  Effective strategies are those that link litigation with grassroots legal action and other forms of political and social activism and advocacy.  Donors need to fund legal advocacy organisations in ways that enable them to select social issues that are locally relevant and political feasible – and allows activists and reformers to work in politically smart and adaptive ways.

Beyond enforcement: communities, governance, incentives and sustainable use in combating wildlife crime. Symposium Report. - Roe, Dilys (IIED, 2015)

The effect of government responsiveness on future political participation - Mellon, Andrew Jonathan; Sjoberg, Fredrik Matias; Peixoto, Tiago Carneiro (The World Bank, 2015)
What effect does government responsiveness have on political participation? Since the 1940s political scientists have used attitudinal measures of perceived efficacy to explain participation. More recent work has focused on underlying genetic factors that condition citizen engagement. The authors develop a ‘calculus of participation’ that incorporates objective efficacy, the extent to which an individual’s participation actually has an impact, and test the model against behavioral data from FixMyStreet.com (n=399,364). The authors find that a successful first experience using FixMyStreet.com (e.g., reporting a pothole and having it fixed) is associated with a 54 percent increase in the probability of an individual submitting a second report. The authors also show that the experience of government responsiveness to the first report submitted has predictive power over all future report submissions. The findings highlight the importance of government responsiveness for fostering an active citizenry, while demonstrating the value of incidentally collected data to examine participatory behavior at the individual level.

Introduction to Project Management - The Case of GIZ Projects - Lacroix, Eric J. (Khulna University, 2015)
In this course, we will try to understand the complexity and the structure of the management of a development project implemented by the GIZ for the German Development cooperation. We will first define the frame of management and project to enter in the map of two logics. Then, we will overfly the 5 success factors of Capacity WORKS, with concrete examples in annexe. It will bring us to the monitoring, the standard processes in GIZ cooperation projects, the basic tools for project implementation and the evaluation. The team building will allow us to enter in a specific aspect of project management.

Adapting development: improving services to the poor - Wild, Leni; Booth, David; Cummings, Clare; Foresti, Marta; Wales, Joseph (ODI, 2015)
On current trends, it will take decades or longer to bring basic services to the world’s most disadvantaged people. Meeting this challenge means recognising the political conditions that enable or obstruct development progress - a radical departure from the approach of the Millennium Development Goals.
Keywords: Practice

Qualitative Comparative Analysis - A Rigorous Qualitative Method for Assessing Impact - Baptist, Carrie; Befani, Barbara (Coffey, 2015)

Who is ICT innovation for? Challenges to existing theories of innovation, a Kenyan Case Study - Marchant, Eleanor (University of Pennsylvania, 2015)
Keywords: IMPORTANT

Transparency, Participation, & Accountability Grantmaking Strategy (THE WILLIAM AND FLORA HEWLETT FOUNDATION, 2015)

Doing Problem Driven Work - Andrews, Matt; Pritchett, Lant; Woolcock, Michael (Harvard University, 2015)
Keywords: IMPORTANT

Institutional Change, Political Economy, and State Capabilities : Learning from Edo State, Nigeria - Bain, Katherine; Porter, Doug; Watts, Michael (World Bank, 2015)
This paper is one of a series aimed at deepening the World Bank’s capacity to follow through on commitments made in response to the World Development Report (WDR) 2011, which gave renewed prominence to the nexus between conflict, security, and development. Nigeria is a remarkable illustration of how deeply intractable the cycle of poverty, conflict, and fragility can become when tied to the ferocious battles associated with the political economy of oil. This paper places the corpus of analytic and programmatic work concerning institutional reform in conversation with a now substantial body of work on resource politics and most especially, the debate over the politico-institutional character (sometimes called political settlements or pacting arrangements associated with the order of power) and reform landscape of the petro-state. Recent institution reform policy writing appears to have little to say about the political and economic conditions in which crises and institutional disjunctures may authorize, and thereby enable, agents to embark on institutional reforms. The authors focus on Edo state for two reasons. First, it does not on its face appear to be an obvious location in which to explore a reform experience, given its entanglement in the Niger Delta conflict and the maladies typically associated with state fragility. Second, Edo is of interest also because of the changes that its experience is contributing to the World Bank country team’s effort to engage operationally across all its instruments with the political economy of institutional reform in Nigeria, its largest client country in Africa.

Impact assessment and the quest for the Holy Grail - Bymolt, Roger (KIT Royal Tropical Institute, 2015)

The Advocacy Strategy Framework. A tool for articulating an advocacy theory of change - Coffman, Julia; Beer, Tanya (Center for Evaluation Innovation, 2015)

Producer-led value chain analysis: The missing link in value chain development - Ghore, Yogesh (Coady, 2015)
Introduction and rationale The concept of a value chain is increasingly being applied in the design and implementation of development programs aimed at poverty reduction. As an analytical tool, it provides a useful framework for understanding key activities, relationships, and mechanisms that allow producers, processors, buyers, sellers, and consumers—separated by time and space—to gradually add value to products and services as they pass from one link of the chain to another, making it a “value chain” (UNIDO, 2009). While it has been popular in the private sector ever since it was conceptualized by Michael Porter in the 1980s, more recently various donors and governments have shown interest in its use and have applied it to a range of development interventions, particularly in the area of sector development, livelihoods promotion, small and medium enterprise (SME) development, and rural and economic development. Academics and development organizations have designed numerous instruments for value chain analysis (VCA) and implementation. Beginning in the early 2000s, international organizations and donor agencies have sponsored the development of these tools, including the United States Agency for International Development (USAID) Deutsche Gesellschaft für Internationale Zusammenarbeit (GIZ), the World Bank, the International Finance Corporation (IFC), and the International Labour Organization (ILO). These guides and tools have been used in many development programs to address organizational, donor and local and regional priorities for development. While VCA guides and tools are important in understanding markets and relationships among key stakeholders, a recent comparative review of popularly used guides by Donovan et al. (2013) suggests that most of these guides are designed to be implemented independently of the local context and do not sufficiently focus on mutual learning, whether related to tool design or to the outcomes and impacts of the designed project activities. VCA is often conducted by external experts and the knowledge generated in the process is often confined to reports. Without the capacity building and effective participation of women and men producers—smallholder farmers who hold critical knowledge about the local context—an important link is missing in the entire process of VCA. The insufficient attention to the human, social, and other contextual factors undermines the full potential of the value chain systems approach, not only for economic outcomes, but for the long term sustainability of the intended benefits. So what are the ways to effectively engage producers in the community in the VCA? The producer-led process described here tries to address this basic question. It introduces simplified tools and an approach that ensures farmer participation in data collection, analysis, and identification of opportunities and constraints, and design of value chain interventions. This participation requires: a) an environment in which they feel comfortable to share their knowledge and insights, which is often not the case when extractive surveys and questionnaires are administered to collect information; and b) the use of simple and participatory tools that will allow them to provide inputs into the process of VCA and also help them to understand complex value chain systems and use this understanding for making livelihood decisions. The key steps and tools described are aimed at empowering the women and men farmers to make informed decisions about their own enterprises and how they relate to the value chain, thereby directly contributing to, and influencing, the overall process of value chain development. The process of VCA involving these steps and tools was developed at Coady Institute and first tested with Oxfam Canada and its local partners in Ethiopia in August 2012.

Logics of Government Innovation and Reform Management in China - Husain, Lewis (STEPS centre, 2015)
Since the beginning of reforms in the late 1970s, China has developed rapidly, transforming itself into a middle-income country, raising hundreds of millions out of poverty and, latterly, developing broad-based social protection systems. The country’s approach to reform has been unorthodox, leading many to talk of a specific Chinese model of development. This paper analyses the role of innovation (chuangxin) and experimentation in the Chinese government repertoire and their contribution to management of change during the rapid, complex and interconnected reforms that China is undergoing. ‘Innovation’ is understood as the process of generation, putting into use, and spread, of new ideas. This contemporary focus on innovation is an extension of an older Chinese government attachment to sub-national initiative in policy formulation and development that goes back to the beginnings of the PRC and before. Central government backing for, and endorsement of, proactive agency on the part of sub-national governments responds to a belief that China is too large, and conditions around the country too diverse, to allow adoption of ‘one size fits all’ policy. Sub-national governments are expected to show initiative in adapting policy locally, and creating locally-useable policy solutions within the overall scope of central policy mandates/ paradigms. The paper argues that innovation by sub-national government is systemically embedded: while central government sets the policy agenda, local governments are frontline managers, and develop a range of policy practices. Differences in conditions between localities mean that multiple variant policy practices are often in circulation at any one time. While innovation is not quantifiable in the aggregate, there is much controlled experimentation, freewheeling innovation, and trial and error, all of which are part of a search for new policy fixes and institutional solutions. Many forms of policy transfer and learning are in evidence, including much central learning from sub-national models, as well as sub-national circulation of a range of innovative policy practices. While much government innovation is not ‘original’, and may be ‘inefficient’ or of little systemic usefulness, overall, the churn of government innovation remains valuable in underpinning systemic adaptation and reform. The paper situates the analysis of government innovation within a larger framework on the functioning of Chinese government and international literature on policy transfer, and outlines an agenda for future research on the structural bases of Chinese government innovation and its contribution to adaptive management.

Enabling citizen-driven improvement of public services. Leveraging Technology to Strengthen Accountability in Nigerian Healthcare - ICT4SA (World Bank and Reboot, 2015)
Effective public service delivery begins with knowing whether the services offered are working as intended. We launched a system to enable citizen input on the delivery of public services.

Mobilizing Accountability: A Learning Agenda for Grassroots Organizations and Social Movements Addressing State Accountability - Joyce, Mary; Halloran, Brendan (Transparency & Accountability Initiative, 2015)

Improving development aid design and evaluation: Plan for sailboats, not trains - Kleinfeld, Rachel (Carnegie Endowment for International Peace, 2015)
How do reforms that require political engagement differ from traditional technical reforms? Why is political engagement different, and what are the implications for design and evaluation? How should development programmes that engage politics be designed? And how can those who fund or implement such programmes evaluate whether their efforts are contributing to reform? This report …
Keywords: IMPORTANT

Adaptive Management: Responding to the evolving needs of PRIME's complex systems - Picon, Diana; Wild, Lorenz (Mercy Corps, 2015)

A Call for Innovation in International Development - R4D (Results for Development, 2015)

Strengthening the Humanitarian Innovation Ecosystem - Ramalingam, Ben; Rush, Howard; Bessant, John; Marshall, Nicholas; Gray, Bill; Hoffman, Kurt; Bayley, Simon; Warren, Kim (University of Brighton, 2015)

Introduction to SAVI's way of working - State Accountability and Voice Initiative (SAVI) - SAVI (DFID, 2015)
We are piloting a new approach to identifying and supporting our partners. This breaks with convention in many ways in order to improve impact. SAVI Approach Papers summarise key aspects of SAVI’s way of working. They explain what we do, and why – and link to relevant tools and frameworks. Our approach is summarized in...
Keywords: Practice

The 2015 State of Scrum Report: How the world is applying the most popular Agile approach to projects - Scrum Alliance (Scrum Alliance, 2015)

Peacebuilding and Statebuilding Strategy for SDC’s work in fragile and conflict contexts - SDC (Swiss Agency for Development and Cooperation SDC, 2015)

USAID/Colombia Introduces Political Economy Analysis to Better Adapt Programming to Local Contexts - Sharp, Preston (USAID, 2015)

Chaos Report 2015 - Standish Group (Standish Group, 2015)

2015 Rhode Island Wildlife Action Plan - Chapter 5 Monitoring and Adaptive Management - Terwilliger Consulting (State of Rhode Island, 2015)

The Operational Guide - for the making markets work for the poor (M4P) approach (Second Edition) - The Springfield Centre (SDC & DFID, 2015)
In 2008, SDC and DFID published three documents aimed at improving the understanding and use of the Making Markets Work for the Poor (M4P) approach: the M4P Synthesis, M4P Perspectives and M4P Operational Guide. Since then the field has grown, diversified and, importantly, learned much more. A second edition was commissioned to capture that learning, maintain the momentum and realise the ambition that development can still 'do better'. The second edition provides an accessible resource to help practitioners put the market systems development approach into practice. It explains the key principles and frameworks which guide effective intervention in – and development of – market systems. It addresses common challenges with examples of good practice based on practitioner experience. Who is it for? The Operational Guide is for people whose job entails trying to make market systems work better for poor women and men. These people include: Individuals currently involved in funding or implementing the approach. Development agencies wishing to incorporate market systems thinking and practice in their work. National stakeholders that wish to play a more strategic and catalytic role within market systems. What is new? Application, application, application! Whilst the key principles and frameworks will be familiar, the advice on their operational application – from start to finish – has been significantly bolstered Enhanced guidance on facilitation. Greater emphasis on what constitutes good facilitation practice, building on a wealth of practitioner experience More real-life examples. The Operational Guide is laced with examples from programmes past and present, working in a diverse array of systems Peer learning focus. Direct insights from practitioners, highlighting 'bumps in the road' and how to avoid them. The Guide aims to provide an accessible operational resource to help practitioners put the market systems development approach into practice. It explains the key principles and frameworks which guide the process of effective intervention in – and development of – market systems, addressing common challenges with examples of good practice based on practitioner experience. HOW SHOULD IT BE USED? The Guide explores, sequentially, the key elements of the implementation process: strategy, diagnosis, vision, intervention, measurement and management. It is not intended to be read from cover to cover; readers can go directly to the chapter most relevant to their needs without having read preceding chapters. However each chapter does build upon the preceding one and may refer to other chapters. THE GUIDE IS STRUCTURED AS FOLLOWS: INTRODUCTION Overview of the objectives, structure and format of the Guide CONTEXT A reminder of what market systems development means 1. STRATEGY Core principles and framework for setting programme strategy 2. DIAGNOSIS Core principles and framework for diagnosing system constraints 3. VISION Core principles and frameworks for defining and planning intervention 4. INTERVENTION Core principles and framework to guide effective intervention 5. MEASUREMENT Core principles and framework for measuring results 6. MANAGEMENT Key considerations in managing market systems development programmes GLOSSARY Definition of key terms used in market systems development

A Practical Guide for Collective Action against Corruption - UN Global Compact (UN Global Compact, 2015)
All anti-corruption professionals looking to better understand the relationship between collective action and corruption should refer to these two comprehensive guides that the UN Global Compact published. The 2015 guide has modules that cover theoretical concepts, practical recommendations for undertaking CA initiatives, and multiple in-depth case studies of CA projects around the world.

Feasibility Study: Crowdsourcing High-frequency food price data in rural Indonesia - UN Global Pulse (2015)

Testing the Waters: A Qualitative Comparative Analysis of the Factors Affecting Success in Rendering Water Services Sustainable Based on ICT Reporting - Welle, Katharina; Williams, Jennifer; Pearce, Joseph; Befani, Barbara (MAVC, 2015)
This research conducted by WaterAid, Itad and IRC aims to understand the factors that facilitate and inhibit the success of ICT-based reporting to improve rural water supply sustainability.

Change in challenging contexts - Williamson, Tim (ODI, 2015)
This report explores how genuine institutional change takes place in fragile and conflict-affected states reforms can be supported.

Learning and Results in World Bank Operations: Toward a New Learning Strategy - World Bank (IEG , The World Bank, 2015)
This report is the second in a program of evaluations that the Independent Evaluation Group (IEG) is conducting on the learning that takes place through World Bank projects. Learning and knowledge are treated as parts of a whole and are presumed to be mutually reinforcing.
Keywords: IMPORTANT

Strategic Framework for mainstreaming citizen engagemente in World Bank Group operations - World Bank (The World Bank, 2014)

Development entrepreneurship: how donors and leaders can foster institutional change - Faustino, Jaime; Booth, David (The Asia Foundation, 2014)
Various communities of practice have been established recently to advance the general idea of thinking and working politically in development agencies. This paper makes a contribution by describing the practice of what has been called development entrepreneurship and explaining some of the ideas from outside the field of development that have inspired it.

Making All Voices Count's Research & Evidence Strategy (2014)
Building an evidence-base on what works in technology for voice, transparency and accountability, how, and why.

Evaluating Complexity. Propositions for improving practice - Preskill, Hallie; Gopal, Srik (FSG, 2014)
9 propositions can help evaluators measure progress on complex social problems.
Keywords: IMPORTANT

Bridging & Bonding: Improving Links between Transparency & Accountability Actors - Wanjiku Kelbert, Alexandra (2014)

The Doing Development Different Manifesto - DDD Community (DDD Community, 2014)
Statement from the October 2014 ‘Doing Development Differently’ workshop Too many development initiatives have limited impact. Schools are built but children do not learn. Clinics are built but sic…

A workshop on Doing Development Differently - CID (CID at Harvard University and ODI, 2014)
October 22-23, 2014 A workshop on Doing Development Differently Harvard Kennedy School, Cambridge, MA Hosted by the Building State Capability (BSC) program at the Center for International Development at Harvard University, and the Overseas Development Institute (ODI) with funding from the Governance Partnership Facility.

Navigating complexity: Adaptive management in the Northern Karamoja Growth, Health & Governance program - Allana, Amir (MercyCorps, 2014)
Development actors increasing agree that managing programs adaptively – especially complex interventions – can improve their effectiveness. But what does adaptive management look like in practice?
Keywords: IMPORTANT; Practice

The SAVI programme in Nigeria: Towards politically smart, locally led devlopment - Booth, David; Chambers, Victoria (ODI, 2014)
 More examples are needed of aid programming that works by being responsive to country realities: politically smart, problem-driven and locally led  DFID’s SAVI programme has revealed a hitherto untapped potential for change leading to better development results at state level in Nigeria  SAVI provides low-profile support to state-level organisations and partnerships, building their capacity to engage constructively with government  It avoid the pitfalls of a donor-driven approach by ‘taking the money off the table’  This illustrates the power of facilitated multi-stakeholder engagement and the disadvantages of seeing this in terms of ‘supply side’ and ‘demand side’ governance  The set-up of the DFID suite of state-level programmes, with separately managed sector support and ‘governance’ initiatives, has limited realisation of the potential  Key enabling conditions were that DFID provided space for an experience-based design process and permitted tangible results to be judged retrospectively, not pre-programmed

Accountability, transparency, Participation, and Inclusion. A New Development Consensus? - Carothers, Thomas; Brechenmacher, Saskia (Carnegie Endowment for International Peace, 2014)
The wide-reaching consensus around the normative and instrumental value of accountability, transparency, participation, and inclusion remains less solid than enthusiasts of these concepts might wish.

Beyond magic bullets in governance reform - de Gramont, Diane (Carnegie Endowment for International Peace, 2014)

Theory of Action: Creating Craddle to Career Proof Points - Strive Together (Strive Together, 2014)
StriveTogether’s Theory of Action provides a framework for improving educational outcomes and ensuring a community transforms how it serves children.

Politically smart, locally led development - Booth, David; Unsworth, Sue (ODI, 2014)
Aid donors have found it hard to move from thinking politically to working differently, but there is evidence that they can do so and that this improves outcomes. This paper presents seven examples of where adopting a politically smart, locally led approach has led to better outcomes.
Keywords: Practice

Taking Innovations to Scale: Methods, Applications and Lessons - Cooley, Larry; Linn, Johannes F. (R4D, 2014)
The international development community increasingly recognizes the need to go beyond fragmented, one-of projects. In response, there is now much talk and some action on scaling up successful innovations and pilot projects with an explicit goal of achieving sustainable impact at scale. However, many questions remain about the practical implications of pursuing a systematic scaling up approach and about how the approaches being pursued by diferent institutions and practitioners relate to each other.

Constituent Voice: Technical Note 1 - Keystone Accountability (Keystone Accountability, 2014)

How useful are RCTs in evaluating transparency and accountability projects? - Leavy, Jennifer (IDS, 2014)

Emerging Opportunities: Monitoring and Evaluation in a Tech-Enabled World - Raftree, Linda; Bamberger, Michael (ITAD & Rockefeller Foundation, 2014)
Monitoring and evaluation practice is using information and communication technologies for more timely data, and more inclusive voice and feedback.

Realist Impact Evaluation: An introduction - Westhorp, Gill (Methods Lab, 2014)
• Realist evaluation is a member of a family of theory-based evaluation approaches which begin by clarifying the ‘programme theory’: the mechanisms that are likely to operate, the contexts in which they might operate and the outcomes that will be observed if they operate as expected. • Realist approaches assume that nothing works everywhere for everyone: context makes a big difference to programme outcomes. A realist evaluation asks not ‘what works?’ but ‘how or why does this work, for whom, in what circumstances?’ • Realist impact evaluation is most appropriate for evaluating new initiatives or programmes that seem to work but where ‘how and for whom’ is not yet understood; programmes that have previously demonstrated mixed patterns of outcomes; and those that will be scaled up, to understand how to adapt the intervention to new contexts.

Adaptive Learning - Guerzovich, Florencia; Poli, Maria (Global Partnership for Social Accountability, 2014)

Theories of change in international development: Communication, learning, or accountability? - Valters, Craig (JSRP, 2014)

Lessons Learned from the Forest, Climate and Communities Alliance - Boshoven, Judy (USAID, 2014)
The Forest, Climate, and Communities Alliance (FCCA) was an initiative funded by USAID’s Bureau for Economic Growth, Education, and the Environment/Office of Forestry and Biodiversity (E3/FAB) under the USAID Global Development Alliance (GDA)1 between 2009 and 2013. FCCA was one of the first USAID-funded projects for reducing emissions from deforestation and degradation (REDD+). The Rainforest Alliance (RA) implemented FCCA in Ghana and Honduras. Operating in two different contexts allowed FCCA to generate important lessons learned that may be useful as countries around the world search for approaches to prepare for and implement successful REDD+ mechanisms. In 2013, as FCCA was coming to an end, USAID tasked the Measuring Impact (MI) initiative with an independent review to generate lessons learned from FCCA. In capturing and disseminating lessons from the FCCA project, the Agency intends to contribute to the understanding of what may be effective, and under what conditions, when promoting REDD+ in conjunction with forest and agricultural product certification. MI is a five-year USAID project executed by the E3/FAB Office designed to improve the Agency’s practice of conservation by building USAID’s capacity to better design, manage, evaluate, and learn from biodiversity conservation initiatives. As such, MI was tasked with carrying out this review of lessons learned from FCCA while using RA’s general hypothesis as the basis to describe the project and its lessons.

ALL - The question of inclusiveness in ICT-mediated citizen engagement - Berdou, Evangelia (MAVC, 2014)

The politics of legal empowerment: Legal mobilisation strategies and implications for development - Domingo, Pilar; O'Neil, Tam (ODI, 2014)
 Legal mobilisation can improve the lives of poor or marginalised people by: o contributing to pro-poor change in policy, law and regulation of service delivery across different sectors; o advancing the realisation of rights, and achieving redress for rights violations; o contesting unjust and illegal practices of resource allocation and power relations, including in relation to land and natural resources; o enabling citizens to exercise social accountability through legal action.  The outcomes of legal empowerment of the poor are not politically neutral and need to be understood within broader social and political environments;  Better coordination between justice, sector and governance interventions will maximise the development and social impact of international support for legal empowerment.

MAKING - Fostering new ideas for social inclusion and accountable, responsive governance - Edwards, Duncan (MAVC and IDS, 2014)

COUNT - Understanding the challenge of government responsiveness - McGee, Rosemary (MAVC and IDS, 2014)

VOICES - On speaking, mediation, representation and listening - Oswald, Katy (MAVC, 2014)

Aiding Institutional Reform in Developing Countries: Lessons from the Philippines on what works, what doesn't and why - Booth, David (The Asia Foundation, 2014)

Thinking and working politically in the transparency and accountability field - Halloran, Brendan (Transparency & Accountability Initiative, 2014)

How DFID learns - ICAI (Independent Commission for Aid Impact (ICAI), 2014)
Excellent learning is essential for UK aid to achieve maximum impact and value for money. We take learning to mean the extent to which DFID uses information and experience to influence its decisions. Each ICAI review assesses how well learning takes place. Our reports to date indicate a mixed performance. This review seeks to identify the way DFID learns and what inhibits it from doing so consistently. We drew on our reviews, assessed data from DFID’s own surveys and carried out interviews inside and outside the department. Review DFID generates considerable volumes of information, much of which, such as funded research, is publicly available. DFID does not clearly or consistently link this investment to how it can deliver better impact. We made five recommendations and gave an amber-red score. Findings DFID does not clearly identify how its investment in learning links to its performance and delivering better impact. DFID has the potential to be excellent at organisational learning if its best practices become common. DFID staff learn well as individuals. They are highly motivated and DFID provides opportunities and resources for them to learn. DFID is not yet, however, managing all the elements that contribute to how it learns as a single, integrated system. DFID does not review the costs, benefits and impact of learning. Insufficient priority is placed on learning during implementation. The emphasis on results can lead to a bias to the positive. Learning from both success and failure should be systematically encouraged. Recommendations DFID needs to focus on consistent and continuous organisational learning based on the experience of DFID, its partners and contractors and the measurement of its impact, in particular during the implementation phase of its activities. All DFID managers should be held accountable for conducting continuous reviews from which lessons are drawn about what works and where impact is actually being achieved for intended beneficiaries. All information commissioned and collected (such as annual reviews and evaluations) should be synthesised so that the relevant lessons are accessible and readily usable across the organisation. The focus must be on practical and easy-to-use information. Knowhow should be valued as much as knowledge. Staff need to be given more time to acquire experience in the field and share lessons about what works and does not work on the ground. DFID needs to continue to encourage a culture of free and full communication about what does and does not work. Staff should be encouraged always to base their decisions on evidence, without any bias to the positive.

Beyond the Pioneer: Getting inclusive industries to scale - Koh, Harvey; Hedge, Nidhi; Karamchandani, Ashish (Deloitte Touche Tohmatsu India Private Ltd (DTTIPL), 2014)

Working Politically: A story of Change about the contribution of research evidence to the new Village Law in Indonesia - Pellini, Arnaldo; Angelina, Maesy; Purnawati, Endah (Austrialian Community Development and Civil Society Strenghtening Scheme (ACCESS), 2014)
On 18 December 2013, the Indonesian House of Representatives passed the new Village Law, a vote that was the culmination of a journey that had started in 2007. This Story of Change takes the passing of the Village Law as its starting point and describes the relative influence that research-based evidence, produced by the Institute for Research and Empowerment (IRE), has had at critical junctions of the legislative process. This Story of Change concludes that good quality, research-based evidence is necessary but not sufficient to influence policy-making processes. Researchers and research organisations need to think and work politically to achieve their influencing goals and to adapt to changes in local circumstances.

Problem-driven iterative approaches and wider governance reform - Rao, Sumedh (GSDRC, University of Birmingham, 2014)
Identify evidence which suggests that problem-driven, iterative approaches to public sector reform can deliver more substantial, wider, long-term governance reform.

Achieving reforms in oligarchical democracies: the role of leadership and coalitions in the Philippines - Sidel, John T. (Developmental Leadership Program, 2014)
This paper examines the role of developmental leadership in two major reforms introduced in the Philippines in 2012: the passage into law in December of excise tax reform which significantly raised taxes on cigarettes and alcohol – generally referred to as the Sin Tax Reform – and, in July, the re-registration of voters in the Autonomous Region of Muslim Mindanao (ARMM). These reforms have a) strengthened government finances and healthcare; and b) improved the quality of elections and promoted good governance and conflict resolution in the southern Philippines. Key points: These reforms were not achieved exclusively through the executive leadership of Philippine President Benigno Aquino III, this paper argues. A broader form of developmental leadership was critical to their passage into legislation and their subsequent implementation, made up of reform coalitions that incorporated elements of government, the legislature, and civil society. While these coalitions were diverse and flexible in their form and composition, their core strength came from established advocacy groups and experienced activists. These groups and activists used highly labour-intensive, specialized and complex forms of mobilization. The success of these reform coalitions, it is argued, has implications for economic and governance reform in the developing world, particularly in systems characterised by oligarchical democracy, where competition for elected office is closely linked to the entrenched interests of business and industry.

Local Systems: a framework for supporting sustained development - USAID (USAID, 2014)

Does Foreign Aid Really Work? An Updated Assessment - Riddell, Roger C. (Social Science Research Network, 2014)
This paper draws widely from the book Does foreign aid really work?, building on that discussion to provide an updated answer to the question based on recent evidence and contemporary debates on aid effectiveness. It starts with a brief discussion (Section 2) of the question: ‘does emergency aid work?’ This is important to the wider debate for two linked reasons. Firstly, the harshest critics of development aid are all supportive of emergency aid, with a number calling for its expansion in spite of evidence of major weaknesses and failures. Secondly, the sharp historical distinction made between emergency and development aid is becoming increasingly strained, as more emergency aid is being used a year or more after emergencies strike to rebuild lives and restore livelihoods, while more development aid is used to directly save lives.The rest of the paper focuses exclusively on development aid. Section 3 provides a rapid overview of the evidence of the impact of individual aid projects, including those of Non-Governmental Organisations (NGOs). The picture is overwhelmingly positive: well over 75% of projects meet their immediate objectives and impact has improved, even though sustaining benefits remains a challenge and there continue to be aid failures. Section 4 reviews the evidence of the wider and long-term impact of aid at the sectoral and country level, including a brief discussion of academic studies on aid and growth. Though there are still major gaps in our knowledge, the quality of the data is improving. However, there is little firm, quantitative evidence to show the specific contribution that aid has made to help achieve the Millennium Development Goals (MDGs), which are a central purpose of many current donor programmes (Section 4.2). More widely, there is plenty of evidence to suggest that aid has contributed positively to both growth and wider development at the sector and country level, though some studies suggest aid has had little or no impact. Overall, the contribution that aid makes to aggregate development is lower than the public has been led to believe (Sections 4.1, 4.3. and 4.4). Assessments of the overall impact of the aid provided by NGOs are rare, but a recent study suggests it has been positive (Section 4.5). More attention is now given to the issues of corruption in aid. Although there is evidence of aid funds being used for corrupt purposes and of aid worsening corruption, on balance it remains a minor issue. Aid has had significant successes in helping the poor affected by corruption (Section 4.6). The second half of the paper shifts focus to the large gap between what aid has done and what it might do. Section 5 discusses a range of inefficiencies within and across the aid system and their costs in terms of reduced impact, including the way aid is allocated, its growing complexity, and the volatility and unpredictability of aid flows (Section 5.1 and 5.2). These inefficiencies place in a different light the evidence of aid’s overall positive impact. The paper looks at the different initiatives that have been mounted to begin to address these weaknesses and failures, including the 2005 Paris Declaration, and discusses why donors have failed to honour the promises they have made to change the ways that they give aid.In Section 5.3, the paper argues that the most critical debate about whether aid works concerns the assessment of whether the short-term, immediate and extensive benefits that aid undoubtedly brings are outweighed by the direct and indirect systemic problems that it risks creating or accentuating. As increasingly over the last decade donors have channelled more of their aid into short-term, quick-impact projects, assessing the wider negative systemic effects of aid has become even more important. Some recent studies suggest that aid’s systemic problems are large and growing, narrowing the gap between aid’s harshest critics and broader analyses of aid impact. Against the backdrop of already too many proposals of how to make aid work better, Section 6 lays out nine concrete proposals for doing this: deepening knowledge of local contexts; ensuring short-term uses of aid are consistent with and supportive of long-term development, and that all aid is more closely related to overall recipient development goals and processes; helping build local capacities for recipients to be able to coordinate aid better; moving from rhetoric to reality in learning lessons from aid; using aid to help the poor in middle income countries; reducing volatility in aid at the country level; encouraging budget support by addressing donor-country political concerns; and rethinking ways of communicating about aid. Section 7 concludes. It suggests that, paradoxically, aid’s impact may well have been harmed by focussing too narrowly on trying to make short-term aid work better, and that the main focus of attention needs to widen to assess how aid can contribute more to a recipient’s own development goals. Additionally, donors need to help build the capacity of developing countries and developing country scholars to enable them to play a bigger role in helping to answer the question of whether aid works; unsettling though their assessments may be.

Assessing the strength of Evidence - DFID (2014)

Global Delivery at the World Bank Group - GDI (Global Delivery Initiative, 2014)

Monitoring and evaluation of policy influence and advocacy - Tsui, Josephine; Hearn, Simon; Young, John (ODI, 2014)
Policy influence and advocacy are increasingly regarded as a means of creating sustainable policy change in international development. It is often also seen as a difficult area to monitor and evaluate. Yet there is an increasingly rich strand of innovation in options to monitor, evaluate and learn from both the successes and failures of policy influence and advocacy interventions. This paper explores current trends in monitoring and evaluating policy influence and advocacy; discusses different theories of how policy influence happens; and presents a number of options to monitor and evaluate different aspects of advocacy interventions. Case studies describe how some organisations have used these options in practice to understand their impact and improve their advocacy strategies.

Making Sense of ‘Messiness’. Monitoring and measuring change in market systems: a practitioner's perspective - Ripley, Matthew; Nippard, Daniel (2014)

Strategic framework for mainstreaming citizen engagement in World Bank Group operations : engaging with citizens for improved results - Rebolledo, Miguel; Seyedian, Aaron; Peixoto, Tiago; Hernandez, Zenaida; Zakhour, Jad; Mahmood, Syed A.; Masud, Harika; Manroth, Astrid; Hamad, Qays (The World Bank, 2014)
The objective of this strategic framework is to mainstream citizen engagement in World Bank Group (WBG)-supported policies, programs, projects, and advisory services and analytics to improve their development results and within the scope of these operations, contribute to building sustainable national systems for citizen engagement with governments and the private sector. This framework will capture the diverse experiences, assess lessons learned, and outline methods and entry points to provide a more systematic and results-focused approach for the WBG. Progress toward this objective will be assessed using indicators included in program, project, and corporate results frameworks. The WBG strategy incorporates citizen engagement, including beneficiary feedback, specifically in its treatment of inclusion, which entails empowering citizens to participate in the development process and integrating citizen voice in development programs as key accelerators to achieving results. This framework builds on stocktaking and lessons learned from WBG-financed operations across regions and sectors. A key lesson is the importance of country context, government ownership, and clear objectives for citizen engagement. The approach to mainstreaming citizen engagement in WBG-supported operations is guided by five principles: 1) it is results-focused; 2) it involves engaging throughout the operational cycle; 3) it seeks to strengthen country systems; 4) it is context-specific; and 5) it is gradual. Under the right circumstances, citizen engagement can contribute to achieving development outcomes in support of the goals the WBG aims to support through all of the operations it funds: eradicating extreme poverty and boosting shared prosperity in a sustainable manner.

Kenya’s tech community will not save journalism - Benequista, Nicholas (Networked News Lab, 2014)

Assessing the use of technological tools and strategies by Oxfam Novib partners in Angola, Burundi, Egypt, Niger, Pakistan, Rwanda and Uganda - The Engine Room (2014)
This report presents findings and insights from the Oxfam Novib pilot module of TechScape, which assessed how Oxfam Novib partners in seven countries related to the use of technology in their work. This report does not aim to draw conclusions about the nature of technology use by Oxfam Novib partners per se, or even the partners included in this assessment. The dramatic difference in organizational activities, contexts and objectives are too great for that. Detailed analysis of how partners are relating to technology in their work, and specific recommendations for greater efficiency and impact are presented in the TechScape Country Briefs. This report aims instead: • to describe the context in which the assessments took place, • to propose insights gained from the cumulative process about measurement, learning and capacity development, and • to identify opportunities for Oxfam Novib to pursue capacity development and knowledge sharing between countries and across the network, through mechanisms that minimize resource demands and directly target documented needs.

Adaptive management in aid programmes (BEAM Exchange, 2014)
Aid works better – especially in complex and conflict-affected environments – when there is scope to trial and adapt programme strategies. So argues Amir Allana in Navigating Complexity, a case study of Northern Karamoja's Growth, Health and Governance (GHG) Programme that is adopting an 'adaptive management' approach. This BEAM Exchange webinar invited Amir, Tim Sparkman and Peter Roggekamp to discuss lessons from their work in Uganda and Cambodia. What does adaptive management look like in practice? And what does it require of managers and funders to make it happen? Tim and Peter share their experiences of applying adaptive management techniques to the Growth, Health and Governance Programme and the Cambodia Agricultural Value Chain Program (CAVAC). Amir highlight insights from the Navigating Complexity report, including the importance of office culture, consistent management signals, and the role of supportive tools and processes.

Making Details Matter: How to Reform Aid Agencies to Generate Contextual Knowledge - Ang, Yuen Yuen (SSRN, 2014)
My essay considers a central problem of reinventing foreign aid in the twenty-first century: how to reform aid agencies to enable a “best-fit” approach to development assistance. For the past decades, the aid community has tried to transplant best practices from the developed world to the developing world. Increasingly, however, it is recognized that copying best practices does not work and may even backfire; rather, aid programs work best when they are tailored to local contexts. Yet while the idea of a best-fit approach is widely embraced in principle, actualizing it is easier said than done. For meaningful changes to take root in practice, we must first identify the obstacles to localizing development assistance and suggest ways to address these problems. To this end, I propose a three-pronged strategy to promote the generation of contextual knowledge among aid professionals, a necessary condition for crafting solutions that can fit various local contexts, namely: (1) build a bank of knowledge about unorthodox practices that work, (2) diversify expertise within aid agencies; and (3) carve experimental pockets. My proposal does not fit neatly into any one of the six themes specified in the GDN competition; rather, it concerns all of the themes. Whether it is to use aid to improve governance, apply information technology, or design financial instruments, the overarching challenge is to empower and incentivize aid professionals to learn and apply contextual knowledge to creatively solve problems in developing societies.
Keywords: Contextual knowledge; Localization; Public policy; foreign aid; international development

Increasing Participation in Evaluation - Baker, A; Bruner, B (Bruner Foundation, 2014)
The Increasing Participation in Evaluation bulletin was developed by Anita Baker with Beth Bruner to help organizations integrate evaluative thinking into their organizational practice. This three page bulletin discusses how Organization Staff, Evaluators, and Funders are typically involved in participatory evaluation. The guide also defines the term "Ripple", and how to accomplish Ripple as well as examining what it looks like when Executive Leaders and Management Staff use Evaluative Thinking. Contents How are Organization Staff, Evaluators, and Funders typically involved in Participatory Evaluation? Organization Managers/Staff Roles Evaluators/Roles Funder Roles Shared Roles What is Ripple Anyway? How do you accomplish Ripple? What does it look like when Executive Leaders and Management Staff Use Evaluative Thinking? How to use these Bulletins

Guidelines for good market development program design. A managers’ perspective - Bekkers, Harald; Roggekamp, Peter (BEAM Exchange, 2014)
Good programme design is a key factor contributing to the success of market development programmes. Unfortunately, too many current designs have major flaws which prevent programmes becoming successful even before they get started. This document considers what makes market development programmes consistently successful and how to prevent programme design from being a hurdle to sustainable and efficient impact as scale.

Assessing Impact in Dynamic and Complex Environments: Systemic Action Research and Participatory Systemic Inquiry - Burns, Danny (IDS, 2014)
This CDI Practice Paper is about the uses of Systemic Action Research (SAR) and Participatory Systemic Inquiry (PSI) for impact assessment (Burns 2006, 2007, 2011, 2012, 2013; Wadsworth 2001, 2010).

Assessing Rural Transformations: Piloting a Qualitative Impact Protocol in Malawi and Ethiopia - Copestake, James; Remnant, Fiona (Centre for Development Studies (CDS), 2014)

Introducción a Capacity WORKS + Proyectos de Cambio - Informe - Cuéllar, Daniel (GIZ, 2014)

The case for thinking and working politically. The implications of 'doing development differently' - DLP (DLP, 2014)
This paper discusses the steps required to build a robust evidence base for 'thinking and working politically' (TWP) in development. It argues that better understanding what works, when and why is an important step in moving TWP into mainstream development programming. The paper reviews the existing evidence base on TWP, building on this and on other literature on public sector reform and 'pockets of effectiveness' to suggest research questions, case study selection criteria, and a four-level analytical framework: 1) political context; 2) sector; 3) organisation; and 4) individual. The framework aims to help build a 'rigorous enough' evidence base to show whether and how TWP happens and whether or not it influences the effectiveness of programme implementation and outcomes. The paper also calls for more focus on gender issues, and on different – and often more fragile – political contexts.

Capacity Works - online training - GIZ (GIZ, 2014)

Participatory Approaches - Guijt, Irene (UNICEF, 2014)
This guide, written by Irene Guijt for UNICEF, looks at the use of participatory approaches in impact evaluation. Using participatory approaches means involving stakeholders, particularly those affected by intervention, in the evaluation process. This includes involvement in the design, data collection, analysis, reporting, and management of the study. Excerpt "By asking the question, ‘Who should be involved, why and how?’ for each step of an impact evaluation, an appropriate and context-specific participatory approach can be developed. Managers of UNICEF evaluations must recognize that being clear about the purpose of participatory approaches in an impact evaluation is an essential first step towards managing expectations and guiding implementation. Is the purpose to ensure that the voices of those whose lives should have been improved by the programme or policy are central to the findings? Is it to ensure a relevant evaluation focus? Is it to hear people’s own versions of change rather than obtain an external evaluator’s set of indicators? Is it to build ownership of the UNICEF programme? These, and other considerations, would lead to different forms of participation by different combinations of stakeholders in the impact evaluation."

Monitoring and Evaluation in the Development Sector - KPMG (KPMG International, 2014)

Political economy analysis: Topic Guide - Mcloughlin, Claire (GSDRC, University of Birmingham, 2014)
Political economy analysis (PEA) aims to situate development interventions within an understanding of the prevailing political and economic processes in society – specifically, the incentives, relationships, and distribution and contestation of power between different groups and individuals. Such an analysis can support more politically feasible and therefore more effective development strategies by setting realistic expectations …

Innovation teams and labs. A Practice Guide - Puttick, Ruth (Nesta, 2014)

From best practice to best fit: understanding and navigating wicked problems in international development - Ramalingam, Ben; Laric, Miguel; Primrose, John (ODI, 2014)

Adapting lean thinking to market systems development: Principles and practices for donors/funders - Rasmussen, Leanne (Engineers without borders, 2014)
The purpose of this paper is to provide funders and implementers of market systems development (MSD) projects with principles, practices, and structures that enables these projects to thrive. It is based on a book that has sold millions of copies worldwide, and a school of thought taught at institutions such as Harvard Business School and practiced in companies ranging from giants such as Toyota to the most successful tech start-ups in Silicon Valley. Published in 2011, Eric Ries wrote the Lean Startup as a remedy to the countless start-ups that create their own demises by getting started with the wrong goals, the wrong structures, and the wrong processes. He outlines an approach that enables the startup to navigate ambiguity and risk while using resources effectively. The conditions a MSD project faces are remarkably similar to those which a startup business faces; it needs structures and processes that are matched to the unpredictable, complex environment which it is attempting to influence. MSD projects can thus benefit from private-sector thought leadership on how start-ups can situate themselves for success. This paper starts with an overview of lean thinking, a concept that derives from lean manufacturing which is widely accepted as the leading approach to modern manufacturing. It explains how Ries’s Lean Startup adapts these principles, and then it in turn adapts these ideas for market systems development. The following three sections provide an overview of Lean Startup concepts, applying them to the development sector as appropriate. The final section summarizes and makes suggestions on next steps for funders who wish to set MSD projects up for success using a lean approach. Overall, the paper builds a case for how the Lean Startup’s approach can enable MSD projects to work successfully in ambiguity and increase their potential for achieving robust and sustainable results, all while using donor resources more efficiently.
Keywords: Adaptive Development; Lean

Theory of Change - Rogers, Patricia (UNICEF, 2014)

Starting out on Social Return on Investment - SROI Network; Aitken, Hall (2014)
This guide, created by the SROI Network in collaboration with Hall Aitken, is aimed at providing guidance to those who have never used SROI on where to begin. The guide provides practical steps to beginning the process of SROI and includes links to tools aimed at supporting its implementation. Excerpt "Most public, private and third sector organisations have a pretty good idea of the costs of what they do. Annual accounts, management accounts, budget reports and a whole accountancy profession add up to a great deal of effort to make sure this is the case. Some organisations are quite good at counting what they do with these resources. They can track the number of users or contacts, or customers. Many can provide some evidence that these activities lead to some sort of change. But very few can explain clearly why all this matters. What would happen if they did not exist? What is the real value of what they do? Social Return on Investment sets out to redress the balance by looking at value not just cost." Contents Part one – why, what and how? Why does social value matter? What is SROI? How do I do SROI? Next steps Part two – progress tool Part three – practical next steps Steps to involve stakeholders Steps to understand what changes Steps to value the things that matter Steps to only include what is material Steps to avoid overclaiming Steps to be transparent Steps to verify the result

Rapid outcome mapping approach: a guide to policy engagement and influence - Young, John; Shaxson, Louise; Jones, harry; Hearn, Simon; Datta, Ajoy; Cassidy, Caroline (ODI, 2014)

Building tech-powered public services - Bickerstaffe, Sarah (Institute for Public Policy Research, 2013)
Given the rapid pace of technological change and take up by the public it is a question of when not if public services become tech powered This new paper asks how we can ensure that innovations are successfully introduced and deployed

Learning about Theories of Change for the Monitoring and Evaluation of Research Uptake - Barnett, Chris; Gregorowski, Robbie (IDS, 2013)
This paper captures lessons from recent experiences on using ‘theories of change’ amongst organisations involved in the research–policy interface. The literature in this area highlights much of the complexity inherent in the policymaking process, as well as the challenges around finding meaningful ways to measure research uptake. As a tool, ‘theories of change’ offers much, but the paper argues that the very complexity and dynamism of the research-to-policy process means that any theory of change will be inadequate in this context. Therefore, rather than overcomplicating a static depiction of change at the start (to be evaluated at the end), incentives need to be in place to regularly collect evidence around the theory, test it periodically, and then reflect and reconsider its relevance and assumptions.
Accepted: 2013-10-01T11:27:00Z

Learning study on 'the users' in Technology for Transparency and Accountability initiatives: Assumptions and realities - McGee, Rosemary; Carlitz, Ruth (IDS, 2013)

Pathways for Change: 10 Theories to Inform Advocacy and Policy Change Efforts - Stachowiak, Sarah (Center for Evaluation Innovation, 2013)
One of our most popular publications, this brief, produced in collaboration with ORS Impact, summarizes 10 theories grounded in social science about how policy change happens. The theories can help to untangle beliefs and assumptions about the inner workings of the policymaking process and identify causal connections supported by research to explain how and why a change may or may not occur. Advocates of all stripes seek changes in policy as a way to achieve impact at a scale and degree of sustainability that differs from what can be achieved through direct services or programs alone. Advocates and funders each come to policy work with a set of beliefs and assumptions about how change will happen, and these beliefs shape their thinking about what conditions are necessary for success, which tactics to undertake in which situations, and what changes need to be achieved along the way. These worldviews are theories of change, whether or not they explicitly have been stated or documented as such. When articulated as theories of change, these strategy and belief system roadmaps can clarify expectations internally and externally, and can facilitate more effective planning and evaluation. This brief lays out 10 theories grounded in diverse social science disciplines and worldviews that have relevance to the world of advocacy and policy change. Global and tactical theories include: - “Large Leaps” or Punctuated Equilibrium theory - “Policy Windows” or Agenda-Setting theory - “Coalition” theory or Advocacy Coalition Framework - “Power Politics” or Power Elites theory - “Regime” theory - “Messaging and Frameworks” theory - “Media Influence” or Agenda-Setting theory - “Grassroots” or Community Organizing theory - “Group Formation” or Self-Categorization theory - “Diffusion” theory or Diffusion of Innovations The theories can help to untangle beliefs and assumptions about the inner workings of the policymaking process and identify causal connections supported by research to explain how and why a change may or may not occur. This piece is not meant to be comprehensive of all possible relevant theories and approaches; rather, it introduces and illustrates a handful of theories that may be useful to advocates, funders, and evaluators.

Designing Participatory Meetings and Brownbags: A TOPS Quick Guide to Linking Development Practitioners - TOPS (Technical and Operational Performance Support Program, USAID, 2013)
Visit this resource for ideas on designing meetings and brownbags that are engaging and build inopportunities for participant dialogue.

Guidelines for the Monitoring System for Development-Related Changes (MERV) - SDC (SDC, 2013)

Citizen-Centered Governance: The Mayor's Office of New Urban Mechanics and the Evolution of CRM in Boston - Crawford, Susan P.; Walters, Dana (Social Science Research Network, 2013)
Over the last three years, the Boston Mayor's Office of New Urban Mechanics, the innovative, collaborative ethos within City Hall fostered by Mayor Menino and h

Lessons for Effective resilience programs: a case study of the RAIN program in Ethiopia - Kleiman, Shanti (Mercy Corps, 2013)
The RAIN program experience affirms the importance of multi-year and flexible funding as key program design features if progress in resilience building is to be supported in areas with high levels of structural and temporal vulnerability. These features enable management to respond effectively to changing circumstances in order to better meet the needs of communities and to create the necessary conditions for relief-to-development strategies to take hold. Finally, this study found that deliberate and effective coordination between donor agencies can ensure that development gains achieved during one program period are directly leveraged by subsequent programs, thereby increasing the effectiveness of resilience programs.

What is democratic evaluation? - Picciotto, R. (2013)
This article from Robert Picciotto provides an overview of democratic evaluation, particularly with reference to its use in the European Union context. "Can evaluation strengthen the democratic process? Speciﬁcally can it help ﬁll the democratic deﬁcit (limited transparency; weak bureaucratic accountability, etc.) often ascribed to the European Union? First and foremost, evaluators operating in the European space should be committed to the ethical and democratic values of the European project and the promotion of its social inclusion and cohesion ideals. But two other sets of challenges must also be met: those that relate to evaluation in democracy and those that relate to democracy in evaluation. The former has to do with the macro positioning of evaluation processes in society. The latter has to do with the evaluation approaches and methods used in deliberative democratic decision making processes." (Picciotto, 2013)

Outcome Measurement in Local Governance Programmes: a Power Dimension - McGee, Rosemary; Pettit, Jethro (IDS, 2013)

It's All About MeE: Using Structured Experiential Learning (“e”) to Crawl the Design Space - Pritchett, Lant; Samji, Salimah; Hammer, Jeffrey (Center for Global Development, 2013)
This paper argues that within-project variations in design can serve as their own counterfactual, reducing the incremental cost of evaluation and increasing the direct usefulness of evaluation to implementing agencies. It suggests combining monitoring (‘M’), structured experiential learning (‘e’), and evaluation (‘E’) so as to facilitate innovation and organisational capability building while also providing accountability …

Retrospective ‘Outcome Harvesting’: Generating robust insights about a global voluntary environmental network - Rassmann, Kornelia; Smith, Richard; Mauremootoo, John; Wilson-Grau, Ricardo (Better Evaluation, 2013)

A problem-focused approach to violence against women: The political-economy of justice and security programming - Denney, Lisa; Domingo, Pilar (ODI, 2013)
The United Nations Commission on the Status of Women ended its 57th session on 15 March 2013 with an outcome document affirming the importance of eliminating violence against women (VAW). The Commission was unable, however, to achieve consensus on a global action plan. The negative reaction of some UN member states to an action plan is a worrying reminder of ongoing resistance to reform. These persistent challenges highlight the continuing struggle to gain a serious global commitment to address VAW and recognise it as a breach of women’s fundamental human rights. Engaging in this struggle, many donors have put addressing VAW generally, and in fragile and conflictaffected situations (FCAS) specifically, at the top of the development agenda and made it a major priority of international policy. But in practice progress remains difficult, not least due to entrenched resistance and discriminatory socio-political norms and gender relations that persist in many societies. The problem of violence against women therefore needs to be addressed from the perspective of the concrete socio-political and cultural conditions that shape its particular features and the relevant context specific dynamics of conflict, post-conflict patterns of violence and fragility. International efforts to support reform in the area of VAW in FCAS need to go beyond prescriptive approaches that focus on what access to protection, justice and redress should look like. We propose here an approach that engages with the specificities of the problem – paying attention to context, and the concrete political-economy dynamics of the drivers of VAW – and takes account of the real options that

Monitoring and Evaluating Conflict Sensitivity: Methodological Challenges and Practical Solutions - Goldwyn, Rachel; Chigas, Diana (DFID, 2013)

Improving the Evaluability of INGO Empowerment and Accountability Programmes - Shutt, Cathy; McGee, Rosemary (Centre for Development Impact, 2013)

Monitoring, Evaluation and Learning in NGO Advocacy - Findings from Comparative Policy Advocacy MEL Review Project - Coe, Jim; Majot, Juliette (ODI and Oxfam, 2013)
For organizations committed to social change, advocacy often figures as a crucial strategic element. How to assess effectiveness in advocacy is, therefore, important. The usefulness of Monitoring, Evaluation and Learning (MEL) in advocacy are subject to much current debate. Advocacy staff, MEL professionals, senior managers, the funding community, and stakeholders of all kinds are searching for ways to improve practices – and thus their odds of success – in complex and contested advocacy environments. This study considers what a selection of leading advocacy organizations are doing in practice. We set out to identify existing practice and emergent trends in advocacy-related MEL practice, to explore current challenges and innovations. The study presents perceptions of how MEL contributes to advocacy effectiveness, and reviews the resources and structures dedicated to MEL.

Impact Evaluation: Taking stock and looking ahead - Conference report - Visser, Irene; Guijt, Irene; Kusters, Cecile (Centre for Development Innovation, 2013)
This report summarises the presentations and discussions of the Conference ‘Impact evaluation. Taking stock and looking ahead’, which took place in Wageningen on March 25 and 26, 2013. The Conference was organised and funded by the Centre for

Systems Innovation - Discussion Paper - Mulgan, Geoff; Leadbeater, Charlie (NESTA, 2013)
Introduction Over the past few years there has been growing interest in systemic innovation. We are defining this as an interconnected set of innovations, where each influences the other, with innovation both in the parts of the system and in the ways in which they interconnect. Yet rather than simply theorising, we want to make this practical. We want to explore the potential of systemic innovation to help tackle some of the key challenges the UK currently faces, from supporting an ageing population to tackling unemployment. We would also like to open up the discussion, to engage with the wide and diverse range of experts already working this space to help sharpen up thinking about systemic innovation and influence practical work to advance it. About this paper The two essays contained within this paper are intended to contribute to this debate. The first paper by Geoff Mulgan addresses a number of questions about systemic – or ‘joined–up’ – innovation, asking what it means to truly transform a system, encompassing the means, methods, and actions needed to realise its potential. The second paper from Charlie Leadbeater argues that companies, governments, cities, and entire societies need to move from seeing innovation in products and services as a source of competitive advantage, to focussing on innovation with entire systems. Our next steps: helping make systemic innovation useful and useable This report is just the start. We want to work with academics, practitioners, policymakers, and others in the field, to help realise the great potential of connecting and joining up innovative products, services and processes so that the whole is more than the parts. Over the coming months we will be building upon the rich and diverse literature available (we have start collating this as an annex to this paper), as well as practitioner experience, to ensure we understand systems more rigorously, as well as the skills, capabilities, methods and means needed to help them change for the better.

Utilization-Focused Evaluation (U-FE) Checklist - Patton, Michael Quinn (The Evaluation Center, 2013)
Utilization-Focused Evaluation begins with the premise that evaluations should be judged by their utility and actual use; therefore, evaluators should facilitate the evaluation process and design any evaluation with careful consideration of how everything that is done, from beginning to end, will affect use. Use concerns how real people in the real world apply evaluation findings and experience and learn from the evaluation process. The checklist is based on Essentials of Utilization-Focused Evaluation (Patton, 2012, Sage Publications). All references in the checklist to exhibits and menus refer to this book. Step 1 Assess and build program and organizational readiness for utilization-focused evaluation. Step 2 Assess and enhance evaluator readiness and competence to undertake a utilizationfocused evaluation. Step 3 Identify, organize, and engage primary intended users. Step 4 Conduct situation analysis with primary intended users Step 5 Identify primary intended uses by establishing the evaluation’s priority purposes. Step 6 Consider and build in process uses if appropriate. Step 7 Focus priority evaluation questions. Step 8 Check that fundamental areas for evaluation inquiry are being adequately addressed. Step 9 Determine what intervention model or theory of change is being evaluated. Step 10 Negotiate appropriate methods to generate credible findings and support intended use by intended users. Step 11 Make sure intended users understand potential controversies about methods and their implications. Step 12 Simulate use of findings. Step 13 Gather data with ongoing attention to use. Step 14 Organize and present the data for use by primary intended users. Step 15 Prepare an evaluation report to facilitate use and disseminate significant findings to expand influence. Step 16 Follow up with primary intended users to facilitate and enhance use. Step 17 Metaevaluation of use: Be accountable, learn, and improve

Scaling Up Development Programmes - UNDP (UNDP, 2013)
Many development organizations, national and local governments and civil society organizations are faced with the issue of scaling up development interventions — the main questions raised time and again are: a) what should be scaled up, and how it can be scaled up; b) is there a strong reason for a particular initiative to be scaled up; and c) what should be the value-added of the scaling up efforts, and how can practitioners ensure that technological and other innovations are being integrated for improved development effectiveness? Answers to these questions depend on a host of complex realities—the relative strengths and weaknesses in national and local systems, political and economic situations, including vulnerability of country systems to shocks, commitments from development partners, power dynamics between various groups and stakeholders in a given country, regional and global environments. For the United Nations Country Teams (UNCT) and UNDP offices, another factor to consider is how to maximize our respective comparative advantages. This guidance note summarizes the main conceptual thinking available from development institutions and academia, and presents a simplified conceptual framework and roadmap for scaling up processes. It also provides UNDP programme staff and UNCTs with practical actions and checklists to consider when designing and implementing programmes that support national scaling up initiatives, and proposes actions that can be undertaken at the regional and global levels. Although intended for internal audiences, the scaling up concept and recommendations can also be used by the public and private sector, civil society and social entrepreneurs. This note benefited from a wide consultative process held in 2011, including practical recommendations and proposals from colleagues at country offices and regional/global centres. Their insights have been instrumental in the distillation of main recommendations presented in this note, and the individuals who provided substantive contributions are gratefully noted in the Acknowledgements section.

Explaining positive deviance in public sector reforms in development - Andrews, Matt (World Institute for Development Economic Research (UNU-WIDER), 2013)
Public sector reforms are commonplace in developing countries. Much of the literature about these reforms reflects on their failures. This paper asks about the successes and investigates which of two competing theories best explain why some reforms exhibi
Keywords: Development; Innovation; Reform; case survey; governance; growth; leadership

Complexity-Aware Monitoring - Britt, Heather; Patsalides, Melissa (USAID's Learning, Evaluation, and Research Office, 2013)

Evaluating Countering Violent Extremism: Practice and Progress - Fink, Naureen Chowdhury; Romaniuk, Peter; Barakat, Rafia (Center on Global Counterterrorism Cooperation, 2013)

The P Process - Five Steps to Strategic Communication - HCCC (Johns Hopkins University, 2013)
The P Process is one of the most respected tools used in designing SBCC programs. Developed in 1982, the P Process is a tool for planning strategic, evidence-based health communication programs. More than thirty years later, communication professionals around the world are using the P Process to design, implement, monitor, and evaluate communication strategies, materials, and programs. The P Process is a step-by-step roadmap that can guide you from a loosely defined concept about changing behavior to a strategic and participatory program that is grounded in theory and has measurable impact. This update of the P Process incorporates lessons learned and acknowledges the real-time, dynamic nature of the strategic process as well as new technologies and the constantly changing nature of communication, social norms, individual behavior, and decision-making. It draws from many other disciplines, including design theory, behavioral economics, social psychology, and anthropology. At its essence though, the P Process is a tool that is only as useful as the data and thinking that go into each step. The attached file contains both the English and French versions.

In the goldfish bowl: science and technology policy dialogues in a digital world - Latta, Susie; Mulcare, Charlotte; Zacharzewski, Anthony (Sciencewise, 2013)

Mapping Context for Social Accountability: A resource paper - O'Meally, Sion C. (The World Bank, 2013)

DAC Peer Review of Switzerland - 2013 - OECD (OECD, 2013)

Power Analysis: A Practical Guide - Pettit, Jethro (Sida, 2013)

Chris Argyris: theories of action, double-loop learning and organizational learning - Smith, Mark K. (The encyclopedia of pedagogy and informal education, 2013)
Chris Argyris: theories of action, double-loop learning and organizational learning. The work of Chris Argyris (1923-2013) has influenced thinking about the relationship of people and organizations…

An inventory and review of countering violent extremism and insurgency monitoring systems - Carter, Lynn; Phyllis, Dinino (USAID, 2012)

Scaling Agile @ Spotify with Tribes, Squads, Chapters & Guilds - Kniberg, Henrik; Ivarsson, Anders (Spotify, 2012)

What about the results? Lessons from long-term process support to strengthen results-based management (RBM) for Swedish framework NGOs operating in Western Balkans - Ørnemark, Charlotte (NCG, 2012)
The observations and lessons outlined in this report should be seen as a contribution to the on-going learning and reflections in a wider debate on how to assess and monitor results from support to CSOs involved in complex social change processes (as opposed to those involved in more ‘classical’ service delivery and more linear development assistance). These lessons, though directly emerging from the consultancy to provide long-term RBM support to the FOs and their local CSO partners in Western Balkans, also draws on previous experiences and similar lessons from on-going initiatives. In brief, these lessons call for a certain ‘paradigm shift’ in the way we (as development professionals) regard and support RBM for actors involved in less tangible social change processes in highly contested political spaces. Some of these shifts, outlined in more detail in different sections of this report, are summarised below:  From a linear, aggregated cause-and-effect thinking around RBM to one that is linked to human factors and is embedded in systems,  From pushing the burden of reporting down in the system to a clearer division of labour between donors, recipient governments, intermediary agents like framework NGOs (FOs) and implementing CSOs with each actor using its comparative advantage and appropriate role in the RBM process and for its own learning,  From operating in programming and results frameworks where outcomes and impact are largely assumed to be predictable to setting up systems that deal with uncertainty and that capture emerging result patterns through tracking of gradual changes,  From a project/programme perspective to a focus on institutions and systems as actors and arenas for change, where organisations are enabled to act as change agents towards clearly identified processes of social transformation,  From SMART to REAL results frameworks – although SMART (specific, measurable, achievable, realistic and time-bound) principles are always good to keep in mind for the formulation of goals and indicators, the danger of coming up with fictive measuring frameworks calls for a more systematic incorporation of ‘real’ concerns, including making RBM processes rights-based, embedded in local realities (and empowering for those involved), aligned to national reform efforts and national and international human rights commitments, and learning-oriented for all different levels of operations.

Escaping Capability Traps through Problem-Driven Iterative Adaptation (PDIA) - Andrews, Matt; Pritchett, Lant; Woolcock, Michael (Harvard University, 2012)
Keywords: IMPORTANT

Budgeting for Developmental Evaluation: An Interview with Michael Quinn Patton - Better Evaluation (2012)

Review of the use of ‘Theory of Change’ in International development - Vogel, Isabel (DFID, 2012)
A new report on Theory of Change (ToC) and its use in International development has just been produced.

Practical place-based initiatives: a better practice guide - Australian Government (Strategy and Delivery Division, Department of the Prime Minister and Cabinet, Australian Government, 2012)

Tiny Tools - Measuring Change in Communities and Groups - Causemann, Bernward; Gohl, Eberhard; Brenner, Verena (Impact+, 2012)
Introduction to the Overview: Tiny Tools Why “Tiny Tools” for assessing change? Currently, change is mostly assessed by NGO staff or external experts. The vision of this paper is that communities assess and reflect change themselves and make use of that reflection with appropriate tools. All the tools presented here are relatively quick and easy to learn (therefore “tiny”). With Tiny Tools we can assess change in one session. They can therefore be used where there are not baselines. They are structured and systematic, and they are all widely tested: Experience shows that these tools lead to new insights, mobilise enthusiasm and increase the capacity of communities to bring about further change. The Tiny Tools are in line with what Participatory Rural Appraisal (PRA) wanted to achieve. Many are slight variations of PRA tools. For a detailed description of concepts, see the NGO-IDEAs Impact Toolbox (www.ngo-ideas.net/impact_toolbox) and the NGO-IDEAs Manual Self-Effectiveness (www.ngo-ideas.net/monitoring_self_effectiveness). The tools are designed to visualise change, but also enable communities to reflect on the reasons of change or verify assessments. They may be implemented once or continuously over time. We know that the time of community members is precious, and limited. Therefore all Tiny Tools can be performed in a relatively short session, provided facilitators (it could be field staff or project officers) are experienced – and the community knows and trusts them. The amount of time spent on the application of the tools may however be prolonged according to the needs of a community or NGO. All of these tools are easy to learn for a facilitator experienced in participatory processes. Which tool should be introduced to which community? It is typically the decision of a development organisation (or external experts) which tools they want to introduce into a community. The staff need to assess which tool will lead to learning and action. It could also be that the staff realise aspects of change that they do not understand well enough. These tools are good for exploring change that we have not planned for and not anticipated. They are also good for exploring change in a context where we have no prior information. Communities are the best experts for their situation, but we emphasise that the tools should be used in ways that benefit and empower the communities or individuals participating. The tool implementations should lead to consequences on the grassroots as well as the NGO level. All Tiny Tools can be integrated into PIAR, the Analysis tool in the NGO-IDEAs Impact Toolbox. Also, the Tiny Tools can help to prepare for the application of the Toolbox tools. They help to make people aware of changes that can be observed. The following box gives some hints how Tiny Tools relate to the Impact Toolbox tools, and to what extent they help to attribute change to development interventions.

A compass in the woods: learning through grantmaking to improve impact - Darling, Marilyn J (Fourth Quadrant Partners, 2012)

Monitoring, Evaluation, Accountability and Learning in Emergencies - Dominique Morel; Hagens, Clara (Catholic Relief Services, 2012)

New communication technologies and citizen-led governance in Africa - Gagliardone, Iginio; Srinivasan, Sharath; Brisset-Foucault, Florence (2012)

Towards excellence: Policy and action research for sustainable development - iied (The International Institute for Environment and Development, 2012)

Ten years of Outcome Mapping adaptations and support - Smith, Richard; Mauremootoo, John; Rassmann, Kornelia (Outcome Mapping Learning Community, 2012)
An analysis of how and where Outcome Mapping has been applied, how users have experienced OM and the support options available and required for its use. This research was commissioned by the OMLC Stewards.

Understanding ‘theory of change’ in international development: A review of existing knowledge - Stein, Danielle; Valters, Craig (LSE, 2012)
This is a review by Danielle Stein and Craig Valters of the concepts and debates within ‘Theory of Change’ (ToC) material, resulting from a search and detailed analysis of available donor, agency and expert guidance documents. The review was undertaken as part of a Justice and Security Research Program (JSRP) and The Asia Foundation (TAF) collaborative project, and focuses on the field of international development.

Broadening the range of designs and methods for impact evaluations - Stern, Elliot; Stame, Nicoletta; Mayne, John; Forss, Kim; Davies, Rick; Befani, Barbara (DFID, 2012)

Appendix 3. ToC Examples - Vogel, Isabel; Stephenson, Zoe (DFID, 2012)

Outcome Harvesting - Wilson-Grau, Ricardo; Britt, Heather (Ford Foundation, 2012)
Outcome Harvesting was developed by Ricardo Wilson-Grau with colleagues Barbara Klugman, Claudia Fontes, David Wilson-Sánchez, Fe Briones Garcia, Gabriela Sánchez, Goele Scheers, Heather Britt, Jennifer Vincent, Julie Lafreniere, Juliette Majot, Marcie Mersky, Martha Nuñez, Mary Jane Real, Natalia Ortiz, and Wolfgang Richert. Over the past 8 years, Outcome Harvesting and has been used to monitor and evaluate the achievements of hundreds of networks, non-governmental organizations, research centers, think tanks, and communitybased organizations around the world. This brief is intended to introduce the concepts and approach used in Outcome Harvesting to grant makers, managers, and evaluators, with the hope that it may inspire them to learn more about the method and apply it to appropriate contexts. Thus, it is not a comprehensive guide to or explanation of the method, but an introduction to allow evaluators and decision makers to determine if the method is appropriate for their evaluation needs. Where possible, we have included examples to illustrate how Outcome Harvesting is applied to real situations. For each case story, organizations were asked to provide a description of the outcome and a summary of the role played by the organization. Sometimes they added other information such as the outcome’s significance. Some details and identifiers were redacted for confidentiality purposes. A draft of this brief was graciously commented on by Bob Williams, Fred Carden, Sarah Earl, Richard Hummelbrunner and Terry Smutylo. The final text is, of course, the sole responsibility of the authors and editor.

Shifting Power? Assessing the Impact of Transparency and Accountability Initiatives - McGee, Rosie; Gaventa, John (IDS, 2011)
This paper arises from a review of the impact and effectiveness of transparency and accountability initiatives which gathered and analysed existing evidence, discussed how it could be improved, and ev...

Learning about New Technologies and the Changing Evidence Base for Social Science Research and Decision Making in International Development - Berdou, Evangelia (IDS, 2011)

System Stewardship. The future of policy making? - Hallsworth, Michael (Institute for Government, 2011)
This working paper looks at the future of policy making in a world of decentralisation and more complex problems. It argues that policy makers need to see themselves less as sitting on top of a delivery chain, but as stewards of systems with multiple actors and decision makers – whose choices will determine how policy is realised. We are keen to open up a debate on what this means.

Making Good Use of Adaptive Management - Doremus, Holly; Andreen, William L.; Camacho, Alejandro E.; Farber, Daniel A.; Glicksman, Robert L.; Goble, Dale D.; Karkkainen, Bradley C.; Rohlf, Dan; Tarlock, A. Dan; Zellmer, Sandra B.; Jones, Shana Campbell; Huang, Yee (Center for Progressive Reform, 2011)

Learning how to learn: eight lessons for impact evaluations that make a difference - Ramalingam, Ben (ODI, 2011)
This Background Note outlines key lessons on impact evaluations, utilisation-focused evaluations and evidence-based policy. While methodological pluralism is the key to effective impact evaluation in development, the focus here is on the factors that need to be considered for impact evaluations to be used in policy and practice – regardless of the method employed.

Mediating Voices, Communicating Realities: Using Information Crowdsourcing Tools, Open Data Initiatives and Digital Media to Support and Protect the Vulnerable - Berdou, Evangelia (IDS, 2011)
This is the final report from a research project, supported by the UK Department for International Development, examining whether and how open ICT projects designed to support the poor can make a diff...

A guide to monitoring and evaluating policy influence - Jones, Harry (ODI, 2011)

Guidance on using the revised Logical Framework - How to note - DFID (DFID, 2011)
The principal changes to the logframe from the earlier (2008) 4 x 4 matrix are: - Objectively Verifiable Indicators (OVIs) have been separated into their component elements (Indicator, Baseline and Target), and Milestones added. - Means of Verification has been renamed ‘Source’. - Inputs are now quantified in terms of funds (expressed in Sterling for DFID and all partners) and use of DFID staff time (expressed as annual Full-Time Equivalents (FTEs). - A DFID Share box now indicates the financial value of DFID’s Inputs as a percentage of the whole. - Assumptions are shown at Purpose and Output level only. - Risks are shown at Activities level only, but also rated at Output level; - At the Output level, the Impact Weighting is now shown in the logframe together with a Risk Rating for individual Outputs. - Activities are now shown separately (so do not normally appear in the logframe sent for approval), although they can be added to the logframe if this is more suitable for your purposes. - A renewed emphasis on the use of disaggregated beneficiary data within indicators, baselines and targets. The Logical Framework (logframe) was significantly re-designed in February 2009. In January 2011 a slightly amended logframe template was introduced at the same time as the launch of the new DFID Business Case. Given the extent of changes that took place in February 2009 it was only necessary to make minor amendments in January 2011. These amendments are as follows: - Results Chain terminology aligned across DFID (the terms Goal and Purpose in the old logframe template have been replaced by Impact and Outcome) - Rows added to allow achieved results to be captured alongside the planned results determined at project design stage - Word version of the logframe template removed – excel is the preferred format for logical frameworks from January 2011 - Indicator numbering introduced within logframe template

Alps - Accountability, Learning and Planning System - 2011 Update - ActionAid (ActionAid International, 2011)
Alps is ActionAid’s over arching accountability framework, containing within it our programme planning system. Alps is distinctive in that it is strongly driven by principles, and sets out necessary personal attitudes and behaviours alongside organisational processes for planning, strategy formulation, learning, monitoring reviews/evaluations and audit. Alps defines our standards, not only about what we do but also how we do it. Alps is part of ActionAid’s human rights-based work.

Foreign policy and complex adaptive systems: Exploring new paradigms for analysis and action - Frej, William; Ramalingam, Ben (Santa Fe Institute, 2011)

NGO-IDEAs Impact Toolbox - Participatory Monitoring of Outcome and Impact - Gohl, Eberhard; Causemann, Bernward (Impact+, 2011)
NGO-IDEAs (NGO – Impact on Development, Empowerment and Actions) NGO-IDEAs is a cooperation of about 40 non-govermental organisations (NGOs) from South Asia, East Africa and the Philippines and 14 German NGOs working in the field of development cooperation. It identifies and develops jointly with all partners, concepts and tools for NGOs in the areas of Outcome and Impact Assessment and Monitoring & Evaluation (M&E). NGO-IDEAs is further being supported by VENRO, the umbrella organisation of development non-governmental organisations in Germany as well as PARITÄT, the legal holder of the project. The Federal Ministry for Economic Cooperation and Development (BMZ) has cofinanced the project. NGO-IDEAs is not just another study evaluating the impact of NGOs’ work – it combines research & development, knowledge management, learning & training as well as advice & coaching to initiate a collective learning process for all partners involved. Additionally, NGOIDEAs intends to create a valuable resource base for use by NGOs. NGO-IDEAs aims at: • Empowering community based organisations or groups and the poor among the rural communities to use and practice impact monitoring for project management • Empowering NGOs to further improve the effectiveness, impact and sustainability of their efforts • Making social changes more visible for implementing and funding NGOs, thus improving development practice • Improving public recognition of NGOs and CBOs and their contribution to development. The NGO-IDEAs “Impact Toolbox” is to enable NGOs and grassroots organisations to monitor projects together with the so called target groups involved, in a manner that will enhance positive outcomes and impacts, and reduce negative ones. It focuses on joint setting of goals, on monitoring them and finally on taking joint decisions about the further design and direction of interventions. The instruments of the NGO-IDEAs “Impact Toolbox” are simple and participatory. Simple means: setting out from people’s knowledge and know-how, therefore, easy to understand and apply. Application can easily be fitted into the “normal” activities of the NGOs or grassroots organisations. The participatory character emerges through democratic elements promoting a “Culture of Learning” that the people can assimilate

Salt River Ecosystem Restoration Project - Adaptive Management Plan - Harvey, H.T. (Humboldt County Resource Conservation District, 2011)

Not everything that connects is a network - Hearn, Simon; Mendizabal, Enrique (ODI, 2011)
In the public policy context, networks have been predominant factors in policy formulation, innovation and global governance. While not a new phenomenon in the development landscape, networks are becoming increasingly prevalent. Donors, for example, are turning to networks to deliver aid interventions, NGOs are working through networks for collective advocacy and researchers collaborate across networks for greater policy influence. But are networks always the most appropriate vehicle? Where they are appropriate, how can we make the best use of them? This Background Note argues for a more rigorous understanding of their nature, particularly their value (and costs), and presents a revised Network Functions Approach as a model for rationalised investment in networks.

Evaluation for equitable development results - Segone, M (UNICEF, 2011)
This document is made up of a range of Evaluation Working Papers (EWP) focused on evaluation for equitable development. Put together by evaluation specialists they present strategic evaluation findings, lessons learned and innovative approaches and methodologies. Part 1:Evaluation and equity Evaluation to accelerate progress towards equity, social justice and human rights 2 Human rights and gender equality in evaluation 13 When human rights is the starting point for evaluation 25 Strengthening Equity- focused evaluations through insights from feminist theory and approaches 39 Decolonizing evaluation in a developing world. Implications and cautions for Equity-focused evaluations 59 Part 2: Methodological implications for Equity-focused evaluations Methodological issues to design and implement equity-focused evaluations 86 Developmental evaluation for Equity-focused evaluations 102 Systems thinking and Equity-focused evaluations 115 Methodological challenges in using programme theory to evaluate pro-poor and equity-focused programmes by Patricia Rogers, Royal Melbourne Institute of Technology University and Richard Hummelbrunner, Independent consultant 142 Case Study and equity in Evaluationby Saville Kushner, University of the West of Englan 172 Values-Engaged Evaluationsby Jennifer Greene, University of Illinois at Urbana-Champaign 192 Part 3: Examples of Equity-focused evaluations Evaluating the contribution of UNDP to equity-focused public policies in Brazil and China 210 Using a human rights approach to evaluate ILO’s discrimination strategy 222 CONEVAL experience in evaluating interventions for Indigenous populations in Mexico 244 UNICEF supported evaluations with elements of equity-focused evaluations 258

Learning to ADAPT: monitoring and evaluation approaches in climate change adaptation and disaster risk reduction – challenges, gaps and ways forward - Silva Villanueva, Paula (IDS, 2011)
This working paper is a methodological contribution to the emerging debate on monitoring and evaluation (M&E) in the context of climate change adaptationand disaster risk reduction. Effectively managing disaster risk is critical for adapting to the impacts of climate change, however disasters risk reduction M&E practice may be limited in capturing progress towards adaptation. The unique nature of adaptation to climate change calls for experience-based learning M&E processes for discovering the key insights into adaptive capacity and its links to adaptation processes, and to risk and vulnerability reduction at large. The ADAPT guiding principles and indicators set the foundations towards this end.

The Alignment, Interest and Influence Matrix (AIIM) guidance note - Mendizabal, Enrique (ODI/RAPID, 2010)
In 2007, on the eve of a workshop to introduce a new version of the RAPID approach to DFID policy teams, Enrique Mendizabal and Ben Ramalingam created the Alignment, Interest and Influence Matrix (AIIM), a stakeholder analysis tool that not only helps to identify key stakeholders, but also suggests a possible course of action towards them.

The unofficial SCRUM checklist - Kniberg, Henrik (CRISP, 2010)

Tips for Conducting a Gender Analysis at the Activity or Project Level - Cozzarelli, Cathy (USAID, 2010)
I. What is gender analysis? II. Relevant sections of the ADS III. Process – What should you do to collect the information you need to conduct a gender analysis at the activity or project level? IV. What questions should you ask in the context of doing your gender analysis? - Access - Knowledge, Beliefs, and Perceptions - Practices and Participation - Time and Space - Legal Rights and Status - Power and Decision-making IV. What should you do after you ask these questions?

Paradigms, Poverty and Adaptive Pluralism - Chambers, Robert (IDS, 2010)
This paper explores participatory methodologies (PMs) associated with a paradigm of people, contrasted with a dominant paradigm associated with things.

Global mapping of technology for transparency and accountability - Avila, Renata; Feigenblatt, Hazel; Heacock, Rebekah; Heller, Nathaniel (Transparency & Accountability Initiative, 2010)

Unpacking the concept of political will to confront corruption - Brinkerhoff, Derek W. (U4 Anti-Corruption Resource Centre, 2010)
Quite often, "lack of political will" is identified as the culprit for poorly performing anti-corruption programmes. Yet despite the frequency with which it is used to explain unsatisfactory reform outcomes, political will remains under-defined and poorly understood. Further, assessments are often conducted retrospectively, looking back at failed programmes. By applying a model of political will that specifies a set of action-based components that are observable and measurable, and amenable to external reinforcement and support, more clarity regarding the degree of political will can be achieved.

An Upside Down View of Governance - IDS (Centre for the Future State, IDS, 2010)
Informal institutions and personalised relationships are usually seen as governance problems. However the research presented in this synthesis paper suggests that they can also be part of the solution...

Grounded Theory Method in HCI and CSCW - Muller, Michael J; Kogan, Sandra (IBM Center for Social Software, 2010)

The Clash of the Counter-bureaucracy and Development - Natsios, Andrew (The Center for Global Development, 2010)
In this essay, Andrew Natsios gives a first-hand account of what he finds most hinders USAID—layers of bureaucracy that misguide and derail development work.

Political Economy Analysis - How to note - DFID (DFID, 2009)
Political economy analysis is a powerful tool for improving the effectiveness of aid. Bridging the traditional concerns of politics and economics, it focuses on how power and resources are distributed and contested in different contexts, and the implications for development outcomes. It gets beneath the formal structures to reveal the underlying interests, incentives and institutions that enable or frustrate change. Such insights are important if we are to advance challenging agendas around governance, economic growth and service delivery, which experience has shown do not lend themselves to technical solutions alone. Political economy analysis is not a magic bullet for the resolution of intractable development problems. However, it can support more effective and politically feasible development strategies, as well as inform more realistic expectations of what can be achieved, and the risks involved. It can also contribute to better results by identifying where the main opportunities and barriers for policy reform exist and how donors can use their programming and influencing tools to promote positive change. This understanding is particularly relevant in fragile and conflict-affected environments where the challenge of building peaceful states and societies is fundamentally political. There are an increasing number of political economy tools available to development agencies for a range of analytical and operational purposes. This note brings together this material with a view to explaining the relevance and uses of political economy analysis. It is intended to be used by a wide range of DFID programme managers and advisers, as well as staff in other HMG departments and partner organisations. The main questions it addresses are: 􀂃 what is political economy analysis? 􀂃 how and why does political economy analysis add value to DFID work? 􀂃 what approaches and tools are available? 􀂃 how should the analysis be prepared, undertaken and applied to DFID’s work? 􀂃 how should we work with other development partners and across HMG on analysis? Key messages include: • Political economy analysis is central to the formulation of sound country plans and sector programmes, and can play a key role in risk mitigation and ensuring that donors avoid harmful practices. • Political economy analysis can help to improve development effectiveness by identifying how and where donors should focus efforts to promote positive change. • There are a growing number of operationally relevant tools which can be used to inform development strategies at the country or sector level, or in relation to particular development problems. • Several DFID country offices have used political economy analysis to improve the quality and impact of aid. This experience provides valuable lessons that should be considered when commissioning and undertaking political economy analysis. • Where possible, analysis should be conducted on an ongoing basis with key partners in HMG and the wider development community to encourage shared understanding and joint action.

The road to results: designing and conducting effective development evaluations - Imas, Morra; G, Linda; Rist, Ray C. (The World Bank, 2009)
The analytical, conceptual, and political framework of development is changing dramatically. The new development agenda calls for broader understandings of sectors, countries, development strategies, and policies. It emphasizes learning and continuous feedback at all phases of the development cycle. As the development agenda grows in scope and complexity, development evaluation follows suit. Development evaluator are moving away from traditional implementation and output-focused evaluation models toward results-based evaluation models, as the development community calls for results and embraces the millennium development goals. As the development community shifts its focus away from projects in order to comprehensively address country challenges, development evaluators are seeking methods with which to assess results at the country, sector, theme, policy, and even global levels. As the development community recognizes the importance of not only a comprehensive but also a coordinated approach to developing country challenges and emphasizes partnerships, development evaluators are increasingly engaged in joint evaluations. These joint evaluations, while advantageous in many respects, add to the complexity of development evaluation (OECD 2006). Additionally, development evaluators increasingly face the measurement challenge of determining the performance of an individual development organization in this broader context and of identifying its contribution. This text is intended as a tool for use in building development evaluation capacity. It aims to help development evaluators think about and explore the new evaluation architecture and especially to design and conduct evaluations that focus on results in meeting the challenges of development.

Strategy Development: Most Signficant Change (MSC) Toolkit - ODI (ODI, 2009)
The Most Significant Change (MSC) technique is a form of participatory monitoring and evaluation. It is participatory because many project stakeholders are involved both in deciding the sorts of change to be recorded and in analysing the data. It is a form of monitoring because it occurs throughout the programme cycle and provides information to help people manage it. MSC contributes to evaluation because it provides data on impact and outcomes which can be used to help assess the performance of the programme as a whole. Essentially, the process involves the collection of significant change (SC) stories emanating from the field level, and the systematic selection of the most important of these by panels of designated stakeholders or staff. The designated staff and stakeholders are initially involved by ‘searching’ for project impact. Once changes have been captured, various people sit down together, read the stories aloud and have regular and often in-depth discussions about the value of the reported changes. When the technique is successfully implemented, whole teams of people begin to focus their attention on programme impact. MSC has had several names since it was conceived, each emphasising a different aspect. Examples are: ‘Monitoring-without-indicators’ – MSC does not make use of predefined indicators, especially ones which have to be counted and measured; or the ‘story approach’ – the answers to the central question about change are often in the form of stories of who did what, when and why, and the reasons the

Evaluation for Improvement: A Seven Step Empowerment Evaluation Approach for Violence Prevention Organizations - Cox, PJ; Keener, D; Woodward, T; Wandersman, A (2009)
This guide, written by Pamela J. Cox, Dana Keener, Tifanee L. Woodard, & Abraham H. Wandersman for the Centers for Disease Control and Prevention (CDC) outlines a seven step process for hiring an evaluator to implement an empowerment evaluation. The process begins with preparing for the hiring and concludes with an assessment of an evaluation to ensure its sustainability. Excerpt "...concerns and experiences with independent evaluation led to the development of participatory evaluation approaches as a way to promote an organization’s use of evaluation for the improvement of its strategies. Although there are many participatory evaluation approaches, empowerment evaluation places an explicit emphasis on building the evaluation capacity of individuals and organizations so that evaluation is integrated into the organization’s day-to-day management processes. Through empowerment evaluation, both individual and organizational evaluation capacity are increased through a “learn-by-doing” process, whereby organizations and their staff evaluate their own strategies. Specifically, organizations hire an evaluator to work with them in conducting an evaluation of their strategies. Rather than evaluating an organization’s strategies and presenting an evaluation “report card,” empowerment evaluators coach individuals and organizations through an evaluation of their own strategy(ies) by providing them with the knowledge, skills, and resources they need to conduct just such an evaluation." Contents Empowerment Evaluation: An Overview 11 Principles of Empowerment Evaluation 11 Step 1: Preparing for the Hiring Process 23 Step 2: Writing a Job Announcement 31 Step 3: Finding Potential Empowerment Evaluators 39 Step 4: Assessing the Candidates 45 Step 5: Writing An Evaluation Contract 55 Step 6: Building an Effective Relationship With Your Evaluator 61 Step 7: Assessing and Sustaining the Evaluation 65 Appendix A: Resources for General Evaluation and Empowerment Evaluation 73 Appendix B: Worksheets for Hiring an Empowerment Evaluator 75 Appendix C: Sample Hiring Committee Confidentiality Statement 89 Appendix D: Sample Job 90 Appendix E: Sample Request For Proposals 91 Appendix F: Sample Interview Questions 93 Appendix G: Sample Budget and Narrative for an Evaluation Team 95

Creating the (organisational) conditions for an OM based M&E and learning practice - Deprez, Steff (Outcome Mapping Learning Community, 2009)

Communities of practice: Linking knowledge, policy and practice - Hearn, Simon; White, Nancy (ODI, 2009)
This paper describes the basic characteristics of CoPs and provides a rationale for their growing importance in international development. It also suggests some ways in which CoPs can be supported by development agencies, research institutes and donors to strengthen the linkages between knowledge, policy and practice.

DAC Peer Review of Switzerland - 2009 - OECD (OECD, 2009)

Glossary of Key Terms in Evaluation and Results Based Management - OECD (OECD, 2009)

Utilization-focused evaluation for agricultural innovation - Patton, Michael Quinn; Horton, D (CGIAR, 2009)
Utilization-focused evaluation (UFE) is based on the principle that an evaluation should be judged by its utility. So no matter how technically sound and methodologically elegant, an evaluation is not truly a good evaluation unless the findings are used. UFE is a framework for enhancing the likelihood that evaluation findings will be used and lessons will be learnt from the evaluation process. This Brief, based on the book Utilization-focused evaluation, introduces this approach to evaluation, outlines key steps in the evaluation process, identifies some of the main benefits of UFE, and provides two examples of UFE in the context of programmes aimed at promoting agricultural innovation.

Developing a performance story report: user guide - Roughly, A.; Dart, J. (Commonwealth of Australia, 2009)
This report is a nuts and bolts guide to developing a Collaborative Outcomes Report (COR)/ Performance Story Report (PSR) produced by the Australian Government and Jess Dart. It includes practical tips, step-by-step process guides and definitions of key concepts surrounding these approaches.

The New Development Economics: We Shall Experiment, but How Shall We Learn? - Rodrik, Dani (SSRN, 2008)
Development economics is split between macro-development economists - who focus on economic growth, international trade, and fiscal/macro policies - and micro-development economists - who study microfinance, education, health, and other social programs. Recently there has been substantial convergence in the policy mindset exhibited by micro evaluation enthusiasts, on the one hand, and growth diagnosticians, on the other. At the same time, the randomized evaluation revolution has led to an accentuation of the methodological divergence between the two camps. Overcoming the split requires changes on both sides. Macrodevelopment economists need to recognize the distinct advantages of the experimental approach and adopt the policy mindset of the randomized evaluation enthusiasts. Micro-development economists, for their part, have to recognize that the utility of randomized evaluations is restricted by the narrow and limited scope of their application. As the Chinese example illustrates, extending the experimental mindset to the domain of economy-wide reforms is not just possible, it has already been practiced with resounding success in the most important development experience of our generation.
Keywords: Economics; Finance; Health Care; International Development; International Economics; International Trade; Macroeconomics; Microeconomics; Social Policy; Welfare

Exploring the science of complexity: Ideas and implications for development and humanitarian efforts - Ramalingam, Ben; Jones, Harry; Reba, Toussaint; Young, John (ODI, 2008)
This paper draws on the science of complexity to outline alternative approaches to analysing and understanding problems faced in humanitarian and development work.

Contribution Analysis: An approach to exploring cause and effect - Mayne, J. (ILAC, 2008)
Questions of cause and effect are critical to assessing the performance of programmes and projects. When it is not practical to design an experiment to assess performance, contribution analysis can provide credible assessments of cause and effect. Verifying the theory of change that the programme is based on, and paying attention to other factors that may influence the outcomes, provides reasonable evidence about the contribution being made by the programme.

Strengthening humanitarian networks: Applying the network functions approach - Ramalingam, Ben; Mendizabal, Enrique; Schenkenberg van Mierop, Ed (ODI, 2008)
This note offers a simple, flexible and powerful methodology — the network functions approach (NFA) — that can be applied to analyse and strengthen humanitarian and development networks. Based on research undertaken at ODI and elsewhere, the NFA suggests there are six overlapping functions that different networks perform in varying combinations. Through reflection on a network’s current activities and how they relate to each of these functions, the NFA helps those facilitating, acting within or supporting networks to work towards an ‘ideal’ functional mix. The aim is for network strategies to be honed, thinking clarified, activities sharpened and ultimately, humanitarian performance improved.

Report on outcomes and get everyone involved: The Participatory Performance Story Reporting Technique - Dart, J. (2008)
This paper outlines the background and philosophy of Collaborative Outcomes Reporting (COR) and Performance Story Reporting (PSR), providing an overview of these emerging techniques and showing how they can be used as a framework for reporting on contribution to long-term outcomes (or targets) using mixed methods and participatory process. The report serves as an introduction to these approaches in evaluation and discusses their bias, limitations and where they might best be applied.

A developmental Evaluation Primer - Gamble, Jamie A.A. (J.W. McConnell Family Foundation, 2008)

Success and Failure in eGovernment Projects - Heeks, Richard (Institute for Development Policy and Management, 2008)

Context-sensitive engagement: Lessons learned from Swiss experiences in South Asia for aid effectiveness in fragile scenarios - Paffenholz, Thania; Jütersonke, Oliver (CCDP, 2008)

HIV, food and drugs: Livelihoods, nutrition and Anti-retroviral Therapy (ART) in Kenya and Zambia - Samuels, Fiona; Rutenberg, Naomi (ODI, 2008)
Key points • ART has restored the health of many people living with HIV, but their livelihoods lag behind • Good nutrition is important for people on ART. Food supplementation can help, but is no substitute for sustainable livelihoods • The livelihoods of people on ART can be bolstered through skills, livelihood networks, assets and cash or food transfers

The Role of Outcome Mapping in Developing a Rural Telemedicine System - Tucker, William; Blake, Edwin (University of the Western Cape, 2008)
We describe the use of Outcome Mapping to guide the design of a rural telemedicine consultation system in South Africa. While Outcome Mapping was not primarily intended to guide design, we show that it tied in well with a cyclical participatory design method for an Information and Communication Technology for Development project

Responding to change: Learning to adapt in development cooperation - Engel, Paul; Keijzer, Niels; Ørnemark, Charlotte (ECDPM, 2007)

Using Emergence to Take Social Innovation to Scale - Wheatley, Margaret; Frieze, Deborah (The Shambhala Institute, 2007)
In spite of current ads and slogans, the world doesn’t change one person at a time. It changes as networks of relationships form among people who discover they share a common cause and vision of what’s possible. This is good news for those of us intent on changing the world and creating a positive future. Rather than worry about critical mass, our work is to foster critical connections. We don’t need to convince large numbers of people to change; instead, we need to connect with kindred spirits. Through these relationships, we will develop the new knowledge, practices, courage, and commitment that lead to broad-based change. But networks aren’t the whole story. As networks grow and transform into active, working communities of practice, we discover how Life truly changes, which is through emergence. When separate, local efforts connect with each other as networks, then strengthen as communities of practice, suddenly and surprisingly a new system emerges at a greater level of scale. This system of influence possesses qualities and capacities that were unknown in the individuals. It isn’t that they were hidden; they simply don’t exist until the system emerges. They are properties of the system, not the individual, but once there, individuals possess them. And the system that emerges always possesses greater power and influence than is possible through planned, incremental change. Emergence is how Life creates radical change and takes things to scale. Emergence has a life-cycle. It begins with networks, shifts to intentional communities of practice and evolves into powerful systems capable of global influence. Since its inception in 1992, The Berkana Institute has striven to learn how living systems work, how they emerge from networks to communities to systems of influence. In our global work—primarily with economically poor communities in many different nations—we have experimented actively with emergence in many different contexts. We have demonstrated what’s possible when we connect people across difference and distance. By applying the lessons of living systems and working intentionally with emergence and its life-cycle, we have become confident that local social innovations can be taken to scale and provide solutions to many of the world’s most intractable issues.

Using Randomization in Development Economics Research: A Toolkit - Duflo, Esther; Glennerster, Rachel; Kremer, Michael (National Bureau of Economic Research, 2006)
This paper is a practical guide (a toolkit) for researchers, students and practitioners wishing to introduce randomization as part of a research design in the field. It first covers the rationale for the use of randomization, as a solution to selection bias and a partial solution to publication biases. Second, it discusses various ways in which randomization can be practically introduced in a field settings. Third, it discusses designs issues such as sample size requirements, stratification, level of randomization and data collection methods. Fourth, it discusses how to analyze data from randomized evaluations when there are departures from the basic framework. It reviews in particular how to handle imperfect compliance and externalities. Finally, it discusses some of the issues involved in drawing general conclusions from randomized evaluations, including the necessary use of theory as a guide when designing evaluations and interpreting results.

Engaging scientists through institutional histories - Prasad, Shambu; Hall, Andrew; Thummuru, Laxmi (CGIAR, 2006)
An institutional history is a narrative that records key points about how institutional arrangements – new ways of working – evolve over time creating more effective ways to achieve goals. It can be used to document institutional innovations in projects and to highlight barriers to change. An institutional history draws out and synthesizes lessons for research organizations and partners as well as for others in similar circumstances.

Building effective research policy networks: linking function and form - Mendizabal, Enrique (ODI, 2006)
This paper looks at the factors that affect the key structural characteristics of research policy networks in an attempt to develop a methodology for studying and understanding what networks do and how. It builds on previous work on the roles and functions that networks carry out and focuses on their structural characteristics. The literature on networks is explored to identify the main and most common factors affecting their structure, and then the paper considers the possible effects of these structural factors on a network's ability to perform one or more of these functions based on some of the lessons learned from a study of seven research policy networks in Peru. The paper provides a menu of factors and characteristics worth considering when intending to build effective networks.
OCLC: 183914885

Herramientas de Conocimiento y Aprendizaje: Una Guía para Organizaciones Humanitarias y de Desarrollo - Ramalingam, Ben (ODI, 2006)
This toolkit presents entry points and references to the wide range of tools and methods that have been used to facilitate improved knowledge and learning in the development and humanitarian sectors.
Keywords: spanish

Tools for Knowledge and Learning: A Guide for Development and Humanitarian Organisations - Ramalingam, Ben (ODI, 2006)
This toolkit presents entry points and references to the wide range of tools and methods that have been used to facilitate improved knowledge and learning in the development and humanitarian sectors.

Understanding Networks: The functions of research policy networks - Mendizabal, Enrique (ODI, 2006)
We are constantly talking about networks. Banks use their networks to offer global services to customers; airlines fly passengers all over the world via their networks of partners; news agencies use media networks to keep us informed every minute of the day; and terrorist networks threaten citizens around the world. The importance of networks extends to the development sector: they organise civil society to advocate for and implement change; they link the local with the global, the private with the public; and they provide spaces for the creation, sharing and dissemination of knowledge. In a way, networks seem to make anything and everything happen. But we have yet to understand what they are and what they can and cannot do. In the development literature, a huge variety of policy and social network concepts and applications exists. This paper attempts to set out a framework to help clarify what research policy networks do.

Alps - Accountability, Learning and Planning System - ActionAid (ActionAid International, 2006)
Alps is ActionAid’s over arching accountability framework, containing within it our programme planning system. Alps is distinctive in that it is strongly driven by principles, and sets out necessary personal attitudes and behaviours alongside organisational processes for planning, strategy formulation, learning, monitoring reviews/evaluations and audit. Alps defines our standards, not only about what we do but also how we do it. Alps is part of ActionAid’s human rights-based work.

Poverty Unperceived: Traps, Biases and Agenda - Chambers, Robert (IDS, 2006)

Conflict Sensitive Programme Management: A quick guide - SDC (SDC, 2006)

Conflict-sensitive programme management (CSPM) - SDC (2006)

Horizontal Evaluation: Stimulating social learning among peers - Thiele, G; velasco, C; Manrique, K (2006)
Horizontal evaluation is a flexible evaluation method that combines self-assessment and external review by peers. We have developed and applied this method for use within an Andean regional network that develops new methodologies for research and development (R&D). The involvement of peers neutralizes the lopsided power relations that prevail in traditional external evaluations, creating a more favourable atmosphere for learning and improvement. The central element of a horizontal evaluation is a workshop that brings together a group of ‘local participants’ who are developing a new R&D methodology and a group of ‘visitors’ or ‘peers’ who are also interested in the methodology. The workshop combines presentations about the methodology with field visits, small group work and plenary discussions. It elicits and compares the perceptions of the two groups concerning the strengths and weaknesses of the methodology; it provides practical suggestions for improvement, which may often be put to use immediately; it promotes social learning among the different groups involved; and it stimulates further experimentation with and development of the methodology in other settings.

The Use and Abuse of the Logical Framework Approach: A Review of International Development NGOs' Experiences - Bakewell, Oliver; Garbutt, Anne (Swedish International Development Cooperation Agency, 2005)
The logical framework approach (LFA) has come to play a central role in the planning and management of development interventions over the last twenty years. Although the logical framework has become universally known, it is far from universally liked. It has been the subject of much criticism over the years, concerning both the theoretical basis of the approach, and the way it is applied in practice. In this review, we have attempted to take stock of the current views of international development NGOs on the LFA and the ways in which they use it.

Appreciative Inquiry: An approach for learning and change based on our own best practices​ - Acosta, Anne; Douthwaite, Boru (Institute of Learning and Change, 2005)
Since it was conceptualized in the late 1980s as a research methodology and change paradigm, the technique of ‘appreciative inquiry’ (AI) has proved to be highly effective for capturing the positive features of an organization or social system and energizing the members to strive for higher levels of performance. This Brief outlines the basic principles and methods of AI, describes various domains in which it has been undertaken and provides a recent example of its use in a centre affiliated with the Consultative Group on International Agricultural Research (CGIAR).

Innovation histories: A method from learning from experience - Douthwaite, B; Ashby, J (2005)
Preparing an ‘innovation history’ is a method for recording and reflecting on an innovation process. People who have been involved in the innovation jointly construct a detailed written account (sometimes referred to as a ‘learning history’) based on their recollections and on available documents. The process of preparing this history stimulates discussion, reflection and learning amongst stakeholders. Subsequent planning can build on the lessons learned, formulate a shared vision and act as a catalyst for change. Based on the initial detailed account of the innovation process, more concise informational products can be prepared that summarize the innovation process for wider dissemination of findings. These may include public awareness materials, policy briefs or articles in professional journals.

Process Monitoring of Impacts: Towards a new approach to monitor the implementation of Structural Fund Programmes - Hummelbrunner, Richard; Huber, Wolf; Arbter, Roland (ÖAR Regionalberatung, 2005)

Outcome mapping: A method for tracking behavioural changes in development programs - Smutylo, Terry (2005)
This guide published by the Institutional Learning and Change (ILAC) Initiative provides a detailed overview of using outcome mapping as an evaluation tool. Contents Expressing results as changes in behaviour Outcome mapping terms The three stages of outcome mapping Figure 1. The three stages and twelve steps of outcome mapping Stage 1. Intentional design Figure 2. The four basic questions of the intentional design stage Stage 2. Outcome and performance monitoring Stage 3. Evaluation planning Using outcome mapping The Ceja Andina Project The Agro-industry and Market Development project for Arracacha

Adaptive management of natural resources: theory, concepts, and management institutions. - Stankey, George H.; Clark, Roger N.; Bormann, Bernard T. (U.S. Department of Agriculture, Forest Service, Pacific Northwest Research Station, 2005)
This report reviews the extensive and growing literature on the concept and application of adaptive management. Adaptive management is a central element of the Northwest Forest Plan and there is a need for an informed understanding of the key theories, concepts, and frameworks upon which it is founded. Literature from a diverse range of ﬁelds including social learning, risk and uncertainty, and institutional analysis was reviewed, particularly as it related to application in an adaptive management context. The review identiﬁes opportunities as well as barriers that adaptive management faces. It concludes by describing steps that must be taken to implement adaptive management.

A mini-primer of critical systems heuristics. - Ulrich, W. (2005)
"Critical Systems Heuristics," also just called "Critical Heuristics" or "CSH," is a framework for reflective practice based on practical philosophy and systems thinking. The basic idea of CSH is to support boundary critique – a systematic effort of handling boundary judgments critically. Boundary judgments determine which empirical observations and value considerations count as relevant and which others are left out or are considered less important. Because they condition both "facts" and "values," boundary judgments play an essential role when it comes to assessing the meaning and merits of a claim. Their systematic discussion can help bridge differences of perspectives across disciplines and between experts and non-experts. They also lend themselves to a specific critical employment, called emancipatory boundary critique, against claims that do not uncover their underlying boundary assumptions. CSH can thus serve as a tool for coproducing knowledge as well as for critical and emancipatory purposes on the part of people concerned by, but not necessarily involved in, the definition of relevant facts and values.

A Brief Introduction to Critical Systems Heuristics (CSH) - Ulrich, Werner (2005)

Democratic governance and evaluation. - Hanberger, A. (2004)
This paper, presented by Anders Hanberger at the Sixth EES (European Evaluation Society) Conference in Berlin, Germany (September 30-October 2, 2004) argues that governance, democracy and evaluation impact each in different ways. It offers a discussion centred around the evaluation of three general democratic governance models and the implications of leading democratic evaluations. "Since governance and democracy are changing phenomena, and evaluation is embedded in these structures, there is a need to illuminate and discuss the role of evaluation. Evaluation is to most people thought of as a democratic tool, but what do we mean with democratic evaluation? Furthermore, what is the role of evaluation in times when democracy and governance are changing? The many meanings of democracy and the shifting role of evaluation in various democratic governance settings are also a motive for discussing governance and democratic evaluation. Governance issues can be discussed in relation to different political systems. However, in this paper the discussion is confined to democratic governance systems. The premise of this paper is that governance, democracy and evaluation affect one another in different ways. Governance is intertwined with democracy, and democracy and governance can be maintained or strengthened by evaluation, for example. Because different models of governance and democracy presuppose one another, evaluating governance models, or programme processes/outcomes where a specific governance model sets up the context, have implications for the model under scrutiny and subsequently for democracy. Accordingly, the implications of democratic evaluations need to be discussed in various ways. For analytical purpose democracy and governance are sometimes kept apart." (Hanberger, 2004)​

ALPS in action: a review of the shift in ActionAid towards a new Accountability, Learning and Planning System | Participatory Methods - Guijt, Irene (2004)

QUIP: Understanding clients through in-depth interviews - IMP-ACT (Institute of Development Studies, 2004)
This Practice Note by Imp-Act gives a step-by-step guide to developing and conducting in-depth interviews using the QUIP approach, and analysing the information and making conclusions based on what you have learned.

Realist Evaluation - Pawson, Ray; Tilley, Nick (2004)
Contents: Introduction 1. The nature of programmes and how they work 2. Basic concepts in the explanation and understanding of programmes 3. Strategies and methods of realist evaluation 4. Realism’s place in the policy cycle: formative, summative and synthetic approaches 5. The nature, presentation and use of findings from realist evaluation 6. Conclusion: strengths, limitations and relationships with other approaches Appendix I – ‘Thinking it through’: an exercise in realist hypothesis making. Appendix II – ‘Varieties of realist evaluation’: pocket illustrations of quantitative, qualitative, formative and synthetic applications. Appendix III – ‘Would it work here?’: a grid to help decide on the feasibility of mounting a programme ‘on your patch’.

Participatory approaches: a facilitator's guide - VSO (VSO, 2004)
This book provides a set of guidelines for people who will be involved in participatory processes and projects with specific design focus on VSO (Voluntary Services Overseas) volunteers. It looks at appropriate levels of participation; pitfalls of participatory approaches (PA); best practice in facilitation; and tools for participation. The guide is organised into three parts (I) Principles, (II) Methods and (III) Toolkit. Part I gives a background to PA with a comparative analysis of PA in relation to top-down approaches, and within the range of PAs; looks at the role PA in VSO; discusses how to facilitate participatory processes with multiple stakeholders; presents a framework for PA on different levels of participation; and examines key facilitation skills needed to support participatory activities. Part II collates a range of participatory methods that have been used successfully in the field by VSO volunteers. Methods are categorised according to this suitability for use at different stages of a project process. Examples are also given of methods that can be used for specific purposes, such as participatory organisational appraisal and gender/diversity analysis. Part III gives tips on how to choose the most appropriate tool and how to organise participatory workshops and small group activities. It also systematically records a range of tools used by development workers all over the world with reference to what tool is appropriate in what situation. A profile of each tool includes guidelines on its purpose, potential applications and variations, as well as possible pitfalls. Illustrative case studies taken form real experiences of development workers in the field are also included.

Most egovernment-for-development projects fail: how can risks be reduced? - Heeks, Richard (Institute for Development Policy and Management, 2003)
Research on the Information Society, the Digital Divide and Information and Communication Technologies for development

Beneficiary Assessment: An Approach Described - Salmen, Lawrence (World Bank, 2002)
Beneficiary assessment is a tool for managers who wish to improve the quality of development operations. This is an approach to information gathering which assesses the value of an activity as it is perceived by its principal users. The approach is qualitative in that it attempts to derive understanding from shared experience as well as observation, and gives primacy to the centrality of the other person’s point of view. As the Bank and others engaged in development activities seek to do their work better, one key indicator will need to be how the ultimate customer, or intended beneficiary, assesses the value of this work, project or policy, as it affects his or her life. The illumination of how an intended beneficiary appreciates a planned or ongoing developmental activity is the primary objective of this approach. Beneficiary assessment is a systematic inquiry into people’s values and behavior in relation to a planned or ongoing intervention for social and economic change. This method draws heavily from the tradition in social science known as "qualitative research...that fundamentally depends on watching people in their own territory and interacting with them in their own language, on their own terms" (Kirk and Miller). Yet beneficiary assessment also includes direct observation, incorporating simple counting, and is expressed in quantitative terms. The ultimate goal of beneficiary assessment is to reveal the meaning people give to particular aspects of their lives so that development activities may better enhance people’s ability to improve their own living conditions, as they see fit. This demands close rapport between the practitioner of this approach, the beneficiary and the development manager. The beneficiary assessment approach is not intended to supplant the questionnaire survey but to provide reliable qualitative, indepth information on the socio-cultural conditions of a beneficiary population which is intended to be of immediate use to managers and policymakers responsible for improving people’s lives.

Failure, Success and Improvisation of Information Systems Projects in Developing Countries - Heeks, Richard (Institute for Development Policy and Management, 2002)
Research on the Information Society, the Digital Divide and Information and Communication Technologies for development

Supporting Pioneering Leaders as Communities of Practice - How to Rapidly Develop New Leaders in Great Numbers - Wheatley, Margaret J (The Berkana Institute, 2002)
Do you ever stand back and try to see the big picture, the view from 50,000 feet of what's going on in organizations, communities, the world? From up there, how would you describe these times? Is it a time of increasing economic and political instability, of growing divisiveness and fear, of failing systems and dying dreams? Is it a time of new possibilities, of great examples of hope, of positive human evolution, of transformation? Are we succeeding in solving major problems, are we creating more? Is it any of these things, is it all of these things? It's important to think about how we answer this question, because that answer affects our choice of actions. If we think that, generally, things are working, that at present we're going through a difficult but temporary downturn, then we don't question current systems or their operating assumptions. Instead, we work hard to revive and improve them. We support initiatives and programs focused on process improvements, developing present systems to work more effectively and more efficiently. If we believe that the old system cannot be repaired, if we expect to see only more system failures, then the work is not to fix. Instead, support needs to be given to radically different processes and methods, new systems based on new assumptions. The work becomes not process improvement but process revolution.

Notes to Accompany ALPS - ActionAid (ActionAid International, 2001)

Manifesto for Agile Software Development - Agile Alliance (Agile Alliance, 2001)

Adaptive Management: A Tool for Conservation Practitioners - Salafsky, Nick; Margoluis, Richard; Redford, Kent (Foundations of Success, 2001)

Spiral Development: Experience, Principles, and Refinements - Boehm, B. W. (Carnegie Mellon University, 2000)

Voices of the poor: can anyone hear us? - Naraya, Deepa*Patel (The World Bank, 2000)
This book is the first in a three-part series, about the common patterns that emerged from the poor people's experiences in many different places. Chapter 1 sets out the conceptual framework and methodology. Chapter 2 discusses poverty from the perspective of the poor. Chapter 3 examines poor people's experience with the state, and includes case studies of access to health care and education. Chapter 4 addresses the nature and quality of poor people's interactions with civil society. Chapter 5 considers the household as a key social institution, and discusses gender relations within households and how these relations affect and are affected by larger institutions of society. Chapter 6 focuses on social fragmentation, and includes a discussion of social cohesion and social exclusion. Chapter 7 concludes the analysis and proposes some policy recommendations. The analysis leads to these conclusions: 1) poverty is multidimensional; 2) the state has been largely ineffective in reaching the poor; 3) the role of nongovernmental organizations (NGOs) in the lives of the poor is limited, forcing the poor to depend primarily on their own informal networks; 4) households are crumbling under the stresses of poverty; and 5) the social fabric - poor people's only

Accountability Learning and Planning System - ActionAid (ActionAid International, 2000)