Discussing Adaptive Approaches for Development Programmes - Pett, Jamie (Medium - LearnAdapt, 2020)
What can adaptive approaches from other sectors teach development practitioners about dealing with uncertainty?

Development and radical uncertainty - Feinstein, Osvaldo (Development in Practice, 2020)
Development strategies, programmes and projects are designed making assumptions concerning several variables such as future prices of outputs and inputs, exchange rates and productivity growth. However, knowledge about the future is limited. Uncertainty prevails. The usual approach to deal with uncertainty is to reduce it to risk. Uncertainty is perceived as a negative factor that should and can be eliminated. This article presents an alternative approach which recognises that radical uncertainty is irreducible to risk, identifying a positive dimension of uncertainty and showing its implications for development practice.

Program Logic Foundations: Putting the Logic Back into Program Logic - Hawkins, Andrew J. (Journal of MultiDisciplinary Evaluation, 2020)
Abstract Background: Program logic is one of the most used tools by the public policy evaluator. There is, however, little explanation in the evaluation literature about the logical foundations of program logic or discussion of how it may be determined if a program is logical. This paper was born on a long journey that started with program logic and ended with the logic of evaluation. Consistent throughout was the idea that the discipline of program evaluation is a pragmatic one, concerned with applied social science and effective action in complex, adaptive systems. It gradually became the central claim of this paper that evidence-based policy requires sound reasoning more urgently than further development and testing of scientific theory. This was difficult to reconcile with the observation that much evaluation was conducted within a scientific paradigm, concerned with the development and testing of various types of theory. Purpose: This paper demonstrates the benefits of considering the core essence of a program to be a proposition about the value of a course of action. This contrasts with a research-based paradigm in which programs are considered to be a type of theory, and in which experimental and theory-driven evaluations are conducted. Experimental approaches focus on internal validity of knowledge claims about programs and on discovering stable cause and effect relationships—or, colloquially, ‘what works?’. Theory-driven approaches tend to focus on external validity and in the case of the realist approach, the search for transfactual causal mechanisms—extending the ‘what works’ mantra to include ‘for whom and in what circumstances’. On both approaches, evaluation aspires to be a scientific pursuit for obtaining knowledge of general laws of phenomena, or in the case of realists, replicable context-mechanism-outcome configurations. This paper presents and seeks to justify an approach rooted in logic, and that supports anyone to engage in a reasonable and democratic deliberation about the value of a course of action. It is consistent with systems thinking, complexity and the associated limits to certainty for determining the value of a proposed, or actual, course of action in the social world. It suggests that evaluation should learn from the past and have an eye toward the future, but that it would be most beneficial if concerned with evaluating in the present, in addressing the question ‘is this a good idea here and now? Findings: In seeking foundations of program logic, this paper exposes roots that extend far deeper than the post-enlightenment, positivist and post-positivist social science search for stable cause and effect relationships. These roots lie in the 4th century BCE with Aristotle’s ‘enthymeme’. The exploration leads to conclusions about the need for a greater focus on logic and reasoning in the design and evaluation of programs and interventions for the public good. Science and research are shown to play a crucial role in providing reasons or warrants to support a claim about the value of a course of action; however, one subordinate to the alpha-discipline of logical evaluation and decision making that must consider what is feasible given the context, capability and capacity available, not to mention values and ethics. Program Design Logic (PDL) is presented as an accessible and incremental innovation that may be used to determine if a program makes sense ‘on paper’ in the design stage as well as ‘in reality’ during delivery. It is based on a configurationalist theory of causality and the concepts of ‘necessary’ and ‘sufficient’ conditions. It is intended to guide deliberation and decision making across the life cycle of any intervention intended for the public good.

Principles for managing in complexity - Proud, Emma (LearnAdapt, 2020)
Written by Toby Lowe and Shaheen Warren (Centre for Public Impact) and Sam Sharp (Overseas Development Institute), with input from Jamie…

Nuru Kenya transition from Nuru International, Kenya - Dillan, Haley; Ouma, Joel; Yamron, David (What Transformation Takes - Evidence of Responsible INGO Transitions to Locally Led Development Around the World, 2020)
This case study is an example of a phased transfer of ownership and responsibility from INGO Nuru International to Nuru Kenya, including the exit of all international staff. Post-transition, Nuru Kenya is managed entirely by Kenyan staff, although it continues to receive financial support from Nuru International. A lot of the elements described are aligned with Adaptive Management ways of working.

Instituciones que aprenden - Informe para la XXVII Cumbre Iberoamericana de Jefes y Jefas de Estado y de Gobierno - Oliván Cortés, Raúl (XXVII Cumbre Iberoamericana, 2020)
La pandemia de la Covid-19 ha brotado súbitamente en un momento de transición para las instituciones y organizaciones sociales de medio mundo. Cuando aún no se había superado la crisis de confianza de la última década entre los ciudadanos y los gobiernos, en un momento de gran impulso de las iniciativas de gobierno abierto, planes de innovación y transformación digital, para hacer más democráticas y eficientes las administraciones a través de programas políticos encaminados a configurar un nuevo contrato social, verde y digital en el marco de la Agenda 2030; el coronavirus ha provocado un shock histórico alterando el decurso del siglo XXI y exigiendo una aceleración de todos los procesos en marcha. La complejidad y dimensión de la pandemia ha puesto de manifiesto la necesidad de arquitecturas institucionales más flexibles, ágiles y resilientes, que sean capaces de incorporar toda la energía civil para aprender de su talento y creatividad, dándole mayor protagonismo a la ciudadanía (activistas, emprendedores, tejido asociativo, academia, makers…) no solo en la toma de decisiones sino también en el diseño e implementación de las estrategias. Conforme se centrifugaba a millones de empleados a teletrabajar desde sus casas, se hacía más poderosa la idea de pensar las organizaciones como flujos más allá de los organigramas estáticos de lugares y personas. Transformar las jerarquías en redes, concebirlas como cuerpos sociales dinámicos, no solo amplía su radio de acción y su conectividad exterior, también reactiva sus fortalezas internas, aflora los liderazgos ocultos, multiplica el valor social producido y maximiza el uso eficiente de los recursos en una época de limitaciones. Los laboratorios de innovación pública, social o ciudadana, o laboratorios de gobierno, junto a otras formas de innovación abierta y diseño social, se reivindican como proyectos inspiradores de un cambio de paradigma: de las instituciones que ordenan a las instituciones que aprenden. Pensar las organizaciones bajo el prisma de la ciencia de redes y la ética de los rizomas –nodos, enlaces, hubs, comunidades…– nos debería permitir una aproximación a la compleja y escurridiza tarea de configurar los ecosistemas de innovación y creatividad en el ámbito de lo público y lo social. Este informe propone un modelo denominado Hexágono de la Innovación Pública (HIP) que promueve un cambio sistémico a través de seis vectores (OPEN_ abierto, TRANS_ transversal, FAST_ ágil, PROTO_ modelado, CO_ colaborativo y TEC_ tecnológico) basados en las propiedades de las redes y en el análisis de 105 metodologías que usan las agencias más innovadoras del mundo. Se incluyen una herramienta de autodiagnóstico y el HIP-SIM, una primera aproximación a un software abierto para visualizar, modelar y simular la creación de ecosistemas innovadores con el que queremos propiciar una comunidad y un debate internacional.

Instituições que aprendem - Relatório para a XXVII Cimeira Ibero-Americana de Chefes de Estado e de Governo - Oliván Cortés, Raúl (XXVII Cimeira Ibero-Americana, 2020)
A pandemia da Covid-19 surgiu de súbito num momento de transição para as instituições e organizações sociais de meio mundo. Antes ainda de se ter ultrapassado a crise de confiança da última década entre os cidadãos e os governos, num momento de grande impulso das iniciativas de governo aberto e de planos de inovação e transformação digital para tornar mais democráticas e eficientes as administrações através de programas políticos encaminhados para configurar um novo contrato social, verde e digital no quadro da Agenda 2030, o coronavírus provocou um choque histórico alterando o curso do século XXI e exigindo uma aceleração de todos os processos em andamento. A complexidade e dimensão da pandemia revelou a necessidade de arquiteturas institucionais mais flexíveis, ágeis e resilientes, capazes de incorporar toda a energia civil para aprenderem com o seu talento e criatividade, dando maior protagonismo à cidadania (ativistas, empreendedores, tecido associativo, academia, makers…) não só na tomada de decisões mas também na configuração e implementação das estratégias. À medida que se afastava do centro a milhões de empregados para teletrabalhar a partir de casa, tornou-se mais poderosa a ideia de pensar nas organizações como fluxos que excedem os organigramas estáticos de lugares e pessoas. Transformar as hierarquias em redes, concebê-las como corpos sociais dinâmicos, não só alarga o seu raio de ação e conectividade externa, mas também reativa as suas forças internas, aflora as lideranças ocultas, multiplica o valor social produzido e maximiza o uso eficiente dos recursos numa época de limitações. Os laboratórios de inovação pública, social ou cidadã, ou laboratórios de governo, a par de outras formas de inovação aberta e configuração social, reivindicam-se como projetos inspiradores de uma mudança de paradigma: de instituições que ordenam para instituições que aprendem. Pensar nas organizações sob a ótica da ciência das redes e da ética dos rizomas – nós, ligações, hubs, comunidades… – deverá permitir-nos uma aproximação à complexa e subtil tarefa de configurar os ecossistemas de inovação e criatividade no âmbito público e social. Este relatório propõe um modelo denominado Hexágono da Inovação Pública (HIP) que promove uma transformação sistémica através de seis vetores (OPEN_ aberto, TRANS_ transversal, FAST_ rápido, PROTO_ modelado, CO_ colaborativo e TEC_ tecnológico) baseados nas propriedades das redes e na análise de 105 metodologias usadas pelas agências mais inovadoras do mundo. Incluem-se uma ferramenta de autodiagnóstico e o HIP-SIM, uma primeira aproximação a um software aberto para visualizar, modelar e simular a criação de ecossistemas inovadores com o qual queremos favorecer a comunidade e o debate internacional.

Risk navigation for Thinking and Working Politically: The work and disappearance of Sombath Somphone - Sims, Kearrin (Development Policy Review, 2020)
Abstract Motivation On December 15, 2012 Sombath Somphone was abducted at a police checkpoint in his home city of Vientiane, the capital of Laos; his whereabouts remain unknown. This article considers his work and disappearance through the lens of Thinking and Working Politically (TWP) approaches to development. The article is supportive of TWP, but emphasizes the significant risks of politicized programming in authoritarian contexts. Purpose By examining the case of Sombath Somphone, the article seeks to offer insights for safer, and more effective, TWP programming. It considers how specific events in authoritarian contexts can suddenly reposition development workers and/or organizations as political dissidents. Approaches and Methods The argument draws on analysis of grey literature; conversational and observational knowledge accrued during 18 months of fieldwork in Laos between 2011 and 2018; on‐going formal and informal interviews with members of Laos’ civil society sector; and extensive dialogue with Sombath Somphone’s wife, Ng Shui Meng. Findings The article identifies four key factors that contributed to the enforced disappearance of Sombath Somphone: international exposure; timing; particular elites; and strategies of oppression. It highlights the need for further consideration of how to anticipate and mitigate the dangers of politically oriented development work, as well as the different forms of risk experienced by local and international development actors working in authoritarian contexts. Policy Implications TWP has much to offer to development practice, but its contributions should not threaten the safety of local development actors. More attention must be given to preventing and mitigating such risks.

Reflections on if and how our partnership is working in Bangladesh - Paul, Sukanta; Snijder, Mieke; Apgar, Marina (CLARISSA, 2020)
Effective partnership working is crucial to producing quality results, especially when working with complex problems such as the worst forms...

Intervention or Collaboration? Redesigning Information and Communication Technologies for Development. - Bon, Anna (2020)
How can we design and build digital technologies to support people in poor and low resource environments to achieve their objectives? And how can we do this inclusively and ethically, while considering the complexity of their living and working environments? This is the central question in my research. One of the grand challenges of international development cooperation is to make digital technologies available for social and economic development of poor regions of the world. To achieve this goal – often referred to as ICT4D – knowledge and technologies are transferred from wealthy countries to poor regions. Nevertheless, these efforts have often turned out unsuccessful and unsustainable, despite large budgets and numerous projects in prestigious international development programs. Mismatch between the transferred technologies and the target environment is a recurrent problem of ICT4D projects. Improvement can be achieved, for example, by involving end-users in the design process. International development organizations are aware of this, and terms like "co-creation", "participation" and "user-oriented design" have nowadays become part of the international development discourse. However, real co-creation and user-centered design are incompatible with unidirectional transfer of technologies and knowledge (this is how ICT4D is commonly organized, in conventional international development). Moreover, the term participation becomes meaningless, in the light of externally formulated development goals. One key question to ask is: what do the envisaged users want? Remarkably, many ICT4D projects, programs and policies do not really ask and (field) investigate this question, which can only be answered by extensive research on-the-ground. This thesis describes the search for and the design of an alternative approach to ICT4D. Ten years of field and action research with partners in Mali, Burkina Faso, and Ghana have led to a collaborative, iterative and adaptive approach, dubbed "ICT4D 3.0". What is novel of this alternative approach and how does it answer the central question? First of all, ICT4D 3.0 is a practical approach for critical investigation and action. It consists of a reconfigurable framework that guides the design and development of information systems, bridging the knowledge gap between developers and users to unlock and integrate different domains of (global, local, indigenous, academic, nonacademic) knowledge. It targets complex, resource-constrained environments where many (for the ICT developers and researchers) unfamiliar conditions or obstacles may exist. It fosters innovative capacity and learning in action, bringing together people with different backgrounds and perspectives in trans-disciplinary and multicultural teams. It is socio-technical, result-oriented, focused on the objectives of the stakeholders and the requirements of their livelihoods. This approach has been validated in various different contexts, by users, ICT developers, practitioners and students. Second, ICT4D 3.0 contributes to a theoretical understanding of ICT4D as a process of networked innovation in complex (adaptive) systems. The underlying idea is that knowledge sharing and diffusion of innovations are complex (non-linear) dynamic processes that evolve and propagate through social networks in rather unpredictable ways, whereby innovation works out differently, depending on context, and whereby contextual (e.g. social, cultural, environmental, political) factors play an important role, and have to be considered. This theoretical framework explains the effectiveness of a collaborative, iterative, adaptive approach in ICT4D. Third, ICT4D 3.0 is built on ethical principles. When reflecting on the meaning and purpose of digital development, it is clear that digital development is not only a question of technology and practice, and collaboration is more than a prerequisite for successful technological innovation and long-term sustainability: collaboration is a fundamental human, ethical value. Therefore, as a reflective practitioner, one has to ask oneself whose interests one is actually looking after, which goals one is trying to achieve, where they come from, how power and political issues play a role and which core values are at stake. This makes ICT4D 3.0 into a democratic process of dialogue and deliberation, in which all voices are heard, in which the local context and complexity are central and in which development goals are determined by the users themselves and not imposed from outside. In this light, the approach proposed in this thesis takes a value position and can be considered a decolonial approach, striving for democracy, emancipation, autonomy and social and economic betterment. Field experience shows that ICT4D can be a meaningful, collaborative, networked process of knowledge sharing, driven by local initiatives, realizing change for the better, in a complex world.

Windows on the world: The power of assumptions in uncertain times - Aston, Thomas (Medium, 2020)
In my last blog on theory-based Monitoring Evaluation and Learning (MEL), I explained why relationships matter, and how to assess change…

Rubrics as a harness for complexity - Aston, Thomas (Medium, 2020)
In this final blog in the series, I want to look at the potential value of rubrics. While evaluability assessments can help us to…

“Real”​ process tracing: part 1 — context - Aston, Thomas (Medium, 2020)
When asserting the value of theory-based methods, you often here words like “black boxes” and “causal mechanisms.” These are commonly…

Reframing, refining, and reconceptualising the worst forms of child labour through participatory adaptive programming - Apgar, Marina; Burns, Danny (Journal of modern slavery, 2021)
This article explores the potential of using participatory action research as an adaptive programming modality to drive learning and innovation to tackle the drivers of (and seek to eliminate) the Worst Forms of Child Labour. We draw on our experience from early phases of implementation of a large-scale action research programme, which despite the constraints covid-19 posed in moving to full implementation and participatory engagement with children and other stakeholders on the ground, is already generating rich learning about the opportunities and challenges of designing programmes that respond to the complex reality of WFCL. We share early learning about what it takes to be fully open to using the lived experience of programme development, and early findings from scoping and mapping of the dynamics of social norms, business practices and urban neighbourhoods and supply chains influencing WFCL in Bangladesh and Nepal, to frame and reframe the questions and response strategies and operationalise a participatory adaptive intent to work with hidden and complex dynamics that characterise the WFCL.

A Systems Framework for International Development: The Data-Layered Causal Loop Diagram - Blair, Courtney; Gralla, Erica; Wetmore, Finley; Goentzel, Jarrod; Peters, Megan (Production and Operations Management, 2021)
Meeting the United Nations Sustainable Development Goals (SDGs) will require adapting or redirecting a variety of very complex global and local human systems. It is essential that development scholars and practitioners have tools to understand the dynamics of these systems and the key drivers of their behavior, such as barriers to progress and leverage points for driving sustainable change. System dynamics tools are well suited to address this challenge, but they must first be adapted for the data-poor and fragmented environment of development work. Our key contribution is to extend the causal loop diagram (CLD) with a data layer that describes the status of and change in each variable based on available data. By testing dynamic hypotheses against the system's actual behavior, it enables analysis of a system's dynamics and behavioral drivers without simulation. The data-layered CLD was developed through a 4-year engagement with USAID/Uganda. Its contributions are illustrated through an application to agricultural financing in Uganda. Our analysis identified a lack of demand for agricultural loans as a major barrier to broadening agricultural financing, partially refuting an existing hypothesis that access to credit was the main constraint. Our work extends system dynamics theory to meet the challenges of this practice environment, enabling analysis of the complex dynamics that are crucial to achieving the SDGs.

Change: How to Make Big Things Happen - Centola, Damon (Little, Brown Spark, 2021)
How to create the change you want to see in the world using the paradigm-busting ideas in this "utterly fascinating" (Adam Grant) big-idea book.​ Most of what we know about how ideas spread comes from bestselling authors who give us a compelling picture of a world, in which "influencers" are king, "sticky" ideas "go viral," and good behavior is "nudged" forward. The problem is that the world they describe is a world where information spreads, but beliefs and behaviors stay the same. When it comes to lasting change in what we think or the way we live, the dynamics are beliefs and behaviors are not transmitted from person to person in the simple way that a virus is. The real story of social change is more complex. When we are exposed to a new idea, our social networks guide our responses in striking and surprising ways. Drawing on deep-yet-accessible research and fascinating examples from the spread of coronavirus to the success of the Black Lives Matter movement, the failure of Google+, and the rise of political polarization, Change presents groundbreaking and paradigm-shifting new science for understanding what drives change, and how we can change the world around us.

Engage. Empower. Enact. - Citizen Engagement & Democratic Innovation Programme White Paper - Doyle, Linda; Smith, Bethan (Cognitive Edge, 2021)
The Cynefin Centre’s Citizen Engagement & Democratic Innovation programme provides tools for collective sense-making in the areas of community development and youth work; civic engagement and democratic innovation; collaborative service/policy design and evaluation; housing/tenant engagement; futures and planning; shared learning and peer to peer knowledge exchange.

Co-Creating Digital Public Services - Jarke, Juliane (Co-creating Digital Public Services for an Ageing Society: Evidence for User-centric Design, 2021)
This chapter reviews key literature and concepts relating to the co-creation of digital public services. For this task, it is firstly important to consider what kind of digital public services may be suitable for co-creation. In order to do so, the first section of this chapter defines what a digital public service is (e.g. with respect to different types of service providers, different types of services and service delivery) and considers what kind of digital public services allow for meaningful citizen participation. To better conceptualise different degrees of participation, the subsequent section reviews Arnstein’s (1969) “ladder of citizen participation” and related work. This allows distinguishing between different degrees of non-participation, (consultative) participation and beyond. Thirdly, the chapter reviews traditional participatory approaches that provide the basis to co-creating of digital public services: (1) co-production of public services, (2) co-design and (3) civic open data use. The chapter summarises and compares the different rationales for participation in these approaches, and reviews how they understand the sharing of control, the sharing of knowledge and the enabling of change.
Keywords: Civic Tech; Co-creation; Co-design; Co-production; Digital public services; Open data; Open government; Participation; Participatory design; User-centric services; e-services

Co-creating Digital Public Services for an Ageing Society: Evidence for User-centric Design - Jarke, Juliane (Public Administration and Information Technology, 2021)
This open access book attends to the co-creation of digital public services for ageing societies. Increasingly public services are provided in digital form; their uptake however remains well below expectations. In particular, amongst older adults the need for public services is high, while at the same time the uptake of digital services is lower than the population average. One of the reasons is that many digital public services (or e-services) do not respond well to the life worlds, use contexts and use practices of its target audiences. This book argues that when older adults are involved in the process of identifying, conceptualising, and designing digital public services, these services become more relevant and meaningful.The book describes and compares three co-creation projects that were conducted in two European cities, Bremen and Zaragoza, as part of a larger EU-funded innovation project. The first part of the book traces the origins of co-creation to three distinct domains, in which co-creation has become an equally important approach with different understandings of what it is and entails: (1) the co-production of public services, (2) the co-design of information systems and (3) the civic use of open data. The second part of the book analyses how decisions about a co-creation project’s governance structure, its scope of action, its choice of methods, its alignment with strategic policies and its embedding in existing public information infrastructures impact on the process and its results. The final part of the book identifies key challenges to co-creation and provides a more general assessment of what co-creation may achieve, where the most promising areas of application may be and where it probably does not match with the contingent requirements of digital public services. Contributing to current discourses on digital citizenship in ageing societies and user-centric design, this book is useful for researchers and practitioners interested in co-creation, public sector innovation, open government, ageing and digital technologies, citizen engagement and civic participation in socio-technical innovation.

Don’t Build It: A Guide For Practitioners In Civic Tech / Tech For Development - Jordan, Luke (Grassroot and MIT Governance Lab, 2021)
If you just remember these... If you can avoid building it, don’t build it; if you have to build it, hire a CTO, ship early, and mature long; and no matter what, draw on a trusted crew, build lean and fast, and get close to and build with your users as soon as possible. --- This guide aims to help you avoid bad projects, structure the team right, ship and learn quicker, and mature longer. The guide starts with project selection, including why the best project to select is no project at all. It moves on to team structure, and the extreme importance of a full-time senior tech lead (or chief technology officer (CTO), understood as an excellent engineering manager). It then covers timelines, emphasizing shipping early but having enormous patience getting to maturity, above all in finding product-use-fit, and avoiding vanity metrics. The guide then goes into some detail on hiring, covering the CTO role, senior contractors, designers and young engineers. The longest section, by some distance, is that on hiring. Hiring is the one thing considered critical in every piece of the lore, by founders and investors and managers alike, across all sectors. It is also the field in which I think I got it mostly right, and for reasons I can explain in ways that I believe will be helpful.

OutNav - Evaluate outcomes and impact - Matter of Focus (Matter of Focus, 2021)
Make better use of your data and information to learn, improve and tell an evidenced story of the difference your organisation is making.

Agile: A New Way of Governing - Mergel, Ines; Ganapati, Sukumar; Whitford, Andrew B. (Public Administration Review, 2021)
The evolving concept of “agile” has fundamentally changed core aspects of software design, project management, and business operations. The agile approach could also reshape government, public management, and governance in general. In this Viewpoint essay, the authors introduce the modern agile movement, reflect on how it can benefit public administrators, and describe several challenges that managers will face when they are expected to make their organizations more flexible and responsive.

Modelo HIP - Hexágono de la innovación pública - Oliván Cortés, Raúl (2021)

Online Learning and collaborative platform for systems thinking and systems change - SI (Systems Innovation, 2021)

Small Foundation Network Partner Evaluation Toolkit: Network Evaluation for Network Coordinators - Small Foundation (2021)
This Toolkit This Toolkit presents an approach to network evaluation that is designed for network coordinators. This Toolkit provides guidance for network coordinators on how to: • More effectively use information they are already collecting as part of their routine coordination duties; • Collect other useful data that would support their network’s health and development; • Integrate network evaluation tasks into network activities, like convenings, to streamline the process; • Use data to effectively coordinate, grow, and sustain their networks. The Toolkit also provides examples of surveys, questionnaires, and dashboards, as well as tips for easy implementation.

Small Foundation Network Partner Evaluation Toolkit Network Evaluation for Network Coordinators Frequently Asked Questions - Small Foundation (Small Foundation, 2021)
This Frequently Asked Questions (FAQ) document is written for Small Foundation network partners. It accompanies Network Partner Evaluation Toolkit, which outlines four evaluation tools that Small Foundation is encouraging, and supporting, its network partners to use to improve their operations and, ultimately, increase their impact. This FAQ document answers typical questions about these evaluation tools and provides additional detailed recommendations for how network coordinators can effectively implement them. This document has five sections: 1) A brief introduction to network theory and the value of evaluation for network coordinators. 2) An overview of the four evaluation tools and answers general questions about incorporating evaluation into network coordination duties. 3) Recommendations regarding the Network Coordinator Administrative Information (tool #1). 4) Information about Social Network Analysis (tool #2). 5) Recommendations regarding the Network Participant Survey (tool #3). 6) Information about the Collaborative Activity Dashboard (tool #4).

Managing complexity (and chaos) in times of crisis. A field guide for decision makers inspired by the Cynefin framework - Snowden, Dave; Rancati, Alessandro (Publications Office of the European Union, 2021)
This field guide helps to navigate crises using the Cynefin framework as a compass. It proposes a four-stage approach through which we can: - assess the type of crisis and initiate a response; - adapt to the new pace and start building sensing networks to inform decisions; - repurpose existing structures and working methods to generate radical innovation; - transcend the crisis, formalise lessons learnt and increase resilience. The guide stresses the importance of setting and managing boundaries, building informal structures, keeping options open, distributing engagement and keeping an ongoing assessment of the evolving landscape. Action items, real life examples and demonstrations complement the references to the developing theoretical framework.

Uniting Against Corruption - A Playbook on Anti-Corruption Collective Action - UN Global Compact (UN Global Compact, 2021)
The 2021 guide provides an easy-to-follow six-step approach on how to develop, implement, and sustain a CA, with respect to the reader’s local corruption landscape and potential stakeholders. The adaptive framework proposed can be used to address corruption challenges, mitigate possible business risks, and achieve optimal results.

Thinking Tools Studio - Waters Center (Waters Center for Systems Thinking, 2021)
Tools to help you Think

What have we learned about learning? Unpacking the relationship between knowledge and organisational change in development agencies - Yanguas, Pablo (DIE - Deutsches Institut für Entwicklungspolitik, 2021)
Development cooperation has spent decades wrangling over the merits, evidence, and implications of what we may term “the learning hypothesis”: the idea that increased knowledge by development organisations must logically lead to increased effectiveness in the performance of their development activities. Organisations of all stripes have built research and monitoring and evaluation (M&E) departments, adopted a multitude of knowledge management systems and tools, and tinkered with different ways to structure their organograms to stimulate knowledge sharing and learning. The topic of organisational learning is particularly significant as the global development community grapples with increasingly complex problems and the aspiration of evidence-based policymaking. This paper presents an analytical framework for interrogating “the learning hypothesis”, breaking it down into causal steps: knowledge causes learning, learning causes organisational change, change causes effectiveness. The framework focuses on the first two sub-hypotheses, mapping out the conceptual space around them by outlining potential relationships between different types of knowledge – tacit and explicit, internal and external – and between different types of learning – operational and strategic. This map provides a foundation for three key research questions: What impact has the rising knowledge agenda had on development organisations? Which factors appear to enable or inhibit organisational learning? What is the relationship between operational and strategic learning and organisational change? A review of available evaluations and studies, including two cases from former UK Department for International Development (DFID) and the World Bank, reveals that there is insufficient evidence to support the causal claim that knowledge leads to learning and thereby to organisational change in development agencies. Sources point to tacit learning prevailing while explicit knowledge management systems flounder, and external advocacy agendas appear more compelling than internal research and evaluation products. It is not entirely clear how, or indeed, whether operational and strategic learning intersect, with delivery-level lessons hardly aggregating into structural or policy shifts. Organisational change – even that aimed at enhancing learning – is rarely based on lessons learned from practice. More research is necessary to fully unpack the learning hypothesis, but what limited evidence is available disproves rather than confirms its central claim. This has significant implications for the future of learning in development agencies as advocated by thought leaders, researchers, and reformers. In particular, the latter should consider an evidence-based reassessment of the function and value-for-money of research and M&E in development practice, and a more critical examination of the politics of external advocacy efforts around innovative aid approaches like thinking and working politically, adaptive management, or results-based management.

Adaptive Management: Learning and Action Approaches to Implementing Norms-shifting Interventions - McLarnon, Courtney; Gayles, Jennifer; Deepan, Prabu (USAID / Passages, 2021)
What Passages has Learned about Adaptive Management: • Be reflective about information that is collected and create a culture of learning. • Be systematic about establishing monitoring and learning systems. • Be strategic about data sources and analysis, prioritizing areas for learning and addressing issues raised. • Be inclusive about information collection: who is collecting what, how, and how is it being used.

USAID ADS 201 - Operational Policy for the Program Cycle (Update 01/23/2021) - USAID (USAID, 2021)
The Program Cycle is USAID’s operational model for planning, delivering, assessing, and adapting development programming in a given region or country to advance U.S. foreign policy. It encompasses guidance and procedures for: 1) Making strategic decisions at the regional or country level about programmatic areas of focus and associated resources; 2) Designing projects and supportive activities to implement strategic plans; and 3) Learning from performance monitoring, evaluations, and other relevant sources of information to make course corrections as needed and inform future programming.

5 reflections on operationalising CLARISSA to generate evidence - Neupane, Sudarshan (CLARISSA, 2021)
As we start a new year, and especially, as 2021 is the International Year for the Elimination of Child Labour, understanding that the Worst Forms of Child Labour (WFCL) is complex is step one – there are no simple solutions. But we must go deeper than that, we must ensure that the programmes designed to intervene, and shape policy take on that complexity and precariousness. This is where the experience of the Child Labour: Action-Research-Innovation in South and South-Eastern Asia (CLARISSA) programme comes in. It is uniquely set up to generate evidence in innovative ways and to co-develop with our stakeholders’ solutions to the WFCL. This is ambitious, yes, but no one said it was going to be easy.

Assumptions and triple loop learning - Aston, Thomas (Medium, 2021)
Triple loop learning

How to make theories of change more useful? - Aston, Thomas (Medium, 2021)
A fair amount has been written recently questioning the value added of theories of change. Have we gone through a hype cycle? Are they…

Linking the prepositions: using power analysis to inform strategies for social action - Gaventa, John (Journal of Political Power, 2021)
This article reviews longstanding debates about the relationship between power over and power to – often posed as the tension between domination and emancipation. It then turns to several frameworks which integrate these approaches to inform strategies for social action. In particular, it focuses on recent empirical studies which apply one such framework, the ‘powercube’, to glean insights into how social actors navigate across multiple forms, spaces and levels of power. In so doing, we gain clues into how relatively powerless groups develop the capacities for agency and action which challenge domination and in turn give new possibilities for emancipation.
Keywords: Lukes; Power; agency; collective action; empowerment

A Political Economy Analysis Framework for EdTech Evidence Uptake - Pellini, Arnaldo; Nicolai, Susan; Magee, Arran; Sharp, Sam; Wilson, Sam (The EdTech Hub, 2021)

The Future of Development: “Make Happen” with Portfolios of Options - CHÔRA Foundation (CHÔRA Foundation, 2021)
This Green Paper intends to review key elements of the problem that Development actors will confront as a new decade opens up ahead of us. It will articulate a solution that we believe should become an inherent feature of Development programs and initiatives. This is the outcome of an intense period of experiences and reflections in the Development space across different geographies and institutional mandates and activities, during which the Foundation has collaborated with institutions such as the UNDP and Grand Challenges Canada. At the heart of our collaborations is a Strategic Innovation and System Transformation Framework, with its associated concepts, working definitions, processes, tools and people. Sourced from CHÔRA Foundation’s knowledge and practice assets, this is a capability we intend to make relevant, customise, scale up and distribute to our partners and stakeholders. We are looking to create with them a space that offers the world a transformational capability. Central to this capability we see a distinctive and robust practice: the design and dynamic management of Portfolios of Strategic Innovation and System Transformation Options. These Portfolios are unique, context relevant, embedded mechanisms for learning, sensemaking and problem solving that social systems leverage to have an impact on themselves and their problems, and to induce the transformations that are necessary to them. It is our view that Portfolios of Options are the most effective means by which human social systems can supply themselves with budgets of possibility that ensure choice and create opportunity. They will also support pragmatic evolutionary outcomes and enable resilience.

FCDO Beneficiary Engagement - FCDO (FCDO, 2021)
This guide has been developed to help build confidence and capability, distilling useful tips and considerations that may help teams think through programme delivery issues and interpret elements of the PrOF Rules. This PrOF Guide lays out: - The definition of beneficiary engagement. - The case for beneficiary engagement. - FCDO’s approach to beneficiary engagement. - Practical tips for how to integrate beneficiary engagement throughout the programme cycle, including guiding questions to ask, rules of thumb to apply, tools to use and challenges and special topics to consider during Design, Mobilisation, Delivery and Closure phases. At its core, beneficiary engagement is about processes that recognise the dignity and support the agency of the people whose lives we are trying to improve. It is about beneficiaries and programme constituents having a say over what assistance they receive and how they receive it. It is about engaging beneficiaries and programme constituents as people with valuable insights and capabilities, rather than a compliance exercise. It is about empowering all beneficiaries and programme constituents to improve their lives by engaging them in helping us make better design and delivery decisions for the programmes that affect their lives. It’s about ensuring that a diverse set of voices are heard. Harnessing the power of beneficiary engagement can also improve outcomes and help programmes reach them more efficiently. It helps define and promote Value for Money, improve transparency and ensure that beneficiaries are safe from harm and empowered to speak out wherever harm does occur. Beneficiary engagement is supported by FCDO policy commitments, PrOF Rules, internal guidance and key international commitments. Beneficiary engagement is applicable to, and valuable in, a range of contexts, including humanitarian contexts. Beneficiary engagement requires time and resources, but programmes can help ensure the benefits of engagement outweigh the costs by Doing No Harm, Engaging Early and Closing the Loop. It is ultimately the Programme Responsible Owner’s responsibility to determine what beneficiary engagement is suitable and feasible for a programme. Quality beneficiary engagement is not about applying the one “right” approach but rather thoughtfully considering key questions and applying key principles and proven tools to the programme’s context to achieve a programme that supports the dignity and agency of all beneficiaries as much as possible.

LearnAdapt: a synthesis of our work on adaptive programming with DFID/FCDO (2017–2020) - Laws, Ed; Pett, Jamie; Proud, Emma; Rocha Menocal, Alina (Briefing Note, 2021)
Key takeaways. • Development is not linear or straightforward, but rather complex, uncertain and context-specific. This calls for international development actors to work differently, in ways that are based on deliberate experimentation, learning and adaptation, to inform decisions and drive effective development. • Although it might go by different names, adaptive programming has been used in a variety of areas and fields in both the public and private sectors. Development practitioners have much to learn from and contribute to these different approaches and experiences. • Trust and relationship-building across all relevant stakeholders are among the most critical enabling factors for adaptive management. They are essential to give partners the space, autonomy and authority needed to try, test, reflect, iterate and feed back at the frontline of implementation, and to give donors the confidence that decisions are being made on the basis of evidence and learning to improve effectiveness. • There is an urgent need to rethink how accountability requirements, results frameworks, value for money considerations, performance markers, procurement and contracting mechanisms and other processes are understood and applied so that they are better aligned with and can support adaptive management more effectively. • The role of senior managers leading adaptive programmes from the donor side should be to create a space for experimentation and learning. Funders should hold their partners accountable for learning and how it feeds into effective programming, rather than for delivering on predetermined targets. • While formal guidance is important, leadership, champions, institutional incentives, a supportive management culture and appropriate mindsets are essential to encourage adaptive ways of working. • Adaptive management is resource-intensive. It requires skill, commitment, time for building trust and investments in learning. It is a journey, not an immediate destination – so it calls for patience, open-mindedness and a more nuanced approach to risk.

Value for money and adaptive programming - Approaches, measures and management - Laws, Ed; Valters, Craig (ODI, 2021)
- The United Kingdom’s Foreign, Commonwealth and Development Office (FCDO)’s standard economy, efficiency, effectiveness/cost-effectiveness and equity (4E) framework is still relevant for approaching, measuring and managing value for money (VfM) for adaptive programmes. • However, this framework needs to be reframed to capture and incentivise flexibility, learning and adaptation. • VfM appraisal and reporting should be done in a way that draws on beneficiary feedback and informs good decision-making, rather than just being a compliance exercise. • If VfM appraisal and reporting cannot be done appropriately for adaptive programmes, it should be avoided or minimised. There is a risk of diverting time and resources from more suitable tools and methods.

Applying Evaluation Criteria Thoughtfully - OECD (OECD, 2021)
Relevance, coherence, effectiveness, efficiency, impact, and sustainability are widely used evaluation criteria, particularly in international development co-operation. They help to determine the merit or worth of various interventions, such as strategies, policies, programmes or projects. This guidance aims to help evaluators and others to better understand those criteria, and improve their use. It starts by describing what they are, and how they are meant to be used. Then the definitions and concepts underpinning each criterion are explained. Finally, examples provide the reader with concrete ideas for using them. The criteria were originally laid out in the early 2000s by the Network on Development Evaluation (EvalNet) of the OECD Development Assistance Committee (DAC). Although they have been widely used in evaluation, and beyond, this document is the first to comprehensively explore the concepts in detail, explain their intended use and offer practical guidance. It captures current thinking and best practice in evaluation, drawing on the inputs of internationally renowned evaluation experts from EvalNet and beyond.

20 years of Outcome Mapping - Evolving practices for transformative change - OM Learning Community (Outcome Mapping Learning Community, 2021)
To mark 20 years of Outcome Mapping, this paper presents the core principles and concepts that are foundational to using the approach. It also presents a set of guiding practices to support transformative change. The OMLC Stewards presented the paper at a special webinar on 29th April 2021 - see the link below for the recording.

Adaptive Learning Guide: A pathway to stronger collaboration, learning, and adapting - Ross, Joey; Karlage, James; Etheridge, James; Alade, Mayowa; Fifield, Jocelyn; Goodwin, Christian; Semrau, Katherine; Hirschhorn, Lisa (USAID MOMENTUM Knowledge Accelerator, 2021)
The purpose of this Adaptive Learning Guide is to provide MOMENTUM project teams with the information and resources to integrate adaptive learning into the design, implementation, and improvement of MNCH/FP/RH programs. This guide provides a conceptual introduction to adaptive learning using links to existing resources and real-world examples of how adaptive learning can drive continuous learning and improvement in project work. The guide is built upon three foundational assumptions: We work in dynamic, often unpredictable environments. Unexpected turns of events will occur. Progress is rarely, if ever, linear. Integrating the principles and practices of USAID’s Collaborating, Learning, and Adapting Toolkit into projects and initiatives requires designing for learning and adaptation. We intend the guide to serve as a “starting point” for interested individuals and teams to begin or strengthen the processes that support the integration of adaptive learning into project work.

Opportunities and challenges for DAC members in ‘adapting to context’ - Sharp, Samuel; Wild, Leni (2021)
Key Messages New principles for OECD DAC members on ‘Managing for Sustainable Development Results’ emphasise tailoring result management approaches to different contexts; balancing internal compliance with empowerment at ground level; and adapting implementation in the pursuit of long-term outcomes. However development organisations face numerous challenges in aligning with these principles in practice. Reporting and evidence collection processes do not consistently encourage adaptive practice, reflecting their orientation towards accountability over learning. Context analysis is common during programme design, but used less on an ongoing basis. Popular tools –such as logical frameworks and theories of change - are often intepreted in linear ways, not as ‘living documents’ that react and change over time. Organisations need to meaningfully empower staff to work adaptively, including examining incentives and cultures that can make staff more comfortable with traditional results management. Even when senior leadership is supportive of adaptive ways of working, they can lack a clear understanding of the resourcing required and appropriate governance and management processes. Development organisations and their partners have attempted to address these challenges through the use of different monitoring and evaluation tools and methods, changes to reporting frameworks and templates, and initiatives to create positive incentives and motivate staff, leadership and partners at different levels.
Keywords: _tablet

Peer reviews - Guidance for facilitators and participants - Wadley, Ian (Mediation Practice Series, 2021)
Essential points for practitioners and donors • Mediation offers a cost-effective and proven method for resolving armed conflict. Between 1985 and 2015, 75 per cent of armed conflicts in the world were resolved through agreement rather than by force. In most cases these processes will have involved third party facilitation or support. • Professional mediators understand the high stakes involved in their work to prevent, mitigate and resolve armed conflict. In addition, they and their financial supporters are increasingly required to demonstrate ‘value-for-money’ to ensure continued funding. • However, traditional monitoring and evaluation (M&E) methods are not well suited to this task, typically imposing artificially linear project models on a dynamic conflict situation, as well as compliance reporting that moves attention away from real value. • Traditional M&E methods tend to focus on documenting the past and generating vast amounts of data, rather than enabling timely adaptation of the project in the present. • Traditional M&E approaches rely heavily on external evaluation consultants. Even in the best of cases this may interfere with the mediation process and impose a heavy time burden on the project team, leading to low acceptance of traditional M&E approaches by mediation practitioners. • In contrast, an ideal M&E approach for mediation should deliver useful insights in even the most dynamic and sensitive mediation environments, impose a light reporting burden, and be readily accepted by mediation teams. It should protect discretion and trust, enable rapid adaptation, and also provide some assurance that donor funds are being well spent.

Doing Development Differently - Wild, Leni (Stanford Social Innovation Review, 2021)
International aid must use different approaches to address the massive systemic problems it seeks to solve.
Keywords: ⛔ No DOI found

Critical System Heuristics : a very, very short introduction - van 't Hof, Sjon (CSL4D/SystemicAgency, 2021)

Theory-Based Approaches to Evaluation: Concepts and Practices - Treasury Board of Canada Secretariat (Government of Canada, 2021)
This document introduces some of the key concepts of theory-based approaches to evaluation. It is hoped that readers will be encouraged by the information and advice provided in this document and will explore the use (e.g., through pilot evaluations) of theory-based approaches to evaluation in a federal setting. To support this, Sections 1.0 to 8.0 of the document describe the general application of theory-based approaches to evaluation, and Sections 9.0 and 10.0 discuss the potential application of theory-based approaches to a range of federal programs. This document is neither an exhaustive training program in theory-based evaluation nor a step-by-step guide to undertaking a theory-based evaluation. Evaluators who wish to integrate theory-based approaches into their practices are encouraged to pursue additional readings (including those referenced in this document) and, as appropriate, to seek additional support in undertaking a theory-based evaluation.

'Development': A visual story of shifting power - Faciolince, Maria; Obando, Hansel (From Poverty to Power, 2021)
The work of shifting power is fundamentally the work of changing our gaze. People act on how they see, and to change how we see, is to radically change how we act. By not exploring other forms of expressing, looking and creating, we’re limiting ourselves.  The development space is fixated on the written word. ... This exhibit, called “‘Development’: a visual story of shifting power”, tells the story of ‘development’, from its origin to its current challenge, from its contradictions to its possible horizons. Our guiding principles were the twin notions of decolonization and intersectionality: moving away from the unequal power structures that reinforce legacies of colonialism, and advancing explicitly anti-racist and feminist agendas.

A Guide to Assessing the Political Economy of Domestic Climate Change Governance - Worker, Jesse; Palmer, Niki (World Resources Institute, 2021)
This paper discusses how understanding the domestic political economy of climate governance is critical for developing informed strategies to build and sustain political ambition. It provides guidance and a methodology for domestic stakeholders to determine the types of institutional reforms, incentives, coalitions, and policy design that can entrench long-term political support for climate ambition. The assessment can also help users identify political barriers to more equitable climate action and identify reforms that may strengthen inclusion and accountability. EXECUTIVE SUMMARY Highlights ▪ There is overwhelming evidence of the social, economic, and environmental case to reduce greenhouse gas (GHG) emissions and rapidly scale up adaptation. Yet, despite a proliferation of climate laws and policies over the last 10–15 years, emissions are still rising, and adaptation needs remain urgent. ▪ This calls for a more sophisticated assessment of the political economy factors that may enable or constrain implementation of policies and actions and sustain political commitment at the country level. ▪ This guide offers an assessment methodology to understand how structural factors, rules and norms, stakeholders and interests, and ideas and narratives influence the political economy of climate action in a given country context. ▪ The methodology was developed on the basis of climate policy, governance, and political economy literature with contributions from subject matter experts. ▪ We intend the assessment to support civil society coalitions, reform-minded civil servants and politicians, international organizations, and other stakeholders. ▪ The resulting analysis should deepen the understanding of context while informing the advocacy, policy design, coalition building, capacity building, and communications of domestic stakeholders.

Lessons learned from PERL and partners' response to the COVID-19 crisis - Sharp, Samuel; Nwachukwu, Tochukwu; Srivatsa, Sripriya Iyengar (PERL Programme, 2021)
The first case of COVID-19 in Nigeria was confirmed on 27 February 2020, with the first lockdown orders issued on 30 March 2020. The pandemic and resultant containment measures have had farreaching socio-cultural, economic, financial and political implications, globally as well as in Nigeria. For the Partnership to Engage, Reform and Learn (PERL) and its partners, the pandemic has required considerable adaptation of their strategic approach and working practices. This report reflects on how COVID-19 changed the operating context for PERL’s partners, how PERL responded and what lessons have been learned across delivery teams. For government partners, the most substantial impacts have been to budgets, struck by falling oil prices and reduced economic activity. Universally, states have had to adjust budgets and reforecast, revising budgets downwards and shifting the focus of expenditure towards healthcare. The World Bank’s State Fiscal Transparency, Accountability and Sustainability (SFTAS) Programme has generated powerful incentives for this budget revision, which PERL has been able to work alongside. A range of new governance structures – such as public response committees and task forces – have been established to deal with various aspects of COVID-19 policy, and PERL has had to grapple to maintain its ongoing engagement with these. For civil society organisations (CSOs), the closure of offices from 30 March 2020 has changed the nature of engagement with government. CSOs often developed innovative approaches to maintaining access, including use of social media and direct calls. But the shift to virtual working has been challenging for many CSOs, both in terms of covering the costs of data for virtual meetings and the risks of disengagement and marginalisation for some organisations. In response to this changed context, from March 2020 PERL began to restrategise. The flexible nature of the programme’s workplans, progress markers and budgets enabled activities to be adjusted in a relatively timely manner, with a new workplan approved by the end of April 2020. Central PERL management developed a COVID-19 response strategy which provided a broad framework for adaptations, but allowed substantial autonomy to state and regional teams to lead on reprioritisation according to their understanding of the local context. This was valued by both management and delivery staff. Challenges manifested themselves more in effectively delivering on these adapted workplans than in the process of restrategising – due to two rounds of budget cuts, the merger of DFID and the FCO to form the Foreign, Commonwealth and Development Office (FCDO) and the difficulties of engaging partners virtually. Overall, the pandemic undoubtedly delayed activities (by roughly three months for deprioritised areas of work), but resulted in an array of new, tailored interventions under its broad categories of work. Interventions relating to the health sector became more prevalent, as did work supporting budget adjustments. Domestic resource mobilisation and education interventions were often adjusted to be more relevant to the COVID context or experienced delay. The report provides short illustrative case studies of PERL’s adaptations to: support budget revisions; work with media partners on COVID-19 sensitisation; tracking and advocacy for palliative distribution; and support for the introduction of tax relief. There is some evidence, albeit partial, that PERL was able to take advantage of windows of opportunity offered by the pandemic to drive ahead with certain ongoing reform initiatives.

Implementing adaptive management: A front-line effort — Is there an emerging practice? - Teskey, Graham; Tyrrel, Lavinia (Working Paper, 2021)
Among the many principles that currently inform donor-funded development initiatives, three appear to stand out: they should be politically informed, locally led, and adaptive. There is as yet little practical guidance for aid implementers regarding how to operationalise these approaches. What will it take to shift practice away from linear and planned approaches, towards models which foster local leadership and which engage with emergent and complex systems? This paper suggests that the answer is not to throw out the discipline of the logical framework, results frameworks, or theories of change. Rather they need to be handled rather more reflectively and ‘elastically’. The purpose of this paper is to set out how this can be achieved, and to propose 15 tools for donors, implementors and front-line staff to apply adaptive management (AM) in practice, at critical stages of the project cycle and within the dominant aid paradigm. This is what we are calling PILLAR: politically informed, locally led and adaptive responses. We are framing PILLAR to cover the full project cycle (design, implementation and review), hence the nomenclature of an ‘end to end’ approach. Our hope is that these tools will eventually replace the current planned, log-frame driven and top-down approach to aid design and delivery which dominates the development sector.

Bringing children into the life story collection process - Shrestha, Kapil (CLARISSA, 2021)
The CLARISSA Social Protection (SP) intervention provided six months of unconditional cash transfers to every household in the Gojmohol neighbourhood,...

How to make a multilayer pie chart in Excel - Karina Adcock (2021)
Create a multilevel donut chart in excel -------------------------------------------------------------------------------- #exceltips #exceltutorials #excelcharts SUBSCRIBE: https://goo.gl/c46YPs Microsoft Office 365, Beta Channel, Version 2104 --------------------------------------------------------------------------------- IF YOU LIKED THIS VIDEO YOU MAY ALSO LIKE: - Recreating a pie chart from a newspaper in PowerPoint    • How to make a pie...

Building a Movement of Public Problem Solvers - Building State Capability - Samji, Salimah (Building State Capability, 2021)
Solving public problems is a hard and thankless job. One that is undertaken with a shortage of time as well as resources, and often under pressure to deliver results. A common approach used to solve public problems is to develop a plan, sometimes with experts, and then to assume that implementation will happen on autopilot. To quote Mike Tyson, “Everyone has a plan ’till they get punched in the mouth.” The question is, what do you do after you get punched? Continue with your existing plan? Or do you learn from the punch?

A top Toolkit on Adaptive Management. But is that a good idea? – FP2P - Green, Duncan (From Poverty to Power, 2021)

How to do Adaptive Management in 15 easy steps – from a top new toolkit – FP2P - Green, Duncan (From Poverty to Power, 2021)

20 years of Outcome Mapping - OM Learning Community (2021)
Celebrate 20 years of Outcome Mapping with the OMLC Stewards plus special guests in this 90 minute webinar. - Exclusive celebration video with Outcome Blues soundtrack - Fireside chat with the authors of the OM manual, Sarah Earl, Fred Carden & Terry Smutylo, to hear their views on OM’s evolution since the 2001 publication - Launch of our new paper presenting core concepts of OM and guiding practices for transformative change, with special guests Sonal Zaveri & Julius Nyangaga as discussants

PDIA in action - Andrews, Matt; Samji, Salimah (Center for International Development, Harvard University, 2021)
Learning from our experience in 2020, we asked the alumni of our HKS Implementing Public Policy (IPP) Executive Education program, if they wanted to work with our students on their nominated problems. Eight IPP alumni, William Keith Young, Adaeze Oreh, Milzy Carrasco, Kevin Schilling, Artem Shaipov, George Imbenzi, David Wuyep, and Raphael Kenigsberg, who had been trained on PDIA and implementation, signed up to work with our students. Thirty-seven students signed up to take the course beginning January 26th, 2021. The students worked across eight teams and adopted a problem driven approach to foster learning that could help their authorizers develop an action learning strategy to their nominated challenge. This book highlights the students’ work drawing from their blogs as well as the event series. There are 8 sections, one for each of the teams and the problems they worked on during the course. We hope you enjoy reading their stories! Scan the QR Code at the end of each section to learn more.

Understanding Strategic Capacity in Constituency-Based Organizations - Jane Booth-Tobin; Kal Munis; Lynsy Smithson-Stanley; Hahrie Han (The P3 Lab, Johns Hopkins University, 2021)
Movement organizations work in inherently uncertain political environments. Whether an organization is advocating for a new minimum wage, working to close a private prison, or seeking to influence an election, the terrain they are operating on shifts nearly every day. That is increasingly true as political uncertainty rises in the 21st century, particularly for historically race-class subjugated communities. Any movement-based organization seeking to build, exercise, and win political power must have sophisticated strategic capacities to be able to navigate these uncertain, dynamic, and constantly shifting political environments. Yet, our knowledge of how movements can nurture the kind of strategic capacities that allows them to build constituencies and leadership that can operate in the flexible ways needed for these dynamic circumstances is limited. This report seeks to synthesize what is currently known about organizations that successfully build and wield strategic capacity, with a particular eye toward how it might apply to constituency-based organizations. The report concludes with an assessment and facilitated conversation guide to support movements and movement organizations in understanding how developed (or not) their strategic capacities are.

Let’s Take the Con Out of Randomized Control Trials in Development - Pritchett, Lant (CID Faculty Working Paper, 2021)
The enthusiasm for the potential of RCTs in development rests in part on the assumption that the use of the rigorous evidence that emerges from an RCT (or from a small set of studies identified as rigorous in a “systematic” review) leads to the adoption of more effective policies, programs or projects. However, the supposed benefits of using rigorous evidence for “evidence based” policy making depend critically on the extent to which there is external validity. If estimates of causal impact or treatment effects that have internal validity (are unbiased) in one context (where the relevant “context” could be country, region, implementing organization, complementary policies, initial conditions, etc.) cannot be applied to another context then applying evidence that is rigorous in one context may actually reduce predictive accuracy in other contexts relative to simple evidence from that context—even if that evidence is biased (Pritchett and Sandefur 2015). Using empirical estimates from a large number of developing countries of the difference in student learning in public and private schools (just as one potential policy application) I show that commonly made assumptions about external validity are, in the face of the actual observed heterogeneity across contexts, both logically incoherent and empirically unhelpful. Logically incoherent, in that it is impossible to reconcile general claims about external validity of rigorous estimates of causal impact and the heterogeneity of the raw facts about differentials. Empirically unhelpful in that using a single (or small set) of rigorous estimates to apply to all other actually leads to a larger root mean square error of prediction of the “true” causal impact across contexts than just using the estimates from non-experimental data from each country. In the data about private and public schools, under plausible assumptions, an exclusive reliance on the rigorous evidence has RMSE three times worse than using the biased OLS result from each context. In making policy decisions one needs to rely on an understanding of the relevant phenomena that encompasses all of the available evidence.

Impact Evidence and Beyond: Using Evidence to Drive Adoption of Humanitarian Innovations - Dodgson, Kate; Crowley, Catie (Scaling series, 2021)
This learning paper provides guidance to humanitarian innovators on how to use evidence to enable and drive adoption of innovation. Innovation literature and practice show time and time again that it is difficult to scale innovations. Even when an innovation is demonstrably impactful, better than the existing solution and good value for money, it does not automatically get adopted or used in mainstream humanitarian programming. Why do evidence-based innovations face difficulties in scaling and how can innovators best position their innovation to scale? This learning paper is for innovators who want to effectively use evidence to support and enable their journey to scale. It explores the underlying social, organisational and behavioural factors that stifle uptake of innovations. It also provides guidance on how to use, prioritise and communicate evidence to overcome these barriers. The paper aims to help innovators generate and present their evidence in more tailored and nuanced ways to improve adoption and scaling of their innovations.

We have experimented with different approaches to systems transformation — here are five insights - Haldrup, Søren Vester (UNDP Innovation, 2021)
At UNDP Innovation we are on a journey to shift our approach to innovation to help tackle complex development challenges. In short, we are moving away from single point solutions, and instead we are…

Applying Systems Thinking to Education: The RISE Systems Framework - Spivack, Marla (Research on Improving Systems of Education (RISE), 2021)
Many education systems in low- and middle-income countries are experiencing a learning crisis. Many efforts to address this crisis do not account for the system features of education, meaning that they fail to consider the ways that interactions and feedback loops produce outcomes. Thinking through the feedback relationships that produce the education system can be challenging. The RISE Education Systems Framework, which is sufficiently structured to give boundaries to the analysis but sufficiently flexible to be adapted to multiple scenarios, can be helpful. The RISE Framework identifies four key relationships in an education system: politics, compact, management, and voice and choice; and five features that can be used to describe these relationships: delegation, finance, information, support, and motivation. This Framework can be a useful approach for characterising the key actors and interactions in the education system, thinking through how these interactions produce systems outcomes, and identifying ways to intervene that can shift the system towards better outcomes.

Non-Violent Communication in everyday life - Aktar, Afrin; Goon, Swmaranika; Pal, Ujjal; Zaman, Thahina (CLARISSA, 2021)
In March, members of the team in Bangladesh (Community Mobilisers and Lead Community Mobilisers) participated in a training course in non-violent communication (NVC). NVC is a communication technique that prioritises listening over speaking. It aims to improve communication by achieving a deeper understanding of our emotions and values and what we observe in the behaviour of others. The training was facilitated by Paul Kahawatte, a UK-based mediator, facilitator and trainer, Roufun Naher, a lecturer at the University of Dhaka and NVC practitioner in Bangladesh, Neil Howard, co-lead of Social Protection for CLARISSA and Jiniya Afroze, CLARISSA Country Coordinator in Bangladesh. The aim of the training was to support the community mobilisers to explore and practice different ways of communicating and facilitating. This will help them support communities to come together and build their collective power for the social protection ‘cash plus’ pilot. Lemon Roy, a community mobiliser was grateful for the experience; “I had little knowledge about Non-Violent Communication before participating in the training. I didn’t know that passive listening and the communication gap between two people could lead to conflict. I have learned about tools and techniques in conflict mediation and the importance of understanding the feelings and needs of other people.”.

Energy Governance in Developing Countries — A New Approach - Mcculloch, Neil (The Policy Practice & Chemonics, 2021)
In 2015, leaders from around the world agreed to 17 Sustainable Development Goals (SDGs) to be achieved by 2030. The seventh goal (SDG7) is: “Ensure access to affordable, reliable, sustainable and modern energy for all.” In the same year, the world’s leaders concluded the Paris Agreement to tackle climate change, which will require a global transition in the energy sector away from the use of fossil fuels. Yet, despite growing investments in clean energy in many developing countries, the transition is happening much more slowly than needed. The central reason for this is poor energy governance. This technical brief shows how poor energy governance damages energy access and efforts to improve the quality and reliability of power. It explains the political reasons why energy governance is so bad in many countries and contrasts this with the current system of procuring technical assistance, which largely ignores the energy governance challenge. It shows that a new approach to tackling energy governance is emerging that is better matched to the nature of the problems faced and provides recommendations on how to implement it.

Why Tackling Energy Governance in Developing Countries Needs a Different Approach - McCulloch, Neil (The Policy Practice & Chemonics, 2021)
Global efforts to improve energy access and quality and to tackle climate change need a different approach to addressing poor energy governance. In 2015, leaders from around the world agreed to 17 Sustainable Development Goals (SDGs) to be achieved by 2030.1 The seventh goal (SDG7) is “Ensure access to affordable, reliable, sustainable and modern energy for all.” In the same year, the world’s leaders concluded the Paris Agreement to tackle climate change, which will require a global transition of the energy sector away from the use of fossil fuels. Yet, in many developing countries, despite growing investments in clean energy, the transition is happening much more slowly than needed to achieve SDG7 and avert damaging climate change. The central reason for this is poor energy governance. This paper outlines the size and nature of the energy challenge, with a focus on electricity. It describes the investments that are currently being made to improve the quality of power and access to electricity — and the growing evidence that investments often fail due to poor energy governance. The paper then delves more deeply into how bad governance influences the quality of and access to electricity, with specific country examples. It shows the importance of understanding how electricity fits into the political settlement of a country and how this affects the incentives of key actors in the sector. Unfortunately, donor projects designed to widen electricity access or to support reform of the power sector in developing countries often pay too little attention to the problem’s political nature; the same is true of measures to improve energy efficiency or to promote renewables. The paper outlines a new way of thinking about energy governance and shows how interventions can be better matched to the different governance challenges that they face. It concludes with recommendations for donors on how energy programs can be better designed and procured — as well as recommendations for implementors on how to improve the chances of successful implementation by adapting to the political realities of the contexts in which they operate.

Learning to adapt & adapting to learn - Using elements of outcome mapping in the ‘Resilient Adolescents in the Syria Crisis’ programme - van Ongevalle, Jan; Kvintradze, Ana; Rennesson, Gaël; Miller, David (Save the Children, 2021)
This learning paper highlights how elements of outcome mapping were used by Save the Children Sweden in a project (2018-2020) that supports adolescents, affected by the Syria crisis, to become more resilient. The paper first outlines how the spheres of influence framework has been applied to develop an actor focused theory of change. It then describes how progress markers, as an alternative to SMART indicators, were formulated to monitor the programme’s results. The paper also outlines how long lists of progress markers were categorised in a more realistic and practical results framework. The paper then continues to elaborate how outcome journals, qualitative data analysis techniques and regular review meetings and reflection workshops were utilised for data collection, for collective learning among programme stakeholders and for informing planning and programme adjustment. Various practical guidelines and tips on how to implement elements of outcome mapping are provided. The final part of the paper explores to what extent outcome mapping was able to foster several key enablers of adaptive programme management and highlights some of the challenges that programme stakeholders faced. Practical recommendations towards the use of outcome mapping in future programmes are also proposed.

Complexity-Aware Monitoring and Evaluation - Hertz, Tilman; Brattander, Eva; Rose, Loretta (Journal of MultiDisciplinary Evaluation, 2021)
Background: Addressing today’s sustainability challenges requires adopting a systemic approach where social and ecological systems are treated as integrated social-ecological systems. Such systems are complex, and the international development sector increasingly recognises the need to account for the complexity of the systems that they seek to transform. Purpose: This paper sketches out the elements of a complexity-aware monitoring and evaluation (M&E) system for international development programmes in the area of sustainable development.  Setting: Not applicable. Data Collection and Analysis: The authors draw on existing literature on complexity and evaluation and on their own experience from working in the field of M&E. Findings: An M&E system should not be seen simply as a tool to track compliance against a pre-determined theory of change. Instead, it is most useful as a real-time approach, constantly defining and re-defining narratives for change that help push systems along trajectories of interest. Dealing with complexity involves embracing uncertainty; and this challenges established notions of accountability—something which funders and implementers must begin to redefine together. Keywords: monitoring; evaluation; complexity; social-ecological systems; international development programmes; narratives for change; theory of change

Impact networks: measuring towards success - Small Foundation (Small Foundation, 2021)
How does Small Foundation think about the impact…

FCDO Programme Operating Framework - FCDO (FCDO, 2021)
The Programme Operating Framework (PrOF) sets the standard for how the Foreign, Commonwealth & Development Office (FCDO) delivers its programmes and projects.

Designing a Participatory Programme at Scale: Phases 1 and 2 of the CLARISSA Programme on Worst Forms of Child Labour - Burns, Danny; Apgar, Marina; Raw, Anna (CLARISSA Working Paper, 2021)
CLARISSA (Child Labour: Action-Research-Innovation in South and South-Eastern Asia) is a large-scale Participatory Action Research programme which aims to identify, evidence, and promote effective multi-stakeholder action to tackle the drivers of the worst forms of child labour in selected supply chains in Bangladesh, Nepal, and Myanmar. CLARISSA places a particular focus on participants’ own ‘agency’. In other words, participants’ ability to understand the situation they face, and to develop and take actions in response to them. Most of CLARISSA’s participants are children. This document shares the design and overarching methodology of the CLARISSA programme, which was co-developed with all consortium partners during and since the co-generation phase of the programme (September 2018–June 2020). The immediate audience is the CLARISSA programme implementation teams, plus the Foreign, Commonwealth & Development Office (FCDO). This design document is also a useful reference point for other programmes trying to build large-scale participatory processes. It provides a clear overview of the CLARISSA programmatic approach, the design, and how it is being operationalised in context.

We created this MEL system for you, now please own it! | From Poverty to Power - Pasanen, Tiina; Yanguas, Pablo (2021)
External consultants, learning partners or critical friends -whatever we call them- can seldom change the system or organisational (learning) culture from outside. So, how can Monitoring Evaluation and Learning (MEL) consultants support real change instead of creating tools or processes that are quickly forgotten without any real institutional ownership? Consultants and learning partners can seldom change a learning system or organisational culture – it just does not work like that. What they can do is interrogate existing practices, introduce new ideas, and support change processes already underway to ensure lasting, transformative impact. They are a multiplier for homegrown change initiatives, not a substitute. Don’t hire MEL consultants to sell you tools or tell you what to do. Hire them because you need a critical friend along the path of organisational change. Own, then learn – not the other way round.

What can Australia learn from the public service revolution taking hold in Europe? - Hurcombe, Sarah (Centre For Public Impact (CPI), 2021)
Sarah Hurcombe shares what she's learning from the public service revolution building momentum in Europe.

How Does Participatory Action Research Generate Innovation? Findings from a Rapid Realist Review - Snijder, Mieke; Apgar, J. Marina (CLARISSA Emerging Evidence Report, 2021)
This Emerging Evidence Report shares evidence of how, for whom, and under what circumstances, Participatory Action Research (PAR) leads to innovative actions. A rapid realist review was undertaken to develop programme theories that explain how PAR generates innovation. The methodology included peer-reviewed and grey literature and moments of engagement with programme staff, such that their input supported the development and refinement of three resulting initial programme theories (IPTs) that we present in this report. Across all three IPTs, safe relational space, group facilitation, and the abilities of facilitators, are essential context and intervention components through which PAR can generate innovation. Implications from the three IPTs for evaluation design of the CLARISSA programme are identified and discussed. The report finishes with opportunities for the CLARISSA programme to start building an evidence base of how PAR works as an intervention modality, such as evidencing group-level conscientisation, the influence of intersecting inequalities, and influence of diverse perspectives coming together in a PAR process.

Supply Chains, the Informal Economy, and the Worst Forms of Child Labour - Aked, Jody (CLARISSA Working Paper, 2021)
As a cohort of people, ‘children in work’ have become critical to the everyday functioning of diverse supply chain systems. This Working Paper considers diverse commodity chains (leather, waste, recycling and sex) to explore the business realities that generate child labour in its worst forms. A review of the literature finds that occurrence of the worst forms of child labour (WFCL) in supply chain systems is contingent on the organising logics and strategies adopted by actors in both the formal and informal economies. Piecing together the available evidence, the paper hypothesises that a supply chain system is sensitive to the use of WFCL when downward pressure to take on business risk cannot be matched by the economic resilience to absorb that risk. Emergencies and persistent stressors may increase risk and reduce resilience, shifting norms and behaviour. There is a need for further work to learn from business owners and workers in the informal economy.

In praise of... Logframes - Teskey, Graham (From Poverty to Power, 2021)
In all the jobs I have held, the only training that has ever stayed with me was a three-day course on logframes, held in a very pleasant beach hotel on Fiji’s beautiful coral coast. This was a few months after I joined what was then the Overseas Development Administration, DFID’s forerunner. Three days on logframes. Yes really. Our Pacific team were gathered together to learn this new skill. The course was designed not only to help us think rigorously about how change happens, but also to ensure that we had a shared understanding of what constitutes good design. Twenty-five years later I am aware that logframes are out of fashion: references to the ‘tyranny of the logframe’ abound, its vertical determinism, and its lack of flexibility. Donors seem to have replaced logframes with things called ‘Program Logics’, as in the schematic below. You don’t have to be able to read it to get the picture… (and it’s logframes that are supposed to be too vertical…?).

Lost Causal: Debunking Myths About Causal Analysis in Philanthropy - Lynn, Jewlya; Stachowiak, Sarah; Coffman, Julia (The Foundation Review, 2021)

Development Policy and Impact Evaluation: Learning and Accountability in Private Sector Development - Ton, Giel (Handbook of Development Policy, 2021)
There is broad recognition of the challenges in evaluating policy and programmes on their contribution to sustainable development. Impact evaluations of PSD programmes are carried out at the behest of a particular configuration of interest groups with different expectations. Some groups want to know whether a programme has worked, others want to know how to do these programmes better, others fear that PSD programmes might result in sub-optimal or adverse development outcomes in recipient countries, and some want to be sure that the programme benefits the private sector in the donors' domestic economies. This chapter discusses these with reference to private sector development programmes and explores how contribution analysis, which is a structured stepwise process of theory-based evaluation, can address these challenges and generate findings to improve learning and accountability.

Learning through and about Contribution Analysis for impact evaluation - Apgar, Marina; Ton, Giel (Institute of Development Studies, 2021)
Over the past five years, colleagues from the Centre for Development Impact (CDI) – a joint initiative between the Institute of Development Studies, Itad and University of East Anglia – have been innovating with and learning how to use Contribution Analysis as an overarching approach to impact evaluation. In this blog series, we share our learning and insights, some of them in raw emergent form, highlight the complexities, nuances, excitements, and challenges of embracing new ways of doing impact evaluation. We begin this series by sharing ideas about ‘causal hotspots’ that first surfaced as ‘aha’ moments in our collaboration on the CDI short course Contribution Analysis for Impact Evaluation – the moments when we found the ways and words to better articulate our ideas and help people navigate the messy realities of theory-based evaluation.

Finding and using causal hotspots: a practice in the making - Apgar, Marina; Snijder, Mieke (Institute of Development Studies, 2021)
This is the second blog in our series ‘Lessons on using Contribution Analysis for impact evaluation’. In our first blog, we introduced Contribution Analysis (CA) as an overarching approach to theory-based evaluation and the idea of causal hotpots as a way to zoom in, unpack and make the hard choices of where to focus evaluation research. Identifying specific links in the theory of change (ToC) with specific evaluation questions enables you to then choose appropriate methods. We have applied the approach in diverse settings, testing how robust Contribution Analysis can really be. Here we walk you through an example.

Exploring a new governance agenda: What are the questions that matter? - Nixon, Nicola; Verhulst, Stefaan; Matin, Imran; Vermonte, Philips J. (From Poverty to Power, 2021)
The 100 Questions Initiative, pioneered by the GovLab, seeks to overcome the chasm between supply and demand. It begins not by searching for what data is available, but by asking important questions about the biggest challenges societies and countries face, and then seeking more targeted and relevant data solutions. In doing this, it narrows the gap between policy makers and constituents, providing opportunities for improved evidence-based policy and community engagement in developing countries. As part of this initiative, we seek to define the ten most important questions across several domains, including Migration, Gender, Employment, the Future of Work, and—now–Governance. On this occasion, we invited over 100 experts and practitioners in governance and data science –whom we call “bilinguals”– from various organizations, companies, and government agencies to identify what they see as the most pressing governance questions in their respective domains. Over 100 bilinguals were encouraged to prioritize potential impact, novelty, and feasibility in their questioning — moving toward a roadmap for data-driven action and collaboration that is both actionable and ambitious.

Lessons for PFM Reform - the PDIA Approach in Africa - Bento, Joana (Public Financial Management Blog - IMF, 2021)
Some important lessons are as follows: First, identifying a policy problem is critical for reforms to get traction both at a political and administrative level. Second, appropriate solutions can emerge from a process of experimentation, iteration, and adaptation. Third, building teams and institutional capabilities is a critical part of solving complex problems in a sustainable way.

Are we there yet? Localisation as the journey towards locally led practice: models, approaches and challenges - Baguios, Arbie; King, Maia; Martins, Alex; Pinnington, Rose (ODI, 2021)
Localisation and locally led international development practice has long been discussed, but has still not been delivered. Systemic barriers have posed challenges, and the term itself is contested. Now, the last tumultuous 18 months could provide a critical juncture to finally move forward with this crucial agenda. The pandemic has highlighted structural inequalities in the global system, and disrupted ways of working in the international development sector. The Black Lives Matter movement has brought conversations about racism and colonialism to the fore. And the climate crisis has highlighted the need for global action on humanity’s challenges that remain rooted in local realities. The emerging analysis in this review aims to set out the key issues in this agenda, building on a wealth of existing knowledge. It aims to span sectors, highlighting many new and existing models and approaches in the humanitarian, development, philanthropic and private sectors. It reviews the barriers and challenges to localisation and locally led practice, with a view to informing a campaign for systemic change to move forward with this agenda. The review is based on: a rapid review of the literature and evidence on localisation and locally led practice; two consultations with over 100 total participants, targeted at Global South actors; and analysis of 28 existing models and approaches.

Building a culture of learning at scale: learning networks for systems change. - McKenzie, Fiona (Orange Compass & Paul Ramsay Foundation, 2021)
This scoping paper explores the question ‘what would it take to build a culture of learning at scale?’. It focuses on systems-wide learning that can help to inform systems change efforts in complex contexts. To answer this question, literature was reviewed from across diverse disciplines and the realms of education, innovation systems, systems thinking and knowledge management. This inquiry was also supported by in-depth interviews with numerous specialists from the for-purpose sector and the examination of several case studies of learning across systems. The goal was to derive common patterns to inform a ‘learning for systems change’ framework. Learning for systems change is critical when working with complexity. The dynamic nature of complex adaptive systems requires an ability to continually sense and learn from the system and adapt accordingly. This is because the nature of the challenge and ‘what works’ to meet the challenge is continually shifting (Lowe and Plimmer, 2019). This requires an ongoing process of iterative inquiry that draws upon wisdom and insights from across the system. Such learning challenges traditional siloes of expertise and organisational boundaries (Clarke et al., 2019). Learning is not simply a nice to have. It is critical for greater impact and improved outcomes, particularly in mission-driven organisations and initiatives (Price et al., 2019). In this paper, a ‘learning networks’ approach is proposed, one that draws upon individual, group and systems-wide learning to build capacity and resilience for systems change in uncertain environments. This fills a gap in the literature where the focus is largely on learning within organisations. Instead, the focus here is on what is required to support learning to occur across scales and boundaries - from the individual to system-wide. A simple meta-framework for developing learning networks is proposed that includes high level guidance on the enabling conditions - the mindsets, relationships, processes and structures- that would enable learning networks to flourish.

Lessons Learned from Applying the Data Powered Positive Deviance - Pawelke, Andreas; Glücker, Andreas; Albanna, Basma; Boy, Jeremy (DPPD, 2021)
This report presents six learnings from four pilot projects conducted by the Data Powered Positive Deviance (DPPD) initiative, a global collaboration between the GIZ Data Lab, the UNDP Accelerator Labs Network, the University of Manchester Center for Digital Development, and UN Global Pulse Lab Jakarta. The pilots seek out grassroots solutions to development challenges that range from the interaction between livestock farming and deforestation to gender-based violence and insecurity in dense urban environments in Ecuador, Mexico, Niger and Somalia. The learnings relate to the early stages of the DPPD method, originally proposed by Albanna & Heeks [1], and focus mainly on the access to, and use of digital data. They are summarized as follows: 1. Remain flexible in the face of data unavailability 2. Leverage existing partnerships for data access 3. Map and fill know-how gaps early 4. Scale with caution 5. Look at deviance over time 6. Look beyond individual or community practices and behavior The report is written for development practitioners, data analysts, domain experts, and more generally anyone interested in using new data sources and technologies to uncover successful local solutions to development challenges.

Twenty years of UK governance programmes in Nigeria - Piron, Laure-Hélène; Cummings, Clare; Williams, Gareth; Derbyshire, Helen; Hadley, Sierd (ODI, 2021)
This Flagship report analyses 20 years of governance programmes in Nigeria funded by the UK Department for International Development (DFID) and the Foreign, Commonwealth and Development Office (FCDO) in the North-western states of Jigawa (since 2001), Kano (since 2005) and Kaduna (since 2006), as well as the North-eastern state of Yobe (since 2011). The report’s main research question is whether, how, under what conditions and for whom UK-funded state-level governance programmes in Nigeria have contributed to sustained changes in governance, and related changes in health and education. ... The report concludes with the following recommendations: To international development partners: 1. Invest for the long term – 10 to 20 years – combining support for both state and nonstate actors. 2. Ensure programmes have the strategic-level mandate, managerial capacity and frontline staff skills to pursue politically savvy opportunities. 3. Take PEA to the next level by unpacking causal mechanisms, understanding incentives and designing interventions to make change happen. 4. Give governance programmes the ability to flex between core governance and service delivery issues. 5. Incentivise greater collaboration between governance and sector programmes. 6. Incentivise greater attention to gender, and to social inclusion beyond disability issues, in governance programming. To FCDO: 7. Empower and resource FCDO teams to enable TWP programmes, ensuring decision-making by country teams to respond to local priorities. 8. Re-imagine TWP for FCDO Nigeria, giving implementers the space to operate in TWP ways. 9. Incentivise stronger collaboration between PERL, Lafiya (health programme) and the Partnership for Learning for All in Nigerian Education. 10. Invest in impact data analysis. To partner governments in Nigeria and beyond: 11. Explicitly set out the objectives for which you would like to receive assistance. 12. Use TWP principles to decide how development partners can support your political objectives and the scope for politically-feasible and mutually-beneficial collaboration. 13. Invest in the coordination of development partners. To non-state partners in Nigeria and beyond: 14. Join coalitions to achieve your priorities. 15. Select development partners which can strengthen your skills, not just fund your activities.

The Systems Work of Social Change: How to Harness Connection, Context, and Power to Cultivate Deep and Enduring Change - Rayner, Cynthia; Bonnici, François (OUP Oxford, 2021)
The issues of poverty, inequality, racial justice, and climate change have never been more pressing or paralyzing. Current approaches to social change, which rely on industrial models of production and power to "solve" social problems, are not helping. In fact, they are designed to entrench the status quo. In The Systems Work of Social Change, Cynthia Rayner and François Bonnici draw on two hundred years of history and a treasure trove of stories of committed social changemakers to uncover principles and practices for social change that radically depart from these approaches. Rather than delivering "solutions," these principles and practices focus on the process of change itself. Through rich storytelling and lucid analysis, Rayner and Bonnici show that connection, context, and power sit at the heart of the change process, ensuring broader agency for people and communities to create social systems that are responsive and representative in a rapidly changing world. Simple yet profound, this book distills a timely set of lessons for practitioners, leaders, scholars, and policymakers.

DPPD Handbook. A step-by-step guide for development practitioners to apply the Data Powered Positive Deviance method - DPPD (DPPD Initiative, 2021)
The Method Positive Deviance (PD) is based on the observation that in every community or organization, there are a few individuals who achieve significantly better outcomes than their peers, despite having similar challenges and resources. These individuals are referred to as positive deviants, and adopting their solutions is what is referred to as the PD approach¹. The method described in this Handbook follows the same logic as the PD approach but uses pre-existing, non-traditional data sources instead of — or in conjunction with — traditional data sources. Non-traditional data in this context broadly refers to data that is digitally captured (e.g. mobile phone records and financial data), mediated (e.g. social media and online data), or observed (e.g. satellite imagery). The integration of such data to complement traditional data sources generally used in PD is what we refer to as Data Powered Positive Deviance² (DPPD). The digital data opportunity Recent developments in the availability of digital data provide an opportunity to look for positive deviants³ in new ways and in unprecedented geographical and on temporal scales. A number of studies⁴ have described the challenges related to the application of the PD approach in development. Given these challenges, there are obvious opportunities for innovation in PD and our particular interest here is in the innovative opportunities offered by non-traditional data, following the increasing “datafication” of development and the growing availability of big datasets in a variety of development sectors⁵. DPPD builds on this and expands our ability to extract value from non-traditional digital data while providing a systematic process for leveraging local know-how and the collective wisdom of communities. Data Powered Positive Deviance The DPPD method described in this Handbook emerged from a process of research and testing and follows the same stages as the PD approach. The difference is that DPPD integrates pre-existing, non-traditional data across the five stages, requiring a series of new and specific methods and practices that are not required in the PD approach. The first stage is also somewhat different because it not only defines the problem, but it also checks if it is suitable and feasible to use the DPPD method for the proposed project. Table 1 lists the five stages of the DPPD method. This Handbook dedicates a section to each stage. Stage 1 Assess problem-method fit Stage 2 Determine positive deviants Stage 3 Discover underlying factors Stage 4 Design and implement interventions Stage 5 Monitor and evaluate

System Innovation on Purpose - Leadbeater, Charles; Winhall, Jennie (The Rockwool Foundation, 2021)
In Building Better Systems, we introduced four keys to unlock system innovation: purpose and power, relationships and resource flows. These four keys make up a set. Systems are often hard to change because power, relationships, and resource flows are locked together in a reinforcing pattern to serve the system’s current purpose. Systems start to change fundamentally when this pattern is disrupted and opened up. Then a new configuration can emerge, serving a new purpose. In this essay series we delve deeper into these four keys and provide practical advice on how they can be put to use. This essay is about the role that purpose plays in orchestrating complex systems and how system innovators can create a new system around a new sense of purpose.

Technical assistance: a practical account of the challenges in design and implementation - Nastase, Alexandra; Rajan, Alok; French, Ben; Bhattacharya, Debarshi (Gates Open Research, 2021)
Technical assistance is provided to country governments as part of international development programmes to support policymaking or strengthen state capability. This article presents the conceptual evolution of ‘technical assistance’ linked to capacity development, starting with programmes aiming exclusively to enhance individual capacity in the 1950s to 1970s and progressing to complex systems approaches in the past ten years. It also presents some of the frequent challenges in designing and implementing technical assistance, drawing from the existing literature and the authors’ experience in international development. The article summarises the latest thinking about delivering more effective development, including the adaptive management practices and the initiatives to strengthen evidence about what works. Finally, we complement this article with a follow-up open letter reflecting on the current policy options and opportunities for change.

Getting Real about Unknowns in Complex Policy Work - Andrews, Matt (Research on Improving Systems of Education (RISE), 2021)
As with all public policy work, education policies are demanding. Policy workers need to ‘know’ a lot—about the problems they are addressing, the people who need to be engaged, the promises they can make in response, the context they are working in, and the processes they will follow to implement. Most policy workers answer questions about such issues within the structures of plan and control processes used to devise budgets and projects. These structures limit their knowledge gathering, organization and sense-making activities to up-front planning activities, and even though sophisticated tools like Theories of Change suggest planners ‘know’ all that is needed for policy success, they often do not. Policies are often fraught with ‘unknowns’ that cannot be captured in passive planning processes and thus repeatedly undermine even the best laid plans. Through a novel strategy that asks how much one knows about the answers to 25 essential policy questions, and an application to recent education policy interventions in Mozambique, this paper shows that it is possible to get real about unknowns in policy work. Just recognizing these unknowns exist—and understanding why they do and what kind of challenge they pose to policy workers—can help promote a more modest and realistic approach to doing complex policy work.

Understanding Governance from the Margins: What Does It Mean In Practice? - Barnes, Katrina, Barnes, Katrina; Anderson, Colin; de Chassy, Stephanie; Ahmed, Affaf; Ali, Mudabbir; Aung, Myo Min; Chaimite, Egidio; Joshi, Anuradha; Khan, Danyal; Loureiro, Miguel; Posse, Lucio; Rowlands, Jo; et al. (Institute of Development Studies (IDS), 2021)
What does governance look like ‘from below’ – from the perspectives of poor and marginalised households? How do patterns of conflict affect that? These were the questions at the heart of the Governance at the Margins research project. Over three years from 2017-2020 we worked to explore this through in-depth study in conflict-affected areas of Mozambique, Myanmar, and Pakistan. Our research teams interviewed the same people regularly over that time, finding out how they resolved problems and interacted with authorities. In this paper we connect what we found to the realities and complexities of development practice, drawing on the input of 20 experienced practitioners working in bilateral and multilateral development agencies and international NGOs, who generously gave their time to help us think through the practical implications of our wealth of findings.

Evaluating complex interventions in international development - Masset, Edoardo; Shrestha, S.; Juden, M. (CEDIL Methods Working Paper 6, 2021)
This paper reviews promising methods for the evaluation of complex interventions that are new or have been used in a limited way. It offers a taxonomy of complex interventions in international development and draws on literature to discuss several methods that can be used to evaluate these interventions. Complex interventions are those that are characterised by multiple components, multiple stakeholders, or multiple target populations. They may also be interventions that incorporate multiple processes of behavioural change. While such interventions are very common and receive a large proportion of development aid budgets, they are rarely subject to rigorous evaluations. The CEDIL Methods Working Paper, ‘Evaluating Complex Interventions in International Development’, reviews promising methods for the evaluation of complex interventions that are new or have been used in a limited way. It offers a taxonomy of complex interventions in international development and draws on literature to discuss several methods that can be used to evaluate these interventions. The paper focuses its attention on methods that address causality and allow us to state conclusively whether an intervention works or not. It shows that several rigorous methods developed in different disciplines can be adapted and used to evaluate complex interventions in international development.

Thinking and working politically: What have we learned since 2013 - Teskey, Graham (TWP Community of Practice, 2021)
The Thinking and Working Politically (TWP) Community of Practice (CoP) was established at a small meeting tacked on at the end of a meeting of Governance Advisers working for the United Kingdom’s Department of International Development (DFID) on South and South-East Asian countries, held in Delhi in November 2013. Since then, a number of meetings have been held throughout the world, each addressing different issues; ‘TWP’ has entered the lexicon of mainstream development; the CoP has expanded to more than 300 people; a Washington DC chapter has been established; and the International CoP has been granted modest funding from DFID’s successor, the Foreign, Commonwealth and Development Office (FCDO) and from Australia’s Department of Foreign Affairs and Trade (DFAT). It is legitimate to ask, however, what has been achieved operationally: how have the ideas underpinning TWP affected operational practice? This short paper traces the evolution of the idea and practice of TWP from 2013 to late 2021, and identifies what we have learned. What has been successful, and what has not? I asked in 2017 whether TWP had become a second orthodoxy (Teskey, 2017). Did this represent hubris or was 2017 in some ways the apogee of what might rather grandly be called the TWP ‘movement’?

Inclusive participatory approaches: A facilitator’s guide - Zaremba, Haley; Elias, Marlène; Devi, J. Tulasi; Priyadarshini, Pratiti (2021)
In community engagement and participatory processes, facilitators must make intentional efforts and adopt inclusive strategies to include marginalized and frequently overlooked groups. Yet, there is a lack of guidance on how to inclusively facilitate participatory processes. Facilitators are therefore often poorly prepared to engage with the power relations that underlie these processes, including those between the facilitator and participants and among participants themselves. This guide addresses this shortcoming by presenting strategies that have been shown to enhance the meaningful participation of women and marginalized groups in participatory processes. The aim is to equip facilitators with tools to create inclusive participatory spaces and community engagement. Although it refers to processes related to natural resource management, the guide is also applicable to the facilitation of participatory processes focusing on other issues or fields of significance to communities.

Social Accountability and Service Delivery Effectiveness: What is the Evidence for the Role of Sanctions? Background Paper - Aston, Tom; Zimmer Santos, Grazielli (2022)
Executive Summary Understanding how civil society can get government to respond to their needs, preferences and demands, and deliver goods and services to citizens is a central concern in social accountability initiatives. It is widely argued that sanctions make a key difference to service delivery outcomes, and that without them, transparency and accountability interventions are less likely to be effective and less likely to be sustainable (Anderson et al., 2020; Arugay, 2016; Fox, 2020; Goetz and Jenkins, 2005; Grandvoinnet et al., 2015; McGee and Gaventa, 2011; Molina et al., 2017; Joshi, 2010; Joshi, 2017; Tsai et al. 2019). In this paper, sanctions refer to the threat or imposition of a punishment for transgressing a rule or norm. Yet, what evidence is there to support the claim that sanctions are king? How much do we actually know about social and formal sanctions and their effectiveness in improving service delivery? Looking at 11 meta-reviews and 35 cases, this background paper sheds light on these questions and the conditions under which sanctions promoted within social accountability interventions may have contributed to improved service delivery. Sanctions, both social and formal, feature very prominently in the scholarly literature related to accountability, so much so that some scholars have argued that it has become synonymous with punishment (Mansbridge, 2014; Schedler, 1999). While it may not be true that these concepts are fully synonymous, this trend in scholarship significantly influenced the thinking of donors such as the Department for International Development (DFID) and the World Bank over the last two decades (Grandvoinnet et al., 2015; Malena and McNeil, 2010; Moore and Teskey, 2006; World Bank, 2003). As a result, sanctions also feature in the dominant models, or theories of change, in the social accountability sector. Yet, in reviewing 11 meta-reviews in the transparency and accountability sector we found that while there is plenty of theoretical argument asserting the potential effectiveness of social and formal sanctions, there was limited empirical evidence to support the claim that sanctions were key. Our review of 35 cases revealed five mechanisms of change related to social and formal sanctions. These were: (i) “sticks” – response to punishment; (ii) “big brother is watching” – response to threat of formal punishment; (iii) diagonal accountability – response to threat of formal punishment by horizontal accountability agencies; (iv) litigation – response to legal investigations or lawsuits supported by community paralegals and legal aid organizations and; (v) response to “naming and shaming” by civil society and/or media. We challenge the dominant view in scholarship that harder social sanctions and enforcement of stronger formal sanctions are either necessary to the achievement of higher-level service delivery outcomes or that they will deliver better and more sustainable outcomes. We find that both social and formal sanctions can contribute to improving service delivery outcomes across a variety of country contexts. In half of the 35 cases reviewed, we were able to establish a likely link between social or formal sanctions and intermediate effects over the short term with some degree of confidence. These effects ranged from increased service provider awareness and motivation, increased availability of funding, staff, and materials, to improved infrastructure quality, and in a minority of cases impact level changes such as improved test scores. However, the role of sanctions in delivering outcomes was often unclear, outcomes were almost never sustainable, and in close to half of the 35 cases reviewed there were substantial negative effects. These effects ranged from reducing transparency and funding, to discrediting, relocating, and reprisals for advocates and whistle-blowers, threats of violence to collaborating government actors, damaging staff morale, reducing attendance, and generating conflict among staff and between staff and community members and between staff and patients, damaging trust. There are therefore some serious ethical dilemmas associated with sanctioning efforts which need to be carefully considered. We argue that imposing sanctions without building relationships or systems to promote good behavior is unlikely to improve service delivery outcomes in a sustainable way. Another, perhaps surprising, finding was that there are some actors that are regular targets of sanctions, and in many cases, these actors are a lot weaker than is commonly assumed. Three broad types of actors were the most common targets of punishment: (i) absentee nurses or teachers who had their pay or allowances reduced; (ii) offending officials who were either suspended, relocated, or fired; and (iii) contractors who had to cover the cost of rejected materials or faced lawsuits, alongside civil servants involved in contracting. We suggest that closer relationships may perhaps reduce stakeholders’ appetite to impose sanctions. Particularly in the health sector, we found that more proximate relationships created disincentives for confrontation, and in such circumstances, a “policing” approach to monitoring was also deemed inappropriate and counterproductive. Conversely, it seems that actors generally prefer to sanction “others,” i.e., when an actor/organization was outside the group. Short-term consultants, contractors, and suppliers were easy (and quite vulnerable) targets for harder sanctions. It has also been argued that there may be productive combinations of collaborative and confrontational tactics — i.e., hybrids. We found that many supposed confrontational and collaborative hybrids were, in fact, dislocated in time and space. Many so-called “inside-outside” strategies, therefore, seem to be a potential mischaracterization. We also found, as Fung and Wright (2003) argued nearly two decades ago, that adversarial forms of engagement cannot easily be redeployed for collaborative purposes. It is widely argued that supposedly ‘weaker’ forms of citizen engagement are less effective than those with ‘strong enforceability (McGee and Gaventa, 2011).’ We found no compelling evidence to support this contention. A quarter of cases reviewed were collaborative for certain periods or in certain locations. On average, these were slightly more successful when compared with periods or locations when imposing sanctions was a key strategic emphasis. So, soft power can also be powerful. Rather than one approach necessarily being superior (confrontational, collaborative, or hybrid) however, we argue that the best approach is likely to be the one most appropriate to the context. While we were unable to identify strong trends of contextual factors which enabled social and formal sanctions to play a role in enhancing service delivery, we were able to identify several conditions which we believe offer the greatest promise. These conditions were: (i) supportive leaders who played a role as champions; opening doors or accompanying civil society efforts; (ii) high capacity and legal authority of oversight agencies; (iii) competitive elections, which provided windows of opportunity for CSOs to combine political and social accountability efforts, and; (iv) vulnerable public servants and service providers already in relatively precarious situations and are thus easy targets. Overall, our study finds that sometimes sanctions can be effective, but punishment is not the answer to all the world’s problems. Given these limitations, we recommend that it is time to reconsider “carrots” and enquire further into the enabling conditions for bureaucrats and service providers. Relatedly, scholars, evaluators, and program teams should look more closely at service providers’ or civil servants’ motivations and take context into account more seriously. To uncover these contextual and motivational features, we believe that scholars, evaluators, and prprogrameams should also make better use of theory-based and participatory methods. And perhaps most importantly, donors and practitioners should carefully consider and mitigate the potential for backlash from sanctions-based approaches.

Guidance Note: Practical introduction to adaptive management - DT Global (DT Global, 2022)
DT Global is proud to introduce our new Guidance Note: Practical Introduction to Adaptive Management There is a growing consensus around adaptive management as an effective (even necessary) approach when programs are tackling complex development problems. While there is no standard definition of adaptive management, there is general agreement that such programs need to routinely engage with and respond to program context; constantly test what works in that context; and adjust approaches, plans, and activities based on continuous learning. However, there remains a more limited body of evidence about what this looks like in practice—the enabling conditions, systems, resourcing, skills, and attitudes to effectively operationalise adaptive management. There is also limited guidance around when adaptive management is required, and to what extent—both critical and often overlooked considerations when planning for successful adaptive management. This Guidance Note draws together lessons and good practice in adaptive management from across DT Global’s diverse portfolio of donor-funded programs. It outlines our conceptual framework for adaptive management, with practical guidance on how it can be applied by our program teams. It is also designed to help our teams distinguish adaptive management from good (non adaptive) project management, consider when adaptive management is most useful on a program, and how adaptive a program (or part of a program) should be.

Evaluating programming that thinks and works politically: Challenges and emerging practice - Jacobstein, David; Swift, Sarah (New Directions for Evaluation, 2022)
Issues of power are not new to program evaluation. What is new is a consideration of how programming uses insights into incentives that shape and adapt implementation. How should one evaluate in a way that explicitly assesses the ways in which a program considers power? One of the innovative topics deriving from the democracy and governance space is the approach of thinking and working politically (TWP) which is seeing increased use in development programming. TWP suggests different mental models and practical approaches to achieving development objectives in ways that are more contextually grounded and informed by power dynamics. This article describes several of the core challenges to evaluation of TWP and also a rubric of considerations for more effective evaluation practices in this emerging field.

Critical friends - real time insights for shaping strategy - Johnson, Debbie; Nathan, Geeta; Rahman, Syahirah Abdul (Chapters, 2022)
This chapter focuses on facilitation given by the Innovation Caucus to provide expert academic critique to inform Innovate UK's strategy for UK business innovation. As Innovate UK's strategy developed, it became evident that several policy domains needed critical insights and evidence. An academic critical friend provided the latest academic insights, evidence and wider perspective of the actors in the UK Innovation system, insights of which policy makers often lack. The chapter gave case studies of Innovation Caucus' work with Innovate UK in facilitating extensive research expertise and helping policy makers to open dialogue. It also describes 'The Innovators' Breakfast Club', a webinar series organised by Innovation Caucus, co-developed with Innovate UK, which created live face-to-face dialogue between academic experts and Innovate UK strategy/policy leads. This example showed how critical friends are important in facilitating the initial dialogues needed to build evidence for policy making. The dialogue builds the foundation for a virtuous cycle of learning. The approach provides an opportunity for academic experts to gain insights into the priorities and dilemmas facing policy makers, to shape future academic research and refine the skill set needed to become critical friends.

Theory amidst complexity – using process tracing in ex-post evaluations - Krueger, Kate; Wright, Molly (New Directions for Evaluation, 2022)
Evaluators who take a complexity-aware approach must consider tradeoffs related to theoretical parsimony, falsifiability, and measurement validity. These tradeoffs may be particularly pronounced with ex-post evaluation designs in which program theory development and monitoring frameworks are often completed before the evaluator is engaged. In this chapter, we argue that theory-based evaluation (TBE) approaches can address unique ex-post evaluation challenges that complexity-aware evaluation (CAE) alone cannot, and that these two sets of approaches are complimentary. We will outline strategies that evaluators may use to conduct rigorous ex-post evaluations of democracy, human rights, and governance (DRG) interventions that merge CAE's inductive approaches with a theory-testing structure. It will illustrate these strategies with two case studies of ex-post evaluation using process tracing (PT).

Human Learning Systems: A practical guide for the curious - Lowe, Toby; Padmanabhan, Chandrima; McCart, Des; McNeill, Karen; Brogan, Andy; Smith, Mark (Centre for Public Impact, 2022)
Our new guide provides practical advice to help any organisation working in public service apply the Human Learning Systems approach to their work. In doing so, they will be better equipped to explore, learn and respond to the unique strengths and needs of each person, family and community they serve.

Context-monitoring for adaptive management - Pickwick, Sarah (World Vision, 2022)

Learning to change: The case for systemic learning strategies in the humanitarian sector - Ramalingam, Ben; Mitchell, John (ALNAP/ODI, 2022)
This paper presents the case for systemic organisational change in the humanitarian system. The paper firstly shows that that organisational learning has tended to reinforce existing ways of working and has not been able to shift a culture that values action over reflection. As a result, the rest of the paper asks about the most significant changes in the humanitarian sector

Learning from Life Story Collection and Analysis with Children who Work in the Leather Sector in Bangladesh - Sayem, Mashrique; Paul, Sukanta; Apgar, Marina; Snijder, Mieke (2022)
CLARISSA (Child Labour: Action-Research-Innovation in South and South-eastern Asia) is a participatory evidence and innovation generating programme. We are generating evidence on the drivers of the worst forms of child labour (WFCL) and exploring how to address them through participatory Action Research (PAR) with children and other stakeholders in the leather supply chain in Dhaka, Bangladesh. Our main intervention modality is Systemic Action Research (SAR) (Burns, 2007), of which life story collection and analysis (LSC&A) is the first step in our participatory design that will inform child-led PAR groups which will become the engines of innovative responses to WFCL. The LSC&A methodology is a storytelling and story listening methodology and was chosen because of the universal power of stories to make sense of complex realities and seek new futures. People across the world like to tell their stories; they like to feel listened to and they are interested in how their story connects and compares to others. By collecting and analysing stories from hundreds of children in WFCL we can visualise the bigger system that each individual story is connected to. We hypothesize that through engagement in the process of telling, listening, collecting, and analysing life stories, children engaged in harmful and hazardous work will use their understanding of systems dynamics to move into creating their own solutions to the drivers of WFCL. In 2021 in Bangladesh, we collected 405 life stories from children living in Hazaribagh, Hemayetpur, Lalbagh, and Bhairab in Dhaka, with more than one hundred of these stories collected by children themselves. Following the story collection and transcription, children were supported by the CLARISSA implementation team to collectively analyse the stories through identifying critical ‘factors’ (events which have causes and consequences) and understanding how they causally relate to each other. The analysis of the 405 life stories resulted in the development of large system maps that illustrate all the causal dynamics that underpin lived experiences of WFCL. Based on the systemic analysis process the children identified themes of the PAR to be set up in their localities. Our experience with the LSC&A methodology is the first in the context of children in WFCL. The methodology has been used in one other project in Bangladesh to date. We therefore intentionally aimed to learn from the implementation process and to evaluate if and how the telling of, listening to, collecting and analysing of life stories is empowering and in turn whether it leads to increased ownership of the problems which motivates collective action (Burns, 2021). In this learning note we share our methodological learning and reflect on operational implications for designing and facilitating an LSC&A process with children which we hope will support adaptation and use of the methodology by others working in participatory programming with children.

Lives Amid Violence: Transforming Development in the Wake of Conflict - Schomerus, Mareike (2022)
Violent conflict and its aftermath are pressing problems, particularly for international development initiatives. However, the results of development in conflict contexts have generally been disappointing and their preventative potential thus questionable. Available Open Access, Lives Amid Violence argues that this is because practitioners adhere to a mental model that emphasises linearity, certainty, and causality, assuming that violence is best addressed through work plans that deliver state-building, stabilisation and services. Based on ten years of multi-method research from, in, and on conflict-affected countries, this book challenges this approach.Drawing on a significant collaborative body of scholarship, this work puts forward original and generalizable conclusions about how lives amid violence persist, offering an invitation to abandon restricting mental models and to embrace creative ways of thinking and working. These include paying attention to the long-term effects of conflict on individual behaviour and decision-making, the social realities of economic life, the role service delivery plays in negotiations between citizens and states, and to creating meaningful relationships. Transformation also requires reflection and therefore the book concludes with constructive suggestions on how to practice these insights to better support those whose lives are shaped by violence.More details are available at www.transformingdevelopment.orgThe eBook editions of this book are available open access under a CC BY-NC-ND 3.0 licence on bloomsburycollections.com.

Nimble adaptation: Tailoring monitoring, evaluation, and learning methods to provide actionable data in complex environments - Serpe, Lauren; Ingram, Mason; Byom, Kate (New Directions for Evaluation, 2022)
This chapter examines good practices in implementing effective Monitoring, Evaluation, and Learning (MEL) systems within complex international development Democracy, Human Rights, and Governance (DRG) programs, which are characterized by challenges of non-linearity, limited evidence of theories of change, and contextual and politically contingent nature of outcomes. The chapter presents three cases of MEL systems in complex projects implemented by Pact across distinct and diverse operating contexts – Zimbabwe, Cambodia, and Somalia – to illustrate those projects’ MEL approaches that enabled continuous adaptation. The authors analyze the cases to respond to two questions: (1) What are the key elements of effective adaptive management-focused MEL systems in complex environments? (2) What is practical guidance for designing and enabling complexity-responsive and effective adaptive management-focused MEL systems? The case studies illustrate three key elements: (1) Information gathering that closely links context, research, and performance data; (2) Systems for reflection that offer scheduled learning moments of varying frequency and intensity, as well as multiple feedback mechanisms; and (3) Enabling structures that promote adaptive mindsets and attitudes within project teams.

Be a participant not a spectator - Sharp, Cathy (The Evaluator - UK Evaluation Society, 2022)
My focus here is on the realities of evaluating in complexity where ‘nothing is clear, and everything keeps changing’. I outline how I use a series of ‘provocations’ that allow people to choose their own starting point. Sharing those choices fuels conversations that discover, explore, and co-create (rather than manage) our mutual expectations and assumptions and track how these might themselves be influenced by the work as it unfolds. This account draws on a review of literature and my practice experience, including reflections from others brought into local, national, and international conversations about what it means for evaluation to recognise complexity.

Storytelling for Systems Change: insights from the field - Thea Snow; David Murikumthara; Teya Dusseldorp; Rachel Fyfe; Lila Wolff; Jane McCracken (Centre for Public Impact, 2022)
Capturing the impact of community-led work The Centre for Public Impact, Dusseldorp Forum, and Hands Up Mallee have been exploring how stories can be used to more effectively communicate the impact of community-led systems change work. Community-led place based initiatives are modelling new ways of working - shifting away from top down, program-focussed approaches towards an approach grounded in systems thinking and community-led innovations. However, while these stories of change are sitting in communities, they’re often not being told or celebrated. We wanted to understand why this is, and what might be done to better enable these stories to be shared and heard. The story of storytelling We talked to a range of people to uncover the story of storytelling - including collective impact backbone team members, community members, storytelling experts, and those working in and around community-led systems change initiatives across Australia. We explored the roles stories play in different communities; what good storytelling looks like; what barriers to storytelling might be; and what role stories can play in supporting systems change. Our findings We have learned through this project that stories can be used both to change the system and to evaluate, understand and showcase the change that is occurring in communities. We have heard that different stories require different approaches – stories that are seeking to enable change look different to those that are seeking to celebrate change.

What is the Outcomes Star? - Triangle (Triangle, 2022)
The Outcomes Stars are a suite of evidence-based outcomes measurement and keywork tools, which drive an ‘enabling help’ approach to service delivery. They support a person-centred, collaborative and trauma informed approach and give service users, workers, managers and commissioners vital information about needs and progress. Since the first version was published in 2006, the unique features of the Outcomes Star contributed to its popularity and widespread use, with over 1,000 organisations with licenses to use the Star including over 500 charities and 170 local authorities

An introductory systems thinking toolkit for civil servants - UK Government (UK Government Office for Science, 2022)
This document is a an Introductory Toolkit for for civil servants. It is one component of a suite of documents that aims to act as a springboard into systems thinking for civil servants unfamiliar with this approach. These documents introduce a small sample of systems thinking concepts and tools, chosen due to their accessibility and alignment to civil service policy development, but which is by no means comprehensive. They are intended to act as a first step towards using systems thinking approaches to solve complex problems and the reader is encourage to explore the wider systems thinking field further.

The UNDP Accelerator Labs enter a year of maturity: let a thousand flowers bloom! - Annual report 2022 - UNDP accelerator labs (UNDP, 2022)
The UNDP Accelerator Labs were designed as an agile and dynamic Network to allow communication and information transfer between 91 Accelerator Labs in 115 countries, and with the global innovation ecosystem, UNDP as a whole, and thousands of partners including grassroots innovators and their communities. The accumulated knowledge of this Network creates new pathways to the solutions that hold the key to sustainable development problems. What it looked like and how it unfolded last year will be illuminated in this annual report: The UNDP Accelerator Labs enter a year of maturity: let a thousand flowers bloom!

CLA Maturity Tool - Card Deck (Implementing Partners version 1) - USAID LEARN (USAID, 2022)
USAID’s Bureau for Policy, Planning and Learning (PPL) and its support mechanism, LEARN, have developed a Collaborating, Learning, and Adapting (CLA) Framework and Maturity tool to help USAID missions think more deliberately about how to plan for and implement CLA approaches that fit the mission’s context and assist them in achieving their development objectives. While the tool is intended primarily for USAID audiences to be used in participatory self-assessment workshops, the CLA Framework and maturity spectrum are relevant to a wider audience. USAID’s CLA Framework identifies key components and subcomponents of daily work that may be opportunities for intentional, systematic, and resourced CLA. The framework recognizes the diversity of what CLA can look like in various organizations and programs while also giving CLA structure, clarity, and coherence across two key dimensions: • CLA in the Program Cycle: how CLA is incorporated throughout Program Cycle processes, including strategy, project, and activity design and implementation; and • Enabling Conditions: how an organization’s culture, business processes, and resource allocation support CLA integration. Recognizing that CLA is not binary—it’s not an issue of “doing it or not doing it”—PPL and LEARN have developed a spectrum of practice for each of the 16 subcomponents in the framework. The spectrum offers examples of what integration might look like at different stages: Not Yet Present, Emergent, Expanding, Advanced and Institutionalized. The maturity stage descriptions are only illustrative and are intended to spark reflection on current practice and opportunities for improvements. In this resource, each CLA subcomponent page describes the key concepts for that topic and includes a description of the maturity stages. Although the descriptions were originally developed for USAID, the majority of the concepts easily transfer or have equivalents in the partner community. For example, although organizations outside of USAID may not hold “Portfolio Reviews” (part of the Pause & Reflect subcomponent), the majority hold some type of meeting to review programmatic progress. This is the seventh version of the CLA Framework and maturity spectrum. PPL and LEARN will continue reviewed and periodically update them based on user feedback, so if you have comments about the content, please email learning@usaid.gov. We would also love to hear how you’ve used this content with your team or organization.

Collaborating, Learning and Adapting (CLA) Maturity Spectrum (v7) - USAID LEARN (USAID, 2022)
USAID’s Bureau for Policy, Planning and Learning (PPL) and its support mechanism, LEARN, have developed a Collaborating, Learning, and Adapting (CLA) Framework and Maturity tool to help USAID missions think more deliberately about how to plan for and implement CLA approaches that fit the mission’s context and assist them in achieving their development objectives. While the tool is intended primarily for USAID audiences to be used in participatory self-assessment workshops, the CLA Framework and maturity spectrum are relevant to a wider audience. USAID’s CLA Framework identifies key components and subcomponents of daily work that may be opportunities for intentional, systematic, and resourced CLA. The framework recognizes the diversity of what CLA can look like in various organizations and programs while also giving CLA structure, clarity, and coherence across two key dimensions: • CLA in the Program Cycle: how CLA is incorporated throughout Program Cycle processes, including strategy, project, and activity design and implementation; and • Enabling Conditions: how an organization’s culture, business processes, and resource allocation support CLA integration. Recognizing that CLA is not binary—it’s not an issue of “doing it or not doing it”—PPL and LEARN have developed a spectrum of practice for each of the 16 subcomponents in the framework. The spectrum offers examples of what integration might look like at different stages: Not Yet Present, Emergent, Expanding, Advanced and Institutionalized. The maturity stage descriptions are only illustrative and are intended to spark reflection on current practice and opportunities for improvements. In this resource, each CLA subcomponent page describes the key concepts for that topic and includes a description of the maturity stages. Although the descriptions were originally developed for USAID, the majority of the concepts easily transfer or have equivalents in the partner community. For example, although organizations outside of USAID may not hold “Portfolio Reviews” (part of the Pause & Reflect subcomponent), the majority hold some type of meeting to review programmatic progress. This is the seventh version of the CLA Framework and maturity spectrum. PPL and LEARN will continue reviewed and periodically update them based on user feedback, so if you have comments about the content, please email learning@usaid.gov. We would also love to hear how you’ve used this content with your team or organization.

Collaborating, Learning, and Adapting Framework & Key Concepts (Implementing Partner Version 1) - USAID LEARN (USAID, 2022)
Although collaborating, learning, and adapting (CLA) are not new to USAID and its implementing partners, they often do not happen regularly or systematically and are not intentionally resourced. The CLA Framework above identifies components and subcomponents to help you think more deliberately about what approach to CLA might be best tailored to your organizational or project context. The framework recognizes the diversity of what CLA can look like in various organizations and projects while also giving CLA structure, clarity, and coherence across two key dimensions: - CLA in the Program Cycle: how CLA is incorporated into planning and design processes throughout the Program Cycle in order to improve their effectiveness; and - Enabling Conditions: how an organization’s culture, daily operating processes, and resource allocation support CLA integration. Organizations need both integrated CLA practices appropriate for their context and conducive enabling conditions to become stronger learning organizations capable of managing adaptively. The framework stresses the holistic and integrated nature of the various components of CLA to reinforce the principle that CLA is not a separate workstream—it should be integrated into existing processes to strengthen the discipline of development and improve aid effectiveness.

Civil society organizations and managerialism: On the depoliticization of the adaptive management agenda - Gutheil, Lena; Koch, Dirk-Jan (Development Policy Review, 2022)
Motivation In the last decade, a movement formed around making aid delivery more adaptive, relying on principles such as context sensitivity, flexibility, and ownership. The approaches seem promising for civil society organizations (CSOs) to fulfil their mission of fostering social transformation. While several donor agencies have started engaging with such approaches, the authors hardly see their political implications in practice. Purpose The article aims to provide evidence on an adaptive project and demonstrate how the social transformative and political nature of adaptive development management is rendered technical and depoliticized in practice. Methods and approach We use a case study of a development programme based on a social transformative policy framework that is implemented through CSOs in Uganda and Vietnam. Data were collected by means of interviews, participant observation and document analysis. Findings We find that, in practice, the social transformative policy framework is competing with managerial logics. We compare this process with the depoliticization of the Paris Declaration on Aid Effectiveness, finding striking similarities. By using practice theory, we show how managerialism remains the dominant paradigm in the civil society aid sector, fuelling the “anti-politics machine.” Policy implications The article shows that policy frameworks do not always work as intended. Donors should therefore not only change policy frameworks, but also start addressing institutional and operational requirements.

Market systems change rubric - Loveridge, Donna (Oxford Policy Management, 2022)
This systems change rubric describes different performance levels according to various systems elements, such as policy (formal rules), practices and relationships and connections. Programmes can use the rubric to assess the performance of systems to help decide where and how to intervene, or during and post-implementation to conduct progress assessments, and assess the effectiveness of interventions and type, breadth and depth of systems change. Each performance level description highlights the type of data and information that needs to be collected. One analysis is completed, users can compare this to the performance descriptions to see which level best matches the analysis. This helps programmes draw conclusions about systems changes. The rubric was developed in 2020 and tested in 2021 and builds on systems change thinking and frameworks from two previous FSG publications. It can be used as: pre-intervention to conduct an assessment during an intervention to conduct progress assessments and reflect on the effectiveness of interventions to change systems and inform decision making post intervention to make judgements about whether interventions were valuable given the resources, time and effort spent Useful for: Implementation managers to determine the effectiveness of interventions, as well as by Monitoring, Evaluation and Learning (MEL) teams to track progress against expected outcomes.

The Relational Work of Systems Change - Millgan, Katherine; Zerda, Juanita; Kania, John (Stanford Social Innovation Review, 2022)
Collective impact efforts must prioritize working together in more relational ways to find systemic solutions to social problems. Sometimes we lose sight of a simple truth about systems: They are made up of people. Despite all of the frameworks and tools at our disposal and all of our learning as a field of practice, purely technical, rational approaches to systems change will not make much of a dent in shifting power or altering our most deeply held beliefs. If most collective impact efforts fall short of supporting people to change in fundamentally consciousness-altering ways, then, the system they are a part of will not significantly change either. However, over the past two decades, the prevailing view among many funders, board members, and institutional leaders has been that only quantifiable and predetermined outcomes can create impact. But if the interrelated, devastating, and deepening crises and divisions over the past two years have taught us anything, it is that complex, adaptive problems defy tidy logic models and reductive technical solutions. It is time to invest our collective energy in more relational and emergent approaches to transforming systems.

MOTION HANDBOOK - Developing a transformative Theory of Change - Palavicino, Carla Alvial; Matti, Cristian; Witte, Jenny (TIPC, Utrecht University, 2022)
A step-by-step guide on how to develop a Transformative Theory of Change, for innovation projects, programmes and organisations working on systems transformation. The MOTION project was initiated with one key question in mind: how can we help projects and organisations be more transformative, using the framework and concept provided by the multi-level perspective? And what kind of tools, methods and frameworks can we co-design that translate scientific concepts into practises relevant for policy practitioners? This led us into a co-creation journey during which researchers from the Transformative Innovation Policy Consortium (TIPC) and EIT Climate-KIC project partners experimented, reflected and learned from each other in building the approach that we share in this handbook. Through this journey, we gained a deeper understanding of what the portfolio approach means in a transformative system change context and which skills and competences are needed to facilitate processes of co-creation in the science-policy-practice interface. We had the opportunity to configure the key building blocks of our theoretical approach, the Transformative Outcomes, into practical insights and actions that can easily be applied by innovation organisations at many levels. This handbook is the culmination of the journey as it translates key learnings from the MOTION project into practical insights that are relevant to practitioners working on systems transformation.

Sensemaking Workshop Preparation Guide and Facilitator Guide and Sensemaking Training - UNDP (UNDP, 2022)
Based on experience from running Sensemaking workshops for UNDP offices and government partners, the Asia-Pacific Regional Innovation Centre developed the Sensemaking Preparation Guide and Facilitator Guide to share its knowledge with teams and organization that are interested in using the Sensemaking process.

Data-powered positive deviance: Combining traditional and non-traditional data to identify and characterise development-related outperformers - Albanna, Basma; Heeks, Richard; Pawelke, Andreas; Boy, Jeremy; Handl, Julia; Gluecker, Andreas (Development Engineering, 2022)
The positive deviance approach in international development scales practices and strategies of positively-deviant individuals and groups: those who are able to achieve significantly better development outcomes than their peers despite having similar resources and challenges. This approach relies mainly on traditional data sources (e.g. surveys and interviews) for identifying those positive deviants and for discovering their successful solutions. The growing availability of non-traditional digital data (e.g. from remote sensing and mobile phones) relating to individuals, communities and spaces enables data innovation opportunities for positive deviance. Such datasets can identify deviance at geographic and temporal scales that were not possible before. But guidance is needed on how this new data can be employed in the positive deviance approach, and how it can be combined with more traditional data to gain deeper, more meaningful, and context-aware insights. This paper presents such guidance through a data-powered method that combines both traditional and non-traditional data to identify and understand positive deviance in new ways and domains. This method has been developed iteratively through six development projects covering five different domains – sustainable cattle ranching, agricultural productivity, rangeland management, research performance, crime control – with global and local development partners in six countries. The projects combine different types of non-traditional data with official statistics, administrative data and interviews. Here, we describe a structured method for data-powered positive deviance developed from the experience of these projects, and we reflect on lessons learned. We hope to encourage and guide greater use of this new method; enabling development practitioners to make more effective use of the non-traditional digital datasets that are increasingly available.

Monitoring and evaluation for thinking and working politically - Aston, Thomas; Roche, Chris; Schaaf, Marta; Cant, Sue (Evaluation, 2022)
This article explores the challenges of monitoring and evaluating politically informed and adaptive programmes in the international development field. We assess the strengths and weaknesses of some specific evaluation methodologies which have been suggested as particularly appropriate for these kinds of programmes based on scholarly literature and the practical experience of the authors in using them. We suggest that those methods which assume generative causality are particularly well suited to the task. We also conclude that factoring in the politics of uncertainty and evidence generation and use is particularly important in order to recognize and value diverse experiential knowledge, integrate understandings of the local context, accommodate adaptation and realistically grapple with the power relations which are inherent in evaluation processes.

PFM Reform Through PDIA: What Works and When it Works - Harris, Jamelia; Lawson, Andrew (Public Financial Management Blog - IMF, 2022)

The Difference Learning Makes - Factors that enable and inhibit adaptive programming - Gray, Stephen; Carl, Andy (Christian Aid, 2022)
Executive Summary When Christian Aid (CA) Ireland devised its multi-country and multi-year Irish Aid funded Programme Grant II (2017-2022), they opted to move away from a linear programme management approach and to explore an adaptive one. Across seven countries: Angola, Colombia, El Salvador, Guatemala, Israel and the occupied Palestinian territory, Sierra Leone, and Zimbabwe, CA and partner organisations support marginalised communities to realise their rights, reduce violence and address gender inequality. Since 2019, Adapt Peacebuilding has accompanied CA Ireland, CA country teams and partner organisations as they experimented with using a deliberate adaptive approach. The authors were also asked to follow up on an initial study by CA Ireland and Overseas Development Institute in 2018, which described the rationale for adopting this new approach and included early lessons from its first year of implementation. The aim of this study is to help deepen CA Ireland, CA country teams’ and partners’ understanding of (a) whether their application of adaptive programming has resulted in better development outcomes, and (b) how they can better understand the factors that enabled or inhibited the effectiveness of using this approach. Over the past three years, this study has found evidence and multiple examples that show adaptive programming contributed to better development outcomes. The main reasons cited were that these were made possible both from improvements to programming strategies based on proactive reflection and learning, as well as those that stem from the reactive capacity of adaptive programmes to change course in response to unanticipated changes in operating conditions. This study found that adaptive programming has enabled better development practice where organisations are enhancing their skills to better respond and be flexible to contextual challenges. 72% of partners surveyed described adaptive programming as the most useful approach to programme management that they have used. The programme approach has meant that CA and partner staff were better able to explore the significance of change in the context and their contributions to them. It also enabled spaces for meaningful engagement with communities in learning and programme planning processes and encouraged opportunities for experimentation in programming. The study also found that adaptive programming has supported flexible delivery. This led to better outcomes that would not have been possible were the programme not able to make flexible adjustments. The main focus has been the analysis of nine factors that can determine the effectiveness and impact (or otherwise) of using an adaptive approach, flagging important issues for understanding. These factors are identified as: 1) Leadership; 2) Organisational culture; 3) Conceptual understanding; 4) Staff capacities; 5) Partnership approaches; 6) Participation; 7) Methods and tools; 8) Administrative procedures; and 9) The operating context. Together these can provide an analytical framework for assessing an organisation’s ‘adaptive scope’, which can be used as a tool for better understanding an organisation’s potential to generate improved development outcomes via adaptive programming and how to strengthen them. The study concludes with several recommendations for CA Ireland, all of which have relevance for a broader community of donors and implementing organisations interested in the potential of adaptive programming.

Guidance Note for Adaptive Management - Ministry of Foreign Affairs of Denmark (Government of Denmark, 2022)
This note explains what adaptive aid management is; why and when it should be considered; and how it should be applied. It covers all Danish development support channels and modalities, including bilateral country assistance, assistance to and through civil society, the private sector and to and through multilateral organisations. This guide has three chapters. Chapter 1 provides an executive overview of what adaptive management is. Chapter 2 goes deeper into five key operational principles of adaptive management. Chapter 3 details the main tenets of adaptive management processes during the programming cycle.

The Patterns of Possibility - How to Recast Relationships to Create Healthier Systems and Better Outcomes - Winhall, Jennie; Leadbeater, Charles (The Rockwool Foundation, 2022)
In Building Better Systems, we introduced four keys to unlock system innovation: purpose and power, relationships and resource flows. These four keys make up a set. Systems are often hard to change because power, relationships, and resource flows are locked together in a reinforcing pattern to serve the system’s current purpose. Systems start to change fundamentally when this pattern is disrupted and opened up so that a new configuration can emerge, serving a new purpose. In this article series we delve deeper into these four keys and provide practical advice on how they can be put to use. This article is about relationships. Systems are defined by the patterns of interactions between their parts: their relationships. Those interactions generate the outcomes of the system as a whole. Transforming the outcomes of a system requires remaking its relationships and then unlocking the other keys to system innovation: purpose, power and resources. This shift in relationships allows all those in the system to learn faster, to be more creative. System innovators redesign the relationships in the system to allow dramatically enhanced learning across the system, and thereby generate far better outcomes.

Thinking and working politically: What have we learned since 2013? – FP2P - Green, Duncan (From Poverty to Power, 2022)
It’s always a red letter day when a new paper from Graham Teskey drops. His most recent is Thinking and working politically: What have we learned since 2013? For those that don’t know him, Graham is a consummate insider-outsider within the aid sector – long stints at DFID (UK), DFAT (Australia) and now Abt (Management Consultants). From this vantage point he has been one of the leading proponents of ‘thinking and working politically’, always ready to call out the hand-wavey academics and demand some practical lessons, please. This paper is part biography of an idea, setting out the timeline, moments and key documents and policy wins in the evolution of TWP (which seems to have involved a lot of seminars that I missed due to Oxfam’s meagre travel budget). The other part is, to be honest, a bit of a lament – a study in ‘Why Change Hasn’t Happened’, because TWP has ‘got lost in the maelstrom’ of the wider, largely negative, changes in the aid sector. Overall, it’s a brilliant summary, and one I’ll be recommending to my increasingly long-suffering activism students….

Systems Thinking and Practice: A Guide to Concepts, Principles and Tools for FCDO and Partners - Woodhill, Jim; Millican, Juliet (K4D, 2022)
This guide is a basic reference on systems thinking and practice tailored to the context and needs of the UK Government’s Foreign, Commonwealth & Development Office (FCDO). It is an output of the FCDO Knowledge for Development Programme (K4D), which facilitated a Learning Journey on Systems Thinking and Practice with FCDO staff during 2021 and 2022. The guide offers a common language and shared framing of systems thinking for FCDO and its partners. It explores what this implies for working practices, business processes and leadership. It also offers links to additional resources and tools on systems thinking. We hope it can support systems thinking to become more commonplace within the culture and practices of FCDO and working relations with partner organisations.

Theory of Change Workbook: A Step-by-Step Process for Developing or Strengthening Theories of Change - Salib, Monalisa (USAID, 2022)
While over time theories of change have become synonymous with simple if/then statements, a strong theory of change should actually be a much more detailed, context-specific articulation of how we *theorize* change will happen under a program.

Evaluating complexity, simplistically - Aston, Thomas (Medium, 2022)
A critical appraisal's of CEDIL papers on Evaluating Complex Interventions... A study was recently published by the Centre of Excellence for Development Impact and Learning (CEDIL) entitled Evaluating complex interventions in international development. This is the sort of title that raises great expectations. Complexity is a hugely popular theme and many of us are keen to know more about how to evaluate efforts that seek to achieve results amid complexity. In April 2021, CEDIL conducted a webinar on the paper, and in July 2021 CEDIL published a blog. In September 2021, I wrote a blog expressing some reservations regarding the focus of the study; its apparent over-emphasis on interventions, under-emphasis on context, as well as its choice of some supposedly under-used methods. These methods were: (1) factorial designs; (2) adaptive trials; (3) Qualitative Comparative Analysis (QCA); (4) synthetic control; (5) agent-based modelling, and system dynamics.

Lessons Learned From the Use of the Most Significant Change Technique for Adaptive Management of Complex Health Interventions - Ohkubo, Saori; Mwaikambo, Lisa; Salem, Ruwaida M.; Ajijola, Lekan; Nyachae, Paul; Sharma, Mukesh Kumar (Global Health: Science and Practice, 2022)
Introduction:The Most Significant Change (MSC) technique is a complex-aware monitoring and evaluation tool, widely recognized for various adaptive management purposes. The documentation of practical examples using the MSC technique for an ongoing monitoring purpose is limited. We aim to fill the current gap by documenting and sharing the experience and lessons learned of The Challenge Initiative (TCI), which is scaling up evidence-based family planning (FP) and adolescent and youth sexual and reproductive health (AYSRH) interventions in 11 countries in Asia and sub-Saharan Africa. Methods:The qualitative assessment took place in early 2021 to document TCI’s use and adaptation of MSC and determine its added value in adaptive management, routine monitoring, and cross-learning efforts. Focus group discussions and key informant interviews were conducted virtually with staff members involved in collecting and selecting MSC stories. Results:TCI has had a positive experience with using MSC to facilitate adaptive management in multiple countries. The use of MSC has created learning opportunities that have helped diffuse evidence-based FP and AYSRH interventions both within and across countries. The responsive feedback step in the MSC process was viewed as indispensable to learning and collaboration. There are several necessary inputs to successful use of the method, including buy-in about the benefits, training on good interviewing techniques and qualitative research, and dedicated staff to manage the process. Conclusion:Our assessment results suggest that the MSC technique is an effective qualitative data collection tool to strengthen routine monitoring and adaptive management efforts that allows for flexibility in how project stakeholders implement the process. The MSC technique could be an important tool for global health practitioners, policy makers, and researchers working on complex interventions because they continually need to understand stakeholders’ needs and priorities, learn from lessons and evidence-based practices, and be agile about addressing potential challenges.

Key Issues in Digitalisation and Governance - Roberts, Tony; Hernandez, Kevin; Faith, Becky; Prieto Martin, Pedro (SDC, 2022)
Digitalisation is perhaps the most important strategic challenge that governance will face over the coming decade. The process is delivering digital dividends as well as new exclusions and injustices, with the rapid but uneven increase in access to mobile and internet technologies transforming how social and economic life takes place. This report highlights the key opportunities and challenges arising from digitalisation.

What, so what, now what? - Aston, Thomas (Medium, 2022)
Getting serious about systems change

System Change: A Guidebook for Adopting Portfolio Approaches - Wellsch, Brent (UNDP, 2022)
This guidebook codifies the principles and methods of applying systems change and portfolio approaches to complex development challenges with practical tools and examples. It is based on the empirical learning generated from the collaborative initiatives in UNDP Country Offices in Bhutan, Pakistan, the Philippines, and Viet Nam with support from Regional Innovation Centre for Asia and the Pacific.

Second (and Third) Thoughts on Adaptive Management and Thinking and Working Politically - Green, Duncan (From Poverty to Power, 2022)
Self-Critical reflections on AM and TWP. Linking it with The Hype Cycle - "it feels like we are heading downward to the ‘trough of disillusionment’ form the initial peak of ‘inflated expectations’, but we will bounce back to something more sustained, that becomes a permanent feature of the aid landscape".

The collaborative exploration of alternative futures: A different approach to Theories of Change - CEDIL programme (2022)
ParEvo is a web application that enables the collaborative construction and exploration of a range of alternative futures: likely and unlikely, desirable and undesirable. These are described in the form of a branching narrative structure, developed over a series of iterations involving the interactions of a group of participants. These detailed storylines about the future contrast with optimistic, skeletal and largely singular views of the future found in diagrammatic ToCs often encountered by evaluators. This webinar will describe a recent ParEvo exercise implemented by the Centre for the Study of Existential Risk (CSER) in Cambridge. In this exercise 11 international experts explored ideas about how global risks associated with biotechnology research could be managed, and mismanaged, in the coming four years. Including how these alternative futures were evaluated by participants and the CSER facilitators. Issues to be discussed by the panel, and others, include comparisons with other representations of Theories of Change and other approaches to the exploration of alternative futures, described variously as scenario planning, futures or foresight work. Speakers: Rick Davies, Lara Mani Tom Hobson An overview of the app can be found here: https://mscinnovations.wordpress.com/ The app website is here: https://parevo.org/

Innovating for inclusive rigour in peacebuilding evaluation - Apgar, Marina; Báez-Silva, Ángela Maria; Deng, Ayak Chol; Fairey, Tiffany; Rohrbach, Livia; Alamoussa, Dioma; Bradburn, Helene; Cubillos, Edwin; Gray, Stephen; Wingender, Leslie (Institute of Development Studies, 2022)
Inclusive and rigorous peacebuilding evaluation is both vital and complex. In this blog we share examples of how we are innovating our methodologies to move towards participatory and adaptive practice.

Capacity Development in a Participatory Adaptive Programme: the Case of the Clarissa Consortium - Widmer, Mireille; Apgar, Marina; Afroze, Jiniya; Malla, Sudhir; Healey, Jill; Constant, Sendrine (CLARISSA Research and Evidence Paper, 2022)
Doing development differently rests on deliberate efforts to reflect and learn, not just about what programmes are doing and achieving, but about how they are working. This is particularly important for an action research programme like Child Labour: Action- Research-Innovation in South and South-Eastern Asia (CLARISSA), which is implemented by a consortium of organisations from across the research and development spectrum, during a rapidly changing global pandemic. Harnessing the potential of diverse skills and complementary strengths across partners in responding to the complex challenge of the worst forms of child labour, requires capacity to work together in novel ways. This Research and Evidence Paper documents how CLARISSA approached capacity development, and what we learnt from our challenges and successes. From the start, the programme incorporated a capacity development strategy resting on self-assessment of a wide range of behavioural and technical competencies that were deemed important for programme implementation, formal training activities, and periodic review of progress through an after-action review (AAR) process. An inventory of capacity development activities that took place during the first year of implementation reveals a wide range of additional, unplanned activities, enabled by the programme’s flexibility and adaptive management strategy. These are organised into eight modalities, according to the individual or collective nature of the activity, and its sequencing – namely, whether capacity development happens prior to, during, or after (from) implementation. We conclude with some reflections on the emergent nature of capacity development. Planning capacity development in an adaptive programme provides a scaffolding in terms of time, resources, and legitimacy that sustains adaptiveness. We also recognise the gaps that remain to be addressed, particularly on scaling up individual learning to collective capabilities, and widening the focus from implementation teams to individuals working at consortium level.

Locally driven development: Overcoming the obstacles - Ingram, George (Working Paper, 2022)
Locally led development is a complex process that the development community, in the U.S. and around the world, has spent several decades trying to get right. Yet, despite all the experience and lessons learned, it feels like we are barely beyond the starting line. This publication aims to contribute to the ongoing dialogue on locally led development, especially as to how the United States can address the obstacles posed by U.S. law, regulation, policy, and practice. It consists of two parts: • An essay by George Ingram that notes the path that has taken us to this point, identifies key obstacles, and invites a discussion of how to overcome impediments and move forward. • A set of 15 commentaries written by development experts that add a range of perspectives and nuances to the discussion

Measuring and Monitoring Adaptive Learning: A Landscape Review - LaFond, Anne; Adrian, Haley (USAID MOMENTUM Knowledge Accelerator, 2022)
This landscape review on measuring and monitoring adaptive learning highlights the learning from five adaptive programming guidelines and toolkits and one implementation science framework to inform the monitoring and evaluation of adaptive learning. The introduction of adaptive learning processes and skillsets in global health programming is part of an emerging strategy to advance a learning culture within projects and teams to improve health program performance. The monitoring and evaluation of adaptive learning is an emergent field aiming to monitor how adaptive learning processes have been introduced, how they are used, and whether they are having the intended results. Although there is a growing body of literature on adaptive programming more generally, there is a limited knowledge base on the monitoring and evaluation of adaptive learning interventions and their impacts. Unlike other implementation strategies or program management approaches, there are no standard metrics or a monitoring and evaluation framework to track the integration, implementation, and effectiveness of adaptive learning in health programming.

Gambling on Development: Why Some Countries Win and Others Lose - Dercon, Stefan (C Hurst & Co Publishers Ltd, 2022)

Can PDIA become a regular part of how a government works? - Building State Capability - Harrington, Peter (Building State Capability, 2022)
Institutional change is part of the theory of change of PDIA – scaling through the diffusion of new ways of thinking and greater problem-solving know-how. And once a community of practice reaches critical mass across an eco-system, a tipping point can happen where the eco-system becomes generally more open to novelty, where success is a more effective route to legitimacy, and where leadership is oriented towards value creation.

Seeing the Combined Effects of Aid Programmes - Burge, Richard; Nadelman, Rachel; McGee, Rosie; Fox, Jonathan; Anderson, Colin (IDS Policy Briefing, 2022)
Multiple aid agencies often try to support change in the same places, at the same time, and with similar actors. Surprisingly, their interactions and combined effects are rarely explored. This Policy Briefing describes findings from research conducted on recent aid programmes that overlapped in Mozambique, Nigeria, and Pakistan, and from a webinar with UK Foreign, Commonwealth & Development Office (FCDO) advisors and practitioners. The research found three distinct categories of ‘interaction effects’: synergy, parallel play, and disconnect. We explore how using an ‘interaction effects’ lens in practice could inform aid agency strategies and programming.

Evaluating complex interventions: What are appropriate methods? - Masset, Eduardo (CEDIL, 2022)
In the CEDIL Methods brief, ‘Evaluating complex interventions: What are appropriate methods?’ we identify four types of complex development interventions: long causal chain interventions, multicomponent interventions, portfolio interventions, and system-level interventions. These interventions are characterised by multiple activities, multiple outcomes, multiple components, a high level of interconnectedness, and non-linear outcomes.

Integrated Governance: Achieving Governance Results and Contributing to Sector Outcomes - Frazer, Sarah; Granius, Mark; Brinkerhoff, Derick W.; McGregor, Lisa (RTI Press Publication, 2022)
Achieving broad-based socio-economic development requires interventions that bridge disciplines, strategies, and stakeholders. Effective sustained progress requires more than simply an accumulation of sector projects, and poverty reduction, individual wellbeing, community development, and societal advancement do not fall neatly into sectoral categories. However, researchers and practitioners recognize key operational challenges to achieving effective integration that stem from the structures and processes associated with the current practice of international development. Integration calls for the intentional linking of intervention designs, implementation, and evaluation across sectors and disciplines to achieve mutually reinforcing outcomes. In this report, we summarize the results of a study we conducted to explore the challenges facing governance programs that integrate with sector interventions to achieve governance outcomes and contribute to sector outcomes. Through a review of policy documents and project reports from recent integrated governance programs and interviews with donor and practitioner staff, we found three integrated governance programming variants, an emphasis on citizen and government collaboration to improve service delivery, interventions that serve as the glue between sectors, and a balancing act for indicators to measure contribution to sectoral outcomes. Our analysis identified four key success factors: contextual readiness, the application of learning and adapting approaches, donor support, and recognition of the limitations of integrated governance. We then discuss recommendations and implications and for answering the challenge of integrating governance and sector programming to achieve development outcomes.

Rethinking monitoring and evaluation in complex systems — when learning is a result in itself - Innovation, UNDP (Medium, 2022)
By Søren Vester Haldrup, UNDP’s Strategic Innovation Unit

Beyond "Lack of Political Will": Elaborating Political Economy Concepts to Advance "Thinking and Working Politically" Comment on "Health Coverage and Financial Protection in Uganda: A Political Economy Perspective" - Ssennyonjo, Aloysius (International Journal of Health Policy and Management, 2022)
Political economy analysis (PEA) has been advanced as critical to understanding the political dimensions of policy change processes. However, political economy (PE) is not a theory on its own but draws on several concepts. Nannini et al, in concert with other scholars, emphasise that politics is characterised by conflict, contestation and negotiation over interests, ideas and power as various agents attempt to influence their context. This commentary reflects how Nannini et al wrestled with these PEA concepts - summarised in their conceptual framework used for PEA of the Ugandan case study on financial risk protection reforms. The central premise is that a common understanding of the PEA concepts (mainly structure-agency interactions, ideas, interests, institutions and power) forms a basis for strategies to advance thinking and working politically. Consequently, I generate several insights into how we can promote politically informed approaches to designing, implementing and evaluating policy reforms and development efforts.

Redefining rigour: using stories to evaluate systems change? - Lowther, Keira (Centre For Public Impact (CPI), 2022)
What might a different way of understanding rigour for work in complex adaptive systems look like?

Thinking and Working Politically on Health Systems Resilience: Learning from the experience of Cameroon, Nepal and South Africa during COVID-19 - Williams, Gareth (TWP Community of Practice, 2022)

(Re)making the case for adaptive management - Aston, Thomas (Medium, 2022)
Great overview of what to read on adaptive management. It’s a long one, so I’ve split it into two – second installment tomorrow. Christian Aid Ireland’s recent publication The Difference Learning Makes by Stephen Gray and Andy Carl made a bit of a splash. The study found that Christian Aid Ireland’s application of adaptive programming contributed to better development outcomes and supported more flexible delivery. The much vaunted MUVA programme in Mozambique is also coming to a close and presenting its results from using and adaptive approach. So, it struck me that we might be at a critical juncture in the conversation on adaptive management. We’ve had the crashing to earth of inflated expectations in recent misanthropic reflections, [misanthropic, moi? – Duncan] alongside a fragile institutionalisation of adaptive management in donor agencies, NGOs, and private sector organisations. However, I’d argue that we’ve reached the point where adaptive management has passed the proof-of-concept stage.

The Learning Power of Listening - Guijt, Irene; Gottret, Maria Veronica; Hanchar, Anna; Deprez, Steff; Muckenhirn, Rita (PRACTICAL ACTION PUBLISHING, 2022)
Steff had the pleasure to co-author the first SenseMaker Practitioner Guide with a group of friends and colleagues supported and published by Oxfam and CRS. This practical guide is for those who wish to use SenseMaker to conduct assessments, monitor progress, and undertake evaluations or research. Drawing on more than a decade of experience, the authors share dozens of examples from international development, providing practical tips and ideas for context-specific adaptations. They show how the method can be used to for difficult-to-measure outcomes related to poverty reduction, social justice, peacebuilding, resilience, gender norms, behavior change, governance and environmental management. ​ SenseMaker is a unique participatory method of inquiry that encourages and enables novel insights not obtained from conventional quantitative and quantitative and qualitative methods. It is action-oriented and, therefore, well-suited for people needing data- informed insights for adaptive management. "Writing this guide together with Irene, Veronica, Anna and Rita was an enormous learning process in itself and has further shaped our thinking and practice. We hope it will support first-time and experienced users to enhance their practice and that it will inspire people to explore and innovate further with the method."

Using stories in evaluation: Participatory Narrative Inquiry and Sensemaking (gLocal Evaluation Week 2022, 2022)
Sensemaking and Participatory Narrative Inquiry (PNI) are similar approaches, based on collecting stories about a real life experience from a large number of stakeholders on a topic of interest to the evaluation, and giving the storytellers themselves the responsibility for analyzing and making sense of the stories. Because a large number of stories are collected and self-interpreted, it is possible to conduct quantitative analysis of recurrent themes, perspectives and feelings emerging in the narratives. By combining elements of qualitative and quantitative research, these approaches help to make sense of complex and evolving realities. This session presented the general approach of Sensemaking and PNI. Panelists discussed their experiences with this method, recently used in IFAD and WFP’s evaluations and invited participants to share their own experiences with using stories in evaluation.

Evidence-led adaptive programming: Lessons from MUVA - Sharp, Samuel; Riemenschneider, Nils; Selvester, Kerry (ODI, 2022)
Calls for more ‘adaptive programming’ have been prominent in international development practice for over a decade. Learning-by-doing is a crucial element of this, but programmes have often found it challenging to become more learning oriented. Establishing some form of reflective practice, against countervailing incentives, is difficult. Incorporating data collection processes that generate useful, timely and practical information to inform these reflections is even more so.This paper explores MUVA - an adaptive female economic empowerment programme in Mozambique. MUVA, we suggest, is atypically evidence-led. It combines systematic, inclusive reflective practice with extensive real-time data collection. We describe the fundamental features of MUVA’s monitoring, evaluation and learning (MEL) approach that supported this. One, how data collection and analysis are synchronised with set cycles for learning and adapting projects. Two, how MEL systems are designed to prioritise actionable learning, with data collection oriented more to the needs of implementing staff than to the reporting requirements of funders.This approach was enabled by building collective ownership over the programme’s objectives and the purpose of MEL from the outset. Implementers are asked about their motivations, and these are related to the programme’s Theory of Change. The evidence culture is supported by the proximity of MEL staff to implementing staff; and through structuring upwards accountability to funders around justifying evidence-based adaptations instead of reporting on more narrow indicators. We conclude by considering the relevance, or not, of MUVA’s approach to programmes in other contexts or issue areas trying to replicate a similarly evidence-informed approach to adaptive management. Key messages Learning-by-doing is essential to adaptive programming, but it can be challenging to establish data collection processes that generate useful, timely and practical information. MUVA – a female economic empowerment programme in Mozambique – has an atypically evidence-led adaptive management approach. This has two fundamental features. One, data collection and analysis are synchronised with set cycles for learning and adapting projects. Two, data collection is oriented more to the needs of implementing staff than to the reporting requirements of funders. Monitoring, evaluation and learning (MEL) systems are designed to prioritise actionable learning. This approach was enabled by building collective ownership over the programme’s objectives and the purpose of MEL from the outset. Implementers are asked about their motivations, and these are related to the programme’s Theory of Change. The evidence culture is supported by the proximity of MEL staff to implementing staff; and through structuring upwards accountability to funders around justifying evidence-based adaptations instead of reporting on more narrow indicators.

Integrating Local Knowledge in Development Programming - Johnson, Madelyn; Maunder, Ishan; Pinga, Andie (USAID, 2022)
The objective of this report, Integrating Local Knowledge in Development Programming is to share knowledge of how development donors and implementing organizations leverage local knowledge to inform programming. In a recent speech at Georgetown University, United States Agency for International Development (USAID) Administrator Samantha Power said, “As Americans with a fraught history living up to our own values, we’ve got to approach this work with intention and humility. But the entire development community needs to interrogate the traditional power dynamics of donor-driven development and look for ways to amplify the local voices of those who too often have been left out of the conversation.” To that end, USAID’s Agency Knowledge Management and Organizational Learning (KMOL) function facilitated conversations with multilateral and bilateral donors and local organizations to understand how organizations define, utilize, and incorporate local knowledge into their programmatic and operational activities. Using qualitative tools to gather data for this report, the research team explored five overarching themes: 1. Local Knowledge Nomenclature and Definitions 2. Best Practices 3. Outcomes 4. Ethics and Power Dynamics 5. Challenges

Achieving disability inclusive employment – Are the current approaches deep enough? - Shaw, Jackie; Wickenden, Mary; Thompson, Stephen; Mader, Philip (Journal of International Development, 2022)
Diverse approaches to promoting disability inclusive employment aim to transform workplaces into truly inclusive environments, usually with intervention strategies targeting two main groups: employers and jobseekers with disabilities. However, they do not always consider other relevant stakeholders or address the relationships and interactions between diverse actors in the wider social ecosystem. These approaches often neglect deeper ‘vexing’ difficulties which block progress towards disability inclusive work environments. Most interventions rightly embrace hegemonic ‘social models of disability’ and use human rights arguments but may neglect entrenched structural factors. Disability inclusive employment is complex, with unaddressed invisible aspects that continue to limit progress. We explore some key relevant disability concepts and then interrogate evidence from the ‘Inclusion Works’ programme working in four middle- and low-income countries, considering some intractable barriers underlying the slow movement towards inclusive employment. Finally, we propose that a more participatory action orientated approach involving disabled people and others is needed to both generate deeper understanding and provide pathways towards new solutions to obstinate problems through progressive action learning processes in context. Programmatic interventions that work across the levels of the ecosystem and address power relations and interactions between stakeholders could lead to more substantial forms of disability inclusive employment.

Where have we got to on Thinking and Working Politically? Update and a Mildly Heretical Thought. – FP2P - Green, Duncan (From Poverty to Power, 2022)
Headed off recently to discuss the state of Thinking and Working Politically within the aid sector. This is a loose network of aid wonks that came together to try and move aid from a pure focus on technical issues, towards taking account of power and politics and why they can facilitate/frustrate attempts to make change happen in any given context. It was great to be in a room with others (50/50 in person and online) – the neurons fire in a way that just doesn’t happen online (but I also need to brush up on my meeting skills, as when I accidentally clicked on a random video in my timeline and it started playing at full volume. Super awkward). On to the content (Chatham House rule, so no names or institutions). Some observations about the evolution of a movement I’ve been connected with for over a decade. I got a lot of pushback from participants on an earlier draft and have made quite a few changes, but they should definitely feel free to set me straight in comments!

How Beijing Commands: Grey, Black, and Red Directives from Deng to Xi - Ang, Yuen Yuen (The China Quarterly, 2022)

How to develop capacity in the international development sector - Prieto Martín, Pedro (CLARISSA, 2022)
In her acclaimed study of effective cross-organizational teamwork, Harvard Business School Professor Amy C. Edmonson concluded that “trying things that...

Designing Contribution Analysis of Participatory Programming to Tackle the Worst Forms of Child Labour - Apgar, Marina; Snijder, Mieke; Prieto Martin, Pedro; Ton, Giel; Macleod, Shona; Kakri, Shanta; Paul, Sukanta (CLARISSA Research and Evidence Paper, 2022)
This Research and Evidence Paper presents the theory-based and participatory evaluation design of the Child Labour: Action-Research- Innovation in South and South-Eastern Asia (CLARISSA) programme. The evaluation is embedded in emergent Participatory Action Research with children and other stakeholders to address the drivers of the worst forms of child labour (WFCL). The report describes the use of contribution analysis as an overarching approach, with its emphasis on crafting, nesting and iteratively reflecting on causal theories of change. It illustrates how hierarchically-nested impact pathways lead to specific evaluation questions and mixing different evaluation methods in response to these questions, critical assumptions, and agreement on causal mechanisms to be examined in depth. It also illustrates how realist evaluation can be combined with contribution analysis to deeply investigate specific causal links in the theory of change. It reflects on learning from the use of causal hotspots as a vehicle for mixing methods. It offers considerations on how to navigate relationships and operational trade-offs in making methodological choices to build robust and credible evidence on how, for whom, and under what conditions participatory programming can work to address complex problems such as child labour.

The Haves and the Have Nots: Civic Technologies and the Pathways to Government Responsiveness - Mellon, Jonathan; Peixoto, Tiago C; Sjoberg, Fredrik M (Policy Research Working Paper, 2022)
“Civic tech” broadly refers to the use of digital technologies to support a range of citizen engagement processes. From allowing individuals to report problems to local government to enabling the crowdsourcing of national legislation, civic tech aims to promote better policies and services – while contributing to more inclusive democratic institutions. But could civic tech affect public issues in a way that benefits some and excludes others? Over the decades, the question of who participates in and who is excluded from participation mediated by technology has been the focus of both civic tech critics and proponents. The latter tend to argue that, by enabling citizens to participate without constraints of time and distance, civic tech facilitates the participation of those who usually abstain from engaging with public issues, leading to more inclusive processes. Critics argue that, given the existing digital divide, unequal access to technology will tend to empower the already empowered, further deepening societal differences. Yet both critics and proponents do tend to share an intuitive assumption: the socio-economic profile of who participates is the primary determinant of who benefits from digitally mediated civic participation. For instance, if more men participate, outcomes will favor male preferences, and if more young people participate, outcomes will be more aligned with the concerns of the youth. In a new paper, we show that the link between the demographics of those participating through digital channels, and the beneficiaries of the participation process, is not necessarily as straightforward as commonly assumed. We review four civic tech cases where data allow us to trace the full participatory chain through: the initial digital divide the participant’s demographics the demands made through the process the policy outcomes We examine online voting in the Brazilian state of Rio Grande do Sul’s participatory budgeting process, the local problem reporting platform Fix My Street (FMS) in the United Kingdom, Iceland’s online crowdsourced constitution process, and the global petitioning platform Change.org. Counterintuitive findings Change.org has been used by nearly half a billion people around the globe. Using a dataset of 3.9 million signers of online petitions in 132 countries, we examine the number of successful petitions and assess whether petitions created by women have more success than those submitted by men. Our analysis shows that, even if women create fewer online petitions than men, their petitions are more likely to be successful. All else equal, when online petitions have an impact on government policy, the agenda being implemented is much closer to the issues women choose to focus on. In Rio Grande do Sul’s digital participatory budgeting (PB), we show that despite important demographic differences between online and offline voters, these inequalities do not affect which types of projects are selected for funding – a consequence of PB’s unique institutional design, which favors redistributive effects. In fact, of all the cases analyzed, none reflect the standard assumption that inequalities in who participates translate directly into inequalities in who benefits from the policy outcomes. Our results suggest that the socio-economic profile of participants predicts only in part who benefits from civic tech. Just as important to policy outcomes is how the platform translates civic participation into policy demands, and how the government responds to those demands. While civic tech practitioners pay a lot of attention to design from a technological perspective, our findings highlight the importance of considering how civic tech platforms function as political institutions that encourage certain types of behavior while discouraging others. Civic tech, it seems, is not inherently good nor bad for democratic institutions. Instead, its effect is a combination of who participates on digital platforms and the choices of platform designers and governments.

How do we know we are doing good work? - Sharp, Cathy; McLaughlin, Dot; Whitley, Janet; Lawson, Karen (Collective Leadership for Scotland, 2022)
Collective leadership feels timely and important in an uncertain, fast changing, and challenging world. This report comes at this heightened moment of urgency and appetite for renewal, bringing potential to do things differently in public services and communities. The opportunities and challenges of true collaboration-in-practice, in the absence of blueprints, mean that it has never been more necessary to change ways of working and foreground learning. As outcomes remain important, and time and resources are scarcer than ever, the approach of collective leadership makes the creation of impact a shared, conscious, and actionable choice. A pathway to ultimate impact at scale is co-created through dialogue about expectations and contributions, and by design, not assuming change will happen because we have good intentions. Collective Leadership for Scotland (CLfS) has a strong vision and enjoys continuing active interest, drawing together participants from a variety of agencies working in public services. People are looking for fresh thinking, space and time for reflection, connection with others, a chance to think about how to tackle difficulties, and to test out what it takes to do, and continue to do, the work of collaborative public service. These motivations are deepened and brought into sharper focus by the pandemic, with an added interest in developing skills in online facilitation. CLfS contributes to building a critical mass for system change, to help to sustain the ambitions of the Christie Commission and the delivery of the National Outcomes for Scotland. There remains further potential to realise wider and deeper impact amongst organisations, communities, and wider systems. The conclusions of this report are likely to have wider resonance beyond interests in the CLfS programmes. This report deepens understanding of some of the challenges of commissioning, convening, and the scope for deeper impact through building reflective and relational leadership practices. It also outlines social and experiential sensemaking and facilitation practices to strengthen the action inquiry approach as a deliberate learning strategy, building cultures that support new forms of collaborative inquiry and systemic action research.

Digital authoritarianism in Bangladesh: Weaponising a draconian law to silence dissent in the pandemic era | Association for Progressive Communications - Siddiki, Zayed (2022)

Thinking and Working Politically in the Land Sector in Mekong Region - Burns, Tony; Ingalls, Micah; Rickersey, Kate (FIG Congress 2022, 2022)
In recent decades, the World Bank and many bilateral development partners have provided funding to support land administration reform. Traditional land administration reform projects focus on the economic and technical design of interventions based on a library of best practice, commonly avoiding the “messy politics” typically involved in land governance. Experience and lessons from land administration reform initiatives have been documented and a recurrent theme is that many projects fail to create effective, transformative change and gain the critical mass, and the community participation, necessary to ensure the sustainability of land administration reform. Over the last decade there have been concerted efforts to develop more politically informed ways of thinking and working using a range of methodologies referred to, variously, as Thinking and Working Politicallyi (TWP), Problem Driven Iterative Adaptation (PDIA) and Doing Development Differently (DDD). There is little evidence that these different approaches have been applied in the land sector.

Learning from Life Story Collection and Analysis With Children Who Work in the Worst Forms of Child Labour in Nepal - Karki, Shanta; Apgar, Marina; Snijder, Mieke; Sharma, Ranjana (CLARISSA Learning Note, 2022)
The CLARISSA Nepal team collected and analysed 400 life stories of children and young people engaged in or affected by the worst forms of child labour (WFCL), particularly in the “Adult Entertainment” sector in Nepal, which includes children working in Dohoris (restaurants playing folk music), dance bars, spa-massage parlours, khaja ghars (tea/snack shop) and guest houses. Stories were also collected from children in CLARISSA’s focus neighbourhoods, children in this category include street connected children and those working in transportation, party palaces, domestic labour and construction sites. Of the 400 stories collected, 350 were collected by adult researchers and 50 were collected by children themselves.

How to do Process Tracing: A Method for Testing “How Change Happened” in Complex and Dynamic Settings - Lynn, Jewlya; Stachowiak, Sarah; Beyers, Jennifer (ORS Impact, 2022)
Process tracing is a causal methodology that can help people understand how a particular large-scale change actually happened within a complex dynamic environment. Much of the existing literature provides important information about the method; we wrote this brief to help more people operationalize the concepts and learn about practical steps for using this method more easily, with quality, and toward a more equitable world. This piece was written based on our experiences implementing process tracing when our experience showed that existing materials on the method had a lot more conceptual than practical information. We’ve approached this as people with some successful (and some unsuccessful) experience with the method itself, alongside deep experience in evaluating initiatives and strategies in complex and dynamic settings. We focus not on the Bayesian side of process tracing but rather on how this can be implemented in a way that’s more participatory and lifts up the experiences and wisdom of those closest to the work and the problems being tackled. We hope this contributes to and helps make more approachable the important work of political scientists and methodologists upon which this work sits.

UNDP Digital Leadership Learning Modules - UNDP (United Nations Development Programme, 2022)
In order to support the digital transformation of government operations Digital Learning Modules for Civil Servants are available, an off-the-shelf package of capacity development in form of replicable training modules to empower public servants at both the local and central government level to be leaders of digital transformation for delivering better public services. The modules cover a multitude of fundamental areas: comprehending digital government and services, human-centered design for inclusivity and agile learning cycles; feature the importance of security and privacy, the value of data and how to manage data and technology related risks; spotlight the key role of supportive leadership and offer practical tools for assessing and overcoming main barriers to ensure a successful digital transformation journey.

Local Capacity Strengthening Policy - USAID (USAID, 2022)

The Art and Craft of Bricolage in Evaluation - Aston, Thomas; Apgar, Marina (CDI Practice Paper, 2022)
This CDI Practice Paper by Tom Aston and Marina Apgar makes the case for ‘bricolage’ in complexity-aware and qualitative evaluation methods. It provides a framework based on a review of 33 methods to support evaluators to be more intentional about bricolage and to combine the component parts of relevant methods more effectively. It discusses two cases from practice to illustrate the value added of taking a more intentional approach. It further argues that navigating different forms of power is a critical skill for bricolage, and that doing so can help to ensure rigour.

Everyday patterns for shifting systems - GCSI (GCSI, 2022)

Everyday patterns for shifting systems - Right scaling - GCSI (GCSI, 2022)

Beneficiary feedback mechanisms - UK Aid (FCDO, 2022)
This guidance seeks to ensure that UK Aid Direct applicants and grant holders understand what the Foreign, Commonwealth & Development Office (FCDO) means by beneficiary feedback mechanisms, and more specifically, that they: • Understand the terms used that relate to beneficiary feedback mechanisms in UK Aid Direct guidance and templates • Understand beneficiary feedback mechanisms and why they are a useful tool for project monitoring and learning • Learn how to use beneficiary feedback mechanisms during project implementation. • Can demonstrate that using beneficiary feedback mechanisms can lead to greater accountability.

Transforming M&E for Uncertain and Complex Contexts: The UNDP’s Innovation Sandbox Approach - Fraser, Dugan; Haldrup, Søren (Powered by Evidence Podcast, 2022)
How can we transform monitoring and evaluation (M&E) into a more adaptive, emergent process to address uncertainty and complexity in todays’ world? How do we move from compliance and accountability to learning – to support better, more timely, decisions? Join GEI Program Manager, Dugan Fraser, as he discusses these questions and others with Special Guest, Søren Haldrup, from UNDP's Strategic Innovation Unit where he manages UNDP's innovation facility and leads a new initiative called the M&E Sandbox.

How and Why Practitioners Think and Work Politically - Evidence from Chemonics Programming Across Sectors - Kantelberg, Renee; Swift-Morgan, Jennifer; Watson, Bryce (Chemonics International, 2022)
Most development practitioners have long recognized that deep contextual knowledge is crucial to understanding how projects interact with their local systems and, in turn, to navigating these systems. Moreover, this knowledge must complement projects' technical solutions, or they will fall flat and may even undercut project objectives as they clatter down. What, then, explains practitioners' particular interest in TWP as an explicit strategy and more than just "doing good development"? This report responds to that question and to the many calls for a more comprehensive picture of TWP by presenting new evidence of the various forms that TWP may take in practice. The evidence comes from a 2022 study that Chemonics undertook to foster more robust learning about TWP. Specifically, we closely examined Chemonics implemented projects that used or are using various forms of TWP in nine countries: Bangladesh, Iraq, Mozambique, the Philippines, Pakistan, Syria, Timor-Leste, and Tunisia. In conducting the study, we interviewed multiple staff from these projects. We complemented what we learned from these projects with a review of eight additional Chemonics- implemented projects applying TWP that had received dedicated support from Chemonics’ Center for Politically Informed Programming (the Center). We consider these findings alongside those of the recent (2022) USAID-Chemonics study on political economy analysis (PEA) usage to identify and articulate what is different and more effective about PEA processes and TWP practices that have received more support.

How Political Contexts Influence Education Systems: Patterns, Constraints, Entry Points - Levy, Brian (Research on Improving Systems of Education (RISE), 2022)
This paper synthesises the findings of a set of country studies commissioned by the RISE Programme to explore the influence of politics and power on education sector policymaking and implementation. The synthesis groups the countries into three political-institutional contexts: - Dominant contexts, where power is centred around a political leader and a hierarchical governance structure. As the Vietnam case details, top-down leadership potentially can provide a robust platform for improving learning outcomes. However, as the case studies of Ethiopia, Indonesia, Nigeria, and Tanzania illustrate, all-too-often dominant leaders’ goals vis-à-vis the education sector can veer in other directions. - In impersonal competitive contexts, a combination of strong formal institutions and effective processes of resolving disagreements can, on occasion, result in a shared commitment among powerful interests to improve learning outcomes—but in none of the case studies is this outcome evident. In Peru, substantial learning gains have been achieved despite messy top-level politics. But the Chilean, Indian, and South African case studies suggest that the all-too-common result of rule-boundedness plus unresolved political contestation over the education sector’s goals is some combination of exaggerated rule compliance and/or performative isomorphic mimicry. - Personalised competitive contexts (Bangladesh, Ghana, and Kenya for example) lack the seeming strengths of either their dominant or their impersonal competitive contexts; there are multiple politically-influential groups and multiple, competing goals—but no credible framework of rules to bring coherence either to political competition or to the education bureaucracy. The case studies show that political and institutional constraints can render ineffective many specialised sectoral interventions intended to improve learning outcomes. But they also point to the possibility that ‘soft governance’ entry points might open up some context-aligned opportunities for improving learning outcomes. In dominant contexts, the focus might usefully be on trying to influence the goals and strategies of top-level leadership. In impersonal competitive contexts, it might be on strengthening alliances between mission-oriented public officials and other developmentally-oriented stakeholders. In personalised competitive contexts, gains are more likely to come from the bottom-up—via a combination of local-level initiatives plus a broader effort to inculcate a shared sense among a country’s citizenry of ‘all for education’.

Development Delusions and Contradictions: An Anatomy of the Foreign Aid Industry - Sims, David (Palgrave Macmillan, 2022)
This book analyses the shortcomings of the Western development aid programme. Through exploring the evolution of aid over more than seven decades, development is examined as an industry with a variety of motives and actors. The driving forces and dynamics in the relationship between aid and economic development are highlighted in relation to faulty development structures and misaligned aims. With a particular focus on Egypt, radical questions are posed on how global aid and development can be improved, including how it can respond to the COVID-19 pandemic.This book aims to present an alternative aid framework to help overcome the dysfunctionality of the current international development system. It will be of interest to researchers and policymakers working within development economics and development policy.

Creative Activities for Work Teams and Communities of Practice. A guide for designing participatory, productive, and engaging sessions - AKF (2023)
This free guide aims to build the skills and confidence of people designing and facilitating participatory, productive, and engaging workshop sessions for work teams and communities of practice by providing them with a series of creative activities that can be run in-person or virtually. By the time they have gone through this guide, session leads will be able to: Describe ways in which creative activities can be of value to teams and communities of practice, Design both in-person and virtual workshop sessions that incorporate creative activities, Facilitate participatory, productive and engaging workshop sessions that incorporate one or several creative activities, and Lead an effective debrief following a creative activity. This toolkit was developed by the Aga Khan Foundation.

Virtual Facilitation Techniques for Work Teams and Communities of Practice. A guide for designing and running dynamic and engaging remote sessions and meetings - AKF (2023)
This free guide aims to build the skills and confidence of people designing and facilitating virtual meetings, including online community of practice sessions. By the time they have gone through this guide, community of practice facilitators will be able to: Design an engaging and participatory virtual session, Develop an effective, varied and realistic agenda for a virtual session, Apply a wide range of virtual facilitation techniques, and Using technology to facilitate virtual sessions. This toolkit was developed by the Aga Khan Foundation.

Si Guides on System Innovation - Colchester, Joss; Si Network (Si Network, 2023)
So you have just hear about systems thinking and systems innovation and are keen to know more, Ok sparky let's get started.We have structured all the content into four main areas that we think you ...

Setting new standards for better MEL. Lessons for grantees & funders - Colnar, Megan; Azevedo, Andrea; Tolmie, Courtney; Caddick, Hannah (Better Evaluation, 2023)
How can donors and grantees work together to create effective monitoring, evaluation, and learning (MEL) practices that drive field-wide transformation? The Open Society Foundation’s Fiscal Governance Program found success by focusing on six key approaches, including empowering grantees and relinquishing power. In 2021, an external close-out evaluation by Intention to Impact of the program (which ran for 7 years and gave over $150 million in grants) revealed something pretty remarkable—the program’s deliberate focus on strengthening field-wide monitoring, evaluation, and learning practices was a success. Substantial capacity increases were observed across key institutions and grantees, new complexity-sensitive practices and methods were being actively championed and deployed, and a growing community of better-connected practitioners were exchanging tips and tricks on how to apply smart, context-specific MEL across fiscal governance issues. What’s more, in this evaluation, most grantees gave high praise to these efforts. So, how did this come about? We detail the six different approaches we used in our new publication Setting new standards for better MEL: Lessons for funders and grantees. The approaches range from checking power dynamics to growing skills for evaluative thinking and seeding peer learning and field-wide research. The publication is paired with a toolkit and showcases resources we used and iterated on across the various approaches.

Two decades of public sector innovation: building an analytical framework from a systematic literature review of types, strategies, conditions, and results - Criado, J. Ignacio; Alcaide-Muñoz, Laura; Liarte, Irene (Public Management Review, 2023)
Scholarly attention in innovation in the public sector is growing rapidly, provoking analytical complexity. We developed a systematic literature review about Public Sector Innovation (PSI), analysing 169 articles published between 2001 and 2021, using PRISMA. We present a comprehensive approach to PSI testing and empirically develop an analytical framework based on the most common combinations of the studied dimensions. Additionally, we propose three main research avenues for the future of PSI: (1) studying PSI in different contexts, (2) expanding the analysis of configurations in PSI initiatives, and (3) analysing ambidextrous strategies to support the practical implementation of PSI.

Hacking by the prompt: Innovative ways to utilize ChatGPT for evaluators - Ferretti, Silva (New Directions for Evaluation, 2023)
“Hacking by the prompt”—writing simple yet creative conversational instructions in ChatGPT's message window—revealed many valuable additions to the evaluator's toolbox for all stages of the evaluation process. This includes the production of terms of reference and proposals for the dissemination of final reports. ChatGPT does not come with an instruction book, so evaluators must experiment creatively to understand its potential. The surprising performance of ChatGPT leads to the question: will it eventually substitute for evaluators? By describing ChatGPT through four personality characteristics (pedantic, “I know it all,” meek, and “speech virtuoso”), this article provides case examples of the potential and pitfall of ChatGPT in transforming evaluation practice. Anthropomorphizing ChatGPT is debatable, but the result is clear: tongue-in-cheek personality characteristics helped hack ChatGPT more creatively while remaining aware of its challenges. This article combines practical ideas with deeper reflection on evaluation. It concludes that ChatGPT can substitute for evaluators when evaluations mostly focus on paperwork and conventional approaches “by the book” (an unfortunate trend in the sector). ChatGPT cannot substitute engagement with reality and critical thinking. Will ChatGPT then be a stimulus to rediscover the humanity and the reality we lost in processes?

Designing & Facilitating Collaborative Learning Networks. A toolkit - Folsom, Amanda (Results for Development, 2023)
Collaborative Learning — an approach which brings together people who face common challenges to share knowledge and jointly problem-solve — is a powerful way to support locally-led development and achieve impact. It provides a structured process in which change agents determine priorities, set the learning agenda, work together to identify strategies to address complex challenges, and provide ongoing implementation support to one another. Collaborative Learning — unlike traditional approaches to technical assistance — centers the expertise of local change agents and captures the valuable tacit knowledge of practitioners to advance systems change. This toolkit synthesizes lessons, tips, and tools accumulated from more than a decade of experience designing and facilitating over 20 Collaborative Learning Networks. The lead author was Amanda Folsom, Senior Program Director and Collaborative Learning Practice Lead, Results for Development (R4D). The toolkit benefited from the contributions of many R4D colleagues, including Katie Bowman, Cheryl Cashin, Tanya Jones, Gina Lagomarsino, Agnes Munyua, Maria Jose Pastor, and Abeba Taddese. R4D has not done this alone. The lessons and examples are drawn from our work with many network partners from the Joint Learning Network for Universal Health Coverage (JLN), Linked Immunisation Action Network (Linked), Health Systems Strengthening Accelerator, Strategic Purchasing Africa Resource Center (SPARC), the Primary Health Care Performance Initiative (PHCPI), the Partnership for Evidence and Equity in Responsive Social Systems (PEERSS), and the School Action Learning Exchange (SALEX), to name a few. We are grateful for the thought partnership and support of our partners and funders, including the Bill & Melinda Gates Foundation, Gavi, Hewlett Foundation, Jacobs Foundation, Rockefeller Foundation, USAID, and World Bank.

The Art of Scaling Deep - Research in Summary - Fraser, Tatiana (The Systems Sanctuary, 2023)
Over the last 15 years the concept of scale has become a foundational part of the apparatus of the social and environmental change sector. A business mindset of growth has been seamlessly transferred to the social and environmental problems we are collectively trying to shift in the world. Scaling up, (influencing policy) has been considered the strategic pathway to systems change. Scaling out (spreading new models) is seen as a pathway to success. The allure of these scaling theories lies, in part, in their tangibility, and the easy way in which they can be measured. However this focus on growth has shifted our attention away from a series of messy truths. Sometimes bigger isn't better. Endless growth is not sustainable and our urgency to try to fix the problem and seek solutions may be part of the crisis we are in. One unintended consequence of this has been that another type of scale has been devalued and as a result, under-resourced. It’s a scale that values the slow steady work of deepening relationships. It recognizes the significance of context, building connections that bridge diverse communities and it prioritizes inner work and healing as integral components of the scaling process. We call this type of scale ‘Scaling Deep’ and we believe that adequately supporting it, and funding it, holds the greatest potential for long lasting systemic change. The purpose of this research has been to delve deeper into the art and craft of Scaling Deep. Ultimately, our goal is for it to become firmly ingrained within the recognized realm of social change, alongside the well-established concepts of scaling up and scaling out. We want practitioners who are Scaling Deep to be able to harness the wisdom and power of this work and to talk about it openly, with confidence and credibility. We want understanding of this approach to flourish and evolve and for it to be appropriately celebrated and supported. We want decision-makers to be informed and inspired by the principles and practices of scaling deep. For it to be embedded in theories of change as an essential component of decision-making processes within the wider change discourse. Importantly we want leaders who are Scaling Deep to have access to sufficient resources and to receive the care from the field that they need to thrive. We would like to see organisations that have the power to invest, to align their efforts with the profound impact that scaling deep aspires to cultivate in the world. Before we begin, let us be crystal clear. We are not opposed to scaling up and out as strategies for change. Innovation and scaling what is working is part of how we evolve as humanity. As social entrepreneurs ourselves, we have both done this twice over. We value it and we know it’s important. We recognize these are strategies for creating widespread impact and effecting positive change. Our intention here is not to dismiss or undermine the value of scaling up and out, but rather to encourage a broader and more holistic perspective that includes other dimensions of scale. There is a need to understand how the different scaling approaches can work together, rather than seeing them as hierarchical and disconnected. Our ultimate goal is to equip the change sector with a more inclusive, expansive, and powerful approach to tackling the myriad challenges we face, by exploring the potential of Scaling Deep as a transformational strategy for systems change.

Building a team culture for Adaptive Management in MSD: 5 Strategies MEL Managers Say Work - Gover, Dun; Nasution, Zulka; Okutu, David; Bolder, Meghan; Henao, Lina (MSD in MEL Brief, 2023)
Keywords: Bolder Meghan; Gover Dun; Henao Lina; Nasution Zulka; Okutu David

Enhancing partner and system-level learning: 8 Tips from MEL Managers - Gover, Dun; Nasution, Zulka; Okutu, David; Bolder, Meghan; Henao, Lina (MSD in MEL Brief, 2023)
Effective learning is a key driver of market systems change, with the potential to enhance system competitiveness, resilience, and inclusiveness. Shifting the Locus of Learning: Catalyzing Private Sector Learning to Drive Systemic Change recently outlined a rationale for enhancing the scale and quality of learning in a system and identifying 10 strategies programs can contextualize to catalyze learning. These strategies are also backed with robust examples from 13 programs doing this work across 11 countries. To deepen insights on what MSD Monitoring, Evaluation and Learning (MEL) Managers have experienced in putting several of those strategies into practice, the Feed the Future Market Systems and Partnerships (MSP) Activity convened a series of peer discussions as part of a larger initiative (see Figure 1). This brief shares the collective learning and experience on this topic of three senior MEL Managers who were interested in and had experience with this topic. The group represented full-time, program-based MEL Leads working on MSD programs funded by USAID and DFAT, based in Fiji, Albania, and Kosovo working for Adam Smith International, SwissContact, and DT Global, respectively. From those discussions, this paper synthesizes eight tips from MEL Managers for practically enhancing partner and system-level learning: 1. Identify the right decision-maker(s) at potential partners. 2. Use a co-creation process to identify learning opportunities. 3. Use diagnostics and assessments to strengthen partner and system capacity for actionable learning. 4. Use a phased capacity strengthening process tied to behavior change. 5. Measure partners’ continued investment in and use of learning—not the continuation of specific learning activities. 6. Work with sector-level institutions for scale but be aware of risks. 7. Leverage informal communities of practice to share learning. 8. Use the right terminology to talk about partner and system-focused learning.

Practioners Guide to Assessing Systems Change - Gover, Dun; Nasution, Zulka; Okutu, David; Bolder, Meghan; Henao, Lina (MSD in MEL Brief, 2023)

The Dynamics of Team Learning: Harmony and Rhythm in Teamwork Arrangements for Innovation - Harvey, Jean-François; Cromwell, Johnathan R.; Johnson, Kevin J.; Edmondson, Amy C. (Administrative Science Quarterly, 2023)
Innovation teams must navigate inherent tensions between different learning activities to produce high levels of performance. Yet, we know little about how teams combine these activities—notably reflexive, experimental, vicarious, and contextual learning—most effectively over time. In this article, we integrate research on teamwork episodes with insights from music theory to develop a new theoretical perspective on team dynamics, which explains how team activities can produce harmony, dissonance, or rhythm in teamwork arrangements that lead to either positive or negative effects on overall performance. We first tested our theory in a field study using longitudinal data from 102 innovation teams at a Fortune Global 500 company; then, we replicated and elaborated our theory in a study of 61 MBA project teams at an elite North American university. Results show that some learning activities can occur within the same teamwork episode to have harmonious positive effects on team performance, while other activities combine to have dissonant negative effects when occurring in the same episode. We argue that dissonant activities must be spread across teamwork episodes to help teams achieve a positive rhythm of team learning over time. Our findings contribute to theory on team dynamics, team learning, and ambidexterity.

Digitally-induced change in the public sector: a systematic review and research agenda - Haug, Nathalie; Dan, Sorin; Mergel, Ines (Public Management Review, 2023)
Digital transformation has become a buzzword that is permeating multiple fields, including public administration and management. However, it is unclear what is transformational and how incremental and transformational change processes are linked. Using the PRISMA method, we conduct a systematic literature review to structure this growing body of evidence. We identified 164 studies on digitally-induced change and provide evidence for their drivers, implementation processes, and outcomes. We derive a theoretical framework that shows which incremental changes happen in public administrations that are implementing digital technologies and what their cumulative, transformative effects are on society as a whole.

Large language model applications for evaluation: Opportunities and ethical implications - Head, Cari Beth; Jasper, Paul; McConnachie, Matthew; Raftree, Linda; Higdon, Grace (New Directions for Evaluation, 2023)
Large language models (LLMs) are a type of generative artificial intelligence (AI) designed to produce text-based content. LLMs use deep learning techniques and massively large data sets to understand, summarize, generate, and predict new text. LLMs caught the public eye in early 2023 when ChatGPT (the first consumer facing LLM) was released. LLM technologies are driven by recent advances in deep-learning AI techniques, where language models are trained on extremely large text data from the internet and then re-used for downstream tasks with limited fine-tuning required. They offer exciting opportunities for evaluators to automate and accelerate time-consuming tasks involving text analytics and text generation. We estimate that over two-thirds of evaluation tasks will be affected by LLMs in the next 5 years. Use-case examples include summarizing text data, extracting key information from text, analyzing and classifying text content, writing text, and translation. Despite the advances, the technologies pose significant challenges and risks. Because LLM technologies are generally trained on text from the internet, they tend to perpetuate biases (racism, sexism, ethnocentrism, and more) and exclusion of non-majority languages. Current tools like ChatGPT have not been specifically developed for monitoring, evaluation, research, and learning (MERL) purposes, possibly limiting their accuracy and usefulness for evaluation. In addition, technical limitations and challenges with bias can lead to real world harm. To overcome these technical challenges and ethical risks, the evaluation community will need to work collaboratively with the data science community to co-develop tools and processes and to ensure the application of quality and ethical standards.

Assessing Value for Money: the Oxford Policy Management Approach - King, Julian; Daniel Wate; Esther Namukasa; Alex Hurrell; Frances Hansford; Patrick Ward; Shiva Faramarzifar (Oxford Policy Management, 2023)
This document offers practical guidance for assessing the Value for Money (VfM) of government- and donor-financed programmes and policy interventions. In line with OPM’s focus and mission, it has been predominantly applied in the international development sector, but the approach upon which it is based is also used in the context of domestic public policy and programmes.1 There is increasing scrutiny on VfM in international development, but a lack of appropriate methods to support its assessment. There is a risk of reaching invalid conclusions if VfM evaluation is tied to a narrow set of indicators devoid of any evaluative judgement—for example, by emphasising the most readily quantifiable measures rather than the most important (but harder to quantify) aspects of performance, or by focusing on the quantification of outputs and outcomes at the expense of more nuanced consideration of their quality, value, and importance. The approach presented in this guide combines theory and practice from evaluation and economics to respond to requirements for accountability and good resource allocation, as well as to support reflection, learning, and adaptive management. It involves developing and implementing a framework for: • organising evidence of performance and VfM; • interpreting the evidence on an agreed basis; and • presenting a clear and robust performance story. This guide sets out a framework for making and presenting judgements in a way that opens both the reasoning process and the evidence to scrutiny. The approach is designed to be used in alignment with broader monitoring, evaluation, and learning (MEL) systems—both for efficiency's sake, and to ensure conceptual coherence between VfM evaluation and wider MEL work. The VfM framework achieves these aims by: • using explicit criteria (aspects of performance) and standards (levels of performance) to provide a transparent basis for making sound judgements about performance and VfM; • combining quantitative and qualitative forms of evidence to support a richer and more nuanced understanding than can be gained from the use of indicators alone; • accommodating economic evaluation (where feasible and appropriate) without limiting the analysis to economic methods and metrics alone; and • incorporating and building on an approach to VfM evaluation which is familiar to international aid donors.

Adaptive Programming and going with the grain: IMAGINE's new water governance model in Goma, DRC - Kirk, Tom; Green, Duncan; Stys, Patrycja; Mosquera, Tom (Development Policy Review, 2023)
Motivation This paper explores adaptive approaches to development programmes that aim at improving service provision in underperforming sectors in fragile and conflict affected states (FCAS). It does this through a case study of the Integrated Maji Infrastructure and Governance Initiative for eastern Democratic Republic of the Congo's (IMAGINE) public-private partnership model for water provision. Purpose The processes and decisions that culminated in IMAGINE's model emphasize the need for programming that is culturally and politically aware, responsive to events, learns in real-time, is entrepreneurial, and works with the grain of local institutions to support change. Detailed case studies of such ways of working are crucial for programmes that seek to challenge and reform the status quo in FCAS. Methods and approach The paper is based on 42 semi-structured interviews conducted in the summers of 2019 and 2020. They reflect the broad spectrum of actors – individuals, public authorities, and organisations – involved in IMAGINE's evolution. Findings The narrative focuses on IMAGINE's attempts to professionalise and commercialise Goma's water sector. It shows how as IMAGINE repeatedly adapted to ground realities, it took on the characteristics of a public authority, thereby, engendering backlashes that threated its longer-term goals. However, by revisiting its initial values and logics it was able to get things done and achieve it aims. Policy implications IMAGINE's story suggests that adaptive programmes should put politically savvy local development professionals in key positions and enable them to carefully construct coalitions of allies across the systems they aim to disrupt. This may also require them to revisit and adapt their initial ideas, guiding principles and values as greater understandings of development problems are gained. A pubic authorities lens, attuned to the logics programmes seek to address and introduce to FCAS, may help analysts to foreground the implications of such adaptations.

Laudes Rubrics - Measurement and Learning Approach - Laudes Foundation (Laudes Foundation, 2023)
How can we measure and learn when promoting systems change? It’s a challenge that has inspired Laudes Foundation to develop a rubrics-based methodology to help us, our partners and the wider field of philanthropy, understand our contribution to change, while learning and adapting to new and unforeseen circumstances. Change cannot be captured by numbers alone because metrics put the focus on what can be counted, not always what’s most important. Rubrics are a framework that set a standard for what ‘good’ looks like – and create a shared language for describing and assessing it using both quantitative and qualitative evidence. At Laudes Foundation, the rubrics are integrated into our grantmaking processes – from the design phase through to measurement, evaluation and learning. The four Laudes Foundation rubrics categories Laudes Foundation has developed 21 rubrics that work across different levels, from processes to long-term impact. When measuring a specific initiative, a smaller set of relevant rubrics are chosen and assessed on a rating scale from ‘harmful’ to ‘thrivable’. The 21 rubrics are categorised into four groups, with some natural overlap between categories B, C and D. Category A focuses on the process-related aspects of initiatives, including design, implementation, monitoring, communication and learning, and organisation and network effectiveness. Category B focuses on the early and later changes that need to happen to create the right conditions to achieve the 2025 outcomes. Category C captures the 2025 outcomes, focusing on how policymakers, financiers, business leaders, and workers and producers behave. Category D captures the 2030 impacts, describing the new reality created as a result of sustained efforts.

How to Sharpen a Nonviolent Movement - McClennen, Sophia; Popovic, Srdja; Wright, Joseph (Journal of Democracy, 2023)
In the past three decades nonviolent social protest has become the most reliable path to democracy. However, not all nonviolent mobilization campaigns succeed. To examine why some nonviolent campaigns are more successful than others, we analyze the use of a particular type of activist campaign tactic, the "dilemma action." The dilemma action is a nonviolent civil-disobedience tactic that provokes a "response dilemma" for the target. Collecting original data on dilemma actions during nonviolent activist campaigns, we find that roughly one-third of mass nonviolent campaigns in the past century deploy this strategy. We theorize four mechanisms linking dilemma actions to nonviolent activist campaign success: facilitating group formation, delegitimizing opponents, reducing fear, and generating sympathetic media coverage. Finally, we assess whether dilemma actions increase campaign success rates, finding that dilemma actions are associated with an increase of 11–16 percent in activist-campaign success.

On the Right Tack: Reflections on Coalition-Building Initiatives across The Asia Foundation - Nixon, Nicola; Yates, Peter; Saluja, Sumaya; Yi, Su Lae; Lucas, Miranda; Bain, Katherine (The Asia Foundation, 2023)
Coalitions—groups of organizations and individuals that work together to pursue a common policy goal or reform—are crucial to development. Some of The Asia Foundation’s longest-standing and most successful development programs and portfolios have used coalition-building as an implementation modality. This paper examines successful initiatives in Bangladesh, Nepal, Philippines, Sri Lanka, Thailand, and Timor-Leste. By delving into each of these, we shed light on this coalition-building modality, sharing insights about how externally supported coalitions emerge and function and contributing ideas about how such support can be adapted to other contexts. We hope that this paper and these case studies contribute to innovations in contemporary development theory and help practitioners use and move beyond some formulaic and technocratic modalities, such as traditional capacity development, knowledge transfer, and technical assistance. People, relationships, and working partnerships can then take center stage, and sustained cooperation will gain primacy over short-term outputs and deliverables.

Developing a Systems Thinking Lens for Collective Leadership - O’Donnell, Joan (Collective Leadership for Scotland, 2023)
Systems thinking approaches are gaining traction as an effective way of understanding and working with increasing complexity. They are being put forward by the Organisation for Economic Co-operation and Development as well as the World Health Organisation as a way to tackle the complex and unpredictable environments we now operate in. As the world has become increasingly interconnected, national or local boundaries cannot isolate and control social problems. The climate emergency, war and political instability have become everyday realities that accompany an ever-widening gap between those who have and control resources, and those who do not have enough to meet basic needs. The usual responses are becoming more and more ineffective; all interventions have unseen consequences or emergent properties that cannot be predicted using an inputoutput outcomes model. Designing interventions therefore requires a fresh lens to manage our way through turbulence and uncertainty. Most complex situations benefit from a complementary mix of a systematic (linear) decisive intervention as much as a systemic (whole-view) understanding of a proposed intervention. In this sense, adopting a systems thinking approach requires both framing the situation using a systems lens, as well as a commitment to take action and learn forward in realtime. When utilised in this way, a systems thinking lens complements rather than replaces traditional management tools. It draws on well-tested concepts and tools and relies on the systems leader to develop their practice, which includes the art of knowing what is needed and when. It is about threading traditional planning methods together with an understanding of the interrelationships, multiple perspectives and boundary judgements that influence the framing of a situation. It is becoming increasingly clear that effective leadership is embedded and invested: objective management science has no place in a world bound by interconnectedness and unintended consequences. Leaders are, as we shall see, an inherent part of both the problem and the proposed intervention. The purpose of this paper is to share some concepts informed by systems thinking to support you, as leaders in your organisations, to bring a systems-informed lens to your work. It puts you as a practitioner-leader at the centre of your own practice and encourages you to reflect critically on your positionality, as well as the lens you use to understand and intervene in complex issues. This paper has been developed on foot of training piloted with public service leaders in Scotland in the summer of 2022, and acts as an introduction to developing your systems literacy skills. While we are all born with a systems sensibility, many of us lose that sense of connectedness and interdependence with the world as we grow up. This paper acts as a prompt to bring a complexity informed approach to your work practice. It also contains some pointers for further resources and tools that may support your learning.

Re-institute website - RE!NSTITUTE (Re!nstitute, 2023)
We are a dynamic non-profit organization with a unique approach to managing change and catalyzing innovation to support global systems transformation. We know that to achieve equal, just, and safe communities, we must collectively challenge and change the systems that people rely on — especially those most vulnerable. Together, we must RE!BUILD a system's resilience and capacity to take on our biggest generational challenges. Justice, housing, health, and other systems in every country are often struggling to come together, across institutions and organizations, to unite under a common vision for change, and implement those changes in effective and innovative ways. Our work and methodology — the 100-Day Challenge — are designed to support these systems to achieve extraordinary results. Our approach is rooted in the philosophy that frontline staff, supported by leadership, can unleash innovation and achieve incredible results by creating new relationships, experimenting with fresh ideas from across their system, and building authentic engagement with those with lived experience.

Using AI to disrupt business as usual in small evaluation firms - Sabarre, Nina R.; Beckmann, Blake; Bhaskara, Sahiti; Doll, Kathleen (New Directions for Evaluation, 2023)
While many knowledge workers may fear that the rise of artificial intelligence (AI) will threaten their jobs, this article argues that small evaluation businesses should embrace AI tools to increase their value in the marketplace and remain relevant. In this article, consultants from a research, evaluation, and strategy firm, Intention 2 Impact, Inc., make a case for using AI tools to disrupt business as usual in evaluation from theoretical and practical perspectives. Theoretically, AI may be another example of technology that was initially feared but is now ubiquitous in society. Using concrete examples, the authors describe how businesses and evaluators have evolved to keep up with changes in supply and demand. Lastly, it is posited that embracing AI will save time for those working in small businesses, which can ultimately increase added value and profitability.

Propel - Strhive (2023)
Learn from experience. Together. In the complex landscape of international development, organisations need a way to learn from their experiences and build on what works. Propel is the software solution that revolutionises the way organisations capture, access, and reuse learnings to adapt, innovate, and create lasting change on a global scale. Let's stop reinventing the wheel and build on what works, together.

MSD Competency Framework - BEAM Exchange (BEAM Exchange, 2023)
MSD Competency Framework A catalogue of the full range of knowledge, skills and aptitudes found in high-performing teams that use the market systems approach. Useful for: Practitioners - identify personal training needs and continue your professional development Trainers - diversify your courses and refine your training / capacity-building offers Managers - strengthen your recruitment and induction processes MSD competencies Competency is a mix of knowledge, skills and attitudes demonstrated through concrete behaviours. This section explores the 17 competencies used by high-performing MSD teams. Arranged in three groups they relate to how we understand the world, make decisions and interact with others. Each competency is defined and has links to useful resources that explain or illustrate the knowledge or skills that feed it. Where available, sources of teaching and learning for that competency are included. Teaching and learning modes An exploration of the most common modes of teaching and learning that support practitioners to acquire knowledge, skills and attitudes. Some modes of learning are more relevant to certain types of competency - as indicated by the box colours. Each page has a summary of the teaching mode, guidance for team leaders and trainers, and examples of how to use this mode for developing specific competencies. Assessment modes Assessing competency is important for recruitment, performance appraisal and continued professional development These pages describe distinct approaches to evaluating individuals. They include a summary of the evaluation mode, guidance for assessors, and examples tailored to specific competencies.