Applying Evaluation Criteria Thoughtfully - OECD (OECD, 2021)
Relevance, coherence, effectiveness, efficiency, impact, and sustainability are widely used evaluation criteria, particularly in international development co-operation. They help to determine the merit or worth of various interventions, such as strategies, policies, programmes or projects. This guidance aims to help evaluators and others to better understand those criteria, and improve their use. It starts by describing what they are, and how they are meant to be used. Then the definitions and concepts underpinning each criterion are explained. Finally, examples provide the reader with concrete ideas for using them. The criteria were originally laid out in the early 2000s by the Network on Development Evaluation (EvalNet) of the OECD Development Assistance Committee (DAC). Although they have been widely used in evaluation, and beyond, this document is the first to comprehensively explore the concepts in detail, explain their intended use and offer practical guidance. It captures current thinking and best practice in evaluation, drawing on the inputs of internationally renowned evaluation experts from EvalNet and beyond.
Type: doi:https://doi.org/10.1787/543e84ed-en

20 years of Outcome Mapping - Evolving practices for transformative change - OM Learning Community (Outcome Mapping Learning Community, 2021)
To mark 20 years of Outcome Mapping, this paper presents the core principles and concepts that are foundational to using the approach. It also presents a set of guiding practices to support transformative change. The OMLC Stewards presented the paper at a special webinar on 29th April 2021 - see the link below for the recording.

Adaptive Learning Guide: A pathway to stronger collaboration, learning, and adapting - Ross, Joey; Karlage, James; Etheridge, James; Alade, Mayowa; Fifield, Jocelyn; Goodwin, Christian; Semrau, Katherine; Hirschhorn, Lisa (USAID MOMENTUM Knowledge Accelerator, 2021)
The purpose of this Adaptive Learning Guide is to provide MOMENTUM project teams with the information and resources to integrate adaptive learning into the design, implementation, and improvement of MNCH/FP/RH programs. This guide provides a conceptual introduction to adaptive learning using links to existing resources and real-world examples of how adaptive learning can drive continuous learning and improvement in project work. The guide is built upon three foundational assumptions: We work in dynamic, often unpredictable environments. Unexpected turns of events will occur. Progress is rarely, if ever, linear. Integrating the principles and practices of USAID’s Collaborating, Learning, and Adapting Toolkit into projects and initiatives requires designing for learning and adaptation. We intend the guide to serve as a “starting point” for interested individuals and teams to begin or strengthen the processes that support the integration of adaptive learning into project work.

Opportunities and challenges for DAC members in ‘adapting to context’ - Sharp, Samuel; Wild, Leni (2021)
Key Messages New principles for OECD DAC members on ‘Managing for Sustainable Development Results’ emphasise tailoring result management approaches to different contexts; balancing internal compliance with empowerment at ground level; and adapting implementation in the pursuit of long-term outcomes. However development organisations face numerous challenges in aligning with these principles in practice. Reporting and evidence collection processes do not consistently encourage adaptive practice, reflecting their orientation towards accountability over learning. Context analysis is common during programme design, but used less on an ongoing basis. Popular tools –such as logical frameworks and theories of change - are often intepreted in linear ways, not as ‘living documents’ that react and change over time. Organisations need to meaningfully empower staff to work adaptively, including examining incentives and cultures that can make staff more comfortable with traditional results management. Even when senior leadership is supportive of adaptive ways of working, they can lack a clear understanding of the resourcing required and appropriate governance and management processes. Development organisations and their partners have attempted to address these challenges through the use of different monitoring and evaluation tools and methods, changes to reporting frameworks and templates, and initiatives to create positive incentives and motivate staff, leadership and partners at different levels.
Keywords: _tablet

Peer reviews - Guidance for facilitators and participants - Wadley, Ian (Centre for Humanitarian Dialogue, 2021)
Essential points for practitioners and donors • Mediation offers a cost-effective and proven method for resolving armed conflict. Between 1985 and 2015, 75 per cent of armed conflicts in the world were resolved through agreement rather than by force. In most cases these processes will have involved third party facilitation or support. • Professional mediators understand the high stakes involved in their work to prevent, mitigate and resolve armed conflict. In addition, they and their financial supporters are increasingly required to demonstrate ‘value-for-money’ to ensure continued funding. • However, traditional monitoring and evaluation (M&E) methods are not well suited to this task, typically imposing artificially linear project models on a dynamic conflict situation, as well as compliance reporting that moves attention away from real value. • Traditional M&E methods tend to focus on documenting the past and generating vast amounts of data, rather than enabling timely adaptation of the project in the present. • Traditional M&E approaches rely heavily on external evaluation consultants. Even in the best of cases this may interfere with the mediation process and impose a heavy time burden on the project team, leading to low acceptance of traditional M&E approaches by mediation practitioners. • In contrast, an ideal M&E approach for mediation should deliver useful insights in even the most dynamic and sensitive mediation environments, impose a light reporting burden, and be readily accepted by mediation teams. It should protect discretion and trust, enable rapid adaptation, and also provide some assurance that donor funds are being well spent.

Doing Development Differently - Wild, Leni (Stanford Social Innovation Review, 2021)
International aid must use different approaches to address the massive systemic problems it seeks to solve.
Keywords: ⛔ No DOI found

A Political Economy Analysis Framework for EdTech Evidence Uptake - Pellini, Arnaldo; Nicolai, Susan; Magee, Arran; Sharp, Sam; Wilson, Sam (The EdTech Hub, 2021)

Linking the prepositions: using power analysis to inform strategies for social action - Gaventa, John (Journal of Political Power, 2021)
This article reviews longstanding debates about the relationship between power over and power to – often posed as the tension between domination and emancipation. It then turns to several frameworks which integrate these approaches to inform strategies for social action. In particular, it focuses on recent empirical studies which apply one such framework, the ‘powercube’, to glean insights into how social actors navigate across multiple forms, spaces and levels of power. In so doing, we gain clues into how relatively powerless groups develop the capacities for agency and action which challenge domination and in turn give new possibilities for emancipation.
Keywords: Lukes; Power; agency; collective action; empowerment

How to make theories of change more useful? - Aston, Thomas (Medium, 2021)
A fair amount has been written recently questioning the value added of theories of change. Have we gone through a hype cycle? Are they…

Assumptions and triple loop learning - Aston, Thomas (Medium, 2021)
Triple loop learning

5 reflections on operationalising CLARISSA to generate evidence - Neupane, Sudarshan (CLARISSA, 2021)
As we start a new year, and especially, as 2021 is the International Year for the Elimination of Child Labour, understanding that the Worst Forms of Child Labour (WFCL) is complex is step one – there are no simple solutions. But we must go deeper than that, we must ensure that the programmes designed to intervene, and shape policy take on that complexity and precariousness. This is where the experience of the Child Labour: Action-Research-Innovation in South and South-Eastern Asia (CLARISSA) programme comes in. It is uniquely set up to generate evidence in innovative ways and to co-develop with our stakeholders’ solutions to the WFCL. This is ambitious, yes, but no one said it was going to be easy.

Adaptive Management: Learning and Action Approaches to Implementing Norms-shifting Interventions - McLarnon, Courtney; Gayles, Jennifer; Deepan, Prabu (USAID / Passages, 2021)
What Passages has Learned about Adaptive Management: • Be reflective about information that is collected and create a culture of learning. • Be systematic about establishing monitoring and learning systems. • Be strategic about data sources and analysis, prioritizing areas for learning and addressing issues raised. • Be inclusive about information collection: who is collecting what, how, and how is it being used.

USAID ADS 201 - Operational Policy for the Program Cycle (Update 01/23/2021) - USAID (USAID, 2021)
The Program Cycle is USAID’s operational model for planning, delivering, assessing, and adapting development programming in a given region or country to advance U.S. foreign policy. It encompasses guidance and procedures for: 1) Making strategic decisions at the regional or country level about programmatic areas of focus and associated resources; 2) Designing projects and supportive activities to implement strategic plans; and 3) Learning from performance monitoring, evaluations, and other relevant sources of information to make course corrections as needed and inform future programming.

Reframing, refining, and reconceptualising the worst forms of child labour through participatory adaptive programming - Apgar, Marina; Burns, Danny (Journal of modern slavery, 2021)
This article explores the potential of using participatory action research as an adaptive programming modality to drive learning and innovation to tackle the drivers of (and seek to eliminate) the Worst Forms of Child Labour. We draw on our experience from early phases of implementation of a large-scale action research programme, which despite the constraints covid-19 posed in moving to full implementation and participatory engagement with children and other stakeholders on the ground, is already generating rich learning about the opportunities and challenges of designing programmes that respond to the complex reality of WFCL. We share early learning about what it takes to be fully open to using the lived experience of programme development, and early findings from scoping and mapping of the dynamics of social norms, business practices and urban neighbourhoods and supply chains influencing WFCL in Bangladesh and Nepal, to frame and reframe the questions and response strategies and operationalise a participatory adaptive intent to work with hidden and complex dynamics that characterise the WFCL.

A Systems Framework for International Development: The Data-Layered Causal Loop Diagram - Blair, Courtney; Gralla, Erica; Wetmore, Finley; Goentzel, Jarrod; Peters, Megan (Production and Operations Management, 2021)
Meeting the United Nations Sustainable Development Goals (SDGs) will require adapting or redirecting a variety of very complex global and local human systems. It is essential that development scholars and practitioners have tools to understand the dynamics of these systems and the key drivers of their behavior, such as barriers to progress and leverage points for driving sustainable change. System dynamics tools are well suited to address this challenge, but they must first be adapted for the data-poor and fragmented environment of development work. Our key contribution is to extend the causal loop diagram (CLD) with a data layer that describes the status of and change in each variable based on available data. By testing dynamic hypotheses against the system's actual behavior, it enables analysis of a system's dynamics and behavioral drivers without simulation. The data-layered CLD was developed through a 4-year engagement with USAID/Uganda. Its contributions are illustrated through an application to agricultural financing in Uganda. Our analysis identified a lack of demand for agricultural loans as a major barrier to broadening agricultural financing, partially refuting an existing hypothesis that access to credit was the main constraint. Our work extends system dynamics theory to meet the challenges of this practice environment, enabling analysis of the complex dynamics that are crucial to achieving the SDGs.
_eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1111/poms.13492

Change: How to Make Big Things Happen - Centola, Damon (Little, Brown Spark, 2021)
How to create the change you want to see in the world using the paradigm-busting ideas in this "utterly fascinating" (Adam Grant) big-idea book.​ Most of what we know about how ideas spread comes from bestselling authors who give us a compelling picture of a world, in which "influencers" are king, "sticky" ideas "go viral," and good behavior is "nudged" forward. The problem is that the world they describe is a world where information spreads, but beliefs and behaviors stay the same. When it comes to lasting change in what we think or the way we live, the dynamics are beliefs and behaviors are not transmitted from person to person in the simple way that a virus is. The real story of social change is more complex. When we are exposed to a new idea, our social networks guide our responses in striking and surprising ways. Drawing on deep-yet-accessible research and fascinating examples from the spread of coronavirus to the success of the Black Lives Matter movement, the failure of Google+, and the rise of political polarization, Change presents groundbreaking and paradigm-shifting new science for understanding what drives change, and how we can change the world around us.

Pushing boundaries: Advocacy evaluation and real-time learning in an HIV prevention research advocacy coalition in sub-Saharan Africa - Dasmariñas, Jules; Schlangen, Rhonda (New Directions for Evaluation, 2021)
Evaluation processes that facilitate learning among advocates must be nimble, creative, and meaningful while transcending putative performance and accountability management. This article describes the experience, lessons, and trajectory of one such approach, Simple, Participatory Assessment of Real Change (SPARC), that a transnational HIV prevention research advocacy coalition pilot-tested in sub-Saharan Africa. Inspired by the pioneering work of the outcome harvesting (OH) and participatory evaluation community, we recuperate advocates' centrality as storytellers, sense-makers, and strategists in advocacy evaluation and describe how we recalibrated SPARC to meet their evaluation and learning needs. This article highlights the normative value of deliberative discourse in evaluation as it contributes to the interpretation of OH and the enrichment of the theory and practice of advocacy evaluation.
_eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1002/ev.20474

Engage. Empower. Enact. - Citizen Engagement & Democratic Innovation Programme White Paper - Doyle, Linda; Smith, Bethan (Cognitive Edge, 2021)
The Cynefin Centre’s Citizen Engagement & Democratic Innovation programme provides tools for collective sense-making in the areas of community development and youth work; civic engagement and democratic innovation; collaborative service/policy design and evaluation; housing/tenant engagement; futures and planning; shared learning and peer to peer knowledge exchange.

Shifting the Evaluation Paradigm: The Equitable Evaluation Framework - EEI (Equitable Evaluation Inititiative, 2021)
This publication provides an overview of the impetus for the Equitable Evaluation Framework™ (EEF) and attempts to document early moments and first steps of engagement with U.S. philanthropic institutions — most often their research, evaluation and learning staff — whom we refer to as foundation partners throughout this publication. The themes shared in this publication surfaced through conversations with a group of foundation staff who have been part of the Equitable Evaluation Project, now referred to as the Equitable Evaluation Initiative (EEI), since 2017 as advisors, investment partners1 and/or practice partners. These are not case studies but insights and peeks behind the curtains of six foundation practice partners. It is our hope that, in reading their experiences, you will find something that resonates, be it a point of view, a mindset or a similar opportunity in your place of work. But as your colleagues share here, once you understand and see how narrowly these constructs have been defined, it cannot be unseen. The only option is to be, think and do evaluative work differently. In keeping with the EEI intention to shift the evaluation paradigm, this publication does not read in a linear fashion, nor does it follow “traditional” evaluation report form and flow. If we are to get someplace new and different, we must embrace new ways of reflecting what we are hearing, gleaning and sharing from those who are in the work. As you read along, notice commonalities in experience, perspective and positioning among the stories, and also consider the ways in which this conversation pushes your own thinking about how evaluative practice can serve its highest aim in U.S. philanthropy. They are thinking about how this influence manifests in all foundation operations and are wondering how others began to think about this reality in the context of evaluation practice. We went back and forth on naming names. We recognize the tendency to be either too specific or too general when telling stories about or from within institutional philanthropy. Because this work is both individual and organizational, we opted to keep names. We think it is important to understand who, with all their identities, took the early steps and were willing to share their experiences. Last, this is not a primer. While we reference concepts essential to the EEF, one must do the work to truly understand them. One might ask, what does it mean to “do the work”? There is a strong tendency for folks to be more aware of and reactive to how we think we should be engaging in evaluative practice based on how others are doing the work or how the work of evaluation has always been done. It is highly intellectualized and decontextualized. In the practice of the EEF, the work begins by acknowledging our own complicities in and contributions to the current paradigm, and accepting that we are the tool for change. The “work” begins with you, right now, where you are. Shifting the evaluation paradigm takes intentionality, discipline and practice. With that in mind, approach this publication with a learning mindset and open heart, and identify the ideas, perspectives and opportunities that resonate and compel you to pursue more practices aligned with the EEF at your organization.

Co-Creating Digital Public Services - Jarke, Juliane (Co-creating Digital Public Services for an Ageing Society: Evidence for User-centric Design, 2021)
This chapter reviews key literature and concepts relating to the co-creation of digital public services. For this task, it is firstly important to consider what kind of digital public services may be suitable for co-creation. In order to do so, the first section of this chapter defines what a digital public service is (e.g. with respect to different types of service providers, different types of services and service delivery) and considers what kind of digital public services allow for meaningful citizen participation. To better conceptualise different degrees of participation, the subsequent section reviews Arnstein’s (1969) “ladder of citizen participation” and related work. This allows distinguishing between different degrees of non-participation, (consultative) participation and beyond. Thirdly, the chapter reviews traditional participatory approaches that provide the basis to co-creating of digital public services: (1) co-production of public services, (2) co-design and (3) civic open data use. The chapter summarises and compares the different rationales for participation in these approaches, and reviews how they understand the sharing of control, the sharing of knowledge and the enabling of change.
Keywords: Civic Tech; Co-creation; Co-design; Co-production; Digital public services; Open data; Open government; Participation; Participatory design; User-centric services; e-services

Co-creating Digital Public Services for an Ageing Society: Evidence for User-centric Design - Jarke, Juliane (Springer International Publishing, 2021)
This open access book attends to the co-creation of digital public services for ageing societies. Increasingly public services are provided in digital form; their uptake however remains well below expectations. In particular, amongst older adults the need for public services is high, while at the same time the uptake of digital services is lower than the population average. One of the reasons is that many digital public services (or e-services) do not respond well to the life worlds, use contexts and use practices of its target audiences. This book argues that when older adults are involved in the process of identifying, conceptualising, and designing digital public services, these services become more relevant and meaningful.The book describes and compares three co-creation projects that were conducted in two European cities, Bremen and Zaragoza, as part of a larger EU-funded innovation project. The first part of the book traces the origins of co-creation to three distinct domains, in which co-creation has become an equally important approach with different understandings of what it is and entails: (1) the co-production of public services, (2) the co-design of information systems and (3) the civic use of open data. The second part of the book analyses how decisions about a co-creation project’s governance structure, its scope of action, its choice of methods, its alignment with strategic policies and its embedding in existing public information infrastructures impact on the process and its results. The final part of the book identifies key challenges to co-creation and provides a more general assessment of what co-creation may achieve, where the most promising areas of application may be and where it probably does not match with the contingent requirements of digital public services. Contributing to current discourses on digital citizenship in ageing societies and user-centric design, this book is useful for researchers and practitioners interested in co-creation, public sector innovation, open government, ageing and digital technologies, citizen engagement and civic participation in socio-technical innovation.

Don’t Build It: A Guide For Practitioners In Civic Tech / Tech For Development - Jordan, Luke (Grassroot and MIT Governance Lab, 2021)
If you just remember these... If you can avoid building it, don’t build it; if you have to build it, hire a CTO, ship early, and mature long; and no matter what, draw on a trusted crew, build lean and fast, and get close to and build with your users as soon as possible. — This guide aims to help you avoid bad projects, structure the team right, ship and learn quicker, and mature longer. The guide starts with project selection, including why the best project to select is no project at all. It moves on to team structure, and the extreme importance of a full-time senior tech lead (or chief technology officer (CTO), understood as an excellent engineering manager). It then covers timelines, emphasizing shipping early but having enormous patience getting to maturity, above all in finding product-use-fit, and avoiding vanity metrics. The guide then goes into some detail on hiring, covering the CTO role, senior contractors, designers and young engineers. The longest section, by some distance, is that on hiring. Hiring is the one thing considered critical in every piece of the lore, by founders and investors and managers alike, across all sectors. It is also the field in which I think I got it mostly right, and for reasons I can explain in ways that I believe will be helpful.

OutNav - Evaluate outcomes and impact - Matter of Focus (Matter of Focus, 2021)
Make better use of your data and information to learn, improve and tell an evidenced story of the difference your organisation is making.

Agile: A New Way of Governing - Mergel, Ines; Ganapati, Sukumar; Whitford, Andrew B. (Public Administration Review, 2021)
The evolving concept of “agile” has fundamentally changed core aspects of software design, project management, and business operations. The agile approach could also reshape government, public management, and governance in general. In this Viewpoint essay, the authors introduce the modern agile movement, reflect on how it can benefit public administrators, and describe several challenges that managers will face when they are expected to make their organizations more flexible and responsive.
_eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1111/puar.13202

Modelo HIP - Hexágono de la innovación pública - Oliván Cortés, Raúl (2021)

Online Learning and collaborative platform for systems thinking and systems change - SI (Systems Innovation, 2021)

Small Foundation Network Partner Evaluation Toolkit: Network Evaluation for Network Coordinators - Small Foundation (2021)
This Toolkit This Toolkit presents an approach to network evaluation that is designed for network coordinators. This Toolkit provides guidance for network coordinators on how to: • More effectively use information they are already collecting as part of their routine coordination duties; • Collect other useful data that would support their network’s health and development; • Integrate network evaluation tasks into network activities, like convenings, to streamline the process; • Use data to effectively coordinate, grow, and sustain their networks. The Toolkit also provides examples of surveys, questionnaires, and dashboards, as well as tips for easy implementation.

Small Foundation Network Partner Evaluation Toolkit Network Evaluation for Network Coordinators Frequently Asked Questions - Small Foundation (Small Foundation, 2021)
This Frequently Asked Questions (FAQ) document is written for Small Foundation network partners. It accompanies Network Partner Evaluation Toolkit, which outlines four evaluation tools that Small Foundation is encouraging, and supporting, its network partners to use to improve their operations and, ultimately, increase their impact. This FAQ document answers typical questions about these evaluation tools and provides additional detailed recommendations for how network coordinators can effectively implement them. This document has five sections: 1) A brief introduction to network theory and the value of evaluation for network coordinators. 2) An overview of the four evaluation tools and answers general questions about incorporating evaluation into network coordination duties. 3) Recommendations regarding the Network Coordinator Administrative Information (tool #1). 4) Information about Social Network Analysis (tool #2). 5) Recommendations regarding the Network Participant Survey (tool #3). 6) Information about the Collaborative Activity Dashboard (tool #4).

Managing complexity (and chaos) in times of crisis. A field guide for decision makers inspired by the Cynefin framework - Snowden, Dave; Rancati, Alessandro (Publications Office of the European Union, 2021)
This field guide helps to navigate crises using the Cynefin framework as a compass. It proposes a four-stage approach through which we can: - assess the type of crisis and initiate a response; - adapt to the new pace and start building sensing networks to inform decisions; - repurpose existing structures and working methods to generate radical innovation; - transcend the crisis, formalise lessons learnt and increase resilience. The guide stresses the importance of setting and managing boundaries, building informal structures, keeping options open, distributing engagement and keeping an ongoing assessment of the evolving landscape. Action items, real life examples and demonstrations complement the references to the developing theoretical framework.

We’re Beating Systems Change to Death - Starr, Kevin (Stanford Social Innovation Review, 2021)
Systems change is more a destination than a journey. “Scalable solutions” might be a better way to make the trip.

Uniting Against Corruption - A Playbook on Anti-Corruption Collective Action - UN Global Compact (UN Global Compact, 2021)
The 2021 guide provides an easy-to-follow six-step approach on how to develop, implement, and sustain a CA, with respect to the reader’s local corruption landscape and potential stakeholders. The adaptive framework proposed can be used to address corruption challenges, mitigate possible business risks, and achieve optimal results.

Thinking Tools Studio - Waters Center (Waters Center for Systems Thinking, 2021)
Tools to help you Think

What have we learned about learning? Unpacking the relationship between knowledge and organisational change in development agencies - Yanguas, Pablo (German Development Institute / Deutsches Institut für Entwicklungspolitik (DIE), 2021)
Development cooperation has spent decades wrangling over the merits, evidence, and implications of what we may term “the learning hypothesis”: the idea that increased knowledge by development organisations must logically lead to increased effectiveness in the performance of their development activities. Organisations of all stripes have built research and monitoring and evaluation (M&E) departments, adopted a multitude of knowledge management systems and tools, and tinkered with different ways to structure their organograms to stimulate knowledge sharing and learning. The topic of organisational learning is particularly significant as the global development community grapples with increasingly complex problems and the aspiration of evidence-based policymaking. This paper presents an analytical framework for interrogating “the learning hypothesis”, breaking it down into causal steps: knowledge causes learning, learning causes organisational change, change causes effectiveness. The framework focuses on the first two sub-hypotheses, mapping out the conceptual space around them by outlining potential relationships between different types of knowledge – tacit and explicit, internal and external – and between different types of learning – operational and strategic. This map provides a foundation for three key research questions: What impact has the rising knowledge agenda had on development organisations? Which factors appear to enable or inhibit organisational learning? What is the relationship between operational and strategic learning and organisational change? A review of available evaluations and studies, including two cases from former UK Department for International Development (DFID) and the World Bank, reveals that there is insufficient evidence to support the causal claim that knowledge leads to learning and thereby to organisational change in development agencies. Sources point to tacit learning prevailing while explicit knowledge management systems flounder, and external advocacy agendas appear more compelling than internal research and evaluation products. It is not entirely clear how, or indeed, whether operational and strategic learning intersect, with delivery-level lessons hardly aggregating into structural or policy shifts. Organisational change – even that aimed at enhancing learning – is rarely based on lessons learned from practice. More research is necessary to fully unpack the learning hypothesis, but what limited evidence is available disproves rather than confirms its central claim. This has significant implications for the future of learning in development agencies as advocated by thought leaders, researchers, and reformers. In particular, the latter should consider an evidence-based reassessment of the function and value-for-money of research and M&E in development practice, and a more critical examination of the politics of external advocacy efforts around innovative aid approaches like thinking and working politically, adaptive management, or results-based management.
Version Number: 1.0

🔮 Seven lessons from building with AI - Azhar, Azeem (2020)
Plus a stack of 40+ AI tools we use at Exponential View

🔮 Six mental models for working with AI - Azhar, Azeem (2020)
The operating manual we wish we'd had

“Real”​ process tracing: part 1 — context - Aston, Thomas (Medium, 2020)
When asserting the value of theory-based methods, you often here words like “black boxes” and “causal mechanisms.” These are commonly…

Rubrics as a harness for complexity - Aston, Thomas (Medium, 2020)
In this final blog in the series, I want to look at the potential value of rubrics. While evaluability assessments can help us to…

Windows on the world: The power of assumptions in uncertain times - Aston, Thomas (Medium, 2020)
In my last blog on theory-based Monitoring Evaluation and Learning (MEL), I explained why relationships matter, and how to assess change…

Intervention or Collaboration? Redesigning Information and Communication Technologies for Development. - Bon, Anna (2020)
How can we design and build digital technologies to support people in poor and low resource environments to achieve their objectives? And how can we do this inclusively and ethically, while considering the complexity of their living and working environments? This is the central question in my research. One of the grand challenges of international development cooperation is to make digital technologies available for social and economic development of poor regions of the world. To achieve this goal – often referred to as ICT4D – knowledge and technologies are transferred from wealthy countries to poor regions. Nevertheless, these efforts have often turned out unsuccessful and unsustainable, despite large budgets and numerous projects in prestigious international development programs. Mismatch between the transferred technologies and the target environment is a recurrent problem of ICT4D projects. Improvement can be achieved, for example, by involving end-users in the design process. International development organizations are aware of this, and terms like "co-creation", "participation" and "user-oriented design" have nowadays become part of the international development discourse. However, real co-creation and user-centered design are incompatible with unidirectional transfer of technologies and knowledge (this is how ICT4D is commonly organized, in conventional international development). Moreover, the term participation becomes meaningless, in the light of externally formulated development goals. One key question to ask is: what do the envisaged users want? Remarkably, many ICT4D projects, programs and policies do not really ask and (field) investigate this question, which can only be answered by extensive research on-the-ground. This thesis describes the search for and the design of an alternative approach to ICT4D. Ten years of field and action research with partners in Mali, Burkina Faso, and Ghana have led to a collaborative, iterative and adaptive approach, dubbed "ICT4D 3.0". What is novel of this alternative approach and how does it answer the central question? First of all, ICT4D 3.0 is a practical approach for critical investigation and action. It consists of a reconfigurable framework that guides the design and development of information systems, bridging the knowledge gap between developers and users to unlock and integrate different domains of (global, local, indigenous, academic, nonacademic) knowledge. It targets complex, resource-constrained environments where many (for the ICT developers and researchers) unfamiliar conditions or obstacles may exist. It fosters innovative capacity and learning in action, bringing together people with different backgrounds and perspectives in trans-disciplinary and multicultural teams. It is socio-technical, result-oriented, focused on the objectives of the stakeholders and the requirements of their livelihoods. This approach has been validated in various different contexts, by users, ICT developers, practitioners and students. Second, ICT4D 3.0 contributes to a theoretical understanding of ICT4D as a process of networked innovation in complex (adaptive) systems. The underlying idea is that knowledge sharing and diffusion of innovations are complex (non-linear) dynamic processes that evolve and propagate through social networks in rather unpredictable ways, whereby innovation works out differently, depending on context, and whereby contextual (e.g. social, cultural, environmental, political) factors play an important role, and have to be considered. This theoretical framework explains the effectiveness of a collaborative, iterative, adaptive approach in ICT4D. Third, ICT4D 3.0 is built on ethical principles. When reflecting on the meaning and purpose of digital development, it is clear that digital development is not only a question of technology and practice, and collaboration is more than a prerequisite for successful technological innovation and long-term sustainability: collaboration is a fundamental human, ethical value. Therefore, as a reflective practitioner, one has to ask oneself whose interests one is actually looking after, which goals one is trying to achieve, where they come from, how power and political issues play a role and which core values are at stake. This makes ICT4D 3.0 into a democratic process of dialogue and deliberation, in which all voices are heard, in which the local context and complexity are central and in which development goals are determined by the users themselves and not imposed from outside. In this light, the approach proposed in this thesis takes a value position and can be considered a decolonial approach, striving for democracy, emancipation, autonomy and social and economic betterment. Field experience shows that ICT4D can be a meaningful, collaborative, networked process of knowledge sharing, driven by local initiatives, realizing change for the better, in a complex world.

Reflections on if and how our partnership is working in Bangladesh - Paul, Sukanta; Snijder, Mieke; Apgar, Marina (CLARISSA, 2020)
Effective partnership working is crucial to producing quality results, especially when working with complex problems such as the worst forms...

Risk navigation for Thinking and Working Politically: The work and disappearance of Sombath Somphone - Sims, Kearrin (Development Policy Review, 2020)
Abstract Motivation On December 15, 2012 Sombath Somphone was abducted at a police checkpoint in his home city of Vientiane, the capital of Laos; his whereabouts remain unknown. This article considers his work and disappearance through the lens of Thinking and Working Politically (TWP) approaches to development. The article is supportive of TWP, but emphasizes the significant risks of politicized programming in authoritarian contexts. Purpose By examining the case of Sombath Somphone, the article seeks to offer insights for safer, and more effective, TWP programming. It considers how specific events in authoritarian contexts can suddenly reposition development workers and/or organizations as political dissidents. Approaches and Methods The argument draws on analysis of grey literature; conversational and observational knowledge accrued during 18 months of fieldwork in Laos between 2011 and 2018; on‐going formal and informal interviews with members of Laos’ civil society sector; and extensive dialogue with Sombath Somphone’s wife, Ng Shui Meng. Findings The article identifies four key factors that contributed to the enforced disappearance of Sombath Somphone: international exposure; timing; particular elites; and strategies of oppression. It highlights the need for further consideration of how to anticipate and mitigate the dangers of politically oriented development work, as well as the different forms of risk experienced by local and international development actors working in authoritarian contexts. Policy Implications TWP has much to offer to development practice, but its contributions should not threaten the safety of local development actors. More attention must be given to preventing and mitigating such risks.

Nuru Kenya transition from Nuru International, Kenya - Dillan, Haley; Ouma, Joel; Yamron, David (What Transformation Takes - Evidence of Responsible INGO Transitions to Locally Led Development Around the World, 2020)
This case study is an example of a phased transfer of ownership and responsibility from INGO Nuru International to Nuru Kenya, including the exit of all international staff. Post-transition, Nuru Kenya is managed entirely by Kenyan staff, although it continues to receive financial support from Nuru International. A lot of the elements described are aligned with Adaptive Management ways of working.

Instituciones que aprenden - Informe para la XXVII Cumbre Iberoamericana de Jefes y Jefas de Estado y de Gobierno - Oliván Cortés, Raúl (XXVII Cumbre Iberoamericana, 2020)
La pandemia de la Covid-19 ha brotado súbitamente en un momento de transición para las instituciones y organizaciones sociales de medio mundo. Cuando aún no se había superado la crisis de confianza de la última década entre los ciudadanos y los gobiernos, en un momento de gran impulso de las iniciativas de gobierno abierto, planes de innovación y transformación digital, para hacer más democráticas y eficientes las administraciones a través de programas políticos encaminados a configurar un nuevo contrato social, verde y digital en el marco de la Agenda 2030; el coronavirus ha provocado un shock histórico alterando el decurso del siglo XXI y exigiendo una aceleración de todos los procesos en marcha. La complejidad y dimensión de la pandemia ha puesto de manifiesto la necesidad de arquitecturas institucionales más flexibles, ágiles y resilientes, que sean capaces de incorporar toda la energía civil para aprender de su talento y creatividad, dándole mayor protagonismo a la ciudadanía (activistas, emprendedores, tejido asociativo, academia, makers…) no solo en la toma de decisiones sino también en el diseño e implementación de las estrategias. Conforme se centrifugaba a millones de empleados a teletrabajar desde sus casas, se hacía más poderosa la idea de pensar las organizaciones como flujos más allá de los organigramas estáticos de lugares y personas. Transformar las jerarquías en redes, concebirlas como cuerpos sociales dinámicos, no solo amplía su radio de acción y su conectividad exterior, también reactiva sus fortalezas internas, aflora los liderazgos ocultos, multiplica el valor social producido y maximiza el uso eficiente de los recursos en una época de limitaciones. Los laboratorios de innovación pública, social o ciudadana, o laboratorios de gobierno, junto a otras formas de innovación abierta y diseño social, se reivindican como proyectos inspiradores de un cambio de paradigma: de las instituciones que ordenan a las instituciones que aprenden. Pensar las organizaciones bajo el prisma de la ciencia de redes y la ética de los rizomas –nodos, enlaces, hubs, comunidades…– nos debería permitir una aproximación a la compleja y escurridiza tarea de configurar los ecosistemas de innovación y creatividad en el ámbito de lo público y lo social. Este informe propone un modelo denominado Hexágono de la Innovación Pública (HIP) que promueve un cambio sistémico a través de seis vectores (OPEN_ abierto, TRANS_ transversal, FAST_ ágil, PROTO_ modelado, CO_ colaborativo y TEC_ tecnológico) basados en las propiedades de las redes y en el análisis de 105 metodologías que usan las agencias más innovadoras del mundo. Se incluyen una herramienta de autodiagnóstico y el HIP-SIM, una primera aproximación a un software abierto para visualizar, modelar y simular la creación de ecosistemas innovadores con el que queremos propiciar una comunidad y un debate internacional.

Instituições que aprendem - Relatório para a XXVII Cimeira Ibero-Americana de Chefes de Estado e de Governo - Oliván Cortés, Raúl (XXVII Cimeira Ibero-Americana, 2020)
A pandemia da Covid-19 surgiu de súbito num momento de transição para as instituições e organizações sociais de meio mundo. Antes ainda de se ter ultrapassado a crise de confiança da última década entre os cidadãos e os governos, num momento de grande impulso das iniciativas de governo aberto e de planos de inovação e transformação digital para tornar mais democráticas e eficientes as administrações através de programas políticos encaminhados para configurar um novo contrato social, verde e digital no quadro da Agenda 2030, o coronavírus provocou um choque histórico alterando o curso do século XXI e exigindo uma aceleração de todos os processos em andamento. A complexidade e dimensão da pandemia revelou a necessidade de arquiteturas institucionais mais flexíveis, ágeis e resilientes, capazes de incorporar toda a energia civil para aprenderem com o seu talento e criatividade, dando maior protagonismo à cidadania (ativistas, empreendedores, tecido associativo, academia, makers…) não só na tomada de decisões mas também na configuração e implementação das estratégias. À medida que se afastava do centro a milhões de empregados para teletrabalhar a partir de casa, tornou-se mais poderosa a ideia de pensar nas organizações como fluxos que excedem os organigramas estáticos de lugares e pessoas. Transformar as hierarquias em redes, concebê-las como corpos sociais dinâmicos, não só alarga o seu raio de ação e conectividade externa, mas também reativa as suas forças internas, aflora as lideranças ocultas, multiplica o valor social produzido e maximiza o uso eficiente dos recursos numa época de limitações. Os laboratórios de inovação pública, social ou cidadã, ou laboratórios de governo, a par de outras formas de inovação aberta e configuração social, reivindicam-se como projetos inspiradores de uma mudança de paradigma: de instituições que ordenam para instituições que aprendem. Pensar nas organizações sob a ótica da ciência das redes e da ética dos rizomas – nós, ligações, hubs, comunidades… – deverá permitir-nos uma aproximação à complexa e subtil tarefa de configurar os ecossistemas de inovação e criatividade no âmbito público e social. Este relatório propõe um modelo denominado Hexágono da Inovação Pública (HIP) que promove uma transformação sistémica através de seis vetores (OPEN_ aberto, TRANS_ transversal, FAST_ rápido, PROTO_ modelado, CO_ colaborativo e TEC_ tecnológico) baseados nas propriedades das redes e na análise de 105 metodologias usadas pelas agências mais inovadoras do mundo. Incluem-se uma ferramenta de autodiagnóstico e o HIP-SIM, uma primeira aproximação a um software aberto para visualizar, modelar e simular a criação de ecossistemas inovadores com o qual queremos favorecer a comunidade e o debate internacional.

Principles for managing in complexity - Proud, Emma (LearnAdapt, 2020)
Written by Toby Lowe and Shaheen Warren (Centre for Public Impact) and Sam Sharp (Overseas Development Institute), with input from Jamie…

Program Logic Foundations: Putting the Logic Back into Program Logic - Hawkins, Andrew J. (Journal of MultiDisciplinary Evaluation, 2020)
Abstract Background: Program logic is one of the most used tools by the public policy evaluator. There is, however, little explanation in the evaluation literature about the logical foundations of program logic or discussion of how it may be determined if a program is logical. This paper was born on a long journey that started with program logic and ended with the logic of evaluation. Consistent throughout was the idea that the discipline of program evaluation is a pragmatic one, concerned with applied social science and effective action in complex, adaptive systems. It gradually became the central claim of this paper that evidence-based policy requires sound reasoning more urgently than further development and testing of scientific theory. This was difficult to reconcile with the observation that much evaluation was conducted within a scientific paradigm, concerned with the development and testing of various types of theory. Purpose: This paper demonstrates the benefits of considering the core essence of a program to be a proposition about the value of a course of action. This contrasts with a research-based paradigm in which programs are considered to be a type of theory, and in which experimental and theory-driven evaluations are conducted. Experimental approaches focus on internal validity of knowledge claims about programs and on discovering stable cause and effect relationships—or, colloquially, ‘what works?’. Theory-driven approaches tend to focus on external validity and in the case of the realist approach, the search for transfactual causal mechanisms—extending the ‘what works’ mantra to include ‘for whom and in what circumstances’. On both approaches, evaluation aspires to be a scientific pursuit for obtaining knowledge of general laws of phenomena, or in the case of realists, replicable context-mechanism-outcome configurations. This paper presents and seeks to justify an approach rooted in logic, and that supports anyone to engage in a reasonable and democratic deliberation about the value of a course of action. It is consistent with systems thinking, complexity and the associated limits to certainty for determining the value of a proposed, or actual, course of action in the social world. It suggests that evaluation should learn from the past and have an eye toward the future, but that it would be most beneficial if concerned with evaluating in the present, in addressing the question ‘is this a good idea here and now? Findings: In seeking foundations of program logic, this paper exposes roots that extend far deeper than the post-enlightenment, positivist and post-positivist social science search for stable cause and effect relationships. These roots lie in the 4th century BCE with Aristotle’s ‘enthymeme’. The exploration leads to conclusions about the need for a greater focus on logic and reasoning in the design and evaluation of programs and interventions for the public good. Science and research are shown to play a crucial role in providing reasons or warrants to support a claim about the value of a course of action; however, one subordinate to the alpha-discipline of logical evaluation and decision making that must consider what is feasible given the context, capability and capacity available, not to mention values and ethics. Program Design Logic (PDL) is presented as an accessible and incremental innovation that may be used to determine if a program makes sense ‘on paper’ in the design stage as well as ‘in reality’ during delivery. It is based on a configurationalist theory of causality and the concepts of ‘necessary’ and ‘sufficient’ conditions. It is intended to guide deliberation and decision making across the life cycle of any intervention intended for the public good.

Development and radical uncertainty - Feinstein, Osvaldo (Development in Practice, 2020)
Development strategies, programmes and projects are designed making assumptions concerning several variables such as future prices of outputs and inputs, exchange rates and productivity growth. However, knowledge about the future is limited. Uncertainty prevails. The usual approach to deal with uncertainty is to reduce it to risk. Uncertainty is perceived as a negative factor that should and can be eliminated. This article presents an alternative approach which recognises that radical uncertainty is irreducible to risk, identifying a positive dimension of uncertainty and showing its implications for development practice.
_eprint: https://doi.org/10.1080/09614524.2020.1763258

Discussing Adaptive Approaches for Development Programmes - Pett, Jamie (Medium - LearnAdapt, 2020)
What can adaptive approaches from other sectors teach development practitioners about dealing with uncertainty?

Agency in the face of path dependence: how organizations can regain scope for maneuver - Fortwengel, Johann; Keller, Arne (Business Research, 2020)
This paper tackles a key problem in path dependence research: how can locked-in organizations regain their scope for maneuver? Leveraging insights from two surprising and thus revelatory cases of organizations that have successfully escaped from path dependence, we develop the theoretical argument that regaining scope for maneuver can be achieved by interrupting the logic of a path’s underlying self-reinforcing mechanisms. More specifically, we argue that, through a targeted interruption of the working of these mechanisms, hyper-stable patterns inscribed in an organization can be gradually rewound—and alternative futures become possible. We position our paper within larger debates around the role of agency in path dependence theorizing, and we outline research frontiers to better understand the necessary antecedents of and exact relationship between mechanisms interruption and pattern unwinding.

Project facilitation as an active response to tensions in international development programmes - Lannon, John; Walsh, John N. (International Journal of Project Management, 2020)
This paper examines the tension within international development programmes between traditional task-oriented approaches to development and the wider view of programmes as sites for adaptability and learning. It characterises it as a set of inter-related tensions between recursive and adaptive tendencies that exist at individual actor, programme and institutional levels. Drawing on a multiple interpretive case study of partnership based programmes between an international non-government organisation and local partners in three countries, it looks at how these tensions play out in practice. Based on the findings, it proposes an active response to the tensions called project facilitation. This is an adaptive and co-created process that incorporates local experience and practice based knowledge to achieve strategic goals, while utilising recognised project management practices to achieve agreed outcomes. By adopting active responses to the tensions that exist within programmes it offers greater potential for effective delivery of long term benefits than the more typical defensive response strategies. Project facilitation is consistent with the social constructionist view of programmes and programme management but broadens our understanding by emphasising the need to actively consider how the tensions inherent in programmes are responded to.
Keywords: Adaptive management; International development; Non-governmental organisations; Partnership; Programme management; Project facilitation; Tensions

Diseño transformacional de proyectos - Kehrer, Daniel (GIZ, 2020)
There are many definitions of the term ‘transformation’ or ‘transformational change’. The first section of the report develops a basic understanding of transformations or transitions (used synonymously) viewed from various perspectives. In this, transformations are defined as processes that use disruptive innovations to change systems into fundamentally new systems that subsequently form the new mainstream. Section two describes existing approaches to environmental and climate finance in international cooperation and discusses them in light of the proposed definition. All of the approaches have the potential to be further refined and in that process often to increase the precision of what is understood by each type of transformation. The definitions already state which criteria and indicators are referred to and how relevant they are. There is wide diversity in the type of criteria and indicators used by the various organisations, and how they are classified, but certain common features can be identified and with the aid of the literature on transformations they can be combined to form a comprehensive framework. With this in mind, the derivation of quality criteria for transformative interventions is explained in section 3.1. Transformational change at one and the same time calls for big decisions and innumerable projects in a particular field of transformation; the projects cannot be planned on the drawing board but still should be coordinated with each other. Section 3.2 offers guidance on this. Section 3.3 argues in favour of focusing more closely on the ‘process promise’ and employing a more iterative and more adaptable commissioning procedure. Finally, section 3.4 introduces two types of indicators under the various criteria: design indicators, which measure the quality of interventions that are aimed at influencing transformations (process orientation), and outcome indicators, which measure the process and/or progress of a transformation itself.

Transformative project design - Kehrer, Daniel (GIZ, 2020)
There are many definitions of the term ‘transformation’ or ‘transformational change’. The first section of the report develops a basic understanding of transformations or transitions (used synonymously) viewed from various perspectives. In this, transformations are defined as processes that use disruptive innovations to change systems into fundamentally new systems that subsequently form the new mainstream. Section two describes existing approaches to environmental and climate finance in international cooperation and discusses them in light of the proposed definition. All of the approaches have the potential to be further refined and in that process often to increase the precision of what is understood by each type of transformation. The definitions already state which criteria and indicators are referred to and how relevant they are. There is wide diversity in the type of criteria and indicators used by the various organisations, and how they are classified, but certain common features can be identified and with the aid of the literature on transformations they can be combined to form a comprehensive framework. With this in mind, the derivation of quality criteria for transformative interventions is explained in section 3.1. Transformational change at one and the same time calls for big decisions and innumerable projects in a particular field of transformation; the projects cannot be planned on the drawing board but still should be coordinated with each other. Section 3.2 offers guidance on this. Section 3.3 argues in favour of focusing more closely on the ‘process promise’ and employing a more iterative and more adaptable commissioning procedure. Finally, section 3.4 introduces two types of indicators under the various criteria: design indicators, which measure the quality of interventions that are aimed at influencing transformations (process orientation), and outcome indicators, which measure the process and/or progress of a transformation itself.

Guidelines for Country Strategic Frameworks Programmes and Projects - Ministry of Foreign Affairs of Denmark (Government of Denmark, 2020)

Extending epistemology for programme evaluation – can After Action Reviews become spaces for critical reflection? - Apgar, Marina; Snijder, Mieke (Methodspace - Sage, 2020)
The term After Action Review (AAR) is becoming more common in the world of development evaluation, particularly in programmes that focus on evaluation as a formative learning process. As structured and facilitated learning moments, AARs take many shapes always built to support a specific team in a specific moment on their journey. Across such a diversity of practice how do we understand the evolution of the AAR as a method with an action research orientation? Being mindful always of the risk of instrumentalising and co-opting participatory methods, there is danger that AARs become an empty programme ritual, remaining at the surface and failing to achieve critical reflection and so falling short of their intention of pushing for deeper change in our practice. As the evaluation and learning team of two large complex, multi-partner projects using research to address development challenges, we have been applying the AAR method while adapting to virtual COVID working with partners across the world. As action researchers we have been thinking about what critical reflection means within a programme AAR process and whether and how they can open up a second person inquiry space.

Rapid Action Learning and COVID-19 - Chambers, Robert (Sanitation Learning Hub, 2020)
A-ha! A moment etched in my memory: 20 or so researchers were...

A Guide To Agile Project Management Methodology & Tools - Aston, Ben (The Digital Project Manager, 2020)
Looking to brush-up on agile? Here's your complete guide to agile project management, agile principles and key components, and the best agile tools for 2020.

Rules of Thumb – good idea or double-edged sword? - Green, Duncan (From Poverty to Power, 2020)
A recent conversation with a good governance programme in Myanmar tried to identify its underlying rules of thumb. Was that a good idea?

Linking constituent engagement and adaptive management - Buell, Stephanie; Campbell, Megan; Pett, Jamie (Overseas Development Institute, 2020)
Constituent engagement is the two-way process of involving constituents in the design, delivery, monitoring and evaluation of programmes. Constituent engagement and adaptive management together can be a powerful combination; high-quality constituent engagement can reinforce effective adaptive management, and vice versa. By highlighting stories from leading practitioners and their organisations, this paper explores how programmes ensure that constituent engagement informs meaningful adaptation. Key messages Constituent engagement and adaptive management are both important tools for implementing responsive and effective development programmes. Together, they can be a powerful combination: input from constituent engagement can be a key source of information and evidence that meaningfully informs programme design and adaptation, and closing the feedback loop in this way increases the quality of future engagement. Both adaptive management and community engagement principles recognise that, for a programme to be effective, it must be responsive to the people meant to ultimately benefit from it. Beyond providing a key source of information for potential programme adaptations, constituent engagement efforts also help build trust with stakeholders, align expectations and promote accountability. This paper explores five key elements for ensuring that constituent engagement and adaptive management are effectively linked within a programme: strong internal systems and external channels; skilled staff that value engagement and adaptation; decision-maker champions; clear points for reflection and action; and a meaningful role for constituents.

Adaptive management: A practical guide to mitigating uncertainty and advancing evidence-based programming - Byom, K.; Ingram, M.; Oakley, A.; Serpe, L. (PACT, 2020)
Pact’s Adaptive Management Guide provides practical guidance to development practitioners globally on the mindsets, behaviors, resources, and processes that underpin an effective adaptive management system. It presents an approach to managing adaptively that is rooted in complexity analysis and program theory. It draws on Pact’s global experiences and work on topics as diverse as health, livelihoods, markets, governance, capacity development, women and youth, and more. This document begins with an introduction to adaptive management, then walks through successive steps to determine how much adaptation a project requires and how to design an appropriate system. The second half of this guidebook contains a toolkit of examples and templates that projects can tailor to their needs.

Using middle-level theory to improve programme and evaluation design - Cartwright, N (CEDIL, 2020)
What can middle-level theory do? Middle-level theory (MLT) has several uses in development planning and evaluation. It helps predict whether a programme can be expected to work in a new setting. It offers insights into what design features are needed for success. It provides invaluable information for monitoring to see if the programme is on track and to fix problems that arise. It reveals the causal processes and related assumptions to be tested in an evaluation and helps identify evaluation questions. It helps in interpreting evaluation findings, assessing their relevance and locating a description of them that is useful for programme design and evaluation in other settings

Inspiring Radically Better Futures - Evidence and Hope for Impact at Scale in a Time of Crisis - Guijt, Irene; Mayne, Ruth (Oxfam, 2020)
The world faces converging crises of health, climate, gender and racial injustice and extreme economic inequality. The calls are mounting to ‘build back better’ to create more inclusive, caring and environmentally sustainable futures. But what evidence exists that this is possible? The Inspiring Better Futures case study series investigates whether radical change at scale is possible and how it was achieved. This paper synthesises 18 cases which show that people are already successfully building better futures, benefitting millions of people, even against the odds in some of the world’s toughest contexts in lower-income countries. Together they offer hope that transformative change and radically better futures after the pandemic are within reach.

Building Better Systems - An introduction to System Innovation - Winhall, Jennie; Leadbeater, Charles (The Rockwool Foundation, 2020)
This paper lays out a series of steps people can take to create the new systems we need to meet shared, public challenges. Systems are ubiquitous and powerful. We rely on them to support our daily lives: every time we turn on a tap, flick a switch for electricity, drop our child at school, jump on a bus or visit a doctor we rely on a wider system. There is a widespread sense, among decision makers and citizens that in the coming decades society will need not just new products, software and services, but new systems for living sustainably in a socially inclusive society. The need for better, different systems will be heightened by the impact and lessons of the Covid-19 pandemic. Systems are productive precisely because they are more than standalone products. A system pulls together all the different ingredients needed to meet a need or to produce an outcome: the shipping container is a product, containerisation is a system; a contactless payment card is a product which only works as part of a payments system; an operation in a hospital can only take place because it is part of a wider health system. To understand how a system works it has to be seen as a whole, from the macro policy frameworks of social security systems right down to how a citizen goes about finding a job. Many of the systems we rely on for care and work, energy and transport, education and health are under pressure to change. Society faces both deeply entrenched and growing challenges that are outpacing the systems we have. We also have opportunities to create new, alternative systems as new knowledge, values and technologies emerge, from artificial intelligence and bitcoin, to circular and renewable systems of production. Rising to the challenge of fixing an existing system and exploring the possibility of creating a new system are different undertakings. The first is about optimising what exists, the second is about creating something different and better. We want this project to yield practical insights for those who want to respond to the systemic challenges of today by stepping into the possibilities of the future. Acting to change systems depends on new ways of seeing both challenge and opportunity: why systems come under strain and what unlocks the potential for alternatives. It depends on better understanding how new systems form, and what and who is part of initiating and driving the transition to them. In putting together this paper and the ones that will follow from it we want to clarify how to assess the need for, invest in and act on the process of deliberate system change.

Locally Led Development: Engaging Local Stakeholders in Building An Evidence Base - OLS (USAID Learning Lab, 2020)
For two months in the Spring of 2020, the Office of Local Sustainability’s Evidence and Learning Team invited staff from across USAID to join us in exploring how the Agency approaches its generation and use of evidence from the perspective of locally led development. Our seven-part Standards of Evidence for Locally Led Development series brought together eight expert presenters and more than 670 participants to engage in conversations ranging from scientific research to complexity-aware monitoring to ethical considerations when conducting research. This blog post outlines some of our key learnings from the series and introduces the topics covered by each presenter. We encourage you to explore the event resources and share your own takeaways in the comment section below. WHAT DID WE LEARN? Three key themes emerged from the series: - Practitioners should draw on a wide range of evidence to inform locally led development programming. - Local actors can provide important contextual knowledge throughout the life of your research programs. - Prioritize local actors as the end users for your evidence to help them build their own self-reliance and increase sustainability of your program’s outcomes.

Four ways development practitioners can borrow from private sector adaptive approaches - Pett, Jamie (LearnAdapt, 2020)
Agile, lean startup and human-centred design can be an answer — if you’re asking the right questions

What is Rapid Action Learning and how was it developed? - Sanitation Learning Hub (2020)
To celebrate the publication of our latest Frontiers of Sanitation, we had a series of conversations with our colleagues and partners on our work on Rapid Action Learning so far. To download the publication in full, head to https://sanitationlearninghub.org/res...

How to Monitor and Evaluate an Adaptive Programme: 7 Takeaways - Sikustahili, Gloria; Adkins, Julie; Makongo, Japhet; Milligan, Simon (From Poverty to Power, 2020)
Monitoring and Evaluation needs to be different to support the new generation of 'adaptive programmes' in aid. 4 M&E specialists in Tanzania explain how.

The Hidden Life of Theories of Change - Green, Duncan (From Poverty to Power, 2020)
A smart new report explores how a good idea - theories of change - is distorted by the way it is implemented in the aid sector

Reflections on the Intersection of Evaluation and Emergent Learning - Pankaj, Veena (InnovationNetwork, 2020)
All too often evaluation, learning, and strategy are disconnected. How many times have you been involved in an evaluation that’s…

5 Principles to Guide Adaptive Leadership - Ramalingam, Ben; Nabarro, David; Oqubuy, Arkebe; Carnall, Dame Ruth; Wild, Leni (Harvard Business Review, 2020)
How you respond to a crisis will have repercussions for years to come. The Covid-19 pandemic is constantly evolving, with leaders facing unpredictability, imperfect information, multiple unknowns, and the need to identify responses quickly — all while recognizing the multi-dimensional (health-related, economic, social, political, cultural) nature of the crisis. Responding to the crisis requires adaptive leadership, which involves what we refer to as the 4 A’s: Anticipation of likely future needs, trends and options. Articulation of these needs to build collective understanding and support for action. Adaptation so that there is continuous learning and the adjustment of responses as necessary. Accountability, including maximum transparency in decision-making processes and openness to challenges and feedback.
Keywords: Crisis management; Leadership

The Long and Short of It: Responding to immediate needs while pursuing long-term goals - Pett, Jamie (LearnAdapt, 2020)
How can we balance our need to respond to a crisis with our long-term goals for systemic change?

Book Review: ‘Thinking and Working Politically in Development’ - Green, Duncan (From Poverty to Power, 2020)
Review of a new book that explores the 'secret sauce' of Coalitions for Change - an unusually successful governance programme in the Philippines

What is Political Economy Analysis (PEA) and why does it matter in development? - Teskey, Graham (From Poverty to Power, 2020)
Graham Teskey shares a great internal links round up and guide to 'political economy analysis'

Rapid Action Learning for Sanitation and Hygiene Programming - Chambers, Robert; Myers, Jamie; Vernon, Naomi (IDS, 2020)
There is a glaring gap and compelling need for approaches and methods that realign to new rigour through timeliness, cost-effectiveness, relevance and being actionable. Over the past few years, the Sanitation Learning Hub, in collaboration with the Government of India, Praxis, WSSCC and WaterAid India, have been developing Rapid Action Learning approaches. Multiple approaches have been trialled, with flexible formats, but the essential criteria is that learning is timely, relevant and actionable. These learning approaches are the focus of the latest edition of the Frontiers of Sanitation series. This Frontiers explains the advantages and disadvantages of the approaches trialled and sets out a challenge to those working in the water, sanitation and hygiene sector to: Reflect on what, for you, constitutes rigour. Adopt and adapt approaches to fit your context and needs. Develop your own approaches. Record your experiences and lessons learnt. Take the time to share your experiences with us

Participación Digital y mejora democrática - un camino sin regreso - Palumbo, Gabriel (Luminate, 2020)
Tal vez como nunca en la historia, la democracia está siendo desafiada. A los viejos problemas, muchos no resueltos, se les agregan otros, surgidos de la incorporación de tecnología, de la emergencia de nuevos actores sociales y de una conformación de la subjetividad política cada vez más compleja e impredecible. Esta investigación pretende reactualizar algunos de los debates clásicos sobre la democracia y, al mismo tiempo, sentar una base empírica que permita pensar de qué modo la tecnología puede ayudar a mejorar la calidad democrática. Según nuestro estudio, la participación digital está promoviendo modificaciones importantes en la relación entre la ciudadanía y los decisores políticos, ampliando los espacios de proximidad. Esto sucede, fundamentalmente, porque baja las barreras de acceso y de incentivos haciendo más fácil, más efectiva y más mensurable la participación ciudadana. La participación digital aumenta el períodos electorales y fomenta el asociacionismo cívico como posibilidad cierta y concreta de presionar sobre los decisores políticos. Lo digital habilita, además, participación offline y genera espacios de pedagogía cívica. La capacidad de implantación de temas en el debate público que se puede generar a través de ella mejora la conversación pública, impactando a su vez la calidad de la democracia. Desde Luminate entendemos que favorecer los espacios de rendición de cuentas desde la ciudadanía hacia el poder político es una apuesta al robustecimiento de la experiencia democrática y que las formas digitales están abriendo posibilidades que otras opciones no pudieron lograr. Esperamos que esta investigación sea un aporte para poder seguir reflexionando y avanzando en esta dirección.

Navigating adaptive approaches for development programmes - Pett, Jamie (ODI, 2020)
This working paper compares six of the most prominent adaptive approaches to emerge over the past two decades. Three come from the world of innovation, largely in the private sector (agile, lean startup and human-centred design), and three from the global development sector (thinking and working politically, forms of adaptive management and problem-driven iterative adaptation). While all of these approaches are valuable when used in the right context, practitioners may be perplexed by the multiplicity of methods and jargon. This paper aims to address some of this confusion by mapping where these approaches have come from and showing how they can be applied across the adaptive programme cycle. Armed with this knowledge, practitioners might experiment with different combinations and sequences of adaptive approaches according to the kind of problem and context faced. In turn, this may help us move beyond a siloed view of approaches linked to innovation, adaptive management or more politically smart ways of working. Key messages: • Adaptive approaches have emerged in several sectors, including software development, product and service design, technology startups and international development. • Adaptive approaches can help practitioners counteract misplaced certainty. By talking to potential users, understanding institutions, interests and ideas and investigating the root causes of a problem, practitioners applying these approaches can illuminate the underlying nature of the problem and context. • Rather than building a whole solution straight away, these approaches commonly encourage practitioners to start small and use structured cycles of testing and learning. There is scope to further consider how different approaches can be better brought together and combined. • Adaptive approaches in development provide a wider range of options for what to create and facilitate – not only products or services, but also forms of collective action. There are also alternative ways to think about scale – considering how others might take up an idea and looking for leverage, rather than quantity.

SystemCraft - a primer: How to Tackle our Toughest Problems - Simpson, Kate; Randall, Ian (Wasafiri, 2020)
Systemcraft is our applied framework to help leaders and organisations get started and keep going when faced with complex problems. It is built on our practical experience. It draws on a broad body of research, action and theory from the worlds of complexity thinking, systems theory, adaptive management, leadership development, social movements, development theory and beyond. Systemcraft has been designed to make systems thinking something any leader can apply when they find themselves faced with a complex problem and asking, ‘So what do I do next?’

Reacting or adapting? Purposeful adaptation and response to contextual change. - Wood, Nik (Rebel with Causation, 2020)
Adaptive management is increasingly on the ticket for development programming, and has been crucial in the wake of covid-19. I’ve been working with adaptive programmes for most of my career, …

International Public Participation Models 1969-2020 - Hussey, Sally (Bang The Table, 2020)
International Public Participation Models 1969 – 2020 provides an essential resource of 60 different models to better map public participation in practice and theory.

W(h)ither sanctions? - Aston, Tom (Thomas Aston LinkedIn pulse, 2020)
Considerations on where and when sanctions fit into the conception of social accountability is a nudge toward better and more granular descriptions of what work is being undertaken effectively in which situations.

How Artifacts Afford: The Power and Politics of Everyday Things - Davis, Jenny L. (MIT Press, 2020)
Technologies are intrinsically social. They reflect human values and affect human behavior. The social dynamics of technology materialize through design features that shape how a technology functions and to what effect. The shaping effects of technology are represented in scholarly fields by the concept of “affordances.” Affordances are the ways design features enable and constrain user engagement and social action. This has been a central construct for designers and technology theorists since foundational statements on the topic from JJ Gibson and Don Norman in the 1970s and 80s. With the rise of digitization and widespread automation, “affordance” has entered common parlance and resurged within academic discourse and debate. Davis provides a conceptual update on affordance theory along with a cogent scaffold that shifts the orienting question from what technologies afford, to how technologies afford, for whom, and under what circumstances? “How Artifacts Afford” introduces the mechanisms and conditions framework of affordances in which technologies request, demand, encourage, discourage, refuse, and allow social action, varying across subjects and circumstances. Underlying this mechanisms and conditions framework is a sharp focus on the politics and power encoded in sociotechnical systems. In this timely theoretical reboot, Davis brings clarity to the affordance concept, situates the concept within a broader history of technology studies, and demonstrates how the mechanisms and conditions framework can serve as a transferrable tool of inquiry, critique, and (re)design.

Webinar - Practicing Thinking and Working Politically (TWP): Voices from the Field - SID (Society for International Development., 2020)
Current thinking on effective international development interventions highlights the importance of “thinking and working politically” (TWP). Among the emerging lessons of experience is that thinking politically, using tools such as political economy analysis, is more easily undertaken than working politically. How can the two pillars of TWP be effectively integrated? What challenges exist and how have practitioners confronted them? This session focuses on listening to voices from the field to explore answers to these questions. The organizers solicited proposals from implementers, reaching out to SID-W members and the Washington, D.C.-based TWP community of practice. They selected the following four projects that illustrate different approaches to practicing TWP. • Mercy Corps: Integrated Maji Infrastructure and Governance Initiative for Eastern Congo, Democratic Republic of the Congo • Asia Foundation: Tourism Strategy Project, Timor-Leste • Counterpart International: Rights and Dignity Project, El Salvador • RTI International: Knowledge Sector Initiative, Indonesia Their voices will be bookended by Sarah Frazer (RTI International), who will summarize a recent study, Thinking and Working Politically: Lessons from Diverse and Inclusive Applied Political Economy Analysis, and Alina Rocha Menocal (Overseas Development Institute), who will provide commentary on the four projects and the study findings. Ann Hudock (Counterpart International) will moderate.

Managing uncertainty when your brain doesn’t like it - Proud, Emma (LearnAdapt, 2020)
Over lockdown there were a lot of tantrums in our household. The tears and tussles were at a dramatic high when the schools closed, and…

The Politics of Uncertainty : Challenges of Transformation - Scoones, Ian; Stirling, Andy; Stirling, Andy (Routledge, 2020)
Why is uncertainty so important to politics today? To explore the underlying reasons, issues and challenges, this book’s chapters address finance and banking,

Webinar - Sustainability Ready: what it takes to support & measure lasting change - Cekan, Jindra (Valuing Voices, 2020)
On June 24th under GLocal’s UNConference, “Co-creating our future stories of hope and action”, Jindra Cekan, Holta Trandafili, and Isabella Jean presented their work on sustainability evaluations and exit strategies via local voices. We chaired a 2-hour discussion session on the following topics: Sustainability of global development projects and exit from them, The importance of valuing local partners’ and participants’ voices, How to embed ex-post evaluation of sustainability into the project cycle, How expectations, benchmarking, and early joint planning in exit strategies, as well as considering long-term ownership & relevance will support projects to be sustained locally, including questioning who will maintain results, Considering power dynamics between donors and ultimate ‘beneficiaries’ and the value of the impact of the project from a variety of perspectives, and more Here is the recording of our presentation or see just the PowerPoint presentation. We harvested lessons from our three presentations: Jindra Cekan: · Fear of learning about failure in our global development industry – INGOs are “waiting for a successful enough project” to commission ex-post project sustainability evaluation · It needs to be a culture of learning, not a culture of success. · Lack of transparency in sharing program evaluation results with communities and local government is widespread, and even more rarely do we come back after many years and share learnings · Participatory approaches are vital, listening to participants about sustained impacts is key · It is never attribution, always contribution. To isolate impacts, we need to look for project sites that haven’t had multiple other organizations overlapping through all phases. · Building sustainability planning throughout the project cycle is key – but often doesn’t happen Holta Trandafili: · Sustainability needs to be planned to be researched, including evaluating why or why not were project elements sustained, and why? What has the project done to enable communities to sustain improvements? · Expectations of sustainability need to be more modest (as most results are mixed good/bad) · We need to ask: How are you defining and measuring sustainability – for how long should the results last? Among how many participants? Have you set benchmarks for success? · We should expand your toolbox on methodology to investigate sustainability. Stories of success are one of a myriad of methods used, including mixed-methods, cost-benefit, etc. · Start with the need for learning not [just] accountability Isabella Jean: · Sustainability investigation/evaluation/learning should be mindful that this is NOT about projects. It is about people. · We have a system that focuses on gaps and needs to be filled vs. existing capacities’ structures to be reinforced. How can our work on measuring sustainability bring this to light and call it out, so that we change the norm? · Planning for sustainability requires the insight to integrate resources and experiences of outsiders with the assets and capacities of insiders to develop context-appropriate strategies for change If you would like to discuss this with any of us, please send us comments and we’re happy to respond. Thanks again to the G-Local UnConference team! Below please find our bios: Jindra Cekan/ova has worked in global development for 33 years focused on participatory design and M&E for global non-profits. She founded Valuing Voices 7 years ago. For details, see: Valuing Voices Founder Holta Trandafili is the Research, Learning, and Analytics Manager with World Vision US and has been leading field research, monitoring, and evaluation since 2007. She has led sustainability measurement studies for World Vision programs in Uganda, Kenya, Sri Lanka, India, Burma, and Bolivia. Her areas of expertise and interest include program and community groups’ sustainability measurements; agency-level measurements; empowerment approaches to development; integrated programming; local capacities for peace; gender analysis; and outcome monitoring. Currently, Mrs. Trandafili serves as an Advisory Committee member for InterAction’s Effectiveness and Program Evaluation Working Group and chairs one of the sub-working groups under The Movement for Community-led Development. Isabella Jean supports international and local organizations and funders to document promising practices, facilitate learning and strengthen capacities for conflict sensitivity, peacebuilding and humanitarian effectiveness. She has facilitated action research, collaborative learning and advisory engagements in over 25 countries, and serves as an advisor to policymakers, senior leadership and program teams. Isabella co-authored the book, Time to Listen: Hearing People on the Receiving End of Aid and developed practical guidance to support accountability to communities, listening and feedback loops, and responsible INGO exits. She teaches graduate-level courses on aid effectiveness, program strategies and M&E of peacebuilding at Brandeis University’s Heller School for Social Policy and Management. Previously, Isabella directed training at a community organizing network and conducted policy research for the Institute for Responsive Education, UNDP, and Coexistence International.

Physically distanced adaptive management - Proud, Emma (LearnAdapt, 2020)
Opportunities and challenges for local leadership

Big Data to Data Science - Moving from “What” to “How” in the MERL Tech Space - Bertermann, Kecia; Robinson, Alexandra; Bamberger, Michael; Higdon, Grace Lyn; Raftre, Linda (MERL Tech, 2020)
This paper probes trends in the use of big data by a community of early adopters working in monitoring, evaluation, research, and learning (MERL) in the development and humanitarian sectors. Qualitative analysis was conducted on data from MERL Tech conference records and key informant interviews. Findings indicate that MERL practitioners are in a fragmented, experimental phase, with use and application of big data varying widely, accompanied by shifting terminologies. We take an in-depth look at barriers to and enablers of use of big data within MERL, as well as benefits and drawbacks. Concerns about bias, privacy, and the potential for big data to magnify existing inequalities arose frequently. The research surfaced a need for more systematic and broader sharing of big data use cases and case studies in the development sector.

Emerging Technologies and Approaches in Monitoring, Evaluation, Research, and Learning for International Development Programs - Bruce, Kerry; Gandhi, Valentine J; Vandelanotte, Joris (MERL Tech, 2020)
Emerging technology is making monitoring, evaluation, research and learning (MERL) more precise and enriching data. However, this evolution is so rapid that it can be difficult to stay informed about the field overall. This paper presents examples of emerging technology that are most often used in MERL for development programs, describes the pros and cons of their use, and discusses technology and ethics concerns that practitioners should keep in mind. The paper covers new types of data sources (application data, sensor data, and drones), new ways to store data (distributed ledger technology and the cloud), and new ways to analyze data (text analytics and supervised and unsupervised learning).

Dynamic knowledge management strategy development in international non-governmental organisations - Walsh, John N.; Lannon, John (Knowledge Management Research & Practice, 2020)
Knowledge management strategies are important for firms’ competitive positioning. This paper examines how knowledge management codification and personalisation strategies are developed in response to environmental and organisational dynamics in an international non-governmental organisation. A longitudinal case study of the organisation’s strategic reformulation of its KM strategy over a 2.5 period is drawn upon. The research examines how pressures in the firm’s operating environment led to the organisation identifying the need to leverage the value of local contextual knowledge. Subsequent reformulation required the organisation to change its strategic mix of codification and personalisation over time. Although efforts were focused on increasing personalisation, developments were supported through codification demonstrating a symbiotic, mutually supporting relationship between the strategies. The strategic reformulation involved processes of reflection, repackaging and support activities.
_eprint: https://doi.org/10.1080/14778238.2020.1785348
Keywords: Knowledge management strategy; case study; international development; knowledge dynamics; non-governmental organisation

Evaluating CLARISSA: Innovation Driven by a Participatory Learning Agenda - Apgar, Marina; Snijder, Mieke; Kakri, Shanta; Macleod, Shona; Paul, Sukanta; Sambo, Anna; Ton, Giel (Institute of Development Studies, 2020)
Children end up in child labour as a result of many, often unknown or hidden, interactions between multiple actors and multiple factors within households, communities, and labour systems, leading to unpredictable outcomes for children and other sector stakeholders and sometimes resulting in the worst forms of child labour (WFCL). It is a complex problem, and interventions aimed at tackling it are also, inevitably, complex and challenging. The way they influence change is non-linear, causality is uncertain, and unintended consequences may result. Programmes such as the Child Labour: Action-Research-Innovation in South and South-Eastern Asia (CLARISSA) that are engaging with such intractable challenges and aim to reach the most left behind (children in WFCL) are operating in conditions of complexity. This complexity poses significant challenges to the way programmes are designed, planned, implemented, and evaluated, and requires a move away from linear and predetermined models. In this Working Paper, we share our experience and early learning about how to design and implement monitoring, evaluation and learning that intentionally embraces the challenge of complexity.
Accepted: 2020-06-25T14:55:41Z

Innovation for Development Impact: Lessons from the OECD Development Assistance Committee - OECD (OECD, 2020)
The development co-operation community needs to innovate to meet the global challenges ahead. Although it has an established track record for innovating partnerships, funding instruments and technologies, they are not enough to deliver the Sustainable Development Goals. This report synthesises the lessons emerging from an OECD Development Assistance Committee peer learning exercise on how innovation efforts can be strengthened, individually and collectively, to achieve the 2030 Agenda. The report is organised around three blocks – strategy, management and culture; organisation and collaboration; and, the innovation process – and provides recommendations on how innovation can best benefit poor and vulnerable people around the world.

Farewell DFID … a personal obituary - Mason, Phil (LinkedIn - Phil Mason, 2020)
Reflection on how DFID was created.

Looking for a way out of aid’s pre-pandemic mess? A model based on cake - Baguios, Arbie (From Poverty to Power, 2020)
Arbie Baguios presents his ideas on how to reform the aid system

Thinking and working politically with technology: State of the art meets art of the state - Thampi, Gopa; Nixon, Nicola (From Poverty to Power, 2020)
Based on its work in Sri Lanka, The Asia Foundation argues for greater attention to the local political dynamics into which digital solutions are introduced

Not Always Movements. Multiple Approaches to Advance Large-Scale Social Change - Stachowiak, Sarah; Gienapp, Anne; Kalra, Nikki (ORS Impact, 2020)

What We Know About Traditional MERL Tech - Insights from a Scoping Review - Tilton, Zach; Harnar, Michael; Raftree, Linda; Perrin, Paul; Bruening, Gretchen; Banerji, Soham; Gordley, John; Foster, Hanna; Behr, Michele (Western Michigan University, 2020)
This paper explores the peer-reviewed evidence base of “traditional” technology-enabled monitoring, evaluation, research, and learning (MERL Tech) in international development assistance from 2015 to 2019. The authors conducted a scoping review that searched seven databases, screened 3,054 reference titles and abstracts, coded 886 abstracts, and extracted and analyzed conclusions and recommendations from the full texts of 256 studies. The findings reveal the most frequently reported technologies, MERL activities, and the sub-sectors, and the geographies where those tech-enabled activities occur. Gaps in the evidence for specific technologies, MERL activities, and sectors are mapped. The data reveals which technologies are trusted more than others and reported barriers to effective MERL Tech implementation and areas that researchers suggest for further investigation. The results suggest that the evidence from peer-reviewed studies is not proportional to estimated MERL Tech activity, significant publication bias exists, and further knowledge synthesis of unindexed grey literature is needed to provide a more comprehensive and possibly accurate description of MERL Tech practice.

Navigating with Action Inquiry (Collective Leadership Animations) - Keira Oliver (2020)
"Have you ever wondered if there might be a better way to work together? Perhaps you would like to find ways to avoid habits, or patterns of behaviour, that whilst you say you don’t like them, you seem unable to escape." Written & narrated by Cathy Sharp (Research for Real) and animated by Keira Oliver (Collective Leadership for Scotland) and her son (10yr) in an animation-sprint style (created quickly, but with care), this film summarises the ideas and practices of action inquiry. It introduces a set of statements designed to stretch, challenge and encourage us to talk about these ideas to find better ways to work together.

Scaling the impact of sustainability initiatives: a typology of amplification processes - Lam, David P. M.; Martín-López, Berta; Wiek, Arnim; Bennett, Elena M.; Frantzeskaki, Niki; Horcea-Milcu, Andra I.; Lang, Daniel J. (Urban Transformations, 2020)
Amplifying the impact of sustainability initiatives to foster transformations in urban and rural contexts, has received increasing attention in resilience, social innovation, and sustainability transitions research. We review the literature on amplification frameworks and propose an integrative typology of eight processes, which aim to increase the impact of such initiatives. The eight amplification processes are: stabilizing, speeding up, growing, replicating, transferring, spreading, scaling up, and scaling deep. We aggregated these processes into three categories: amplifying within, amplifying out, and amplifying beyond. This integrative typology aims to stimulate the debate on impact amplification from urban and rural sustainability initiatives across research areas to support sustainability transformations. We propose going beyond an understanding of amplification, which focuses only on the increase of numbers of sustainability initiatives, by considering how these initiatives create transformative change.
Keywords: City; Rural; Scaling; Transformation; Transition; Transition initiative; Urban

Smart Implementation of complex change processes - Beier, Christoph; Kirsch, Renate (Global Solutions Journal, 2020)
Cooperation management facilitates the recoupling of progress toward sustainable development

Uncertainty and COVID-19: A turning point for Monitoring Evaluation, Research and Learning? - A discussion note for aid actors, policymakers and practitioners - Tyrrel, Lavinia; Roche, Chris; Jackson, Elisabeth (Abt Associates, 2020)
The COVID-19 pandemic has significantly shifted the context in which aid and development is being delivered. The global scale of the pandemic and the speed at which it is spreading mean that the ‘normal’ economic, ideological and organisational influences which shape (if not determine) aid delivery are in flux. This means that – for a relatively short-period – there is scope for aid actors to work collectively to embed more locally-led, politically-informed and adaptive forms of MERL in aid and development practice. These forms of Monitoring Evaluation Research and Learning (MERL) are not only well-suited to the current global pandemic. They also offer ways for aid program decision makers and practitioners to make sense of the complex and uncertain contexts in which much development work takes place. Applying locally-led, politically-informed and adaptive forms of MERL in the COVID-19 context and beyond requires a shift in mindset and approaches. Situations of complexity, in which it is difficult to predict the relationships between cause and effect, do not lend themselves to linear approaches and fixed indicators. Instead, they require ‘navigation by judgement’, ongoing learning and adaptation and greater privileging of local knowledge, and of the perspectives of those who are often excluded. Rather than being focused on upwards accountability, simple numbers and good news stories, the core function of MERL in this context is to support a better understanding – in real-time – of the changing operating context, to generate learning about the immediate impact of policy and program responses and their longer-term effects, and to inform decision making by front line staff. Whether the opportunities afforded by this ‘critical juncture’ are realised will depend on the degree to which those in the aid and development sector use this opportunity to promote a shift in the deep incentive structures within which development agencies are embedded. On the one hand, the pandemic underscores the limits of the linear understandings of change which underpin many orthodox approaches to planning, design and associated MERL. On the other hand, there is a vested interest in the status quo amongst many organisations, consultants, researchers and MERL practitioners. This is because approaches which promote locally-led development inevitably require those in power to relinquish control. While a range of factors make this shift difficult, there is more scope to change internal ways of working in development agencies than is commonly acknowledged. There is no time like the present to advocate for a ‘new normal’ for MERL.

A pragmatic approach to assessing system change - Webinar (DCED, 2020)
Few topics inspire as much confusion and debate as systemic change. What is it? How do you measure it? Does it even matter? Assessing changes in systems might be more doable than you think. This webinar explored a back-to-basics approach to assessing system change. In November 2019, thirty results measurement specialists, managers and consultants got together in Bangkok. They took part in workshops on a back-to-basics approach to assessing system changes, applying it to cases from participants’ programmes. Since then, the insights from the workshop have been further developed into a pragmatic approach to assessing system changes that builds on what programmes are actually doing and learning from practice. It can be: applied by programmes using a variety of different systemic change frameworks applied across a variety of sectors implemented with internal resources using familiar methods for information gathering The speakers walked through the approach using examples from the 2019 workshop, including PRISMA in Indonesia and Skills for Jobs (S4J) in Albania.

Deciding Well in Tumultuous Times - Moss, Ian David (Smarter decisions for a better world, 2020)
Practical advice for donors and institutions responding to COVID-19

Software Engineering at Google: Lessons Learned from Programming Over Time - Winters, Titus; Manshreck, Tom; Wright, Hyrum (O'Reilly Media, 2020)
Today, software engineers need to know not only how to program effectively but also how to develop proper engineering practices to make their codebase sustainable and healthy. This book emphasizes this difference between programming and software engineering. How can software engineers manage a living codebase that evolves and responds to changing requirements and demands over the length of its life? Based on their experience at Google, software engineers Titus Winters and Hyrum Wright, along with technical writer Tom Manshreck, present a candid and insightful look at how some of the world's leading practitioners construct and maintain software. This book covers Google's unique engineering culture, processes, and tools and how these aspects contribute to the effectiveness of an engineering organization. You'll explore three fundamental principles that software organizations should keep in mind when designing, architecting, writing, and maintaining code: How time affects the sustainability of software and how to make your code resilient over time How scale affects the viability of software practices within an engineering organization What trade-offs a typical engineer needs to make when evaluating design and development decisions

LEARN End of Contract Report - Learning Lab (USAID, 2020)
In September of 2014, USAID’s Office of Learning, Evaluation & Research (LER) awarded the Learning and Knowledge Management (LEARN) contract to Dexis Consulting Group and subcontractor RTI International.1 This document—the End of Contract Report—captures five and half years of results and reflections for our stakeholders. Our intention is to share the good and the bad, and while this report would not be considered a “tell all,” we think we have a story worth sharing, particularly to USAID CORs and AORs, activity managers, and other implementers of institutional support contracts.2 LEARN’s primary purpose was to support organizational change at USAID. More specifically, the contract was focused on helping USAID staff integrate collaborating, learning, and adapting (CLA) approaches into the design, implementation, monitoring, and evaluation of programs (what is known at USAID as the Program Cycle). It was clear that most USAID staff, whether they realized it or not, were already integrating CLA into their work to some extent. The focus of our efforts, therefore, was to make those practices more systematic, intentional, resourced, and ultimately more widespread throughout the Agency, which would have a ripple effect on implementing partners and even other stakeholders, such as host country governments. This was based on the theory—later borne out by evidence—that by becoming a better learning organization, USAID could be a more effective development organization. And that theory brought the USAID CLA team within USAID’s Bureau for Policy, Planning & Learning (PPL) and LEARN contractors together, driven by a shared purpose of improving how USAID does business. LEARN was designed based on this belief and, as you might expect from a learning-oriented contract, began with more questions than answers. Primary among those questions was: could an institutional support contract do more than carry out requested services—could it actually accelerate positive organizational change at USAID? And if so, how and under what conditions?

How Do Donor-led Empowerment and Accountability Activities Take Scale into Account? Evidence from DFID Programmes in Contexts of Fragility, Conflict and Violence - Anderson, Colin; Fox, Jonathan; Gaventa, John (IDS, 2020)
Development donors invest significantly in governance reform, including in contexts characterised by conflict and fragility. However, there is relatively little comparative study of their change strategies, and little understanding of what works and why. This paper explores the strategies of six recent DFID-funded programmes in Mozambique, Myanmar, and Pakistan with empowerment and accountability aims. Document review and field interviews are used to analyse the application of multi-scalar or multi-level change strategies, since such approaches are hypothesised to potentially generate more leverage for public accountability reforms. Analysis suggests that these strategies can strengthen citizen ability to navigate governance systems to resolve problems and claim accountability, and can bolster pro-accountability coalitions’ internal solidarity and external legitimacy. Multi-level strategies also appear associated with establishing more significant pressure for reform than exclusively local or national approaches. Yet conventional project reporting focuses on counting activities and outputs rather than analysing the dynamic, interactive processes at work in these strategies, and few evaluations are publicly accessible. To fully understand what kinds of action strengthen citizen demands for accountability requires a more transparent and rigorous approach to learning from donor-led governance interventions.

Monitoring for problem-solving, adaptive management, reporting and learning - Dominique Morel; Dzino-Silajdzic, Velida; Hagens, Clara (Catholic Relief Services, 2020)
Internal and external stakeholders have different information needs over a project’s life, for purposes that include adaptive management, accountability, compliance, reporting and learning. A project’s monitoring, evaluation, accountability and learning, or MEAL, system should provide the information needed by these stakeholders at the level of statistical reliability, detail and timing appropriate to inform data use. In emergency contexts where the situation is still fluid, ‘informal monitoring’ has proved helpful to staff’s ongoing assessment of the broader environment in order to identify changes in the situation, in other actors’ responses, and in priority unmet needs that would require corresponding changes in the response.2 The same distinction between informal monitoring of possible changes in the project’s operating context—whether identified as project assumptions and risk factors or not—and formal monitoring of the activities included in the response and project indicators, is relevant for development contexts too. Informal monitoring: Ongoing assessment of changes in operating context Formal monitoring: Tracking progress against project activities and indicators Within formal monitoring, it is useful to further differentiate between light monitoring and rigorous monitoring: - Light monitoring aims to provide timely feedback on new activities (or new locations or target groups) or aspects of the project’s theory of change (activity-to-output or output-to-IR change) logic that staff are less confident about, to check for early signs that progress is being made and that assumptions are holding true while there is still ample time to make adjustments if necessary.3 - Rigorous monitoring aims to collect representative data for evidence-based project management, reporting and learning, not just at midterm but throughout project implementation.

MERL Tech State of the Field - The evolution of MERL Tech - Raftree, Linda (MERL Tech, 2020)

Adaptive leadership in the coronavirus response - Ramalingam, Ben; Wild, Leni; Ferrari, Matt (ODI, 2020)
The coronavirus pandemic poses unprecedented challenges to science, policy and the interface between the two. How – and how quickly – policy-makers, practitioners and researchers react to this emerging and complex crisis is making a profound difference to people’s lives and livelihoods (WHO, 2020). But how can we ensure effective collective decision-making on the basis of emerging evidence, changing trends and shifting scientific understanding, all in the face of considerable uncertainty? Recent experience highlights the need for adaptive leadership in national and global responses to the outbreak. This briefing paper sets out key principles for what this might look like, and proposes a roadmap for policy-makers, practitioners and researchers to move towards such an approach as they tackle the unfolding crisis. Key messages Tackling the coronavirus outbreak requires adaptation at operational and leadership levels. Operationally, there is scope to strengthen evidence-based adaptive management practices, to adjust the mix and type of interventions being implemented and learn as we go so as to achieve shared goals. This requires adaptive leadership capacities, being open and transparent about learning, using collective decision-making processes and building trust with communities and individuals.

Cynefin St David’s Day 2020 (1 of 5) - Snowden, Dave (Cognitive Edge, 2020)
Series of blog posts discussing the last version (as of 2020) of the Cynefin framework.

Adaptation in practice: lessons from teenage pregnancy programmes in Sierra Leone - Castillejo, Clare; Buell, Stephanie (ODI, 2020)
A discussion of initial learning emerging from the SLRC ’Adaptive approaches to reducing teenage pregnancy in Sierra Leone’ action research project.

Politically informed approaches to gender in fragile and conflict-affected settings - Castillejo, Clare; Domingo, Pilar; George, Rachel; O’Connell, Shannon (ODI, 2020)
This report summarises the discussions at a meeting held in September 2019 of a group of global development research and policy experts and practitioners, convened by ODI and the Westminster Foundation for Democracy, to share experiences and knowledge, reflect on what we already know about working politically on gender in fragile and conflict-affected settings (FCAS), and identify what further evidence would be useful. Key messages: - It is important to identify, document and learn from politically informed and adaptive ways of working in practice on gender in FCAS. - Different analytical and monitoring, evaluation and learning tools are required for politically informed programming on gender. These should be embedded within programme teams and processes, and be both gender-responsive and responsive to the political economy context. - There is a need for more politically smart use of quantitative and qualitative data in order to identify plausible entry points and ways of working on gender in FCAS. This should include increasing the capacity of programme staff to use data to inform, adapt and correct programmes. - Staff promoting politically informed work on gender equality are often isolated, with little opportunity to share experiences or learn from others. Platforms should therefore be created to share experience and knowledge, and to bring together donors and implementers involved in this work.

The Magenta Book - Central Government guidance on evaluation - HM Treasury (UK Government, 2020)
HM Treasury guidance on what to consider when designing an evaluation.

The Magenta Book - Suplementary guide: Handling Complexity in Policy Evaluation - HM Treasury (UK Government, 2020)
HM Treasury guidance on what to consider when designing an evaluation.

Human trafficking in South Asia: Assessing the effectiveness of interventions - Jensen, Charity; Oosterhoff, P.; Pocock, N. (FCDO, 2020)
This Rapid Evidence Assessment (REA) examines current evidence on the effectiveness of interventions to combat human trafficking in four South Asian countries (Bangladesh, India, Nepal, and Pakistan). This REA is being carried out as part of a wider assignment for the UK Department for International Development (DFID), with the overall objective of synthesising evidence on the effectiveness of interventions that tackle modern slavery in South Asia. Two REAs were conducted on different types of modern slavery, one on human trafficking for labour and sexual exploitation, and another on child labour (Idris et al., 2020). The research question for this REA is: ‘What has been the effect of interventions to combat and/or reduce sexual and labour exploitation in Bangladesh, Pakistan, India, and Nepal?’ Trafficking in persons is a form of ‘modern slavery’, which is an umbrella term for the variety of situations in which someone is forcibly controlled by an individual or group for the purpose of exploitation. The Global Slavery Index estimated that, on any given day in 2016, 40.3 million people were victims of modern slavery, including 24.9 million people in forced labour and 15.4 million people in forced marriage. Of the estimated almost 25 million people in modern slavery in Asia, 66% were exploited for labour (Global Slavery Index, 2018).

Mainsreaming gender in an adaptive, politically smart governance programme - Lessons from Institutions for Inclusive Development in Tanzania - Laws, Ed (ODI, 2020)
• This paper looks at the experience of gender mainstreaming in the Institutions for Inclusive Development (I4ID) programme – an adaptive, politically smart governance programme in Tanzania. • When development programmes try to engage with political stakeholders and align with the priorities of wider coalitions there is a danger that gender equality is de-prioritised. • It is important that formal political economy analysis, as well as other data collection, analysis and consultation exercises, are gender-sensitive. Teams should also look for ways to make gendered political and power analysis part of the everyday routine practice of staff. • Working politically and adaptively to advance gender objectives calls for staff with a specific skillset, as well as links to appropriate networks and political stakeholders. It also implies establishing checks and incentives to hold staff and partners accountable for gender objectives, and strong and consistent messaging from team leaders.

Rethinking value for money for adaptive, politically smart programmes - Lessons from Institutions for Inclusive Development in Tanzania - Laws, Ed (ODI, 2020)
- This short paper draws out lessons for working effectively with and through partners, based on the experience of the Institutions for Inclusive Development (I4ID) programme – an adaptive, politically smart governance programme in Tanzania. • Cultivating effective partnerships can be a key part of delivering locally legitimate projects that have the potential to create sustainable change. Adaptive and politically informed ways of working create specific opportunities and challenges for doing this well. • Flexible and adaptive programmes are deliberately designed to experiment and to make small investments in different areas, to see what will work. While this is often important for making headway on complex challenges, it can also leave partners exposed and can undermine trust. • It can also be challenging to balance the need to meet accountability commitments to donors while allowing local partners to take the lead in pursuing their own objectives. • Co-creating plans, priorities and activities with partners has the potential to resolve some of these tensions. But the time and patience required to do this successfully should not be underestimated, and can be difficult to maintain in the face of pressure from donors to see results within a confined timeframe.

Working effectively through partnerships - Lessons from Institutions for Inclusive Development in Tanzania - Laws, Ed (ODI, 2020)
• This paper looks critically at the approach to value for money (VfM) in the Institutions for Inclusive Development (I4ID) programme – an adaptive, politically smart governance programme in Tanzania. • Adaptive, politically smart programmes like I4ID aim to deliver VfM by learning about what will work in complex environments, and quickly incorporating those lessons into delivery. When functioning properly, they can rapidly wind down activities as new information emerges and divert funding to more effective alternatives. • This means that adaptive programmes will achieve their potential to deliver strong VfM when their processes are good – when appraisal of experimental efforts is timely, consistent, knowledgeable and politically astute. As these programmes mature in their implementation phase, VfM evaluation should be focused on checking for a culture of adaptation and learning supported by strong adaptive processes. • While economy is important for adaptive programmes, it is also important that keeping costs low does not deprive teams of the resources, staff, and management and administration time they need to gather information, experiment, learn and adapt.

Evidence base for Collaborating, Learning, and Adapting: summary of the literature review (2020 update) - Learning Lab (USAID, 2020)
The LEARN contract and the United States Agency for International Development/Bureau of Policy, Planning, and Learning (USAID/PPL) are managing an area of work known as the Evidence Base for Collaborating, Learning, and Adapting (EB4CLA). The purpose of this work is to answer the following key learning questions: ● Does an intentional, systematic, and resourced approach to collaborating, learning, and adapting (CLA) contribute to organizational effectiveness and/or development outcomes? ● If so, how? And under what conditions? ● How do we know? How do we measure any contribution that CLA makes to development results? As we began this work, we identified the need to conduct a literature review looking at these questions to understand what is known, what remains unknown, and how others have tried to answer these questions to date. We were primarily interested in answering these questions: ● What evidence is there, if any, that collaborating, learning, and/or adapting contributes to organizational effectiveness, development outcomes, or both? What are the strongest pieces of evidence? ● Does the literature identify any factors critical to CLA that are not currently included in the CLA framework? ● Who else is working on measuring the impact of collaborating, learning and adapting? ● What methods and measures did researchers use to study the effects of collaborating, learning, and adapting? ● Where are there gaps in the research relevant to collaborating, learning, and adapting? ● When taken together, what practical guidance does the evidence for collaborating, learning, and adapting offer to practitioners and policy makers to improve organizational effectiveness and development outcomes?

What difference does CLA make to development? Key findings from a recent literature review (2020 update) - Learning Lab (USAID, 2020)
USAID’s Bureau for Policy, Planning and Learning and its LEARN support contract are working to integrate systematic, intentional and resourced collaborating, learning and adapting (CLA) throughout program planning and implementation to achieve more effective development programs. As part of this effort, USAID is exploring several approaches to understand whether and how strategic collaboration, continuous learning and adaptive management make a difference to organizational effectiveness and development outcomes. To begin this work, we have undertaken a foundational literature review of academic and gray literature to answer our key learning questions: • Does an intentional, systematic and resourced approach to collaborating, learning and adapting contribute to organizational effectiveness? To development outcomes? • If so, how? And under what circumstances? • How do we measure the contribution? The 13 key findings, mapped to the CLA Framework below, are described in greater detail in the following pages.

Process Tracing as a Practical Evaluation Method: Comparative Learning from Six Evaluations - Wadeson, Alix; Monzani, Bernardo; Aston, Tom (2020)

What we’ve learned about adaptive management from our Somalia education programming - Heales, Charlotte (Care Insights, 2020)
Adaptive management in its various incarnations has long been a focus of a development community that is more and more frequently bumping up against the barriers of complexity, and looking for ways to overcome its challenges. In a field where we consistently have to deal with multifaceted problems,...

Decide Madrid: A Critical Analysis of an Award-Winning e-Participation Initiative - Royo, Sonia; Pina, Vicente; Garcia-Rayado, Jaime (Sustainability, 2020)
This paper analyzes the award-winning e-participation initiative of the city council of Madrid, Decide Madrid, to identify the critical success factors and the main barriers that are conditioning its performance. An exploratory case study is used as a research technique, including desk research and semi-structured interviews. The analysis distinguishes contextual, organizational and individual level factors; it considers whether the factors or barriers are more related to the information and communication technology (ICT) component, public sector context or democratic participation; it also diﬀerentiates among the diﬀerent stages of the development of the initiative. Results show that individual and organizational factors related to the public sector context and democratic participation are the most relevant success factors. The high expectations of citizens explain the high levels of participation in the initial stages of Decide Madrid. However, the lack of transparency and poor functioning of some of its participatory activities (organizational factors related to the ICT and democratic dimensions) are negatively aﬀecting its performance. The software created for this platform, Consul, has been adopted or it is in the process of being implemented in more than 100 institutions in 33 countries. Therefore, the ﬁndings of this research can potentially be useful to improve the performance and sustainability of e-participation platforms worldwide.

Adaptive Bureaucracies: lessons from DFID to the world - Simpson, Lea (LearnAdapt, 2020)
How many times have you worked on something that you know is failing, or just isn’t going as well as we’d all have hoped, but haven’t had…

Governance Diaries: An Approach to Governance Research from the Ground Up - Loureiro, Miguel; Joshi, Anuradha; Barnes, Katrina; Chaimite, Egídio (IDS, 2020)
Research on empowerment and accountability tends to focus on collective action and its potential for empowering citizens undertaking the action and on achieving state accountability. In fragile, conflict and violence-affected settings (FCVAS) collective action is rare and risky. So how do citizens, particularly the chronically poor and most marginalised, interact and make claims on the different public authorities that exist in these settings, and how do these interactions contribute to citizens’ sense of empowerment and accountability? Given the current agenda of ‘leave no one behind’, an understanding of how such populations interact with public authorities to meet their governance needs can help identify the constraints to achieving development for all in these challenging settings. We developed ‘governance diaries,’ a cross between a panel survey and multi-sited ethnographies, as an iterative approach to capture their experiences around governance issues over time. We explain here how this approach works, and the challenges and opportunities it offers for research.

Getting intentional about M&E: choosing suitable approaches for adaptive programmes - Management, Global Learning for Adaptive (GLAM Blog, 2020)
Does the choice of monitoring and evaluation (M&E) approaches and tools matter for adaptive programmes? In short, yes: monitoring, evaluation and learning (MEL) and adaptive management (AM) are intertwined. While programme monitoring data and evaluation results are not the only sources of evidence that programmes use for learning and iteration, they often are amongst most important ones — or at least they should be. Selecting what type of information to collect and analyse — and how — is critical for any type of programme. However, what AM especially focuses on is intentionally building in opportunities for structured and collective reflection, ongoing and real-time learning, course correction and decision-making in order to improve effectiveness.

Actually Navigating by Judgment: Towards a new paradigm of donor accountability where the current system doesn’t work - Honig, Dan (Center for Global Development, 2020)
This working paper explores how donors can move towards greater Navigation by Judgment, highlighting the actions people inside and outside aid agencies can work to make change—encouraging more Navigation by Judgment on the margin, starting today.

Managing Better: What All of Us Can Do to Encourage Aid Success - Honig, Dan (Center for Global Development, 2020)
Management by way of top-down controls and targets sometimes gets in the way of aid donors’ aims, undermining project success. These unhelpful controls often stem from a need to account for performance; legislatures or executive boards induce agencies to exercise tight process controls and orient projects towards what is measurable and reportable.

Transforming our work: Getting ready for transformational projects - Kehrer, Daniel; Flossmann-Kraus, Ursula; Ronco Alarcon, Sabrina Valeria; Albers, Vivien; Aschmann, Gwendolin (GIZ, 2020)

Change theory and theory of change: what’s the difference anyway? - Reinholz, Daniel L.; Andrews, Tessa C. (International Journal of STEM Education, 2020)
This commentary focuses on the difference between a theory of change and change theory, as it relates to systemic change projects in STEM higher education. A theory of change is project-specific and related to evaluation. It makes the underlying rationale of a project explicit, which supports planning, implementation, and assessment of the project. In addition, a theory of change is often required by funding agencies as part of grant proposals. In contrast, change theories represent theoretical and empirically grounded knowledge about how change occurs that goes beyond any one project. Ideally, a theory of change is informed by change theories. This essay describes the connections between a theory of change and change theory and provides examples of how change theory can inform a project’s theory of change. Grounding projects in change theory allows change agents to draw on existing knowledge and to better contribute to our collective knowledge about how to achieve meaningful change in STEM higher education.
Keywords: Change; Change theory; Education reform; Reform; Theory of change; Undergraduate STEM

Self Sabotage: Why Doing the Right Thing Results in Failure - Newell, Peter (War on the Rocks, 2020)
In January 2010 improvised explosive device (IED) attacks against dismounted infantry squads in Afghanistan numbered in the single digits — with only two

Practising change together – where nothing is clear, and everything keeps changing - Sharp, Cathy (Action Learning: Research and Practice, 2020)
This paper explores the thinking and practice of ‘action inquiry’ an embedded learning practice that can help navigate complexity when practising change together. The paper uses examples from social contexts where there are concerns about community wellbeing and health care. These are drawn from collaborative or collective leadership development programmes within public services that seek to bring new attention to the qualities of how people think, converse and interact, as part of their collective professional practice. This treats social action as a relational and dialogical practice, something that we do together as professionals by engaging in reflective inquiry and action. The paper suggests that action inquiry offers a prospect of rekindling the links between ‘action learning’ and collaborative leadership by developing a co-mission and a mutual commitment to a new type of learning partnership. Action inquiry can be wrapped around and enmeshed within initiatives and programmes that work with complexity, anywhere where effective social action will depend on the quality of relationships that can be developed. This research was funded by two separate Scottish Government commissions, where the author was a learning partner. The paper also draws on the further reflections of some of the practitioners most centrally involved.
_eprint: https://doi.org/10.1080/14767333.2020.1712838

A New Policy Narrative for Pastoralism? Pastoralists as Reliability Professionals and Pastoralist Systems as Infrastructure - Roe, Emery (ESRC STEPS Centre, 2020)
This paper proposes that pastoralist systems are better treated, in aggregate, as a global critical infrastructure. The policy and management implications that follow are significant and differ importantly from current pastoralist policies and recommendations. A multi-typology framework is presented, identifying the conditions under which pastoralists can be considered real-time reliability professionals in systems with mandates preventing or otherwise avoiding key events from happening. The framework leads to a different policy-relevant counternarrative to pastoralism as understood today. Some features of the counternarrative are already known or have been researched. The paper’s aim is to provoke further work (including case research and interactions with decisionmakers) on how robust the counternarrative is as a policy narrative for recasting today’s pastoralist policy and management interventions.
Accepted: 2020-01-14T10:22:59Z

Embracing complexity - Towards a shared understanding of funding systems change - Ashoka (Ashoka, 2020)

Contribution Rubrics: A simple way to assess influence - Aston, Tom (Independent Consultant, 2020)
This document explores how organisations can measure the level of influence that they had over an outcome. It summarises influence in terms of the significance of the outcome, the level of contribution and the strength of evidence.

Adaptive Management in SDC: Challenges and Opportunities - Prieto Martin, Pedro; Apgar, Marina; Hernandez, Kevin (IDS, 2020)
Adaptive management (AM) is a programme management approach that helps international development organisations to become more learning-oriented and more effective in addressing complex development challenges. AM practices have been applied for decades within other sectors as varied as logistics, manufacturing, product design, military strategy, software development and lean enterprise. At its core, AM is not much more than common sense, as it essentially recognises that the solutions to complex and dynamic problems cannot be identified at the outset of a programme but need to emerge throughout the process of implementation as a result of systematic and intentional monitoring and learning. The generic AM process typically involves an iterative cycle of design, implementation, reflection and adaptation activities, supported both by system monitoring and stakeholder involvement to obtain a better understanding of the evolving system and improve how the intervention is managed. A favourable context for AM in development. During recent decades, the international development sector has aimed to increase its results and impact orientation. As a result, a growing number of development organisations and governments have become increasingly aware of the limitations of traditional ‘linear and prescriptive’ programming approaches. They are now recognising the need to handle complexity better, and have begun to adapt their policies and practices to facilitate adaptive approaches. The World Bank, for example, now acknowledges that aid agencies need to increase flexibility of implementation, tolerate greater risk and ambiguity, devolve power from aid providers to aid partners, and avoid simplistic linear schemes for measuring results. Multilateral and bilateral organisations such as the World Bank, the United Kingdom’s (UK) Department for International Development (DFID) and the United States Agency for International Development (USAID) are currently experimenting with adaptive approaches. A multitude of adaptive approaches and communities of practice have emerged that aim to improve the effectiveness of aid, including Collaborating, Learning, and Adapting, Thinking and Working Politically, Doing Development Differently, Market Systems Development, Conflict-Sensitive Programme Management, and Science of Delivery. Since generic AM approaches have existed for decades in other sectors, AM has the potential to act as a neutral ‘bridge language’ that facilitates exchange and learning among the different communities and donors. This report is the result of a learning partnership between the Swiss Agency for Development and Cooperation (SDC) and the Institute of Development Studies (IDS). It assesses the relevance of AM to SDC, how it relates to working practices across SDC, and the key challenges and opportunities for SDC. Its process of elaboration involved a literature review on AM, an exploration of AM approaches from several bilateral donors, a series of 6 interviews with SDC staff and partners working in different countries and thematic domains, and a learning workshop at SDC headquarters (HQ), where staff from several SDC divisions reflected on AM and on how to advance the organisation’s capacity for adaptive programming and learning.

Thinking and Working Politically in development - Coalitions for Change in The Philippines - Sidel, John T.; Faustino, Jaime (The Asia Foundation, 2020)
The Asia Foundation and the Australian Embassy in the Philippines today released a new publication, Thinking and Working Politically in Development: Coalitions for Change in the Philippines. Written by London School of Economics and Political Science Professor John T. Sidel and The Asia Foundation’s Jaime Faustino, the book examines the first phase of the Coalitions for Change program (2012-2018) and the contributions to key development policy reforms in the Philippines. The book is a rigorous treatment of the Coalitions for Change program’s transformative policy reforms—alongside lessons from its failures—across diverse policy arenas and in a wide variety of cities and provinces. The chapters are organized thematically: excise tax reform (Chapter 2), land governance reform (Chapter 3), education (Chapter 4), electoral reform (Chapter 5), disaster risk reduction and management (Chapter 6), and subnational governance reform and conflict resolution in Mindanao (Chapter 7). The co-authors together combine an independent, academic perspective on the program’s impacts (Sidel) with a front-row view of doing policy reform in the Philippines – both its political and technical dimensions (Faustino). Based on the empirical research and comparative analysis undertaken by the authors, the book articulates – and substantiates – a strong set of arguments that help to explain the program’s mixed pattern of achievements and disappointments. Overall, the book concludes that the seven-year program achieved significant and sustainable impact using problem-driven, adaptive, and iterative approaches to developmental change. The authors assert the program was at the forefront of notable development approaches: aid effectiveness and development around thinking and working politically, doing development differently, and adaptive programming. Graham Teskey, an early advocate of ‘thinking and working politically’ explores how development agencies can replicate the conditions for success in his Afterword. With illustrative case studies and analyses, the book provides valuable lessons for policymakers, scholars, bilateral agencies, think-tanks, and anyone interested in successfully maneuvering the shifting dimensions of development in the Philippines and elsewhere.

Contribution analysis for adaptive management - Apgar, Marina; Hernandez, Kevin; Ton, Giel (IDS/GLAM, 2020)
This briefing note shares practical learning on the use of contribution analysis for adaptive management (CA4AM). It examines how the approach enables programmes to work with theories of change in a practical, reflexive way, and how, combined with assessing evidence of a programme’s contribution to change, its findings can inform programme adaptation. It also examines both how and to what extent CA enables AM through the experiences of four large complex programmes all working towards systems-level change and employing a structured process of reflection on theories of change. Key messages CA4AM can enable programmes to work with theories of change in a practical, reflexive way. It is particularly useful for programmes operating in conditions of complexity, when it is difficult to discern attribution and when systems-level change is the goal. A range of enabling factors help CA4AM to be used most effectively, including contractual flexibility; embedded monitoring, evaluation and learning (MEL); and supportive leadership.

Unpacking Mechanisms in Climate Resilient Agriculture Interventions - Barrett, Sam; D'Errico, Stefano; Anderson, Simon; Nebsu, Bayu (New Directions for Evaluation, 2020)
The investigation of causal mechanisms has the capacity to provide donors and implementing institutions with a greater understanding of people's reasoning and reactions as they work with interventions. This chapter contributes to the literature by identifying behavioral mechanisms generated through engagement with climate-resilient agriculture interventions within a larger livelihood project in Ethiopia. It works through the steps that enabled the study to unpack the black box between the climate-smart interventions and the outcome of crop production. The first step was a matching-based sampling design, following households over 3 years with six biannual surveys, and setting up a quasi-experimental evaluation setting. The second step used the data generated from the surveys in difference-in-difference models to assess the impact of the intervention on crop production. Third, to gain insight into the mechanisms at work, Focus Group Discussions (FGD) were conducted with smallholder farmer project beneficiaries. The FGDs revealed what the beneficiaries themselves considered the key mechanisms generated from the intervention, thus forming the bridge between the interventions and outcome. The result was an evaluation design enabling deeper insight into attribution claims. The findings offered novel insights for policymakers about how the climate-resilient interventions worked for the people themselves and shedding light on the inner workings of the climate-smart technologies. Finally, they provided key stakeholders (commissioning agency and implementing organizations) with a powerful means by which to learn about the last project, so to better plan for the next iteration and improve the climate resilience of smallholder farmers.

Letting Evidence Speak for Itself: Measuring Confidence in Mechanisms - Befani, Barbara; D'Errico, Stefano (New Directions for Evaluation, 2020)
This chapter argues that the credibility of causal mechanisms can be greatly increased by formulating them as statements that are both empirically falsifiable and empirically confirmable. Whether statements can be so depends on the potential availability of the relevant evidence (e.g., no evidence exists that can prove or disprove the existence of God, but good quality evidence is potentially available in many other cases). The Bayes formula can be used to measure the extent to which a given set of empirical observations supports or weakens the belief that a causal mechanism exists. With this approach, confidence in the existence of a mechanism is increased or decreased through a process that can be open, transparent, and shared with the public or groups of stakeholders, reducing cognitive biases, and improving internal validity and consensus around the existence of given mechanisms. The approach is showcased in the evaluation of a learning partnership whereby a knowledge product released by a research organization influenced policy at the municipal level.

Using a Participatory Theory Driven Evaluation Approach to Identify Causal Mechanisms in Innovation Processes - Blundo‐Canto, Genowefa; Devaux‐Spatarakis, Agathe; Mathé, Syndhia; Faure, Guy; Cerdan, Claire (New Directions for Evaluation, 2020)
Applied agricultural research institutes play different roles in complex agricultural innovation processes, contributing to them with other actors. To foster learning and usable knowledge on how research actions influence such lasting innovation processes, there is a need to identify the causal mechanisms linking these actions and the effects of the changes they enable. A participatory, theory-driven, ex-post evaluation method, ImpresS, was developed by the French Agricultural Research Center for International Development (Cirad). ImpresS reconstructs the innovation history and its impact pathway by analyzing behavioral mechanisms linked to stakeholders' individual reactions and responses, and underlying process mechanisms at a group level. ImpresS relies on iterative updating and refinement and on triangulating data sources and collection methods to ensure internal validity and to increase credibility by enabling different actors to express their viewpoints. Drawing on an in-depth case study, we discuss how ImpresS makes it possible to draw robust conclusions on causal mechanisms while posing challenges linked to the group dynamics and power imbalances commonly encountered in participatory methods. As demonstrated by the case study, ImpresS generates policy-relevant knowledge for future research projects. It also demonstrates how research actions can help coconstruct lasting dynamics that can survive fluctuating institutional support. Distinguishing between behavioral and process mechanisms benefits knowledge use as it makes it possible to disentangle the conditions that trigger changes in a given context while generating research questions concerning the external validity of mechanism hypotheses.

Outcome mapping: learning brief - Buell, Stephanie; Malallah, Haneen; Mason, Paige (ODI/GLAM, 2020)
Adaptive programmes recognise that certain changes, particularly in behaviours, are complex, non-linear and difficult to measure. This briefing note explores the use of outcome mapping (OM) as a monitoring, evaluation and learning (MEL) approach to track behavioural change and inform adaptation for two programmes: the Pathways to Resilience in Semi-arid Economies (PRISE) research consortium and the Accountability in Tanzania programme (AcT I and AcT II). It discusses the implementation of OM, the ways in which it has enabled adaptation and enabling contexts in order to identify key considerations for MEL specialists and programme managers as they determine whether OM may be the right fit, and how best to use the approach. Key messages OM has a number of different benefits as a MEL approach, including unpacking different uses of information at different levels of programme implementation; helping to develop a common language around progress markers; and going beyond monitoring to inform adaptation throughout implementation. These benefits are important aspects of monitoring, evaluation and learning for adaptive management (MEL4AM), as they provide richer evidence for decision-making at a frequency that could mean real-time learning and change. OM works best when it is embedded throughout the organisation, and as part of programme and organisational culture, rather than tasked to a MEL unit or individual.

Using the Business Model Canvas for Social Enterprise Design - Burkett, Ingrid (The Yunus Centre, Griffith University, 2020)
What should come first? Your business plan or your business model? I strongly believe it’s option B — your business model. But so many social enterprises I’ve seen, get it backwards which means they struggle to articulate how they create, deliver and capture value. This is what first inspired me to create the Business Model Canvas for Social Enterprise Design in 2013. It outlined how I had used, tested and learnt about the Business Model Canvas, created by Alex Osterwalder and Yves Pigneur, adapted for designing and growing social enterprises. Since then I’ve applied the Canvas in a range of contexts and learnt more about some of the nuances of using it in relation to social and impact enterprises. Note, I now tend to use the nomenclature of ‘impact enterprises’ rather than just social enterprises, but for ease of continuation from the first edition I’ve stuck with the terminology. It should however be recognised that the ‘impact’ referenced in the document could refer to social, environmental, cultural or economic outcomes that impact people, communities or societies. How to use the Canvas The Business Model Canvas for Social Enterprise Design and accompanying booklet is a companion to your strategy and planning. It’s a tool for you to get clear on how you’re creating revenue, how you’re creating impact, for who, and how both sides interact. You can use it at any stage of an enterprise’s lifecycle — before or at start-up, for testing a ‘minimum viable product’ model, for growth and development, and when pivots are required to build viability or sustainability. Completing a business model is not a desk activity, it is an immersive, practical, collective and engaged exercise that needs to be repeated over time. I like to use large forms of the BMC, in group settings and I like to display the canvas following a workshop so that team members and stakeholders of the social enterprise can engage, understand and continue to co-design the business model. Impact enterprises are extraordinary entities. If we can design their business models to reflect the best blends of commerce and impact we have the opportunity to truly transform people, places and planetary futures. What better way is there to do business?

Learning From Failure 2020 - What CARE’s evaluations tell us about how to improve our work - CARE (CARE, 2020)
“Ever failed. No matter. Try again. Fail again. Fail better.” –Samuel Beckett Here’s my favorite part of that quote: the ultimate goal is not a lack of failure; it’s better failures. That’s good news for CARE, because we just published round two of our Learning From Failure initiative, and…I know this will surprise everyone…we haven’t stopped failures yet. We do have some hopeful signs that we’re failing better; or at least, that we’re improving on some concrete weaknesses we identified in the first round. It’s an interesting process to launch the second phase of learning from failure. The first round, we didn’t know what we were going to find. We spent as much time talking about how it was the first-ever report of its kind as we did about the actual failures. Our case study admitted, “It's still very early to see specific development impacts.” Round two isn’t quite the same. It’s not new anymore, so there’s less excitement at having invented something. We’re not discovering data and themes for the first time. In a lot of ways, the stakes are higher. Round two of learning from failure becomes an exercise in continuous performance improvement, rather than a journey of discovery. If we don’t see improvements, we don’t have the excuse that it’s too early to tell. It also takes a sustained commitment. Launching an exploratory exercise at a small scale is easy, especially when no one quite knows what the answers will be. Pulling together a few pieces of content over a few months is pretty straightforward. It takes some staying power—and real support from leadership—to keep up the work over time, especially in the middle of a pandemic. That’s even more true once we’ve seen one round of results and had a chance to understand the work that it takes to improve.

Listen carefully. Tread lightly. Adapt quickly. Approaching Adaptive Management: Examples from our Somalia Education Programming - CARE (CARE, 2020)
Adaptive management approaches potentially offer us opportunities to deliver high quality results in circumstances where change is complex, including in fragile, unstable or conflict affected places. However, building adaptive programming continues to be a challenge for the sector. For CARE, our Department for International Development -UK Aid funded Girls’ Education Challenge (GEC) programming has provided useful learning on how to operationalise adaptive approaches. In this paper we expand on our learning from this project and offer some recommendations for how to create more opportunities for truly adaptive programming in the future. In particular: • Projects that are designed to adapt need budget structures, results frameworks and governance that enable the process of adaptation. In our GEC projects the approaches employed by DFID, including the introduction of Review and Adaptation meetings have served to support meaningful adaptation. • Adaptive projects require both strong participatory elements and flexible governance and accountability structures. Whilst rigorous and comprehensive Monitoring, Evaluation and Learning (MEL) systems are important without these other elements appropriate adaptation can be hindered. • Adaptive Management requires resources. Where the expected change is complex, adaptation is frequently necessary to ensure we are responding to context and evidence. This should be adequately resourced if we are to expect results. In an environment where many INGOs work consistently within complex environments, the sector also needs more opportunities to trial these approaches and could benefit from more funding streams available which include the kinds of approaches used by DFID in current GEC programming

Exit for Sustainability Checklists - Cekan, Jindra (Valuing Voices, 2020)
These Checklists for Exit along the Sustainability Project Cycle are based on lessons from ex-post evaluations that Valuing Voices has done or researched. Ex-post project completion evaluations are rare and even more so those that consulted partners and participants in the field about sustained impacts. Over $3.5 trillion has been spent on public foreign aid projects in the past 70 years (OECD 2019) yet we have evaluated less than 1% of these projects for sustainability. Our Valuing Voices ex-post research of 39 organizations’ evaluations of sustainability shows that most project results decrease (20-90%) as early as two years ex-post in addition to An Asian Development Bank study of post-completion sustainability found that “some early evidence suggests that as many as 40% of all new activities are not sustained beyond the first few years after disbursement of external funding,” Most project exits are in the last quarter and sustainability handover assumptions are not validated expost. Learning from what was sustained helps us know how to exit for sustainability from the very onset of the project (green slices) as compared to the typical project cycle (orange), above. We encourage those tasked with funding, designing, implementing, monitoring & evaluating projects to use these longer checklists and view the full recording shared with participants. A partial PowerPoint can also be found on „Sustaining All of our Hard Work“ presentation for the Vienna Evaluation Network (10/20). These checklists are aimed at donors/designers and implementers of foreign aid projects outcomes and impacts and can be adapted by local NGOs, national governments, private sector, academics, to create exit plans. Local participation in creating these and feedback on how well exit is going will help them sustain results.

Changeroo - Theory of Change platform - Changeroo (2020)
Changeroo assists organisations, programs and projects with a social mission to develop and manage high-quality Theories of Change. It allows you to - together with stakeholders co-create Theories of Change, and - present these in an interactive and engaging way. It helps keep a Theory of Change alive, enables reflective monitoring, and supports capacity building among social organisations. Changeroo helps you utilize your Theory of Change for strategic learning, communication, stakeholder engagement, impact measurement, scaling, planning, monitoring and evaluation. Thus you build a culture of critical thinking, constant analysis, co-creation and continuous learning. In sum, a truly strategic approach to societal value creation that helps you to assist your target groups to flourish!

Theory of Change: From novice to master changemaker - Changeroo (2020)
Theory of Change is more than just a tool or instrument. It is a mindset! A mindset to optimise impact management. As the cornerstone of impact management it is indispensable to any purpose-driven organisation targeting social or environmental value creation. It offers you the cornerstone of a learning approach toward change and impact. A way to build a mindset and organisational culture of critical reflection, co-creation with stakeholders and constant analysis of what works, why, for whom and under what circumstances. Are you ready to take impact management to the next level with the help of Theory of Change? We offer you the premier online course to learn everything about this approach.

ParEvo - Web-assisted participatory evolution of scenarios - Davies, Rick (2020)
ParEvo is a method of exploring alternative futures or histories, using a participatory evolutionary process (hence ParEvo). The process is designed to be used by multiple people, to produce a branching structure of storylines about what did, or could, happen. Participants are anonymous and can choose the extent to which they collaborate with others. Participants are also able to tag, comment on, and evaluate the storylines that are generated. Completed exercises can be analysed using downloaded data describing the content of storylines and the structure of people’s participation. Outcomes can be both cognitive (e.g. how we think about the future) and behavioural (e.g. how we respond to the future).

How to design and plan public engagement processes: a handbook - Faulkner, Wendy; Bynner, Claire (What Works Scotland, 2020)
There is a growing hunger for more, and more meaningful, citizen participation in decisions that affect our lives. Across the world, calls for open government, workplace democracy, community empowerment are gaining support, as are innovative developments in deliberative democracy. The current COVID-19 pandemic makes these calls more pressing than ever, given the deepening inequalities it has caused and the complex challenges of building a progressive road out of the crisis. So, now more than ever we need people capable of designing and planning public engagement processes that are empowering and worthwhile. Experienced practitioners know that, without considerable forethought, care and preparation, public engagement processes risk achieving little or, worse, alienating people so that they never engage (with you or anyone else) again. This Handbook seeks to deepen people’s skills in designing and planning effective public engagement processes, by providing a structured four-stage framework for tackling the task. It draws on the authors’ extensive practical experience of training and working with public engagement facilitators across sectors as well as international expertise. You may be a citizen, a community or public engagement practitioner, an elected or government representative, or some other sponsoring organisation or stakeholder. You may be new to this kind of work or experienced but wanting to review and improve your practice. Or you may be studying public participation in democratic processes. Wherever you are coming from, and whatever type of public engagement you are doing, this Handbook promises to be a useful addition to your toolbox.

Plans Are Worthless but Planning Is Everything: A Theoretical Explanation of Eisenhower’s Observation - Garcia Contreras, Angel F.; Ceberio, Martine; Kreinovich, Vladik (Decision Making under Constraints, 2020)
The 1953–1961 US President Dwight D. Eisenhower emphasized that his experience as the Supreme Commander of the Allied Expeditionary Forces in Europe during the Second World War taught him that “plans are worthless, but planning is everything”. This sound contradictory: if plans are worthless, why bother with planning at all? In this paper, we show that Eisenhower’s observation has a meaning: while directly following the original plan in constantly changing circumstances is often not a good idea, the existence of a pre-computed original plan enables us to produce an almost-optimal strategy—a strategy that would have been computationally difficult to produce on a short notice without the pre-existing plan.

Shifting Relationships with Grantees: Advancing Trust-Based Philanthropy through Emergent Learning - Gaspari, Brittney (The Emergent Learning Community Project, 2020)

Beneficiary Feedback Mechanisms - Hayman, Rachel (INTRAC, 2020)
There is a growing emphasis among NGOs and donors on ensuring that the voices of beneficiaries are heard. A range of beneficiary feedback mechanisms (BFMs) exist which enable beneficiary perspectives and suggestions to be gathered and used within an M&E system. BFMs are tools designed to enable a continuous cycle of interaction between those receiving and those delivering aid-funded interventions.

The Hidden Life of Theories of Change - Ho, Wenny; Tamas, Peter; van Wessel, Margit (Hivos, 2020)
Theory of Change is thought to be very useful for learning and adaptive management of complex interventions such as advocacy. Nevertheless, the use of Theory of Change is also under critique. One common criticism is that Theory of Change is often used as a framework that fixes agreements rather than as a living, guiding tool that helps reflection and adaptation. However, while such criticism stresses forms of control, little research has looked at the way Theory of Change and advocacy practice relate. This is a pertinent issue considering that formally agreed Theories of Change and realities on the ground can be very different. This raises questions: Do advocates work in ways different from what Theory of Change states, and if so, how, and why? How does the way they strategize relate to formal Theories of Change? With what implications? In this brief, we explore these more hidden aspects of the life of Theories of Change.

Examining the results and adaptation ideas in foreign aid - Janus, Heiner (University of Manchester, 2020)
This thesis applies ideational and institutional theories to analyse how two specific ideas, results and adaptation, have changed the theory and practice of development cooperation. The thesis addresses the question of why the results and adaptation ideas are often treated as binaries and how this debate has evolved historically. In a first theoretical paper, the evolution of results and adaptation is conceptualised as a combination of institutional layering and diffusion within development organisations. The second theoretical paper applies ideational theory, in particular, the coalition magnet framework, to China as a donor country. The empirical papers apply ideational and institutional theories to study aid projects funded by the World Bank and China in the Rwandan agriculture sector. The third paper analyses through which mechanism, results-based principal-agent relationships or problem-driven iterative adaptation, the World Bankâs Program for Results in the agriculture sector in Rwanda has led to increased agricultural productivity. The paper combines causal process tracing and contribution analysis to investigate two underlying theories of change of the Program for Results. The fourth paper applies the same framework and methodology to the Chinese Agricultural Technology Demonstration Center in Rwanda. The fifth paper compares both projects, the World Bank project and the Chinese project. The thesis finds that the ideas of results and adaptation are often presented as mutually exclusive mainly at the general level of public philosophies or paradigms, but show overlap and potential for integration on the level of framing policy problems and policy solutions. The thesis also demonstrates that there is unexplored potential for convergence between China and Development Assistance Committee donors around âcoalition magnetâ ideas. The empirical part of the thesis reveals how results-based and adaptive causal mechanisms co-exist within given aid interventions by the World Bank and China, how these interact and how they ultimately contribute to achieving development outcomes. The key finding is that the broader political context of the Rwandan agricultural sector is the main factor for determining development outcomes, which neither the World Bank project nor the Chinese projects take into account. The comparison of the World Bankâs and Chinaâs interventions finds that donor organisations need to address how results-based ideas in combination adaptive development ideas can be better tailored to fit into the specific context of the Rwandan agriculture sector.

The Actor-Based Change Framework: A Pragmatic Approach to Developing Program Theory for Interventions in Complex Systems - Koleros, Andrew; Mulkerne, Sean; Oldenbeuving, Mark; Stein, Danielle (American Journal of Evaluation, 2020)
Despite a wide body of literature on the importance of program theory and the need to tackle complexity to improve international development programming, the use of program theory to underpin interventions aimed at facilitating change in complex systems remains a challenge for many program practitioners. The actor-based change framework offers a pragmatic approach to address these challenges, integrating concepts and frameworks drawn from complexity science and behavioral change literature to develop robust program theory for complex interventions. This article presents the conceptual framework for the approach and describes how it has been applied in practice on an evaluation of a security and justice program in Nepal. It concludes with a discussion of the strengths and weaknesses of the approach in practice and how it can be applied more widely to improve program theory for interventions in complex systems.

Crisis relief for adults & families everywhere - Featuring the mighty micro-VCoL - Lectica (Lectica Live, 2020)
In response to the COVID-19 pandemic and the resulting economic crisis, we're open-sourcing our most effective stress- and crisis-management learning tools. We call these tools micro-VCoLs™. All of the micro-VCoLs shared here can be practiced effectively just by following the instructions.

Virtuous cycles of learning (VCoL) and the +7 skills - Lectica (Lectica Live, 2020)
VCoL is a cycle of goal setting, information seeking, application, and reflection. Its +7 skills include reflectivity, awareness, seeking and evaluating information, making connections, applying knowledge, seeking and working with feedback, and recognizing and overcoming built-in biases. VCoLing engages the whole learner. By this, we mean that it engages learners emotionally, physically, and intellectually, leveraging both conscious and unconscious mental processes. VCoLing ensures that new knowledge is integrated into existing knowledge in a way that makes it useful and "sticky." When people work with our assessments, they're not only building knowledge, they're also nurturing the dispositions and skills required for a lifetime of learning and development.

What Is This Thing Called a Mechanism? Findings From a Review of Realist Evaluations - Lemire, Sebastian; Kwako, Alexander; Nielsen, Steffen B.; Christie, Christina A.; Donaldson, Stewart I.; Leeuw, Frans L. (New Directions for Evaluation, 2020)
Realist evaluation has, over the past two decades, become a widely used approach in evaluation. The cornerstone of realist evaluation is to answer the question: What works, for whom, under what circumstances, and why. This is accomplished by explicating the causal mechanisms that, within a particular context, generate the outcomes of interest. Despite the central role of mechanisms in realist evaluation, systematic knowledge about how the term mechanism is conceptualized and operationalized is limited. The aim of the present chapter is to examine how mechanisms are defined and applied in realist evaluations. Informed by the findings of the review, further conceptual and practical developments for future applications of mechanisms in realist evaluation are considered.

111 Evaluation Cartoons for Presentations and Blog Posts - Lysy, Chris (2020)

AdaptScan - Improving your Team's Adaptive Management - Mercy Corps (Mercy Corps, 2020)

A Pragmatic Approach to Assessing System Change: How to put it into practice - Miehlbradt, Alexandra; Shah, Rachel; Posthumus, Hans; Kessler, Adam (DCED, 2020)
Planning for and assessing system change is a strategic management issue. It is critical for everything from developing a strategy and designing interventions, to adapting strategy, improving implementation and reporting impact. But many programmes get stuck when it comes to assessing system change. The private sector development field has struggled to agree on an approach that programmes can implement and stakeholders can understand. However some mature programmes are starting to assess system change more effectively. Building on these emerging practices, this paper outlines a process that programmes can use to assess system changes regularly and practically. Two complementary papers: Overview and How to put it into practice The Overview summarises the approach and How to put it into practice provides more detailed implementation guidance, worked examples, and useful tips. The Overview explores how to: develop a system change strategy and intervention plans that lay the groundwork for system change assessment, including how to set system boundaries and how to identify the system changes a programme aims to catalyse assess system changes using both: - an intervention lens focused on changes introduced by specific interventions - a helicopter lens that provides a whole system view By analysing findings from both lenses, programmes can improve their strategy and report on their contribution to system change. How to put it into practice uses two case examples for illustration throughout the paper - PRISMA’s work in the maize system in East Java and Indonesia and S4J’s work in the Vocational Education and Training (VET) system in Albania. It targets practitioners responsible for facilitating and/or assessing system change. The paper explains how to: articulate the system changes that a programme aims to catalyse assess those changes use the results to inform decision making and reporting The approach described in the paper builds on the practices outlined in the DCED Results Measurement Standard. The guidance provided has been designed to be useful to programmes that aim to catalyse system changes whether or not they apply the DCED Standard.

Leveraging Experimental Evaluations for Understanding Causal Mechanisms - Peck, Laura R. (New Directions for Evaluation, 2020)
Experimental evaluations—especially when grounded in theory-based impact evaluation—can provide insights into the mechanisms that generate program impacts. This chapter details variants of experimental evaluation designs and also analytic strategies that leverage experimental evaluation data to learn about causal mechanisms. The design variants are poised to illuminate causal mechanisms related to program implementation and the contribution of selected components of multifaceted programs. The analysis strategies lend themselves to illuminating causal mechanisms related to participants’ responses to program components as well as to the contributions of selected program components themselves. The chapter offers an example from one, theory-based impact evaluation, which embedded both design and analytic strategies to examine the extent to which specific program components and participant experiences might be identified as causal mechanisms. The particular value in using this theory-based experimental strategy is that the results are rigorous and potentially highly relevant to policy and practice.

A Theory of Change of citizen participation: an update - Peña-López, Ismael (ICTlogy, 2020)
when it was reduced to a subsidiary internal service lacking all kind of political attributions. The work done in those years had been formidable, but too many things had passed since, especially the 15M Spanish Indignados Movement, the raise of technopolitics… and the raise of populism and fascism all across Europe. We urgently needed a theoretical framework in which to substantiate our political strategy, so I came up with a Theory of Change of citizen participation (see Figure 1) which defined four expected impacts of our political action: 1. Efficiency, efficacy and legitimacy of public decisions improves. 2. Populism has decreased in institutions and the public sphere. 3. Citizens understand the complexity of public decision-making. 4. Citizen participation and political engagement clearly shifts towards a technopolitical paradigm. These impacts were expected to be achieved after some outcomes resulting from some outputs grouped in five programmes: 1. Programme of citizen participation. 2. Programme of internal participation. 3. Programme of collaboration. 4. Programme of intermediaries, facilitators and infomediaries. 5. Programme of e-participation, e-voting and technopolitics. 20 months after, the Theory of Change of Citizen Participation has worked quite well. But it does have some limitations, especially at the operational level-which is what the whole thing was about, to help in putting some order in our daily work.

Measuring Governance, Advocacy and Power: A Database of Existing Indicators, Tools and Indices - Poirrier, Caroline; Tolmie, Courtney (R4D, 2020)
Measuring Governance, Advocacy, and Power is an excel sheet that brings together existing indicators, tools, and indices that may be useful to practitioners responsible for the measurement of outcomes in the field of governance, advocacy, and power in an easily accessible and filterable format.

A new bird's eye view on the agile forest - Portman, Henny (PM World Journal, 2020)

A Pragmatic Approach to Assessing System Change - Posthumus, Hans; Shah, Rachel; Miehlbradt, Alexandra; Kessler, Adam (DCED, 2020)
Planning for and assessing system change is a strategic management issue. It is critical for everything from developing a strategy and designing interventions, to adapting strategy, improving implementation and reporting impact. But many programmes get stuck when it comes to assessing system change. The private sector development field has struggled to agree on an approach that programmes can implement and stakeholders can understand. However some mature programmes are starting to assess system change more effectively. Building on these emerging practices, this paper outlines a process that programmes can use to assess system changes regularly and practically. Two complementary papers: Overview and How to put it into practice The Overview summarises the approach and How to put it into practice provides more detailed implementation guidance, worked examples, and useful tips. The Overview explores how to: develop a system change strategy and intervention plans that lay the groundwork for system change assessment, including how to set system boundaries and how to identify the system changes a programme aims to catalyse assess system changes using both: - an intervention lens focused on changes introduced by specific interventions - a helicopter lens that provides a whole system view By analysing findings from both lenses, programmes can improve their strategy and report on their contribution to system change. How to put it into practice uses two case examples for illustration throughout the paper - PRISMA’s work in the maize system in East Java and Indonesia and S4J’s work in the Vocational Education and Training (VET) system in Albania. It targets practitioners responsible for facilitating and/or assessing system change. The paper explains how to: articulate the system changes that a programme aims to catalyse assess those changes use the results to inform decision making and reporting The approach described in the paper builds on the practices outlined in the DCED Results Measurement Standard. The guidance provided has been designed to be useful to programmes that aim to catalyse system changes whether or not they apply the DCED Standard.

Keeping it Real: Using Mechanisms to Promote Use in the Realist Evaluation of the Building Capacity to Use Research Evidence Program - Punton, Melanie; Vogel, Isabel (New Directions for Evaluation, 2020)
This chapter explores the use of mechanisms within the realist evaluation of the Building Capacity to Use Research Evidence (BCURE) program, a £15.7 million initiative aiming to improve the use of evidence in decision-making in low and middle-income countries. The evaluation was commissioned to establish not just whether BCURE worked but also how and why capacity building can contribute to increased use of evidence in policymaking in the very different contexts in which the program operated. This chapter argues that using mechanisms helped provide nuanced and robust insights into these questions, while also strengthening the usefulness and policy relevance of the evaluation. Drawing primarily on qualitative data, including interviews with more than 500 stakeholders over 3 years, the evaluation explored the mechanisms that promote capacities to use evidence in decision-making, through developing and testing realist context-intervention-mechanism-outcome configurations (CIMOs). Uncovering the value of mechanisms for policy and program learning was not easy, and the chapter sets out some of the thorny challenges faced and how the BCURE evaluation navigated these. Ultimately, the use of mechanisms in the BCURE evaluation helped to generate practical and nuanced insights that fed directly into the design of a £17 million follow-up program. The seven mechanisms uncovered are continuing to inform our own and others’ work on institutional capacity change in a wide range of fields.

Getting Practical With Causal Mechanisms: The application of Process-Tracing Under Real-World Evaluation Constraints - Raimondo, Estelle (New Directions for Evaluation, 2020)
Over the past decade, the field of development evaluation has seen a renewed interest in methodological approaches that can answer compelling causal questions about what works, for whom, and why. Development evaluators have notably started to experiment with Bayesian Process Tracing to unpack, test, and enhance their comprehension of causal mechanisms triggered by development interventions. This chapter conveys one such experience of applying Bayesian Process Tracing to the study of citizen engagement interventions within a conditional cash transfer program under real-world evaluation conditions. The chapter builds on this experience to discuss the benefits, challenges, and potential for the applicability of this approach under real-world evaluation conditions of time, money, and political constraints.

No Mechanism Without Context: Strengthening the Analysis of Context in Realist Evaluations Using Causal Loop Diagramming - Renmans, Dimitri; Holvoet, Nathalie; Criel, Bart (New Directions for Evaluation, 2020)
Realist evaluation is an approach with a strong emphasis on causal mechanisms and the context in which they are triggered. However, recent reviews of published realist evaluations show that context is often understudied. This is problematic, as a thorough understanding of the relationship between context and causal mechanisms is crucial in assisting policymakers to make appropriate and targeted decisions that improve the intervention. Therefore, we set out to test whether combining realist evaluation with the “systems thinking” approach and, more specifically, causal loop diagramming, could help strengthen the analysis of context. We did this through a study of a performance-based financing (PBF) intervention in the Ugandan health care sector by the Belgian development agency, Enabel. PBF allocates funds to health workers and/or health facilities based on their performance, and introduces additional management support tools, provides extra monitoring and supervision, and promotes community participation in management issues, among other activities. In this case, we found that the proposed combined methodological approach indeed adds value to the analysis, as it leads to insights into the role played by the underlying system that otherwise may have been overlooked. Moreover, such information may provide clear directions to policymakers on how to improve the intervention in a sustainable way. Finally, causal loop diagrams help to visualize complex causal interactions and to communicate them to policymakers.

Pando - Root Change (Root Change, 2020)
Pando is a platform that gives organizations a way to visualize, learn from, and engage with the social systems in which they work. We have designed Pando to help users build trust, strengthen relationships, and work together to achieve greater social impact. Grounded in social network analysis, Pando allows users to collect and visualize organizations and their relationships in real-time on simple and easy-to-use relationship maps. Relationship maps are managed by map administrators. These are actors interested in learning about a particular system and making the tool available to those working in the system. Pando is integrated with Keystone Accountability’s Feedback Commons, an online tool that allows map administrators to collect and analyze qualitative feedback about levels of trust and relationship quality among map participants. Key features of Pando include relationship mapping, feedback surveys, and dashboards.

Special Issue: Causal Mechanisms in Program Evaluation - Schmidt, Johannes (New Directions for Evaluation, 2020)
_eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1002/ev.20357

The Causal Mechanism Claim in Evaluation: Does the Prophecy Fulfill? - Schmitt, Johannes (New Directions for Evaluation, 2020)
Despite increased discussions in the community and a common understanding about the virtue of mechanism-based explanation, little is known about the true benefits and challenges of applying causal mechanism analysis in practice. This chapter aims to introduce the reader to the topic of causal mechanisms and synthesize significant findings on this special issue. It begins by laying out definitions and concepts of causal mechanisms in evaluation literature and proposes a two-way classification of causal mechanisms along which the chapters to this issue are structured. The chapter continues by introducing the Causal Mechanism Claim and elaborates on how analyzing causal mechanisms is expected to increase policy relevance and causal capacity in evaluations. Drawing on this issue's rich corpus of firsthand practical experience, this introduction synthesizes key lessons that support or contradict the Causal Mechanism Claim. We find that both parts of the claim—increased policy relevance and strengthened causal capacity—are supported by the authors' experiences as bundled in this issue. However, we also identify challenges related to cross-cutting issues such as communication and practical applicability and point to the importance of method integration.

System Innovation Network - Si Network (Si Network, 2020)
The Si Network is an online platform for building the developing of systems innovation - connecting people around the world to learn and apply the ideas and methods of systems innovation towards addressing complex challenges and building better systems that work for all. Not sure what systems innovation is? Systems innovation is a new approach to innovation that tries to tackle complex social and environmental challenges through the use of more holistic & innovation driven approaches. It is a kind of innovation that aims to change the underlying structure of a system, thus potentially enabling a more transformational kind of change - systems change - rather than incremental "innovation as usual". What Do You Do? We are building an ecosystem of individuals and organizations co-learning and co-creating systems innovation across geographies and sectors. This ecosystem is enabled by our online platform which provides educational content, toolkits, organizes events and projects as well as provides various support services for organizations. Purpose Statement Our purpose is to build the world’s capacity for systems innovation - so as to better understand and address complex challenges and co-create a world where systems work for all. We envision a world where everyone thinks in systems and has an understanding of complex systems. From this understanding, we are able to design and develop regenerative systems that work for everyone. Our mission is to grow the field of systems innovation as a pathway to co-learn the ideas of systems thinking and apply them to co-creating new systems Who's Involved? We are a networked organization of some 17K+ members forming part of 20+ hubs in major cities around the world. Our community is broad and diverse in areas of work and interest from designers, innovators, and entrepreneurs, to researchers and management but they all share a common interest in learning and applying systems thinking ideas. The Si platform is developed and managed by a small core team based in London UK. We are registered as a business but operate as a social enterprise focused on our purpose of advancing the area of systems innovation in theory and practice. Core Value Creativity - We put creativity at the centre of what we do. Holding a space for curiosity, diversity of views, exploration and critical thinking as a pathway to transformative innovation. Openness - We strongly believe in openness in our ways of being, thinking and organizing as a sustainable pathway to creating an adaptive, scalable and dynamic community. Growth Mindset - We embrace challenges as opportunities for continuous personal and collective learning and development, with a never-ending potential to grow and start with a new beginning. Perseverance - When facing uncertainty and failure, perseverance is what is needed to maintain commitment and a resilient pathway aligned with our purpose. Care - Care is one of our core principles. We foster relationships based on honesty and empathy, striving to be present and conscious in what we do and taking responsibility for the effects of our actions.

Learning and Accountability - Simister, Nigel; Scholz, Vera (INTRAC, 2020)
Most people agree that monitoring and evaluation (M&E) should be used for both learning and accountability. However, there is no consensus about which one is more important. The debate matters as there is sometimes tension between the two purposes. In the past there has often been a disconnect between M&E and learning. Many M&E systems are primarily designed to enable accountability to donors.

Chaos Report 2020 - Standish Group (Standish Group, 2020)

Your Software, Their Hardware: Stacking the Deck for High-Quality Replication - Starr, Kevin (Stanford Social Innovation Review, 2020)
Your awesome model doesn’t get to serious scale unless others replicate it, too. Here’s how to make it happen.

Guide for Adopting Remote Monitoring Approaches During COVID-19 - USAID (USAID, 2020)
USAID maintains staff and operations in more than 80 countries around the world, all of which the COVID-19 pandemic will disrupt. USAID remains committed to protecting the health and safety of our staff, while continuing appropriate oversight of our programs and ensuring the accountable and effective use of U.S. taxpayer funds. In the current operating environment, USAID and implementing partners face new challenges in implementing activities, monitoring progress, collecting data, and tracking indicators. As we adapt our approaches, we will work with implementing partners to find innovative, responsible, and safe ways to monitor and evaluate programming. Digital tools can support novel approaches to remote monitoring. Responsible use of digital tools also supports operating unit alignment with the Digital Data Collection mandate in the Agency’s new Digital Strategy. This guide provides information for Agency staff and implementing partners on remote monitoring techniques and when they can be employed. We encourage use of this guide to identify and pursue appropriate remote-monitoring approaches for your needs. CORs/AORs and Activity Managers should work with implementing partners to document updated approaches in each agreement’s plans for monitoring, evaluation, and learning (MEL), and should upload these amended Activity MEL Plans into the Agency Secure Image and Storage Tracking System (ASIST) as soon as possible. This can be done in collaboration with Mission and Washington Bureau M&E Specialists, as appropriate

Evaluation of International Development Interventions: An Overview of Approaches and Methods - Vaessen, Jos; Lemire, Sebastian; Befani, Barbara (World Bank, 2020)
This guide provides an introductory overview of a range of methods that have been selected for their actual and potential use in the field of international development evaluation. For each method, a detailed guidance note presents the method’s main features and procedural steps, key advantages and disadvantages, as well as its applicability. Each guidance note includes references for relevant background readings (basic and advanced) as well as references to other additional resources of interest. Both the choice of approaches and methods and the associated guidance are by no means definitive. IEG plans to periodically update the guide as evaluation practices evolve.

Applying Rigorous Impact Evaluation in GIZ Governance Programmes: Results of a GIZ Initiative on Impacts in Governance - von Schiller, Armin (GIZ GmbH, 2020)
Pressure is mounting on international development cooperation agencies to prove the impact of their work. Private and public commissioners as well as the general public are increasingly asking for robust evidence of impact. In this context, rigorous impact evaluation (RIE) methods are increasingly receiving attention within the broader German development system and in GIZ. Compared to other implementing agencies such as DFID or USAid, the Deutsche Gesellschaft für Internationale Zusammenarbeit (GIZ) GmbH has so far relatively little experience in systematically applying rigorous methods of impact evaluation. This is particularly true in the governance sector. In order to gain more experience and to understand which methods and formats are best suited for GIZ governance programmes, the Governance and Conflict division and the Africa department launched the ‘Impact Initiative Africa’ in 2016, a cooperative effort with several programmes in Africa. The Initiative set out to apply the experiences from GIZ governance programmes to design and conduct RIEs, and to use the results to steer programme implementation. Initially, the Initiative included three countries: Benin (Programme for Decentralisation and Local Development), Malawi (Support to Public Financial and Economic Management) and Mozambique (Good Financial Governance in Mozambique). During its implementation, the Initiative also benefitted from the experience of two additional governance programmes which had already undertaken RIEs, namely Peru (Citizen-oriented State Reform Programme) and Pakistan (Support to Local Governance Programme II). This report summarizes the insights gained from these experiences and discusses opportunities and limitations regarding the use and usability of RIEs in GIZ governance programmes as well as proposals on how to organise RIEs to maximise learning potential and benefits for the specific programmes and the GIZ Governance sector at large.

Rigorous Impact Evaluation in GIZ Governance Programmes - von Schiller, Armin (Bonn, 2020)
Why should I integrate an impact assessment in my programme? How will the programme benefit from it? Are the benefits worth the effort and expenses? How do I design and implement it in detail? Who is addressable for support? What should I pay attention to in order to get the most out of it? This note is meant to answer these questions. It addresses leaders and project staff of governance programmes who are interested in using this tool within their specific governance programme or project. This note provides a guideline and good practice recommendations on how to design and conduct an impact assessment and on how to fully utilise the benefits of the results for the programme and for communication with commissioners, partners and other donors. Additionally, this note will point to indirect benefits that can arise and that should not be ignored. Results of impact assessments are highly relevant for the GIZ as an institution. However, in this note we stress the benefits for the programme or project itself. In particular this note addresses the following aspects: • What are rigorous impact assessments and why should GIZ Governance programmes conduct them more often within their programmes? • Which phases does an impact assessment include? How do I set one up and which aspects deserve special attention in each phase to maximise the benefits for my programme? • What are the benefits I can expect from implementing and impact assessment? • Whom to ask at headquarters in case I need support? This note complements the policy brief “Strategic use of Rigorous Impact Evaluation” and the corporate strategic review on “Rigorous Impact Evaluation” written by the GIZ evaluation unit which focuses on the strategic use of rigorous impact evaluations (RIE) at GIZ. Based on the review findings, the policy brief presents recommendations for strategic planning and implementation of purpose-sensitive RIE using a number of key levers. By adopting central coordination and needs-based support mechanisms, the evaluation unit intends to promote the strategic use of RIE for evidence-based learning and decision-making within the organisation.

Habits of a system thinker - WCST (Waters Center For Systems Thinking, 2020)
From the Waters Center For Systems Thinking. See: https://thinkingtoolsstudio.waterscenterst.org/courses/habits https://thinkingtoolsstudio.waterscenterst.org/cards

USAID Wildlife Asia as a case study in adaptive rigour - Ziegler, Jessica (ODI/GLAM, 2020)
This briefing note looks at how the United States Agency for International Development (USAID) Wildlife Asia programme has operationalised the concepts of adaptive rigour and adaptive management as part of its approach to collaborating, learning and adapting. As described by the Global Learning for Adaptive Management (GLAM) initiative, adaptive rigour is about ensuring that the data, information, methods, processes and systems that underpin adaptive management are robust, systematic and high‑quality. Key messages When faced with programmatic complexity, it is important to take an adaptive approach driven by continuous and iterative monitoring, evaluation and learning (MEL). USAID Wildlife Asia, which works to advance regional action towards ending illegal wildlife trafficking in Southeast Asia and China, has embraced this approach as a way of operationalising adaptive rigour. Throughout programming, MEL approaches should reflect the characteristics of adaptive rigour: comprehensiveness, usefulness, practicality, timeliness and support. Utilising performance monitoring and research in order to test and revise technical approaches and employing mixed methods to collect both qualitative and quantitative data, as well as looking for relevant lessons generated by others, can ensure access to the most useful information for decision-making throughout implementation. For adaptive management, it is not enough to monitor, evaluate and learn; it is also essential to pause and reflect in order to analyse and process evidence gained through MEL with colleagues and stakeholders to reach the right conclusions and make good decisions.

Adapting data collection and utilisation to a Covid-19 reality: Monitoring, evaluation and learning approaches for adaptive management - Ziegler, Jessica; Mason, Paige (ODI/GLAM, 2020)
This briefing note focuses on the remote collection and use of data for adaptive management during the Covid-19 pandemic, setting out key considerations to help practitioners think through a transition from more ‘traditional’ monitoring, evaluation and learning (MEL) to MEL for adaptive management (MEL4AM) that reflects the unique data collection challenges presented by Covid-19. The brief provides an overview of some key considerations in remote data collection, when this is required, and identifies other sources that address these issues in more detail. It concludes with a discussion of how to bring the information resulting from remote monitoring into decision-making to enable adaptive management. Key messages - When planning for remote data collection during the Covid-19 pandemic, first determine what information is still necessary, because data needs may have changed, e.g. if programming has pivoted or needs to pivot due to Covid-19. Then identify how the programme’s information needs align with existing data sources and what gaps remain, which will guide the need for remote data collection. - Also consider what data is ‘good enough’ for current decision-making needs in order to provide sufficient information to the right people at the right time to an acceptable standard of rigour. - There may be pragmatic reasons to reduce the number or scope of MEL activities, such as logistical constraints or ethical considerations introduced by the pandemic. - MEL activities should be accompanied by frequent feedback loops and pause points to reflect on emergent needs and challenges, information needs that have been met, and contextual changes that may affect MEL. - Be clear with decision-makers about the assumptions and gaps in the data, including proxies used and their limitations, sampling changes, and how these changes and assumptions may affect the decisions/options being discussed.

Evaluability Checklist for Post Project Evaluation - Zivetz, Laurie; Cekan, Jindra (Valuing Voices, 2020)
Considerations for planning a post project evaluation during the project, at the end, or after it has closed.

Handbook of Democratic Innovation and Governance - Elstub, Stephen; Escobar, Oliver (Edward Elgar Publishing, 2019)
Democratic innovations are proliferating in politics, governance, policy, and public administration. These new processes of public participation are reimagining the relationship between citizens and institutions. This Handbook advances understanding of democratic innovations, in theory and practice, by critically reviewing their importance throughout the world. The overarching themes are a focus on citizens and their relationship to these innovations, and the resulting effects on political equality. The Handbook therefore offers a definitive overview of existing research on democratic innovations, while also setting the agenda for future research and practice.
Google-Books-ID: P7bEDwAAQBAJ
Keywords: Democracy; Public Affairs & Administration

From purposeful to meaningful adaptive programming: how about adaptive operations? - Nixon, Nicola (Devpolicy Blog from the Development Policy Centre, 2019)
Embracing adaptive programming approaches but sticking with rigid, traditional systems-based operations? It's time to take a long hard look at all the boring stuff, says Nicola Nixon.

Supporting adaptive management. Monitoring and evaluation tools and approaches - Pasanen, Tiina; Barnett, Inka (ODI, 2019)
Key messages • This working paper introduces a set of monitoring and evaluation (M&E) tools and approaches, discussing their potential usefulness in supporting adaptive management in development and humanitarian programmes. It emphasises adaptive programmes characterised by complex aspects, such as: (1) they are innovative; (2) they have uncertain pathways for change; or (3) they operate in uncertain or unstable environments. • The majority of these tools have been used in international development for many years. However, adaptive management brings additional challenges for monitoring and evaluating programmes, as they require intentional M&E design from the start that is oriented towards both learning and accountability. • All of the tools and approaches introduced in this paper have potential to support learning and adaptation, although in various ways and at different stages of a programme. Some tools can support strategic planning and diagnosing throughout a programme – especially during design and inception – while others can help analyse causal relationships at specific points in a programme. It is important to tailor the approach used for its intended purpose. However, whether learning and adaptation happens depends also on factors other than the choice of M&E methods. • For some of these approaches a considerable body of evidence already exists but, for many, more practical examples and systematic analysis is needed. In addition to building the evidence base concerned with which approaches are suitable for different types of adaptive programmes and why, it is also important to improve understanding of the enabling environmental conditions necessary for the tools and approaches outlined here to facilitate and strengthen evaluative thinking, evidence-informed decision-making and ongoing programme iteration.

Big Enough. Simple Enough. Cheap Enough. - Starr, Kevin (Stanford Social Innovation Review, 2019)
Daunting social problems need scalable solutions. Here’s how to know if you’ve got one.

What are the Simple Rules that guide our Strategies? - Green, Duncan (From Poverty to Power, 2019)
A management treatise on 'simplre rules' for companies facing unpredictable situations could provide useful guidance to aid organizations

Applying Power Analysis: Using the ‘Powercube’ to explore forms, levels and spaces - Gaventa, John (Power, Empowerment and Social Change, 2019)
In a complex, globalised and rapidly changing world, power dynamics are multidimensional, constantly evolving, and full of complexity. The ‘powercube’ (Gaventa, 2006) is an approach to power analysis which can be used to examine the multiple forms, levels and spaces of power, and their interactions. Building on earlier work on power, and elaborated and popularised in collaboration with other colleagues through the web site powercube.net and numerous other resources, the powercube has been widely used around the world for analysis of power, education and awareness building, context analysis, programme and strategy development, and monitoring and evaluation. This article briefly outlines the evolution of the powercube, and provides examples of the issue areas in which it has been used, and for what purposes. Drawing on these, we then offer eight lessons of how to apply the powercube framework for analysing and transforming power relations.

Power, Empowerment and Social Change - McGee, Rosemary; Pettit, Jethro (2019)
This book uncovers how power operates around the world, and how it can be resisted or transformed through empowered collective action and social leadership. The stakes have never been higher. Recent years have seen a rapid escalation of inequalities, the rise of new global powers and corporate interests, increasing impunity of human rights violations, suppression of civil society, and a re-shaping of democratic processes by post-truth, populist and nationalist politics. Rather than looking at power through the lenses of agency or structure alone, this book views power and empowerment as complex and multidimensional societal processes, defined by pervasive social norms, conditions, constraints and opportunities. Bridging theory and practice, the book explores real-world applications using a selection of frameworks, tools, case studies, examples, resources and reflections from experience to support actors to analyse their positioning and align themselves with progressive social forces. Compiled with social change practitioners, students and scholars in mind, Power, Empowerment and Social Change is the perfect volume for anyone involved in politics, international development, sociology, human rights and environmental justice who is looking for fresh insights for transforming power in favour of relatively less powerful people.

Towards evidence-informed adaptive management - Hernandez, Kevin; Ramalingam, Ben; Wild, Leni (ODI/GLAM, 2019)
Development and humanitarian organisations seeking to be adaptive have emphasised the need to be transparent about complexity and uncertainty; to be honest about their inability to control what happens; and to design programmes that change over time to become more appropriate and relevant. At their heart, adaptive management approaches emphasise the ability to lean, 'unlearn' and adapt programming accordingly. The cornerstone of effective learning is the creation, gathering, accumulation, interpretation and use of data and evidence. This working paper provides development professionals with tools, strategies and ideas to help them use evidence for adaptive management in practical and evidence-informed ways. Key messages - Evidence is central to effective and rigorous adaptive management. However, despite this central importance, exactly how evidence has been used to inform decisions for adapting development and humanitarian programmes in the past remains unobservable to many. - There is a need to strengthen and document evidence-informed adaptive management. This working paper proposes a roadmap to do this. - Those seeking to use evidence for adaptive management will need to manage trade-offs between ensuring a rigorous, documented (and auditable) trail of evidence-informed actions, being pragmatic about the time and resources allocated to documentation and recognising that it may be necessary to proceed without rigorous evidence when it is unavailable.

Scrambling after Moving Targets: Monitoring & Evaluation Applied to Adaptive Management Approaches in Peacebuilding - Woodrow, Peter; Jean, Isabella (New Directions in Peacebuilding Evaluation, 2019)
In this landmark collection, the voices of pathMakers and innovators in peacebuilding evaluation are assembled to provide new direction for the field.
Keywords: IMPORTANT

The good and the ‘not so good’ of our experiences with SenseMaker - Mager, Franziska (Oxfam Views & Voices, 2019)
When we purchased a license for the SenseMaker research method in early 2017 (a proprietary data collection and analysis software), the excitement in our more “geeky” teams was palpable. SenseMaker ... Read More

Livelihoods in the Za'atari Camp: Impact evaluation of Oxfam’s Cash for Work activities in the Za’atari camp (Jordan) - Lombardini, Simone; Mager, Franziska (http://hdl.handle.net/10546/620883, 2019)

Stakeholder and Social Network Analysis - Guidance Note - IRC (International Rescue Committee, 2019)
Stakeholder Analysis is used to identify the actors and relationships that influence project outcomes. This guidance, from IRC, can help you determine how to work and who to partner, coordinate or engage with in order to best achieve the outcome. It builds on existing stakeholder information and typically will include a participatory internal meeting or workshop.

Agile Transformation: Structures, Processes and Mindsets for the Digital Age - Perkin, Neil Neil (Kogan Page, 2019)
LONGLISTED: CMI Management Book of the Year 2020 - Technology and Patterns at Work Transformed CategoryTraditional organizational structures and cultures are no longer fit for purpose in a digitally empowered world. The number of new and disruptive technologies is increasing, the speed of change shows no sign of slowing down and business leaders and organization development practitioners need to act urgently to enable their companies to succeed in the digital age. Agile Transformationis the much-needed guide to achieving this success. Packed full of practical advice, this book covers everything: why new operating models are needed, how to apply agile principles at scale, leveraging digital-native processes and why change managers need to think big but start small.Agile Transformationalso looks at how to build and engage high-performing teams for change, how to tackle the employee mindset that can hinder agile adoption and why developing an agile business is not a reason to fail to plan. There is also guidance on how to develop fast and focused high-velocity decision making, build momentum for change, and ensure that leadership behaviours and organizational culture catalyze true organizational agility. Featuring case studies from organizations including Amazon, Netflix and Vodafone, this book is crucial reading for businesses wanting to effectively compete in the new world of work.

Theory of change in ten steps - Noble, James (NPC, 2019)

Citizen engagement: emerging digital technologies create new risks and value - Peixoto, Tiago; Steinberg, Tom (World Bank, 2019)
The recent rapid evolution of digital technologies has been changing behaviors and expectations in countries around the world. These shifts make it the right time to pose the key question this paper explores: Will digital technologies, both those that are already widespread and those that are still emerging, have substantial impacts on the way citizens engage and the ways through which power is sought, used, or contested? The authors address this question both to mitigate some of the World Bank’s operational risks, and to initiate a conversation with peers about how those risks might require policy shifts. The overall framing question also is being explored in case theapproaches to citizen engagement advocated by the World Bank are changing and may require different advice for client countries. Despite the lower technology penetration levels in developing countries, their more malleable governance contexts may be more influenced by the effects of emerging technologies than older states with greater rigidity. Digitally influencedcitizen engagement is, in short, one of those “leapfrog” areas in which developing nations may exploit technologies before the wealthier parts of the world. But countries can leapfrog to worse futures, not just better ones. This paper explores what technology might mean for engagement, makespredictions, and offers measures for governments to consider.

Tale of the converted: how complex social problems have made me question the use of data in driving impact - Burgoyne, John (Impact of Social Sciences, 2019)
15 Shares In practice the way in which research impacts and influences policy and society is often thought to be a rational, ordered and linear process. Whilst this might represent a ‘common sense’ understanding of research impact, in this cross-post John Burgoyne reflects on how upending the primacy of data and embracing complexity can lead to a more nuanced and effective understanding of research impact.

ICT4D Blog » A citizen participation ecosystem - Peña-López, Ismael (ICTlogy, 2019)
Research on the Information Society, the Digital Divide and Information and Communication Technologies for development