Evidence-led adaptive programming: Lessons from MUVA - Sharp, Samuel; Riemenschneider, Nils; Selvester, Kerry (ODI, 2022)
Calls for more ‘adaptive programming’ have been prominent in international development practice for over a decade. Learning-by-doing is a crucial element of this, but programmes have often found it challenging to become more learning oriented. Establishing some form of reflective practice, against countervailing incentives, is difficult. Incorporating data collection processes that generate useful, timely and practical information to inform these reflections is even more so.This paper explores MUVA - an adaptive female economic empowerment programme in Mozambique. MUVA, we suggest, is atypically evidence-led. It combines systematic, inclusive reflective practice with extensive real-time data collection. We describe the fundamental features of MUVA’s monitoring, evaluation and learning (MEL) approach that supported this. One, how data collection and analysis are synchronised with set cycles for learning and adapting projects. Two, how MEL systems are designed to prioritise actionable learning, with data collection oriented more to the needs of implementing staff than to the reporting requirements of funders.This approach was enabled by building collective ownership over the programme’s objectives and the purpose of MEL from the outset. Implementers are asked about their motivations, and these are related to the programme’s Theory of Change. The evidence culture is supported by the proximity of MEL staff to implementing staff; and through structuring upwards accountability to funders around justifying evidence-based adaptations instead of reporting on more narrow indicators. We conclude by considering the relevance, or not, of MUVA’s approach to programmes in other contexts or issue areas trying to replicate a similarly evidence-informed approach to adaptive management. Key messages Learning-by-doing is essential to adaptive programming, but it can be challenging to establish data collection processes that generate useful, timely and practical information. MUVA – a female economic empowerment programme in Mozambique – has an atypically evidence-led adaptive management approach. This has two fundamental features. One, data collection and analysis are synchronised with set cycles for learning and adapting projects. Two, data collection is oriented more to the needs of implementing staff than to the reporting requirements of funders. Monitoring, evaluation and learning (MEL) systems are designed to prioritise actionable learning. This approach was enabled by building collective ownership over the programme’s objectives and the purpose of MEL from the outset. Implementers are asked about their motivations, and these are related to the programme’s Theory of Change. The evidence culture is supported by the proximity of MEL staff to implementing staff; and through structuring upwards accountability to funders around justifying evidence-based adaptations instead of reporting on more narrow indicators.

Capacity Development in a Participatory Adaptive Programme: the Case of the Clarissa Consortium - Widmer, Mireille; Apgar, Marina; Afroze, Jiniya; Malla, Sudhir; Healey, Jill; Constant, Sendrine (Institute of Development Studies, 2022)
Doing development differently rests on deliberate efforts to reflect and learn, not just about what programmes are doing and achieving, but about how they are working. This is particularly important for an action research programme like Child Labour: Action- Research-Innovation in South and South-Eastern Asia (CLARISSA), which is implemented by a consortium of organisations from across the research and development spectrum, during a rapidly changing global pandemic. Harnessing the potential of diverse skills and complementary strengths across partners in responding to the complex challenge of the worst forms of child labour, requires capacity to work together in novel ways. This Research and Evidence Paper documents how CLARISSA approached capacity development, and what we learnt from our challenges and successes. From the start, the programme incorporated a capacity development strategy resting on self-assessment of a wide range of behavioural and technical competencies that were deemed important for programme implementation, formal training activities, and periodic review of progress through an after-action review (AAR) process. An inventory of capacity development activities that took place during the first year of implementation reveals a wide range of additional, unplanned activities, enabled by the programme’s flexibility and adaptive management strategy. These are organised into eight modalities, according to the individual or collective nature of the activity, and its sequencing – namely, whether capacity development happens prior to, during, or after (from) implementation. We conclude with some reflections on the emergent nature of capacity development. Planning capacity development in an adaptive programme provides a scaffolding in terms of time, resources, and legitimacy that sustains adaptiveness. We also recognise the gaps that remain to be addressed, particularly on scaling up individual learning to collective capabilities, and widening the focus from implementation teams to individuals working at consortium level.
Accepted: 2022-04-25T10:25:54Z

Adaptive Learning Guide: A pathway to stronger collaboration, learning, and adapting - Ross, Joey; Karlage, James; Etheridge, James; Alade, Mayowa; Fifield, Jocelyn; Goodwin, Christian; Semrau, Katherine; Hirschhorn, Lisa (USAID MOMENTUM Knowledge Accelerator, 2021)
The purpose of this Adaptive Learning Guide is to provide MOMENTUM project teams with the information and resources to integrate adaptive learning into the design, implementation, and improvement of MNCH/FP/RH programs. This guide provides a conceptual introduction to adaptive learning using links to existing resources and real-world examples of how adaptive learning can drive continuous learning and improvement in project work. The guide is built upon three foundational assumptions: We work in dynamic, often unpredictable environments. Unexpected turns of events will occur. Progress is rarely, if ever, linear. Integrating the principles and practices of USAID’s Collaborating, Learning, and Adapting Toolkit into projects and initiatives requires designing for learning and adaptation. We intend the guide to serve as a “starting point” for interested individuals and teams to begin or strengthen the processes that support the integration of adaptive learning into project work.

Measuring and Monitoring Adaptive Learning: A Landscape Review - LaFond, Anne; Adrian, Haley (USAID MOMENTUM Knowledge Accelerator, 2022)
This landscape review on measuring and monitoring adaptive learning highlights the learning from five adaptive programming guidelines and toolkits and one implementation science framework to inform the monitoring and evaluation of adaptive learning. The introduction of adaptive learning processes and skillsets in global health programming is part of an emerging strategy to advance a learning culture within projects and teams to improve health program performance. The monitoring and evaluation of adaptive learning is an emergent field aiming to monitor how adaptive learning processes have been introduced, how they are used, and whether they are having the intended results. Although there is a growing body of literature on adaptive programming more generally, there is a limited knowledge base on the monitoring and evaluation of adaptive learning interventions and their impacts. Unlike other implementation strategies or program management approaches, there are no standard metrics or a monitoring and evaluation framework to track the integration, implementation, and effectiveness of adaptive learning in health programming.

Adaptive Evaluation: A Complexity-based approach to Systematic Learning for Innovation and Scaling in Development - Gokhale, Siddhant; Walton, Michael (Center for International Development, Harvard University, 2023)
Nearly all challenges in international development tend to be complex because they depend on constantly evolving human behaviour, systems, and contexts, involving multiple actors, entities, and processes. As a result, both the discovery and scaling of innovations to address challenges in development often involve changes in system behaviour or even system-level transformation. This is rarely a linear process over time and can result in unexpected outcomes. Existing evaluation techniques commonly used in international development, including Randomized Control Trials (RCT) and quasi-experimental methods, are good at assessing specific effects of interventions but are not designed for the change processes inherent to innovation and scaling within a system. There is a need to reconstruct how we use existing measurement tools, techniques, and methodologies so that they capture the complexity of the environment in which an intervention or change occurs. We introduce Adaptive Evaluation, designed to learn at various levels of complexity while supporting the transformation needed to foster sustainable change. An Adaptive Evaluation uses three main approaches to work with complex questions—systems diagnosis, theorybased assessment of change processes, and iterative designs. An Adaptive Evaluation typically builds hypotheses from field-based interactions, emphasizes learning over testing, advocates open-mindedness with techniques, and appreciates the value of dialogue and participation in navigating complex processes. It can use RCT or similar techniques to analyse specific processes within a system or a development cycle, but these are embedded in a broader approach to assessment and interpretation. It is designed to be flexible and adjust to shifting contexts. Finally, an Adaptive Evaluation can be applied at any stage in a complex intervention's lifecycle, from the interpretation of the system and change processes to rapid experimentation, prototyping, and testing of select interventions, and then adaptation to different settings for impact at scale. This paper provides the theoretical basis for an Adaptive Evaluation—the main approaches, core ideology, process, and applications.

What have we learned about learning? Unpacking the relationship between knowledge and organisational change in development agencies - Yanguas, Pablo (German Development Institute / Deutsches Institut für Entwicklungspolitik (DIE), 2021)
Development cooperation has spent decades wrangling over the merits, evidence, and implications of what we may term “the learning hypothesis”: the idea that increased knowledge by development organisations must logically lead to increased effectiveness in the performance of their development activities. Organisations of all stripes have built research and monitoring and evaluation (M&E) departments, adopted a multitude of knowledge management systems and tools, and tinkered with different ways to structure their organograms to stimulate knowledge sharing and learning. The topic of organisational learning is particularly significant as the global development community grapples with increasingly complex problems and the aspiration of evidence-based policymaking. This paper presents an analytical framework for interrogating “the learning hypothesis”, breaking it down into causal steps: knowledge causes learning, learning causes organisational change, change causes effectiveness. The framework focuses on the first two sub-hypotheses, mapping out the conceptual space around them by outlining potential relationships between different types of knowledge – tacit and explicit, internal and external – and between different types of learning – operational and strategic. This map provides a foundation for three key research questions: What impact has the rising knowledge agenda had on development organisations? Which factors appear to enable or inhibit organisational learning? What is the relationship between operational and strategic learning and organisational change? A review of available evaluations and studies, including two cases from former UK Department for International Development (DFID) and the World Bank, reveals that there is insufficient evidence to support the causal claim that knowledge leads to learning and thereby to organisational change in development agencies. Sources point to tacit learning prevailing while explicit knowledge management systems flounder, and external advocacy agendas appear more compelling than internal research and evaluation products. It is not entirely clear how, or indeed, whether operational and strategic learning intersect, with delivery-level lessons hardly aggregating into structural or policy shifts. Organisational change – even that aimed at enhancing learning – is rarely based on lessons learned from practice. More research is necessary to fully unpack the learning hypothesis, but what limited evidence is available disproves rather than confirms its central claim. This has significant implications for the future of learning in development agencies as advocated by thought leaders, researchers, and reformers. In particular, the latter should consider an evidence-based reassessment of the function and value-for-money of research and M&E in development practice, and a more critical examination of the politics of external advocacy efforts around innovative aid approaches like thinking and working politically, adaptive management, or results-based management.
Version Number: 1.0

Building a culture of learning at scale: learning networks for systems change. - McKenzie, Fiona (Orange Compass & Paul Ramsay Foundation, 2021)
This scoping paper explores the question ‘what would it take to build a culture of learning at scale?’. It focuses on systems-wide learning that can help to inform systems change efforts in complex contexts. To answer this question, literature was reviewed from across diverse disciplines and the realms of education, innovation systems, systems thinking and knowledge management. This inquiry was also supported by in-depth interviews with numerous specialists from the for-purpose sector and the examination of several case studies of learning across systems. The goal was to derive common patterns to inform a ‘learning for systems change’ framework. Learning for systems change is critical when working with complexity. The dynamic nature of complex adaptive systems requires an ability to continually sense and learn from the system and adapt accordingly. This is because the nature of the challenge and ‘what works’ to meet the challenge is continually shifting (Lowe and Plimmer, 2019). This requires an ongoing process of iterative inquiry that draws upon wisdom and insights from across the system. Such learning challenges traditional siloes of expertise and organisational boundaries (Clarke et al., 2019). Learning is not simply a nice to have. It is critical for greater impact and improved outcomes, particularly in mission-driven organisations and initiatives (Price et al., 2019). In this paper, a ‘learning networks’ approach is proposed, one that draws upon individual, group and systems-wide learning to build capacity and resilience for systems change in uncertain environments. This fills a gap in the literature where the focus is largely on learning within organisations. Instead, the focus here is on what is required to support learning to occur across scales and boundaries - from the individual to system-wide. A simple meta-framework for developing learning networks is proposed that includes high level guidance on the enabling conditions - the mindsets, relationships, processes and structures- that would enable learning networks to flourish.

Enhancing partner and system-level learning: 8 Tips from MEL Managers - Gover, Dun; Nasution, Zulka; Okutu, David; Bolder, Meghan; Henao, Lina (USAID, 2023)
Effective learning is a key driver of market systems change, with the potential to enhance system competitiveness, resilience, and inclusiveness. Shifting the Locus of Learning: Catalyzing Private Sector Learning to Drive Systemic Change recently outlined a rationale for enhancing the scale and quality of learning in a system and identifying 10 strategies programs can contextualize to catalyze learning. These strategies are also backed with robust examples from 13 programs doing this work across 11 countries. To deepen insights on what MSD Monitoring, Evaluation and Learning (MEL) Managers have experienced in putting several of those strategies into practice, the Feed the Future Market Systems and Partnerships (MSP) Activity convened a series of peer discussions as part of a larger initiative (see Figure 1). This brief shares the collective learning and experience on this topic of three senior MEL Managers who were interested in and had experience with this topic. The group represented full-time, program-based MEL Leads working on MSD programs funded by USAID and DFAT, based in Fiji, Albania, and Kosovo working for Adam Smith International, SwissContact, and DT Global, respectively. From those discussions, this paper synthesizes eight tips from MEL Managers for practically enhancing partner and system-level learning: 1. Identify the right decision-maker(s) at potential partners. 2. Use a co-creation process to identify learning opportunities. 3. Use diagnostics and assessments to strengthen partner and system capacity for actionable learning. 4. Use a phased capacity strengthening process tied to behavior change. 5. Measure partners’ continued investment in and use of learning—not the continuation of specific learning activities. 6. Work with sector-level institutions for scale but be aware of risks. 7. Leverage informal communities of practice to share learning. 8. Use the right terminology to talk about partner and system-focused learning.

LEARN End of Contract Report - Learning Lab (USAID, 2020)
In September of 2014, USAID’s Office of Learning, Evaluation & Research (LER) awarded the Learning and Knowledge Management (LEARN) contract to Dexis Consulting Group and subcontractor RTI International.1 This document—the End of Contract Report—captures five and half years of results and reflections for our stakeholders. Our intention is to share the good and the bad, and while this report would not be considered a “tell all,” we think we have a story worth sharing, particularly to USAID CORs and AORs, activity managers, and other implementers of institutional support contracts.2 LEARN’s primary purpose was to support organizational change at USAID. More specifically, the contract was focused on helping USAID staff integrate collaborating, learning, and adapting (CLA) approaches into the design, implementation, monitoring, and evaluation of programs (what is known at USAID as the Program Cycle). It was clear that most USAID staff, whether they realized it or not, were already integrating CLA into their work to some extent. The focus of our efforts, therefore, was to make those practices more systematic, intentional, resourced, and ultimately more widespread throughout the Agency, which would have a ripple effect on implementing partners and even other stakeholders, such as host country governments. This was based on the theory—later borne out by evidence—that by becoming a better learning organization, USAID could be a more effective development organization. And that theory brought the USAID CLA team within USAID’s Bureau for Policy, Planning & Learning (PPL) and LEARN contractors together, driven by a shared purpose of improving how USAID does business. LEARN was designed based on this belief and, as you might expect from a learning-oriented contract, began with more questions than answers. Primary among those questions was: could an institutional support contract do more than carry out requested services—could it actually accelerate positive organizational change at USAID? And if so, how and under what conditions?

A Framework for Embedding Learning in Systems Change - Bertermann, Kecia; Coffman, Julia (Center for Evaluation Innovation, 2024)
This five-step framework, developed and tested by a foundation, embeds learning in emergent systems change strategies. It prioritizes the testing of hypotheses and assumptions, uses learning questions, and calls for examining both confirming and disconfirming evidence. — A framework for embedding learning in systems change strategies and for testing strategic uncertainties. Learning and evaluation approaches that accompany systems change efforts need to fit with and support the emergent nature of this work. Systems change efforts call for examining, with evidence, the hypotheses and assumptions that undergird the work and regularly adapting in response to what is learned. They require staff to be highly engaged in testing their ideas by gathering evidence, interpreting it, and then acting on its implications. This brief presents a learning approach—referred to here as the Learning Framework—developed and tested by a foundation to support its systems change strategies. It is intended for other practitioners working on approaches for integrating learning with systems change efforts. Because a wealth of conceptual work exists about why learning is critical to systems change but practical advice about how to do it is much less available, we describe the approach we took and how the Framework’s core elements fit together in practice. The primary purpose of this approach is to interrogate our strategic uncertainties. This approach to learning is different from the dominant approach to learning and evaluation in philanthropy that primarily looks for evidence (often static metrics) that expected outcomes have been achieved (an approach that is more suitable for linear and predictable change strategies). This Learning Framework is driven by, and organized around, hypotheses and assumptions. This is especially important for systems change efforts where the path forward cannot be determined with complete certainty, but where robust evidence collection and regular sensemaking can increase our confidence in chosen directions and ultimately increase our chances of success.

The Dynamics of Team Learning: Harmony and Rhythm in Teamwork Arrangements for Innovation - Harvey, Jean-François; Cromwell, Johnathan R.; Johnson, Kevin J.; Edmondson, Amy C. (Administrative Science Quarterly, 2023)
Innovation teams must navigate inherent tensions between different learning activities to produce high levels of performance. Yet, we know little about how teams combine these activities—notably reflexive, experimental, vicarious, and contextual learning—most effectively over time. In this article, we integrate research on teamwork episodes with insights from music theory to develop a new theoretical perspective on team dynamics, which explains how team activities can produce harmony, dissonance, or rhythm in teamwork arrangements that lead to either positive or negative effects on overall performance. We first tested our theory in a field study using longitudinal data from 102 innovation teams at a Fortune Global 500 company; then, we replicated and elaborated our theory in a study of 61 MBA project teams at an elite North American university. Results show that some learning activities can occur within the same teamwork episode to have harmonious positive effects on team performance, while other activities combine to have dissonant negative effects when occurring in the same episode. We argue that dissonant activities must be spread across teamwork episodes to help teams achieve a positive rhythm of team learning over time. Our findings contribute to theory on team dynamics, team learning, and ambidexterity.

Track to Change | Home - Clear Horizon (Clear Horizon, 2025)
Track to Change homepage

Pushing boundaries: Advocacy evaluation and real-time learning in an HIV prevention research advocacy coalition in sub-Saharan Africa - Dasmariñas, Jules; Schlangen, Rhonda (New Directions for Evaluation, 2021)
Evaluation processes that facilitate learning among advocates must be nimble, creative, and meaningful while transcending putative performance and accountability management. This article describes the experience, lessons, and trajectory of one such approach, Simple, Participatory Assessment of Real Change (SPARC), that a transnational HIV prevention research advocacy coalition pilot-tested in sub-Saharan Africa. Inspired by the pioneering work of the outcome harvesting (OH) and participatory evaluation community, we recuperate advocates' centrality as storytellers, sense-makers, and strategists in advocacy evaluation and describe how we recalibrated SPARC to meet their evaluation and learning needs. This article highlights the normative value of deliberative discourse in evaluation as it contributes to the interpretation of OH and the enrichment of the theory and practice of advocacy evaluation.
_eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1002/ev.20474