Adaptive Management Across Project Cycles: Look into Coherence in Time - Guerzovich, Florencia (Medium, 2023)
This is a series about Monitoring, Evaluating and Learning (MEL) whether sets of interventions/portfolios are adding more together than each one would produce on their own. In post 1, I pointed to coherence, the new OECD-DAC evaluation criteria as a way to bridge the ambition of bringing bigger change with the MEL world. In post 2, I shared 3 of 4 practical lessons I’ve learned in experimenting with MEL systems and exercises that focus explicitly on interactions of interventions/portfolios. In the third post, I bring Paul Pierson’s groundbreaking argument for social science to MEL. Paraphrasing, most contemporary MEL takes a “snapshot” view of interventions and portfolios, distorting their effects and meaning by ripping them from their temporal context. Instead, we should place in time interventions/portfolios with the ambition to add more than the sum of the part by constructing MEL systems looking at “moving pictures” rather than taking snapshots. ...

Assessing Value for Money: the Oxford Policy Management Approach - King, Julian; Daniel Wate; Esther Namukasa; Alex Hurrell; Frances Hansford; Patrick Ward; Shiva Faramarzifar (Oxford Policy Management, 2023)
This document offers practical guidance for assessing the Value for Money (VfM) of government- and donor-financed programmes and policy interventions. In line with OPM’s focus and mission, it has been predominantly applied in the international development sector, but the approach upon which it is based is also used in the context of domestic public policy and programmes.1 There is increasing scrutiny on VfM in international development, but a lack of appropriate methods to support its assessment. There is a risk of reaching invalid conclusions if VfM evaluation is tied to a narrow set of indicators devoid of any evaluative judgement—for example, by emphasising the most readily quantifiable measures rather than the most important (but harder to quantify) aspects of performance, or by focusing on the quantification of outputs and outcomes at the expense of more nuanced consideration of their quality, value, and importance. The approach presented in this guide combines theory and practice from evaluation and economics to respond to requirements for accountability and good resource allocation, as well as to support reflection, learning, and adaptive management. It involves developing and implementing a framework for: • organising evidence of performance and VfM; • interpreting the evidence on an agreed basis; and • presenting a clear and robust performance story. This guide sets out a framework for making and presenting judgements in a way that opens both the reasoning process and the evidence to scrutiny. The approach is designed to be used in alignment with broader monitoring, evaluation, and learning (MEL) systems—both for efficiency's sake, and to ensure conceptual coherence between VfM evaluation and wider MEL work. The VfM framework achieves these aims by: • using explicit criteria (aspects of performance) and standards (levels of performance) to provide a transparent basis for making sound judgements about performance and VfM; • combining quantitative and qualitative forms of evidence to support a richer and more nuanced understanding than can be gained from the use of indicators alone; • accommodating economic evaluation (where feasible and appropriate) without limiting the analysis to economic methods and metrics alone; and • incorporating and building on an approach to VfM evaluation which is familiar to international aid donors.

Building a team culture for Adaptive Management in MSD: 5 Strategies MEL Managers Say Work - Gover, Dun; Nasution, Zulka; Okutu, David; Bolder, Meghan; Henao, Lina (USAID, 2023)
Keywords: Bolder Meghan; Gover Dun; Henao Lina; Nasution Zulka; Okutu David

How adaptive M&E from the peace sector can help demonstrate the value of aid - Kratzer, Sebastian (2023)
Over the last decade, the peace sector has been developing and adapting Monitoring and Evaluation (M&E) systems and tools to fit their contexts and ways of working. This evolution may hold some insights for the aid community in how to go beyond more traditional, backwards-looking M&E to navigate today’s volatile, interest-based world of politics and aid.

Systems-aware social accountability: Harnessing Practitioner Insights for More Responsive Governance - Guerzovich, Florencia (Pact, 2023)
The framework’s four principles, described below, are not exclusive to social accountability interventions; for example, Pact has published tools on how to use political economy analysis or integrate adaptive management principles into programming. Rather, the principles highlight how – by including the right combination of mindsets, management approaches, analytic approaches and tools – practitioners can develop and apply a deep understanding of a system to improve relationships within it, in support of greater accountability and improved service delivery. - Use a Holistic Lens - Recognize that there is always a system; understand and engage with it​: Developing a strong understanding of the system in which a project works and effectively integrating this knowledge into implementation increases the chances that local actors will sustain the interventions. - Right Fit the Approach​ - Equipped with a holistic understanding of the context, teams identify where they have a comparative advantage in a specific system. In other words, they identify where their resources, expertise and partners have the greatest potential for impact, then set realistic targets that they revise periodically. -Orchestrate Across The System -​ Practitioners take a networked, rather than a top-down, approach to capitalize on their convening power and maximize opportunity to facilitate the most-promising alignments across and within their projects, portfolios and systems. Orchestrators use their individual power, relationships and insight into the local system to enable, not impose, promising interactions and mitigate the risks of damaging ones. - Manage Adaptively - Monitor, evaluate and reflect on emerging patterns of data, evidence and learning to inform decision-making within and, equally importantly, across project cycles. Where possible, practitioners should consider how to use short-term projects to play a long-term game by linking new projects to the lessons and results of their predecessors and ensuring that they are responsive to the evolving context, particularly opportunities, limits and risks.

Adaptive Evaluation: A Complexity-based approach to Systematic Learning for Innovation and Scaling in Development - Gokhale, Siddhant; Walton, Michael (Center for International Development, Harvard University, 2023)
Nearly all challenges in international development tend to be complex because they depend on constantly evolving human behaviour, systems, and contexts, involving multiple actors, entities, and processes. As a result, both the discovery and scaling of innovations to address challenges in development often involve changes in system behaviour or even system-level transformation. This is rarely a linear process over time and can result in unexpected outcomes. Existing evaluation techniques commonly used in international development, including Randomized Control Trials (RCT) and quasi-experimental methods, are good at assessing specific effects of interventions but are not designed for the change processes inherent to innovation and scaling within a system. There is a need to reconstruct how we use existing measurement tools, techniques, and methodologies so that they capture the complexity of the environment in which an intervention or change occurs. We introduce Adaptive Evaluation, designed to learn at various levels of complexity while supporting the transformation needed to foster sustainable change. An Adaptive Evaluation uses three main approaches to work with complex questions—systems diagnosis, theorybased assessment of change processes, and iterative designs. An Adaptive Evaluation typically builds hypotheses from field-based interactions, emphasizes learning over testing, advocates open-mindedness with techniques, and appreciates the value of dialogue and participation in navigating complex processes. It can use RCT or similar techniques to analyse specific processes within a system or a development cycle, but these are embedded in a broader approach to assessment and interpretation. It is designed to be flexible and adjust to shifting contexts. Finally, an Adaptive Evaluation can be applied at any stage in a complex intervention's lifecycle, from the interpretation of the system and change processes to rapid experimentation, prototyping, and testing of select interventions, and then adaptation to different settings for impact at scale. This paper provides the theoretical basis for an Adaptive Evaluation—the main approaches, core ideology, process, and applications.

Enhancing partner and system-level learning: 8 Tips from MEL Managers - Gover, Dun; Nasution, Zulka; Okutu, David; Bolder, Meghan; Henao, Lina (USAID, 2023)
Effective learning is a key driver of market systems change, with the potential to enhance system competitiveness, resilience, and inclusiveness. Shifting the Locus of Learning: Catalyzing Private Sector Learning to Drive Systemic Change recently outlined a rationale for enhancing the scale and quality of learning in a system and identifying 10 strategies programs can contextualize to catalyze learning. These strategies are also backed with robust examples from 13 programs doing this work across 11 countries. To deepen insights on what MSD Monitoring, Evaluation and Learning (MEL) Managers have experienced in putting several of those strategies into practice, the Feed the Future Market Systems and Partnerships (MSP) Activity convened a series of peer discussions as part of a larger initiative (see Figure 1). This brief shares the collective learning and experience on this topic of three senior MEL Managers who were interested in and had experience with this topic. The group represented full-time, program-based MEL Leads working on MSD programs funded by USAID and DFAT, based in Fiji, Albania, and Kosovo working for Adam Smith International, SwissContact, and DT Global, respectively. From those discussions, this paper synthesizes eight tips from MEL Managers for practically enhancing partner and system-level learning: 1. Identify the right decision-maker(s) at potential partners. 2. Use a co-creation process to identify learning opportunities. 3. Use diagnostics and assessments to strengthen partner and system capacity for actionable learning. 4. Use a phased capacity strengthening process tied to behavior change. 5. Measure partners’ continued investment in and use of learning—not the continuation of specific learning activities. 6. Work with sector-level institutions for scale but be aware of risks. 7. Leverage informal communities of practice to share learning. 8. Use the right terminology to talk about partner and system-focused learning.

Strengthen Your Team’s CLA Practices: Introducing the CLA Maturity Tool for USAID Implementers - Social Impact , Inc (2023)
The Collaborating, Learning & Adapting (CLA) Maturity Tool has been used by USAID operating units since 2015. The tool enables staff to self-assess their current CLA practice and plan ways to improve their internal and external collaboration, organizational learning efforts, and adaptive management practices. Recently, SI built on this resource to create the CLA Maturity Tool for implementing partners (IPs) to provide a version that speaks more directly to the experience of international and local organizations. Learn more here.

Practicing what we preach — our experience rethinking M&E in complex settings - Bina, Andrea; Vester, Søren (UNDP Strategic Innovation, 2023)
The UNDP has been exploring new approaches to M&E to better understand and address complex systems challenges. Traditional linear project-based planning and evaluation methods are insufficient for dealing with the unpredictable nature of such systems. The UNDP’s new Strategic Plan calls for a shift in M&E practices to align with the complexity of today’s global challenges. Key points from the blog post include: Learning and Adapting: There’s a need for continuous learning and adaptation in M&E practices to effectively address complex problems like transitioning to a circular economy. Longer Time Horizons: Substantive change takes time, and it’s often unclear what the results will look like, making it difficult to track progress. Aggregate Impact: Evaluating interconnected interventions as a portfolio is crucial because they cannot be assessed in isolation. Contribution vs. Attribution: Focusing on the contribution to broader change processes is more important than attributing specific outcomes to individual actions. The UNDP has initiated the M&E Sandbox to experiment with and learn from innovative M&E practices. This collaborative space has attracted participants from over 180 organizations, highlighting a shared interest in evolving M&E to better understand change in contexts of uncertainty. A partnership with the Bill and Melinda Gates Foundation has also been formed to focus on M&E in food systems. The blog post emphasizes the importance of embracing uncertainty, experimenting, and collaborating to contribute to positive change in complex settings1.

Knowledge Retention and Transfer (KRT) - Model overview presentation - USAID (USAID, 2023)
What problem is the KRT model trying to solve? For workforces that experience continuous staff turnover, the lack of systematic knowledge transfer can often lead to: - Loss of programmatic momentum, - Duplication of efforts and frustration, and - Wasted time and resources. The Knowledge Retention and Transfer (KRT) model provides tools, processes, and practices to individuals, teams, offices, and organizations to improve knowledge handover, which in turn improves efficiency and programmatic and operational learning.

How are Research for Development Programmes Implementing and Evaluating Equitable Partnerships to Address Power Asymmetries? - Snijder, Mieke; Steege, Rosie; Callander, Michelle; Wahome, Michel; Rahman, M. Feisal; Apgar, Marina; Theobald, Sally; Bracken, Louise J.; Dean, Laura; Mansaray, Bintu; Saligram, Prasanna; Garimella, Surekha; et al. (The European Journal of Development Research, 2023)
The complexity of issues addressed by research for development (R4D) requires collaborations between partners from a range of disciplines and cultural contexts. Power asymmetries within such partnerships may obstruct the fair distribution of resources, responsibilities and benefits across all partners. This paper presents a cross-case analysis of five R4D partnership evaluations, their methods and how they unearthed and addressed power asymmetries. It contributes to the field of R4D partnership evaluations by detailing approaches and methods employed to evaluate these partnerships. Theory-based evaluations deepened understandings of how equitable partnerships contribute to R4D generating impact and centring the relational side of R4D. Participatory approaches that involved all partners in developing and evaluating partnership principles ensured contextually appropriate definitions and a focus on what partners value.

How to Build Movements with Cyclical Patterns in Mind - Non Profit News | Nonprofit Quarterly - Algoso, Dave; Guerzovich, Florencia; Gattoni, Soledad (Nonprofit Quarterly, 2023)
The world changes too much for anyone who is invested in social change work to imagine that this work is linear and predictable. Opportunities come and go, whether caused by a pandemic or political shifts. This much most social movement leaders and activists intuitively understand. But what can be done with this realization? How might movement groups better prepare for moments of opportunity? We want to explore how we can create the changes we want to see by responding to the changes that are outside our control.

A Guide to Participatory Outcome-Focused Monitoring with Most Significant Change, Outcome Harvesting, and Outcome Mapping - Smith, Richard (CARE, 2023)
Audience This guidance is primarily aimed at development and humanitarian professionals who design or manage monitoring and evaluation processes to support design and implementation of qualitative monitoring using Most Significant Change (MSC), Outcome Harvesting (OH), or Outcome Mapping (OM), which are participatory, outcome-focused qualitative methods. Purpose This guidance is a resource to strengthen qualitative monitoring, specifically addressing commonplace gaps or weakness in the monitoring of complex development and humanitarian projects. Importantly, these gaps include: 1. Lack of evidence about unexpected, as well as expected, changes or outcomes, especially during project implementation. 2. Limited documentation about whether and how a project contributed to change. Such information is vital to learning about what worked and how to build on successes. 3. Limited data to inform the review of theories of change, which limits their function and relevance. Addressing these monitoring gaps has the potential to contribute to more informed and effective adaptive management and learning. This guidance can be used to support the design and implementation of monitoring exercises using MSC, OH, and OM. Specifically, it supports users in choosing among MSC, OH, and OM or blending elements from these approaches. Additionally, it provides practical tips and examples - including expertise gained during the two pilot qualitative monitoring activities conducted in Niger and Bangladesh - as well as other publicly available materials. 1. Relevant CARE guidance and tools include: 1. Qualitative Methods Guidance Note for Monitoring, Evaluation and Learning (MEL) at CARE; 2. Outcome Mapping Manual: Development of a monitoring system for the Right to Food project using Outcome Mapping, CARE Nepal; 3. Tipping Point MEL Methods Brief: Outcome Mapping; 4. The Rolling Profile Tool.

Theory of Change in complex Research for Development programmes: challenges and solutions from the Global Challenges Research Fund - Chapman, Sarah; Boodhoo, Adiilah; Duffy, Carren; Goodman, Suki; Michalopoulou, Maria (The European Journal of Development Research, 2023)
The United Kingdom Research and Innovation (UKRI) Global Challenges Research Fund (GCRF) aimed to address global challenges to achieve the United Nations (UN) Sustainable Development Goals through 12 interdisciplinary research hubs. This research documents key lessons learned around working with Theory of Change (ToC) to guide Monitoring, Evaluation and Learning (MEL) within these complex research for development hubs. Interviews and document reviews were conducted in ten of the research hubs. The results revealed that only one hub invested in an explicit visual system mapping approach, and that funder timelines, budget constraints and issues with capacity and expertise limited the application of these approaches across all hubs. In contrast, many hubs attempted to deal with visual complexity by means of ether constructing multiple, nested ToCs, or a conscious simplification of complexity through reducing their ToC towards a straightforward and uncomplicated chain model or spherical model. While the former approach had some value, most hubs struggled to find capacity to support the full articulation of nested ToCs. In contrast, the latter approach resulted in ToCs which lacked detail or mechanism articulation, but which nevertheless were often ‘fit for purpose’ in ensuring effective communication and coherence across diverse stakeholders and sub-projects. We conclude that in instances where the reporting, funding and management cycles of complex research for development programmes cannot be adapted to properly support learning-based approaches to ToC development, imposing simplicity in the ToC might be fit for purpose. This might also be preferable to more complex visual approaches that are only partially realised.