Supporting adaptive management. Monitoring and evaluation tools and approaches - Pasanen, Tiina; Barnett, Inka (ODI, 2019)
Key messages • This working paper introduces a set of monitoring and evaluation (M&E) tools and approaches, discussing their potential usefulness in supporting adaptive management in development and humanitarian programmes. It emphasises adaptive programmes characterised by complex aspects, such as: (1) they are innovative; (2) they have uncertain pathways for change; or (3) they operate in uncertain or unstable environments. • The majority of these tools have been used in international development for many years. However, adaptive management brings additional challenges for monitoring and evaluating programmes, as they require intentional M&E design from the start that is oriented towards both learning and accountability. • All of the tools and approaches introduced in this paper have potential to support learning and adaptation, although in various ways and at different stages of a programme. Some tools can support strategic planning and diagnosing throughout a programme – especially during design and inception – while others can help analyse causal relationships at specific points in a programme. It is important to tailor the approach used for its intended purpose. However, whether learning and adaptation happens depends also on factors other than the choice of M&E methods. • For some of these approaches a considerable body of evidence already exists but, for many, more practical examples and systematic analysis is needed. In addition to building the evidence base concerned with which approaches are suitable for different types of adaptive programmes and why, it is also important to improve understanding of the enabling environmental conditions necessary for the tools and approaches outlined here to facilitate and strengthen evaluative thinking, evidence-informed decision-making and ongoing programme iteration.

Towards evidence-informed adaptive management - Hernandez, Kevin; Ramalingam, Ben; Wild, Leni (Working Paper, 2019)
Development and humanitarian organisations seeking to be adaptive have emphasised the need to be transparent about complexity and uncertainty; to be honest about their inability to control what happens; and to design programmes that change over time to become more appropriate and relevant. At their heart, adaptive management approaches emphasise the ability to lean, 'unlearn' and adapt programming accordingly. The cornerstone of effective learning is the creation, gathering, accumulation, interpretation and use of data and evidence. This working paper provides development professionals with tools, strategies and ideas to help them use evidence for adaptive management in practical and evidence-informed ways. Key messages - Evidence is central to effective and rigorous adaptive management. However, despite this central importance, exactly how evidence has been used to inform decisions for adapting development and humanitarian programmes in the past remains unobservable to many. - There is a need to strengthen and document evidence-informed adaptive management. This working paper proposes a roadmap to do this. - Those seeking to use evidence for adaptive management will need to manage trade-offs between ensuring a rigorous, documented (and auditable) trail of evidence-informed actions, being pragmatic about the time and resources allocated to documentation and recognising that it may be necessary to proceed without rigorous evidence when it is unavailable.

Livelihoods in the Za'atari Camp: Impact evaluation of Oxfam’s Cash for Work activities in the Za’atari camp (Jordan) - Lombardini, Simone; Mager, Franziska (http://hdl.handle.net/10546/620883, 2019)

Stakeholder and Social Network Analysis - Guidance Note - IRC (International Rescue Committee, 2019)
Stakeholder Analysis is used to identify the actors and relationships that influence project outcomes. This guidance, from IRC, can help you determine how to work and who to partner, coordinate or engage with in order to best achieve the outcome. It builds on existing stakeholder information and typically will include a participatory internal meeting or workshop.

Theory of change in ten steps - Noble, James (NPC, 2019)

Citizen engagement: emerging digital technologies create new risks and value - Peixoto, Tiago; Steinberg, Tom (World Bank, 2019)
The recent rapid evolution of digital technologies has been changing behaviors and expectations in countries around the world. These shifts make it the right time to pose the key question this paper explores: Will digital technologies, both those that are already widespread and those that are still emerging, have substantial impacts on the way citizens engage and the ways through which power is sought, used, or contested? The authors address this question both to mitigate some of the World Bank’s operational risks, and to initiate a conversation with peers about how those risks might require policy shifts. The overall framing question also is being explored in case theapproaches to citizen engagement advocated by the World Bank are changing and may require different advice for client countries. Despite the lower technology penetration levels in developing countries, their more malleable governance contexts may be more influenced by the effects of emerging technologies than older states with greater rigidity. Digitally influencedcitizen engagement is, in short, one of those “leapfrog” areas in which developing nations may exploit technologies before the wealthier parts of the world. But countries can leapfrog to worse futures, not just better ones. This paper explores what technology might mean for engagement, makespredictions, and offers measures for governments to consider.

LAC MEL Specialists Use Peer Network to Improve Quality and Use of Evidence - Anderson, Todd M; Prevatt, Amy (CLA Case Competition, 2019)
Based on consultations and stocktakings with LAC Missions, the LAC Bureau identified Mission Monitoring, Evaluation and Learning (MEL) Specialists as influential decision makers regarding data and evidence use. The LAC Bureau aims to improve the quality and use of evidence for decision making, and ultimately to improve development outcomes.

Failing Forward: How CARE is focusing on what goes wrong to improve impact - Janoch, Emily (2019 CLA Case Competition, 2019)
CARE's Failing Forward initiative is sparking opportunities to showcase the ideas that don't work so we can spend more time implementing the ones that do. It's changing the conversation inside the organization, and leading to changes in the way we design and implement programs. It's also allowing us to make connections across a global portfolio of more than 900 projects, and with new and different partners to learn from their experience.

The COM-B Theory of Change Mode - v.5 - Mayne, John (Palladium, 2019)
ResearchGate is a network dedicated to science and research. Connect, collaborate and discover scientific publications, jobs and conferences. All for free.

Adaptive Programming in Fragile, Conflict and Violence-Affected Settings, What Works and Under What Conditions?: The Case of Institutions for Inclusive Development, Tanzania - Green, Duncan; Guijt, Irene (Action for Empowerment and Accountability Research Programme, 2019)
Adaptive Management involves a dynamic interaction between three elements: delivery, programming and governance. This case study focuses on a large DfID governance project, the Institutions for Inclusive Development (I4ID), a five-year initiative in Tanzania. The study forms part of a research project to examine whether and how adaptive approaches can strengthen aid projects promoting empowerment and accountability in fragile, conflict and violence-affected settings (FCVAS). The research examines some of the assertions around the adaptive management approach and explores if and how adaptive approaches, including rapid learning and planning responses (fast feedback loops and agile programming) are particularly relevant and useful for citizen empowerment and government accountability (E&A) in FCVAS.
Keywords: A4EA; Adaptive Development; Economy; Fishery

Supporting the policy environment for economic development (SPEED+) - Strategic Review Report - MSI (USAID, 2019)
Donor-funded policy reform has a long history in Mozambique, with United States Agency for International Development (USAID) efforts dating to the mid-1990s. Many laws and regulations adopted during the past quarter century are the consequence of these efforts. And yet, even with several years of robust economic growth, Mozambique has not experienced the broad economic transformation that policy reforms can trigger: per capita income in 2017 was $519 and more than 80 percent of the country lived on less than two dollars per day. The country ranks 180 out of 189 on the Human Development Index and 135 out of 180 on the World Bank’s Doing Business Index. Supporting the Policy Environment for Economic Development (SPEED+), a $37 million, four-year, USAID-funded program, represents the latest iteration of USAID’s policy reform efforts. SPEED+ began in August 2016 and is scheduled to run through August 2020. As the program passed the midway point, USAID commissioned Management Systems International (MSI) to conduct a strategic review (SR) of the SPEED+ program, with the objectives of advising on: • how to maximize the impact of SPEED+ during its final 18 months; • what type of follow-on program USAID should consider; and • what broader lessons can be drawn from SPEED+’s experience in Mozambique.

Top Tips: How to design and manage adaptive programmes - Valters, Craig; Wild, Leni (Overseas Development Institute, 2019)
Within DFID, there is now a commitment to more flexible and adaptive programming. This recognises that: • DFID works in contexts that continuously evolve and change, sometimes in unpredictable ways. To respond to this, the agency needs to remain flexible – to expect change and have a good understanding of context, with resources that can be adjusted and scope to change direction if needed. All DFID programmes should be able to do this. • Some DFID programmes aim to support change in complex systems, behaviours and incentives. Efforts to address women’s empowerment, improve sanitation or build more sustainable health systems, for instance, all require engagement with the way in which complex systems operate and the people and behaviours within them. Trying to deliver reforms in these circumstances is challenging because the pathway to reform itself will be unclear: as a reform is rolled out, the system itself will react and respond. These types of programmes therefore need to build in from the start deliberate processes of learning and testing, to allow for adaptations as more information is gathered for what works over time. These top tips are concerned with programmes that aim to be flexible and adaptive – which work in dynamic contexts and are trying to address complex problems. While there is growing commitment to these approaches, feedback suggests staff still have questions about how to do this well. This document highlights some of the commonly reported issues related to adaptive programming and a set of tips, strategies and examples to help in addressing them. It is aimed at programme managers and advisors who may be Senior Responsible Owners (SROs) or those managing and supporting adaptive programming in a range of ways. We have collated these lessons from discussions with country offices and SROs, feedback from surveys and the wider evidence. We have looked specifically at the adaptive programmes we can find in the DFID portfolio, but this is not an exhaustive list. It should be noted that there is as yet no wide variety of case law to review, but there is a growing set of examples within DFID that can provide continuous learning for the organisation.

Beyond Tweets and Screams: Action for Empowerment and Accountability in Nigeria – The Case of the #BBOG Movement - Aina, Tade Akin; Atela, Martin; Ojebode, Ayo; Dayil, Plangsat; Aremu, Fatai (IDS, 2019)
This paper explores the nature, role and dynamics of new forms of social and political action as pathways to empowerment and accountability in fragile conflict- and violence-affected settings in Africa. Through an in-depth analysis of the case of the Bring Back Our Girls (#BBOG) movement in Nigeria and a multi-methods approach, the paper provides new knowledge that addresses evidence gaps in the following areas: (1) the multiple ways through which social and political action play out in fragile, conflict- and violence-affected settings; (2) whether the conditions in which new forms of social and political action applicable to fragile, conflict- and violence-affected settings – the settings most dominant in African countries – emerge as currently projected in the literature; and (3) whether these social and political actions necessarily produce accountability and empowerment in fragile, conflict- and violence-affected settings. Although often expressed as contentious and/or unruly politics, experiences from the BBOG movement suggest that the new forms of social and political action possess a wide range of implications for citizen action and governance, including leading to multiple forms of empowerment in fragile settings.

Exploring the new world: practical insights for funding, commissioning and managing in complexity - Lowe, Toby; Plimmer, Dawn (Collaborate & Newcastle University, 2019)
The world is complex. If we want to contribute to creating positive social outcomes, we must learn to embrace this complexity. This is the New World that funders and commissioners are discovering: • People are complex: everyone’s life is different, everyone’s strengths and needs are different. • The issues we care about are complex: issues – like homelessness – are tangled and interdependent. • The systems that respond to these issues are complex: the range of people and organisations involved in creating ‘outcomes’ in the world are beyond the management control of any person or organisation. Building on the findings from our previous report, A Whole New World, we have spent the last 12 months working with a growing movement of funders, public sector commissioners, and organisations working on the ground to begin to explore this New World: to find examples of practice that will help people to navigate it effectively. This report explores the key features of their response: working in a way that is human, prioritises learning and takes a systems approach.

The Case for an Adaptive Approach to Empowerment and Accountability Programming in Fragile Settings: Synthesis report - Christie, Angela; Green, Duncan (Action for Empowerment and Accountability Research Programme, 2019)
Fragile, conflict and violence-affected settings (FCVAS) are messy and ambiguous contexts in which to plan and implement development initiatives. To work there, external actors are increasingly adopting an adaptive approach to empowerment and accountability (E&A) programming, whatever the setting. This means using a compass rather than map, where real-time political economy analysis (PEA) in relation to context and programme monitoring and evidence-informed learning in relation to intervention are used in combination and in shorter-than-usual planning cycles to maintain and adapt strategic direction. This paper brings together three case studies of large Department for International Development (DFID) governance projects in Myanmar, Nigeria and Tanzania.
Keywords: A4EA; Adaptive Development; Economy; Fishery

Implementing Adaptive Approaches in Real World Scenarios: A Nigeria Case Study, with Lessons for Theory and Practice - Bridges, Kate; Woolcock, Michael (The World Bank, 2019)
How does adaptive implementation work in practice? Drawing on extensive interviews and observations, this paper contrasts the ways in which an adaptive component of a major health care project was implemented in three program and three matched comparison states in Nigeria. The paper examines the bases on which claims and counterclaims about the effectiveness of these approaches were made by different actors, concluding that resolution requires any such claims to be grounded in a fit-for-purpose theory of change and evaluation strategy. The principles of adaptive development may be gaining broad acceptance, but a complex array of skills, expectations, political support, empirical measures, and administrative structures needs to be deftly integrated if demonstrably positive operational results are to be obtained, especially when undertaken within institutional systems, administrative logics, and political imperatives that are predisposed to serve rather different purposes.

Knowledge Retention and Transfer (KRT) - Model and Matsurity Matrix - USAID (CLA Toolkit, 2019)
The Maturity Matrix is not a standalone tool. It is one of the three major components of the KRT Model. The Maturity Matrix is meant to be used in conjunction with the KRT Toolkit and Implementation Plan.

Managing staff transistions through CLA: preserving institutional memory as staff come and go - USAID (CLA Toolkit, 2019)
This document is relevant for any position or hiring mechanism. While this document does not explicitly address what happens when someone arrives into a newly-created position, which has its own set of challenges, many of the principles, actions, and resources can be applied in that context. Section 1 offers guidance for how to set up and implement systems at Mission/OU or office to ensure that all staff help preserve institutional memory and enable continuity of relationships. This section is for any staff in a Mission/OU working to build systems that improve handovers and knowledge capture and sharing across the Mission or OU. If a Mission or OU is just beginning to develop a system to manage staff transitions, you may want to start by developing a system within an office or team and then expand it out to other offices or to the Mission/OU writ large. Section 2 includes key principles and actions individuals could consider following when arriving or departing a position, regardless of hiring mechanism and position. This includes staff new to the Mission/OU and staff who are currently in the Mission/OU and are moving into a new position. Section 3 provides a select number of resources to help you take a systematic and comprehensive approach to manage staff transitions as effectively and efficiently as possible at the individual or organizational level.

Broad Agency Announcement for Locally Led Development Innovation - USAID (USAID, 2019)
This BAA allows USAID Operating Units (OUs) to co-create, co-design, co-invest, and collaborate in the research, development, piloting, testing, and scaling of innovative, practical, and cost-effective interventions to catalyze locally led development. The BAA aligns with a number of Agency priorities and policies, including the Journey to Self-Reliance, resilience, procurement innovation, and expanding and diversifying the partner base - as well as the New Partnerships Initiative and Acquisition and Assistance Strategy. E3/LS has put up the BAA’s first Addendum, "New Directions in Advancing Locally Led Development," which seeks Expressions of Interest on the following: - Engaging the Local Private Sector - Conflict, Post-Conflict, and Non-Permissive Environments - Effective Partnerships - The Changing Role of the Donor - Local Giving, Philanthropy, and Other Private Resources

The Limits of Accounting-Based Accountability in Education (and Far Beyond): Why More Accounting Will Rarely Solve Accountability Problems - Honig, Dan; Pritchett, Lant (Center for Global Development, 2019)
Accountability is rightly at the center of the conversation regarding how to improve governance systems, particularly health and education systems. But efforts to address accountability deficits often focus primarily on improving what can be counted and verified—what we term “accountingbased accountability.” We argue that introducing greater accounting-based accountability will only very rarely be the appropriate solution for addressing accountability problems. We illustrate this by exploring the role of Accountability ICT in (not) improving education system performance. Strengthening “real” accountability is not the same as improving data systems for observation and verification, and often attempts at the latter undermine the former. The development discourse’s frequent semantic misunderstanding of the term “accountability” has pernicious effects on system reform efforts and ultimately global welfare.

Monitoring and learning for country-level portfolio decision-making and adaptation - Buffardi, Anne L; Mason, Paige; Hutchings, Claire; Sharp, Samuel (Briefing Note, 2019)
Most measurement and adaptive management approaches were developed for and from individual projects. This briefing aims to guide measurement and management of country-level portfolios of work. It identifies potential purposes portfolio-level analyses can fulfil, types of adaptation, and the relative role of monitoring, learning and evaluation (MEL). Drawing on reviews of practice from the United States Agency for International Development (USAID) and the United Kingdom Department for International Development (DFID), guidance notes, and experiences of members of the Global Learning for Adaptive Management (GLAM) initiative, it offers recommendations and considerations that are particularly relevant for this level of analysis and management. Key messages - Portfolio-level analyses can serve eight potential purposes, each of which answers different questions, involves adaptation at different times and levels, and requires different types of evidence. Identifying the purpose(s) and how the component parts relate to each other should guide the development of monitoring and learning systems. - Portfolios are oriented more towards breadth than depth, involve more people with different perspectives, and draw on multiple sources of evidence with potentially greater variation in quality. - In practice, four activities appear to be applied most frequently at a portfolio level: alignment of indicators and aggregation of monitoring data; synthesis of multiple sources and types of information to provide a summary of outputs, outcomes, common observations and trends; periodic review and reflection sessions; and strategic planning, design or refresh of the portfolio strategy. - The extent to which evidence-informed portfolio management is facilitating learning and adaptation has not been well documented to date, and we suggest potential indicators to do so.

Barrio Mio and Katye: PCI’s neighbourhood approach in cities - Campbell, Leah (AKNAP case study, 2019)

Help! I’m hiring new staff and I want them to work adaptively. A guide to hiring adaptive employees. - Salib, Monalisa (USAID LEARN, 2019)
If you are involved in hiring, this tool can help you increase your chances of selecting staff members skilled in adaptive management. It will help answer the questions: Which competencies should I recruit for in order to hire more adaptive employees? Which desired qualifications should I incorporate into position descriptions to attract adaptive employees? Which interview questions should I ask to screen for adaptive employee competencies? You should use this tool when you decide to hire a new individual - whether that be a direct hire, contractor, or consultant - or when you are changing an individual’s current position description or scope of work. Specifically, this tool will help you determine which qualifications are most important for the position and offers interview questions you can use with candidates. --- You’re in luck! We just released a new guide for hiring adaptive employees. This visual and user-friendly tool will help increase your chances of selecting staff skilled in adaptive management. Here are the details: What do you mean by “work adaptively” or an “adaptive employee”? An adaptive employee is an individual who, in collaboration with relevant stakeholders, systematically acquires and uses knowledge to make decisions and adjustments in their work in order to achieve greater impact. Why focus on adaptive employees? Why does it matter? With industries, technologies, and organizations in a near constant state of flux, leaders are recognizing the importance of adaptability as a critical capacity. Because of this, it’s not surprising that a recent LinkedIn report found that adaptability was the most important soft skill hiring managers were screening for. In addition, evidence shows that teams that apply more data-driven and adaptive leadership practices perform better than those that focus less on these practices. It makes sense, then, that managers want to recruit more team members who are skilled in adaptive management - it helps achieve results. Who should use this tool? Anyone involved in hiring, responsible for developing or approving position descriptions, participating in interviews, and/or approving new hires. When should I use this tool? Use it as soon as you decide to hire a new staff person or when you’re adjusting existing position descriptions. What does the tool help me do? It will help you think through: Which competencies should I recruit for in order to hire more adaptive employees? Which desired qualifications should I incorporate into position descriptions to attract adaptive employees? (You can even copy and paste qualification language from the tool into scopes of work!) Which interview questions should I ask to screen for adaptive employee competencies? (You can copy and paste interview questions into your screening, interview, and reference check protocols!) Has the tool been tested? Yes, staff within USAID’s Global Development Lab and on the USAID LEARN contract tested the tool, providing feedback on the minimum viable product and subsequent versions. Testers confirmed finding the tool useful, and gave it a 9/10 score when asked if they would recommend it to colleagues. Some specific feedback from testers included: "The tool prompts deeper thought on what you might need and want in a job candidate" and "All [of the interview questions I used from the tool] worked well." What about the existing staff? How can they gain greater skills in adaptive management? Existing USAID staff and implementing partners can access online training in collaborating, learning, and adapting (CLA) and find resources in the CLA Toolkit. USAID staff also have access to an internal CLA community of practice and five-day, in-person CLA training.

A Guide to Digital Feedback Loops. An approach to strengthening program outcomes through data for decision making - Whittle, Dennis; Campbell, Megan (USAID, 2019)
Feedback is information about what happens as a result of what you do. Using that information to adapt what you do or how you do it creates what is called a feedback loop. A digital feedback loop uses digital technology at some stage of the feedback loop. Digital feedback loops help USAID missions improve the effectiveness and efficiency of their activities and can support partner countries on their journey to self-reliance through increased information sharing and improved government and civil society capacity. USAID missions and their partners have used digital feedback loops to improve HIV treatment targeting in Zimbabwe, engage three million young people in Nigeria in policy, and reduce field reporting times from one month to one day. Digital feedback loops provide access to information faster and more cheaply than ever before. As part of an adaptive management approach, digital feedback loops increase our ability to act on opportunities and respond to challenges. That in turn can improve outcomes and reduce the risk of waste and failure. This guide outlines the case for integrating digital feedback loops into your work, and provides tools, resources and guiding questions to support you as you get started. The guide includes examples of how USAID missions are using digital feedback loops to improve their programs in various sectors and provides worksheets to help you apply digital feedback loops to your own context.

How Does the World Bank Build Citizen Engagement Commitments into Project Design? Results from Pilot Assessments in Mozambique, Myanmar, Nigeria, and Pakistan - Nadelman, Rachel; Le, Ha; Sah, Anjali (Working Paper 525, 2019)
How and to what degree is the World Bank putting its new institutional citizen engagement (CE) commitments into practice? This question guides an independent assessment that the Accountability Research Center (ARC) at American University has undertaken as part of the Institute of Development Studies-led Action for Empowerment and Accountability (A4EA) research programme’s investigation into how external actors can best support local processes of and conditions for empowerment and accountability. This report investigates the World Bank’s incorporation of CE into project design, the critical early stage of donor engagement. To accomplish this, ARC reviewed the World Bank’s fiscal year 2015–17 investment project portfolios for four A4EA priority countries, Mozambique, Myanmar, Nigeria, and Pakistan, which covers 57 projects that range from US$19 million to U$600 million. The analysis determines whether projects commit to seeking a strategic approach to CE, which involves combining multiple tactics so that the whole could be greater than the sum of the parts. This assessment of CE commitments is intended to help to inform possible national, civil society organisation strategies to monitor whether and how these commitments are actually implemented in practice.

A study of the sustained uptake developmental evaluation: How does developmental evaluation work in the USAID context, what factors help and hinder its success, and what is its value to stakeholders? - Baylor, Rebecca; Fatehi, Y. K.; Esper, H. (USAID, 2019)

DFID’s partnerships with civil society organisations - A performance review - ICAI (Independent Commission for Aid Impact (ICAI), 2019)
DFID values civil society organisations (CSOs), but its funding and partnership practices do not fully support the long-term health of the civil society sector.

Making adaptive rigour work - Principles and practices for strenghening monitoring, evaluation and learning for adaptive management - Ramalingam, Ben; Wild, Leni; Buffardi, Anne L (ODI/GLAM, 2019)
Adaptive programmes can be accountable, rigorous and high quality in how they use evidence by taking an ’adaptive rigour’ approach. Core development and humanitarian challenges are complex, and require processes of testing, learning and iteration to find solutions – adaptive management offers one approach for this. Yet large bureaucracies and development organisations can have low tolerance for experimentation and learning, and adaptive management can be viewed as an excuse for ‘making things up as you go along’. This briefing from the Global Learning for Adaptive Management (GLAM) initiative argues that adaptive programmes can be accountable, rigorous and high quality in how they use evidence – but this requires rethinking some key assumptions about how they are practised. The paper sets out three key elements of an ‘adaptive rigour’ approach: - Strengthening the quality of monitoring, evaluation and learning data and systems. - Ensuring appropriate investment in monitoring, evaluation and learning across the programme cycle. - Strengthening capacities and incentives to ensure the effective use of evidence and learning as part of decision-making, leading ultimately to improved effectiveness.

Learning from Results-Based Management evaluations and reviews - Vähämäki, Janet; Verger, Chantal (Development Co-operation Working Papers, 2019)
What have we learned from implementing results-based management in development co-operation organisations? What progress and benefits can be seen? What are the main challenges and unintended consequences? Are there good practices to address these challenges? To respond to these questions this paper reviews and analyses the findings from various evaluations and reviews of results-based management systems conducted by members of the Development Assistance Committee (DAC), the OECD/DAC Results Community Secretariat and other bodies in the past four years (2015-2018). It also draws on emerging lessons from new methods for managing development co-operation results. This analytical work aims to: identify recent trends in results-based management, explore challenges faced by providers when developing their results approaches and systems, select good practices in responding to these challenges that can be useful for the OECD/DACResults Community, considering new approaches, new technologies and evolving contexts. This body of evidence will inform the development of a core set of generic guiding principles for results-based management in development co-operation.

Knowing When to Adapt - A Decision Tree - USAID (USAID LEARN, 2019)
This job aid will help you determine how best to resolve a challenge/issue in programming. It is intended for use by USAID, partners, and others working in international development.

Bringing adaptive management to life: Insights from practice - Arora, Anmol; Gogoi, Elizabeth; Joy, Divya; Kumar, Pankaj; Luthra, Rajni; Pal, Uma; Pervaiz, Arif (Oxford Policy Management, 2019)
The increasing popularity of adaptive programmes in the development sector in the past decade is partly a response to the complex and interconnected challenges facing many developing countries, including the impacts of climate change. The pathway to adapting to climate change is unknown, and there are many deep-rooted institutional, political, economic and social barriers. Adaptive programmes provide the flexibility to allow those delivering the technical assistance to support governments to experiment with different entry points and to adapt or change course when some are not successful. The Action on Climate Today (ACT) programme is a five-year adaptive programme that aims to strengthen systems of planning and delivery for adaptation to climate change in South Asia. As the programme draws to a close, this Learning Paper reflects on some of the major lessons learnt on the challenges and opportunities of using an adaptive programme management approach to support governments to adapt to climate change. The authors are members of the ACT implementation team, representing the management, operational and technical teams, from across different locations. The paper therefore reflects their first-hand account of the reality of implementing adaptive programmes. The paper sets out a framework for the essential ingredients of an adaptive programme management approach. These include a set of essential core principles: an evolving theory of change; a locally led and politically savvy delivery approach; experimentation and learning; and stakeholder alignment. It also includes two sets of essential resources required: management flexibility and adequate financial resources. The paper then provides learning from ACT on how to operationalise these core principles and mobilise the necessary resources. It gives detailed examples of challenges faced by ACT with regard to each component of the framework, and how the programme has overcome them, as well as retrospective reflections of additional actions the programme could have taken. A set of case studies unpacks how adaptive programme management has been operationalised in practice in particular locations. The authors are confident that the impact ACT has had owes in large part to it being an adaptive programme. The programme’s flexibility, decentralised structure and strong emphasis on learning have meant the team has been able to respond to opportunities as they have emerged and to invest time and resources where there was most potential for impact. However, this paper is also honest about the challenges ACT has faced as an adaptively managed programme. The paper therefore concludes with a set of lessons learnt from ACT for the benefit of others designing and delivering adaptive programmes on climate change and other issues.

How DFID can better manage complexity in development programming - Sharp, Samuel; Valters, Craig; Whitty, Brendan (ODI, 2019)
The United Kingdom’s (UK) Department for International Development (DFID) is an ambitious government department that is committed to reducing poverty and conflict overseas. Many of the issues on which DFID works are complex; whether focused on climate change, gender equality, health or other priorities, simple solutions rarely exist. And to tackle these complex challenges, DFID staff must interact with unpredictable systems of political, organisational and individual behaviours and incentives. There is a risk that complex problems spur more complicated programmes; that the complexity of development challenges is addressed through designing programmes with too many projects and implementers. While there may be valid reasons for this, too many of these complicated programmes will overburden staff. This briefing note is the outcome of an ongoing process within DFID to confront these issues and answer the question: how can DFID design and manage programmes to address complex development challenges without creating too much staff workload? Key messages DFID deals with complex problems, which require flexible systems to support testing, learning and adaptation. • Complex problems do not necessarily require workload-heavy delivery structures, but simpler approaches depend on delivery partners’ experience and competence. • However, programmes that tackle complex problems do require more hands-on engagement and face more workload from inflexible compliance requirements. We suggest that DFID and similar agencies: • Pay closer attention to delivery options in programme design, making use of existing options where possible and, where not, fostering local organisations through long-term investments. • Encourage programme designers to articulate how ambition matches resources and consider ‘good enough’ design. • Reduce the burden of compliance by cultivating partner autonomy, reassessing results and valuefor-money requirements and promoting more flexible contracting and procurement. • Plan and prioritise management resources across a portfolio of programmes to make sure they can be focused in the right areas, where the complexity of the problem requires greater engagement.

Short Course - Adaptive Management, an overview - Bazaz Smith, Komal (American University, 2019)

No Royal Road: Finding and Following the Natural Pathways in Advocacy Evaluation - Coe, Jim; Schlangen, Rhonda (Center for Evaluation Innovation, 2019)
As efforts at driving change become more diffuse, involve more actors, and have more transformational goals, we need a radically different approach to thinking about and assessing effective advocacy. Clear answers, certain judgments, and simple tools won't result in good representations of reality. This brief proposes ideas for adjusting how we approach advocacy monitoring, evaluation, and learning.

Monitoring, Evaluating and Learning for Complex Programs in Complex Contexts: Three Facility Case Studies - Davda, Tara; Tyrrel, Lavinia (Governance Working Paper 6, 2019)

AID Programming Guide - DFAT (Australian Government, 2019)

Peacebuilding design, monitoring, and evaluation: A Training Package for participants and trainers at intermediate to advanced levels - Ernstorfer, Anita; Barnard-Webster, Kiely (Carnegie Corporation, 2019)
This training package includes 7 Training Modules and a set of Annexes (Annexes A-O). The Training Modules build on each other and should ideally be used in a sequenced way in a training setting. However, for groups with specific training needs around particular areas, modules can also be used individually, but need to be tailored by the trainers and facilitators to meet the needs of specific audiences. The annexes provide worksheets and hand-outs that can be used as resources during the training for specific modules and exercises.

Designing Monitoring and Evaluation Approaches for Learning An FOS How-To Guide - FOS (FOS, 2019)

Local Works Guidance - Round 4 - USAID (USAID, 2019)

Integrating Big Data and Thick Data to Transform Public Services Delivery - Ang, Yuen Yuen (IBM Center for The Business of Government, 2019)
This report explains how public organizations can combine big data and thick data to transform public services delivery—a strategy called mixed analytics. Governments can greatly enhance the value of big data by combining it with “thick” data—rich qualitative information about users, such as their values, goals, and consumption behavior, obtained by observing or interacting with them in their daily lives. Big data holds great promise for improving public services delivery and innovation in government, but they are not a panacea . Having lots of data can be overwhelming or have little utility if the data are “thin”—that is, they lack meaning for users or fail to capture issues that matter most . By yielding insights into what citizens really care about and how they consume services, thick data can inform both the collection and analysis of big data. This report introduces the concept of “mixed analytics,” integrating big data and thick data to transform government decision making, public services delivery, and communication. The report presents three case studies of organizations that employ mixed analytics at the international, federal, and city level, respectively. Together, this research offers a set of transferable lessons for agencies at all levels of government: Lesson 1: Big data is a means to an end, rather than an end. Lesson 2: Thick data can identify unexpected problems or previously unexpressed needs. Lesson 3: Thick data can inform the analysis of big data. Lesson 4: Mixed analytics can offer both scale and depth. Lesson 5: Applying technology is a social activity, not an isolated technical task. Lesson 6: The best solutions are not always high-tech. The report concludes with five actionable recommendations for public managers. Make data and technology relevant to the people who use it. Leverage thick data at appropriate stages of the problem-solving process. Build an interdisciplinary team of quantitative and qualitative experts who work closely with stakeholders. Combine big and thick data to improve communication Improve government agencies’ knowledge of mixed research methods. This report builds on multiple past IBM Center reports about how government can leverage data and analytics to improve decisions, including Data-Driven Government: The Role of Chief Data Officers, by Jane Wiseman; Ten Actions to Implement Big Data Initiatives: A Study of 65 Cities, by Alfred Ho and Bo McCall, and Realizing The Promise of Big Data, by Kevin DeSouza. The report also complements several chapters that assess the past and potential use of data across the public sector in our recent book, Government For The Future: Reflection and Vision For Tomorrow’s Leaders. At a time when the US and other governments continue to enhance their use of data as a strategic asset for transformation, we hope that Professor Ang’s report provides useful insights for government managers and stakeholders.

Developmental Evaluation: How Barriers & Enablers Emerge Over Time - Baylor, Rebecca; Fatehi, Y. K.; Esper, H. (USAID, 2019)

Inclusive Social Accountability - Counterpart International (Counterpart International, 2019)
Counterpart International’s Inclusive Social Accountability (ISA) developmental framework integrates elements of social inclusion and community accountability into one comprehensive approach.

Pocket Guide to Systems Wayfinding - Davidson, Seanna (The Systems School, 2019)

DFID Smart Rules: Better Programme Delivery - version X (2019.01) - DFID (DFID, 2019)

Breaking the Mould: Alternative approaches to monitoring and evaluation - Dillon, Neil (ALNAP, 2019)
This paper looks at a range of M&E innovations that are designed specifically to provide input to ongoing iterative decision-making and learning at the project level. It identifies three key areas for potential innovation: 1) timing of M&E data provision; 2) flexibility of M&E frameworks to evolve with programme change; and 3) approaches to integrate diverse perspectives on project implementation in a meaningful way. It then looks at a collection of approaches currently being used in each of these three areas through a series of ‘practice examples’, considering the key lessons learned. Finally, the paper discusses the major opportunities and challenges for applying and scaling up the use of these approaches inside the humanitarian sector.

Back to the Drawing Board: How to improve monitoring of outcomes - Dillon, Neil; Sundberg, Amelie (ALNAP Paper, 2019)

Empowerment and Accountability in difficult settings: What are we learning? - Gaventa, John; Oswald, Katy (Key messages emerging from theAction for Empowerment and Accountability Programme, 2019)
Empowerment and Accountability in Difficult Settings: What Are We Learning? Key Messages Emerging from the Action for Empowerment and Accountability Programme Empowerment and accountability have long been part of the international development vocabulary and a core part of governance, social development and civil society programmes. Yet, much of what has been learnt about these approaches has been drawn from studies in somewhat stable, open and middle-income places around the world. Less is known about how empowerment and accountability are achieved through social and political action in more difficult settings – those faced by institutional fragility, conflict, violence, and closing civic space. This document highlights key messages emerging from the work of the Action for Empowerment and Accountability Research Programme (A4EA), and the implications for how donors, policy makers and practitioners support strategies for empowerment and accountability in fragile, conflict and violence affected settings (FCVAS). Our eight key messages have strong implications for the theories of change used for effective programming in the field.

Adaptive Management Practice in the UK Government - The MUVA case - HLS (Human Learning Systems, 2019)
Case study about the MUVA programme in Mozambique. (Maybe it "misses the point of AP which is not learning for learning. Is learning for impact. The word impact doesn’t even come up once!") Adaptive Management programming within the Foreign & Commonwealth Development Office demonstrates that the UK Government has examples of optimising for learning within its existing management practice. However, currently, the adaptive management practices are unhelpfully framed by an approach which limits both their effectiveness and scope. The programmes use learning as their metastrategy. They succeed in connecting practicebased learning with strategic learning, and make a shift towards reframing accountability as accountability for learning. This learning strategy is enabled by funding and contract management arrangements which prioritise learning. Both at macro-level of programme management and micro level of de-risking experimentation and enabling necessary failure by decoupling people’s job security from potential failure.

Understanding Innovation Ecosystems: A Framework for Joint Analysis and Action. Cambridge: MIT D-Lab - Hoffecker, Elizabeth (MIT D-Lab, 2019)
Place-based innovation ecosystems play a crucial role in driving local and regional economic development. This role has been documented and understood for over 40 years in industrialized economies but is only starting to be appreciated in the context of emerging and developing economies. However, the past several years have seen an intensification of interest in innovation ecosystems among global development actors as well as practitioners and policymakers working across the Global South. Whether at the municipal level, in places such as Medellín, Colombia and Guadalajara, México, or the national level in Rwanda, South Africa, or India, we see governments, multi-laterals, donors, and civil society actors embarking on initiatives to strengthen local innovation ecosystems. The International Development Innovation Alliance, for example, which includes many of the largest public and private global development agencies, has created a set of recommendations for why and how actors investing in economic development should support the strengthening of innovation ecosystems. Within the past year, MIT D-Lab has been invited to play a role in some of these ecosystem-strengthening efforts. In contexts ranging from Oaxaca, México to Accra, Ghana, we have been asked to convene ecosystem actors and stakeholders in order to facilitate joint ecosystem strengthening work. In preparing for these engagements, we have researched the state of the field regarding both innovation and entrepreneurial ecosystems as well as existing ecosystem frameworks, models, and tools. In doing so, it has become clear that much of the current thinking and practice related to these concepts is drawn from research on innovation processes and entrepreneurial clusters in highly developed economies — places like Silicon Valley and Kendall Square, Boston. Much less is known about innovation ecosystems in less-developed contexts in terms of how they can be characterized, how they function, and — most importantly — how they can be strengthened. To that end, the Local Innovation Group at D-Lab has been conducting multi-year research on local innovation ecosystems in the types of contexts where D-Lab and our partners engage. This involves learning about diverse processes of ecosystem development through primary and secondary case study research. Based on this research, we have developed a framework for understanding local innovation ecosystems, which we share in this publication. We have found the framework, and accompanying visual model, to be a useful tool for orienting and organizing conversations among ecosystem actors on how particular innovation ecosystems are functioning, what their strengths and weaknesses are, and where opportunities for further development might lie. This document shares this framework and how we have used it over the past year to catalyze ecosystem-strengthening efforts. We start by clarifying the concept of a “local innovation ecosystem” and presenting the core ideas informing the visual model. We then describe the model and each of its individual components. We follow with guidance from our research on best practices for conducting ecosystem strengthening work, and share three examples of how we have used the model to facilitate ecosystem-strengthening conversations in distinct local contexts.

Monitoring and Result Measurement for Adaptive Programming - How to Use Data to Manage a MSD Program: Lessons from PRISMA - Khan, Khaled; Seely, Kevin; Ridwan, Mustika; Mulya, Bodhiya (PRISMA, 2019)
Using monitoring data to improve interventions is harder than it seems. Decision-makers are often busy implementing activities, unclear about their roles in data collection and analysis, and uncertain what data matters most or when. PRISMA, an AUD77 million agricultural Market Systems Development (MSD) programme funded by DFAT Australia, has encountered these challenges. With the programme completing its first five year phase, this case study shares ten key lessons divided into three sections: shaping the culture, developing systems, and top management decisions. These lessons aim to help program, sector and intervention managers make better use of monitoring data to improve interventions.

Joint Programming in Conflict-Affected and Fragile States - Koenig, Sibylle; Brusset, Emery (Joint Programming - EU, 2019)
On the Joint Programming Practices from the EU in fragile settings

DAC Peer Review of Switzerland - 2019 - OECD (OECD, 2019)

Planning & Navigating Social Change - Tools for Pacific voyagers - Orr, Douglas Epeli; Cavatore, Maria; Aston, Tom (Oxfam, 2019)
Those working to advance social change in the Pacific must understand, work with and respond to the complex and changing relationships and dynamics of power that exist within such networks, and situate their work in the context of decolonisation and self-determination. But the approaches typically used by NGOs to plan our projects and programs have failed to give due attention to these dynamics. The result is that our projects fall short of achieving their promise, despite being technically sound and logical. In this guide, we aim to provide a process and tools that prioritise and draw out the rich, often implicit, knowledge that Pacific Islanders have about our contexts to plan and manage social change initiatives in a manner that values and responds to this ocean of relationships. Drawing from these relationship-based and voyaging traditions, we emphasise the need to: • frame change in terms of change in people; e.g. change in the situation of groups of people and change in terms of the actions of people; • identify a course towards a particular ‘change destination’; • identify the signs to read that will help you determine where you are on that course; • be constantly attentive to the context as you make the journey, and • be ready to adjust your course in response to changes in your context. To support this way of working, we have selected a number of tools from approaches that we believe fit. In particular, we have drawn from the following approaches: • Thinking and Working Politically and other ‘politically aware’ approaches that recognise that the success of a development initiative will be determined by the interplay of relationships of power within a particular context, not just by having the right technical solution; • Outcome mapping because of its focus on people and relationships, and on identifying and supporting needed changes in the behaviour of specific actors in relation to others. This is key, because it’s individuals who deliver services, individuals who decide whether to allocate budget and resources to address a problem, and individuals who make the choice to believe whether a service is relevant to solving their issues; • Adaptive management because of its recognition that development initiatives take place within complex and changing systems and need to adjust to these and that this requires ongoing learning and adaptation.

Monitoring And Evaluation Tools and Approaches to Support Adaptive Management - Pasanen, Tiina (GLAM, 2019)

The Rapid Evaluation, Action, and Learning (REAL) Approach: A toolkit to measure and refine changes and interventions in health campaigns - PATH (PATH, 2019)
Evaluative Tools for Improving Campaign Effectiveness Public health campaigns aim to control disease or deliver essential health services and products through time-limited and periodic channels. Many countries use campaigns to augment or replace routine service delivery, to target certain populations, or to accelerate progress towards coverage targets. The Root Cause Analysis and Rapid Evaluation, Action, and Learning toolkits present a systematic but flexible approach to identifying the root causes of campaign bottlenecks and then designing, testing, and refining solutions to optimize potential impact. These toolkits respond to a growing need for “fit-for-purpose” rapid-testing, adaptive learning approaches to evaluation and the need for a culture shift toward iterative adaptation and improvement that integrates measurement and evidence-informed decision-making into daily practice. The comprehensive package of toolkits, in addition to the individual Root Cause Analysis toolkit and Rapid Evaluation, Action, and Learning toolkit are available below.

What is Uncertainty and Why Does it Matter? - Scoones, Ian (STEPS Working Paper 105, 2019)
Uncertainty defines our times. Whether it is in relation to climate change, disease outbreaks, financial volatility, natural disasters or political settlements, every media headline seems to assert that things are uncertain, and increasingly so. Uncertainty, where we do not know the probabilities of either likelihoods or outcomes, is different to risk, the implications of which are explored in this paper through five different ways of thinking about uncertainty, derived from highly diverse literatures encompassing societal, political, cultural, practice and individual perspectives. The paper continues by examining how these perspectives relate to four domains: finance and banking; critical infrastructures; disease outbreaks and climate change; natural hazards and disasters. Reflecting on these experiences, the paper argues that embracing uncertainty raises some fundamental challenges. It means questioning simple, linear perspectives on modernity and progress. It means rethinking expertise and including diverse knowledges in deliberations about the future. It means understanding how uncertainties emerge in social, political and economic contexts, and how uncertainties affect different people, depending on class, gender, race, age and other dimensions of social difference. And, if uncertainty is not reducible to probabilistic risk, it means a radically different approach to governance; one that rejects control-oriented, technocratic approaches in favour of more tentative, adaptive, hopeful and caring responses. The paper concludes by asking whether we can learn from those who live with and from uncertainty – including pastoralists in marginal settings – as part of a wider conversation about embracing uncertainties to meet the challenges of our turbulent world.

Managing and Adapting a development program: Lessons from PRISMA - Seely, Kevin (PRISMA, 2019)
This case study is part of the AIP-Rural Learning Series. Funders and implementers of international development programs largely agree that adaptive management is industry best practice. Most development experts also broadly agree on what ‘adaptive management’ means. In this case study, we use a common definition of ‘adaptive management’, including the following features: Flexibility. Implementers create, modify and drop interventions when circumstances change or new information emerges, in order to tailor strategies to context and maximise impact. Purposeful experimentation. Implementers test different activities at the same time, monitoring them to learn what will achieve the desired impact. Since 2015, the number of publications calling for and praising adaptive management in development programs has grown fast. These publications give us an idea of the state of adaptive management in development programs. They reveal progress in some areas, such as tailoring interventions to local context and using evidence for decision-making. Yet they also reveal multiple, ongoing, real-world constraints to adaptive management. This case study explores how one development program, PRISMA, has avoided and overcome some of these constraints, whilst continuing to battle others. Four major constraints revealed in the adaptive management literature are discussed in this case study: Programs are designed in ways that make it hard to adapt interventions, target regions or sectors based on new learning or changing circumstances. Programs struggle to create an organisational culture that encourages learning and adapting. When staffing programs, recruiters prioritise sectoral expertise and length of experience over adaptive managerial competence. Program managers face pressures to spend their budgets predictably and before their program ends. Learning and testing take time and cost little, so managers feel pressure to deprioritise them. The first of these constraints is explored in the next section, ‘Design’, which looks at how the selection of target sectors and performance targets affects PRISMA implementers’ ability to manage adaptively. The third section, ‘Organisational Culture’, shares lessons on encouraging staff to test and improve interventions. Section 4, ‘Staffing’, looks at hiring adaptive managers, and freeing up their time to improve interventions. The final section summarises this case study’s key lessons. Throughout this paper, findings are based on 16 in-depth interviews with PRISMA’s junior, mid-level and senior staff, and its funders. To contextualise these findings, the authors reviewed the ‘adaptive management’ literature, and drew on their experience with development programs worldwide.

Doing adaptive management at Sida - Tim Ruffer; Helen Bailey; Stefan Dahlgren; Patrick Spaven; Mark Winters (Evaluation Brief, 2019)
Lessons from the market systems development approach The evaluation focuses on Sida’s management of MSD projects. The projects are applying the MSD approach with an aim to: - contribute to improved MSD programming by Sida through better management practices across the project cycle - generate recommendations on how Sida can create conducive conditions for systems approaches and adaptive programming more generally. The brief assesses Sida’s organisational capacity for adaptive management in three dimensions: - leadership and culture - staff capacities - skills, and systems and procedures