Strengthening the results chain: Synthesis of case studies of results-based management by providers - Zwart, Rosie (OECD, 2017)
This paper presents an analysis of evidence from seven case studies of results-based management by development co-operation providers. Analysis of themes from the case studies reveal six interrelated challenges which providers face in their efforts to manage for the results of development co-operation. The six challenges are: linking results to development goals, ensuring the purpose of results systems is well-defined, weighing up the benefits of aggregating and attributing results from standard indicators, enabling country ownership, using performance information alongside results information, and building and embedding a results culture. Providers continue to adopt a range of approaches to pursue a focus on results and there are many examples of good practice. The case studies suggest that in a complex development landscape some providers are prioritising the use of aggregated results information for domestic accountability, communication, and performance management over their use for learning and policy direction. In this context, the paper argues that in agency-wide results frameworks development co-operation results are often detached (or de-contextualised) from development results and discusses the implications of this, including the use of standard indicators to measure aggregated outputs. The paper uses case study evidence to discuss and suggest practical ways providers can build and maintain a strong results focus which enables analysis, insights and learning, and has achievement of development outcomes as its objective.

Participatory evaluation. What is it? Why do it? What are the challenges? - Zukoski, A.; Luluquisen, Mia (Community-based public health policy & practice, 2002)
Yogi Berra was right when he remarked, “If you don’t know where you’re going, you’ll wind up somewhere else.” When we collectively apply our hopes and energies to improving our communities, how do we know if we’re making the right choices along the way? How will we notice when we are spinning our wheels and what changes to make? How will we know that we are making a difference? And how will we know what success looks like to everyone in the group? By taking a community-based public health approach to our work, we create an opportunity to engage in a particular type of evaluation—participatory evaluation – that can help answer those questions. For those groups that are interested in this approach, this policy brief discusses the key concepts of participatory evaluation and some tips for applying it. We also present some real-life examples from two evaluators who work with grantees of the Partnership for the Public’s Health Initiative.

Evaluability Checklist for Post Project Evaluation - Zivetz, Laurie; Cekan, Jindra (Valuing Voices, 2020)
Considerations for planning a post project evaluation during the project, at the end, or after it has closed.

Adapting data collection and utilisation to a Covid-19 reality: Monitoring, evaluation and learning approaches for adaptive management - Ziegler, Jessica; Mason, Paige (Briefing Paper, 2020)
This briefing note focuses on the remote collection and use of data for adaptive management during the Covid-19 pandemic, setting out key considerations to help practitioners think through a transition from more ‘traditional’ monitoring, evaluation and learning (MEL) to MEL for adaptive management (MEL4AM) that reflects the unique data collection challenges presented by Covid-19. The brief provides an overview of some key considerations in remote data collection, when this is required, and identifies other sources that address these issues in more detail. It concludes with a discussion of how to bring the information resulting from remote monitoring into decision-making to enable adaptive management. Key messages - When planning for remote data collection during the Covid-19 pandemic, first determine what information is still necessary, because data needs may have changed, e.g. if programming has pivoted or needs to pivot due to Covid-19. Then identify how the programme’s information needs align with existing data sources and what gaps remain, which will guide the need for remote data collection. - Also consider what data is ‘good enough’ for current decision-making needs in order to provide sufficient information to the right people at the right time to an acceptable standard of rigour. - There may be pragmatic reasons to reduce the number or scope of MEL activities, such as logistical constraints or ethical considerations introduced by the pandemic. - MEL activities should be accompanied by frequent feedback loops and pause points to reflect on emergent needs and challenges, information needs that have been met, and contextual changes that may affect MEL. - Be clear with decision-makers about the assumptions and gaps in the data, including proxies used and their limitations, sampling changes, and how these changes and assumptions may affect the decisions/options being discussed.

USAID Wildlife Asia as a case study in adaptive rigour - Ziegler, Jessica (Briefing Paper, 2020)
This briefing note looks at how the United States Agency for International Development (USAID) Wildlife Asia programme has operationalised the concepts of adaptive rigour and adaptive management as part of its approach to collaborating, learning and adapting. As described by the Global Learning for Adaptive Management (GLAM) initiative, adaptive rigour is about ensuring that the data, information, methods, processes and systems that underpin adaptive management are robust, systematic and high‑quality. Key messages When faced with programmatic complexity, it is important to take an adaptive approach driven by continuous and iterative monitoring, evaluation and learning (MEL). USAID Wildlife Asia, which works to advance regional action towards ending illegal wildlife trafficking in Southeast Asia and China, has embraced this approach as a way of operationalising adaptive rigour. Throughout programming, MEL approaches should reflect the characteristics of adaptive rigour: comprehensiveness, usefulness, practicality, timeliness and support. Utilising performance monitoring and research in order to test and revise technical approaches and employing mixed methods to collect both qualitative and quantitative data, as well as looking for relevant lessons generated by others, can ensure access to the most useful information for decision-making throughout implementation. For adaptive management, it is not enough to monitor, evaluate and learn; it is also essential to pause and reflect in order to analyse and process evidence gained through MEL with colleagues and stakeholders to reach the right conclusions and make good decisions.

Inclusive participatory approaches: A facilitator’s guide - Zaremba, Haley; Elias, Marlène; Devi, J. Tulasi; Priyadarshini, Pratiti (2021)
In community engagement and participatory processes, facilitators must make intentional efforts and adopt inclusive strategies to include marginalized and frequently overlooked groups. Yet, there is a lack of guidance on how to inclusively facilitate participatory processes. Facilitators are therefore often poorly prepared to engage with the power relations that underlie these processes, including those between the facilitator and participants and among participants themselves. This guide addresses this shortcoming by presenting strategies that have been shown to enhance the meaningful participation of women and marginalized groups in participatory processes. The aim is to equip facilitators with tools to create inclusive participatory spaces and community engagement. Although it refers to processes related to natural resource management, the guide is also applicable to the facilitation of participatory processes focusing on other issues or fields of significance to communities.

Data and monitoring needs for a more ecological agriculture - Zaks, David P. M.; Kucharik, Christopher J. (Environmental Research Letters, 2011)
Information on the life-cycle environmental impacts of agricultural production is often limited. As demands grow for increasing agricultural output while reducing its negative environmental impacts, both existing and novel data sources can be leveraged to provide more information to producers, consumers, scientists and policy makers. We review the components and organization of an agroecological sensor web that integrates remote sensing technologies and in situ sensors with models in order to provide decision makers with effective management options at useful spatial and temporal scales for making more informed decisions about agricultural productivity while reducing environmental burdens. Several components of the system are already in place, but by increasing the extent and accessibility of information, decision makers will have the opportunity to enhance food security and environmental quality. Potential roadblocks to implementation include farmer acceptance, data transparency and technology deployment.

Engaging Stakeholders in Planning for Sea Level Rise and Resilience - Yusuf, Juita-Elena (Wie); John, Burton St; Covi, Michelle; Nicula, J. Gail (Journal of Contemporary Water Research & Education, 2018)
This case study describes a region-wide, multi-sectoral, and whole-of-community stakeholder engagement approach for addressing sea level rise (SLR) and flooding. This approach was implemented through a university-led community engagement event, the Hampton Roads Resilient Region Reality Check (H4RC), which allowed an examination of its effectiveness as a mechanism for capturing community-wide perceptions regarding SLR, flooding, and associated risks; engaging stakeholders in discussion within and across different groups; and assessing community willingness to address flooding and SLR. The results show that the event helped participants broaden their perspectives and understanding of flooding and SLR. In an approach that called for participants to engage in social learning across social networks, the event had some effect on individual efficacy. However, there was little impact on participants' already-established perception that the region does not possess significant willingness to take action.
Keywords: Social learning; flooding; resilience; social capital; stakeholder engagement

Rapid outcome mapping approach: a guide to policy engagement and influence - Young, John; Shaxson, Louise; Jones, harry; Hearn, Simon; Datta, Ajoy; Cassidy, Caroline (ODI, 2014)

Case Study Research and Applications: Design and Methods - Yin, Dr Robert K. (SAGE Publications, Inc, 2018)
Winner of the 2019 McGuffey Longevity Award from the Textbook & Academic Authors Association (TAA) Recognized as one of the most cited methodology books in the social sciences, the Sixth Edition of Robert K. Yin′s bestselling text provides a complete portal to the world of case study research. With the integration of 11 applications in this edition, the book gives readers access to exemplary case studies drawn from a wide variety of academic and applied fields. Ultimately, Case Study Research and Applications will guide students in the successful use and application ofthe case study research method.

Barriers to Political Analysis in Aid Bureaucracies: From Principle to Practice in DFID and the World Bank - Yanguas, Pablo; Hulme, David (World Development, 2015)
Politics has become a central concern in development discourse, and yet the use of political analysis as a means for greater aid effectiveness remains limited and contested within development agencies. This article uses qualitative data from two governance “leaders” – the United Kingdom Department for International Development and the World Bank – to analyze the administrative hurdles facing the institutionalization of political analysis in aid bureaucracies. We find that programing, management, and training practices across headquarters and country offices remain largely untouched by a political analysis agenda which suffers from its identification with a small cross-national network of governance professionals.
Keywords: DFID; United Kingdom; World Bank; aid effectiveness; foreign aid; political economy analysis

What have we learned about learning? Unpacking the relationship between knowledge and organisational change in development agencies - Yanguas, Pablo (DIE - Deutsches Institut für Entwicklungspolitik, 2021)
Development cooperation has spent decades wrangling over the merits, evidence, and implications of what we may term “the learning hypothesis”: the idea that increased knowledge by development organisations must logically lead to increased effectiveness in the performance of their development activities. Organisations of all stripes have built research and monitoring and evaluation (M&E) departments, adopted a multitude of knowledge management systems and tools, and tinkered with different ways to structure their organograms to stimulate knowledge sharing and learning. The topic of organisational learning is particularly significant as the global development community grapples with increasingly complex problems and the aspiration of evidence-based policymaking. This paper presents an analytical framework for interrogating “the learning hypothesis”, breaking it down into causal steps: knowledge causes learning, learning causes organisational change, change causes effectiveness. The framework focuses on the first two sub-hypotheses, mapping out the conceptual space around them by outlining potential relationships between different types of knowledge – tacit and explicit, internal and external – and between different types of learning – operational and strategic. This map provides a foundation for three key research questions: What impact has the rising knowledge agenda had on development organisations? Which factors appear to enable or inhibit organisational learning? What is the relationship between operational and strategic learning and organisational change? A review of available evaluations and studies, including two cases from former UK Department for International Development (DFID) and the World Bank, reveals that there is insufficient evidence to support the causal claim that knowledge leads to learning and thereby to organisational change in development agencies. Sources point to tacit learning prevailing while explicit knowledge management systems flounder, and external advocacy agendas appear more compelling than internal research and evaluation products. It is not entirely clear how, or indeed, whether operational and strategic learning intersect, with delivery-level lessons hardly aggregating into structural or policy shifts. Organisational change – even that aimed at enhancing learning – is rarely based on lessons learned from practice. More research is necessary to fully unpack the learning hypothesis, but what limited evidence is available disproves rather than confirms its central claim. This has significant implications for the future of learning in development agencies as advocated by thought leaders, researchers, and reformers. In particular, the latter should consider an evidence-based reassessment of the function and value-for-money of research and M&E in development practice, and a more critical examination of the politics of external advocacy efforts around innovative aid approaches like thinking and working politically, adaptive management, or results-based management.

Why We Lie About Aid: Development and the Messy Politics of Change - Yanguas, Pablo (Zed Books, 2018)
Foreign aid is about charity. International development is about technical fixes. At least that is what we, as donor publics, are constantly told. The result is a highly dysfunctional aid system which mistakes short-term results for long-term transformation and gets attacked across the political spectrum, with the right claiming we spend too much, and the left that we don't spend enough. The reality, as Yanguas argues in this highly provocative book, is that aid isn't – or at least shouldn't be – about levels of spending, nor interventions shackled to vague notions of ‘accountability’ and ‘ownership’. Instead, a different approach is possible, one that acknowledges aid as being about struggle, about taking sides, about politics. It is an approach that has been quietly applied by innovative development practitioners around the world, providing political coverage for local reformers to open up spaces for change. Drawing on a variety of convention-defying stories from a variety of countries – from Britain to the US, Sierra Leone to Honduras – Yanguas provides an eye-opening account of what we really mean when we talk about aid.

Evaluating development projects: exploring a synthesis model of the logical framework approach and outcome mapping - Yang, Ting (2018)
Under the current results-driven development agenda, sound evaluation, and a corresponding evaluation toolkit, need to be in place to examine whether and to what extent development interventions have achieved their targeted objectives and results, and to generate lessons for further development learning and improvement. My review of the literature shows that innovative and appropriate evaluation approaches are needed to address key challenges in evaluation such as the tension between learning and accountability objectives, the need to unpack the mechanisms linking outputs and outcomes or goal, and to add an actor perspective. Irrespective of project type, the Logical Framework Approach (LFA) is often a standard requirement of major official donor agencies on projects they fund, so as to fulfil bureaucratic imperatives. However, it is often considered inadequate in addressing key challenges in development evaluation. Given the dominant status of the LFA with such strong support from donors, it is helpful to seek a ‘middle way’: a combination of the LFA with other approaches in order to address some of its inadequacies, while satisfying donor agencies’ requirements. A synthesis of the LFA and Outcome Mapping (OM) is one such option. This thesis explores the practical value and usefulness of a synthesis model empirically. Applying the model in two case study aid projects, I found that it serves well as a theory-based evaluation tool with a double-stranded (actor strand and results chain) theory of change. The model helps reconcile learning and accountability and add explanatory power and an explicit actor perspective. It also helps establish causation and enable attribution claims at various results levels with its different elements. The model has some limitations but my results suggest it can be usefully adopted. The choice of its application depends on project evaluation context and purpose in specific cases.

Future Democracies - Laboratory of Collective Intelligence for Participatory Democracy - Yago Bermejo (MediaLab Prado, 2019)
The Laboratory of Collective Intelligence for Participatory Democracy (2016-2019) is a project that arose out of Medialab Prado in coordination with the Government Area of Citizen Participation, Transparency and Open Government of the city of Madrid. Its work has been very connected with the analysis, reflection and innovation found on the digital participation platform Decide Madrid. The project has also organised many workshops and conferences that have brought together hundreds of people from the world of participatory democracy. This publication summarises the project’s core working principles as well as the open activities it has hosted over the course of these three years. These pilot experiences offer a possible vision of the future of democratic governance.
Keywords: Democracy

Evaluation and learning in complex, rapidly changing health systems: China’s management of health sector reform - Xiao, Yue; Husain, Lewis; Bloom, Gerald (Globalization and Health, 2018)
Healthcare systems are increasingly recognised as complex, in which a range of non-linear and emergent behaviours occur. China’s healthcare system is no exception. The hugeness of China, and the variation in conditions in different jurisdictions present very substantial challenges to reformers, and militate against adopting one-size-fits-all policy solutions. As a consequence, approaches to change management in China have frequently emphasised the importance of sub-national experimentation, innovation, and learning. Multiple mechanisms exist within the government structure to allow and encourage flexible implementation of policies, and tailoring of reforms to context. These limit the risk of large-scale policy failures and play a role in exploring new reform directions and potentially systemically-useful practices. They have helped in managing the huge transition that China has undergone from the 1970s onwards. China has historically made use of a number of mechanisms to encourage learning from innovative and emergent policy practices. Policy evaluation is increasingly becoming a tool used to probe emergent practices and inform iterative policy making/refining. This paper examines the case of a central policy research institute whose mandate includes evaluating reforms and providing feedback to the health ministry. Evaluation approaches being used are evolving as Chinese research agencies become increasingly professionalised, and in response to the increasing complexity of reforms. The paper argues that learning from widespread innovation and experimentation is challenging, but necessary for stewardship of large, and rapidly-changing systems.

Future oriented conservation: knowledge governance, uncertainty and learning - Wyborn, Carina; van Kerkhoff, Lorrae; Dunlop, Michael; Dudley, Nigel; Guevara, Oscar (Biodiversity and Conservation, 2016)
Despite signiﬁcant progress in understanding climate risks, adaptation efforts in biodiversity conservation remain limited. Adaptation requires addressing immediate conservation threats while also attending to long term, highly uncertain and potentially transformative future changes. To date, conservation research has focused more on projecting climate impacts and identifying possible strategies, rather than understanding how governance enables or constrains adaptation actions. We outline an approach to futureoriented conservation that combines the capacities to anticipate future ecological change; to understand the implications of that change for social, political and ecological values; and the ability to engage with the governance (and politics) of adaptation. Our approach builds on the adaptive management and governance literature, however we explicitly address the (often contested) rules, knowledge and values that enable or constrain adaptation. We call for a broader focus that extends beyond technical approaches to acknowledge the sociopolitical challenges inherent to adaptation. More importantly, we suggest that conservation policy makers and practitioners can use this approach to facilitate learning and adaptation in the context of complexity, transformational change and uncertainty.

Learning to Learn Collectively - Wrigley, Rebecca (Capacity, 2008)
CABUNGO, a Malawian NGO, recently evaluated its own performance using the Most Significant Change approach. Rebecca Wrigley describes how, with the support of stakeholders,CABUNGO learned to improve its services.

The five trademarks of agile organizations - Wouter Aghina; Karin Ahlbäck; Aaron De Smet; Clemens Fahrbach; Christopher Handscomb; Gerald Lackey; Michael Lurie; Monica Murarka; Olli Salo; Elizabeth Seem; Jannik Woxholth (McKinsey anc Company, 2017)
Our experience and research demonstrate that successful agile organizations consistently exhibit the five trademarks described in this article. The trademarks include a network of teams within a people-centered culture that operates in rapid learning and fast decision cycles which are enabled by technology, and a common purpose that co-creates value for all stakeholders. These trademarks complement the findings from The McKinsey Global Survey Results: How to create an agile organization.

Using beneficiary feedback to improve development programmes: findings from a multi-country pilot - World Vision (World Vision, 2016)
This briefing summarises the findings from Beneficiary Feedback Mechanisms pilots and is intended to inform organisations and their funders about the development and implementation of feedback mechanisms.

The precautionary principle: protecting public health, the environment and the future of our children - World Health Organization (World Health Organization, Regional Office for Europe, 2004)

Forward Look - A vision for the World Bank Group in 2030 - World Bank (World Bank, 2016)

Learning and Results in World Bank Operations: Toward a New Learning Strategy - World Bank (IEG , The World Bank, 2015)
This report is the second in a program of evaluations that the Independent Evaluation Group (IEG) is conducting on the learning that takes place through World Bank projects. Learning and knowledge are treated as parts of a whole and are presumed to be mutually reinforcing.
Keywords: IMPORTANT

Strategic Framework for mainstreaming citizen engagemente in World Bank Group operations - World Bank (The World Bank, 2014)

World Development Report 2015: Mind, Society, and Behavior - World Bank (The World Bank, 2015)
Development economics and policy are due for a redesign. In the past few decades, research from across the natural and social sciences has provided stunning insight into the way people think and make decisions. Whereas the first generation of development policy was based on the assumption that humans make decisions deliberatively and independently, and on the basis of consistent and self-interested preferences, recent research shows that decision making rarely proceeds this way. People think automatically: when deciding, they usually draw on what comes to mind effortlessly. People also think socially: social norms guide much of behavior, and many people prefer to cooperate as long as others are doing their share. And people think with mental models: what they perceive and how they interpret it depend on concepts and worldviews drawn from their societies and from shared histories.The World Development Report 2015 offers a concrete look at how these insights apply to development policy. It shows how a richer view of human behavior can help achieve development goals in many areas, including early childhood development, household finance, productivity, health, and climate change. It also shows how a more subtle view of human behavior provides new tools for interventions. Making even minor adjustments to a decision-making context, designing interventions based on an understanding of social preferences, and exposing individuals to new experiences and ways of thinking may enable people to improve their lives.The Report opens exciting new avenues for development work. It shows that poverty is not simply a state of material deprivation, but also a “tax” on cognitive resources that affects the quality of decision making. It emphasizes that all humans, including experts and policy makers, are subject to psychological and social influences on thinking, and that development organizations could benefit from procedures to improve their own deliberations and decision making. It demonstrates the need for more discovery, learning, and adaptation in policy design and implementation. The new approach to development economics has immense promise. Its scope of application is vast. This Report introduces an important new agenda for the development community.

World Development Report 2017: Governance and the Law - World Bank (The World Bank, 2017)

A Guide to Assessing the Political Economy of Domestic Climate Change Governance - Worker, Jesse; Palmer, Niki (World Resources Institute, 2021)
This paper discusses how understanding the domestic political economy of climate governance is critical for developing informed strategies to build and sustain political ambition. It provides guidance and a methodology for domestic stakeholders to determine the types of institutional reforms, incentives, coalitions, and policy design that can entrench long-term political support for climate ambition. The assessment can also help users identify political barriers to more equitable climate action and identify reforms that may strengthen inclusion and accountability. EXECUTIVE SUMMARY Highlights ▪ There is overwhelming evidence of the social, economic, and environmental case to reduce greenhouse gas (GHG) emissions and rapidly scale up adaptation. Yet, despite a proliferation of climate laws and policies over the last 10–15 years, emissions are still rising, and adaptation needs remain urgent. ▪ This calls for a more sophisticated assessment of the political economy factors that may enable or constrain implementation of policies and actions and sustain political commitment at the country level. ▪ This guide offers an assessment methodology to understand how structural factors, rules and norms, stakeholders and interests, and ideas and narratives influence the political economy of climate action in a given country context. ▪ The methodology was developed on the basis of climate policy, governance, and political economy literature with contributions from subject matter experts. ▪ We intend the assessment to support civil society coalitions, reform-minded civil servants and politicians, international organizations, and other stakeholders. ▪ The resulting analysis should deepen the understanding of context while informing the advocacy, policy design, coalition building, capacity building, and communications of domestic stakeholders.

A Practical Guide to Distributed Scrum - Woodward, Elizabeth; Surdek, Steffan; Ganis, Matthew (IBM Press, 2010)
Succeed with Scrum in Even the Largest, Most Complex Distributed Development Projects Forewords by Ken Schwaber, Scott Ambler, Roman Pichler, and Matthew Wang This is the first comprehensive, practical guide for Scrum practitioners working in large-scale distributed environments. Written by three of IBM’s leading Scrum practitioners--in close collaboration with the IBM QSE Scrum Community of more than 1000 members worldwide--this book offers specific, actionable guidance for everyone who wants to succeed with Scrum in the enterprise. Readers will follow a journey through the lifecycle of a distributed Scrum project, from envisioning products and setting up teams to preparing for Sprint planning and running retrospectives. Each chapter presents a baseline drawn from “conventional” Scrum, then discusses additional issues faced by distributed teams, and presents specific best-practice solutions, alternatives, and tips the authors have identified through hard, empirical experience. Using real-world examples, the book demonstrates how to apply key Scrum practices, such as look-ahead planning in geographically distributed environments. Readers will also gain valuable new insights into the agile management of complex problem and technical domains. Coverage includes•  Developing user stories and working with Product Owners as a distributed team•  Recognizing and fixing the flaws Scrum may reveal in existing processes•  Engaging in more efficient Release and Sprint planning•  Conducting intense, brief daily Scrum meetings in distributed environments•  Managing cultural and language differences•  Resolving dependencies, performing frequent integration, and maintaining transparency in geographically distributed environments•  Successfully running remote software reviews and demos•  Brainstorming what worked and what didn’t, to improve future Sprints This book will be an indispensable resource for every team leader, member, product owner, or manager working with Scrum or other agile methods in any distributed software development organization.

Scrambling after Moving Targets: Monitoring & Evaluation Applied to Adaptive Management Approaches in Peacebuilding - Woodrow, Peter; Jean, Isabella (New Directions in Peacebuilding Evaluation, 2019)
In this landmark collection, the voices of pathMakers and innovators in peacebuilding evaluation are assembled to provide new direction for the field.
Keywords: IMPORTANT

Fast Forwarding to Systems Maps of Corruption: Getting to Usable Analysis More Quickly - Woodrow, Peter (CJL, 2023)
For AC practitioners, systems mapping is essential but also difficult. To lessen obstacles, Peter Woodrow proposes a scaffolded approach. An Experiment in “Fast Forwarding” Drawing upon our near-decade work on corruption, we recently decided to try an experiment: we would present “common patterns” of corruption as tentative models to adapt and add to—rather than try to teach people to do systems mapping from scratch. In this teaching experiment, each common pattern would function as a kind of “scaffolding” or framework that participants can build on to generate a cogent systems map as the basis for identifying possible points of intervention and subsequent program planning.

Systems Thinking and Practice: A Guide to Concepts, Principles and Tools for FCDO and Partners - Woodhill, Jim; Millican, Juliet (K4D, 2022)
This guide is a basic reference on systems thinking and practice tailored to the context and needs of the UK Government’s Foreign, Commonwealth & Development Office (FCDO). It is an output of the FCDO Knowledge for Development Programme (K4D), which facilitated a Learning Journey on Systems Thinking and Practice with FCDO staff during 2021 and 2022. The guide offers a common language and shared framing of systems thinking for FCDO and its partners. It explores what this implies for working practices, business processes and leadership. It also offers links to additional resources and tools on systems thinking. We hope it can support systems thinking to become more commonplace within the culture and practices of FCDO and working relations with partner organisations.

Reacting or adapting? Purposeful adaptation and response to contextual change. - Wood, Nik (Rebel with Causation, 2020)
Adaptive management is increasingly on the ticket for development programming, and has been crucial in the wake of covid-19. I’ve been working with adaptive programmes for most of my career, …

Agile Metrics: The Good, the Bad, and the Ugly - Wolpers, Stefan (dzone.com, 2017)
Suitable agile metrics reflect either a team’s progress in becoming agile or your organization’s progress in becoming a learning organization.

Building Better Systems - An introduction to System Innovation - Winhall, Jennie; Leadbeater, Charles (The Rockwool Foundation, 2020)
This paper lays out a series of steps people can take to create the new systems we need to meet shared, public challenges. Systems are ubiquitous and powerful. We rely on them to support our daily lives: every time we turn on a tap, flick a switch for electricity, drop our child at school, jump on a bus or visit a doctor we rely on a wider system. There is a widespread sense, among decision makers and citizens that in the coming decades society will need not just new products, software and services, but new systems for living sustainably in a socially inclusive society. The need for better, different systems will be heightened by the impact and lessons of the Covid-19 pandemic. Systems are productive precisely because they are more than standalone products. A system pulls together all the different ingredients needed to meet a need or to produce an outcome: the shipping container is a product, containerisation is a system; a contactless payment card is a product which only works as part of a payments system; an operation in a hospital can only take place because it is part of a wider health system. To understand how a system works it has to be seen as a whole, from the macro policy frameworks of social security systems right down to how a citizen goes about finding a job. Many of the systems we rely on for care and work, energy and transport, education and health are under pressure to change. Society faces both deeply entrenched and growing challenges that are outpacing the systems we have. We also have opportunities to create new, alternative systems as new knowledge, values and technologies emerge, from artificial intelligence and bitcoin, to circular and renewable systems of production. Rising to the challenge of fixing an existing system and exploring the possibility of creating a new system are different undertakings. The first is about optimising what exists, the second is about creating something different and better. We want this project to yield practical insights for those who want to respond to the systemic challenges of today by stepping into the possibilities of the future. Acting to change systems depends on new ways of seeing both challenge and opportunity: why systems come under strain and what unlocks the potential for alternatives. It depends on better understanding how new systems form, and what and who is part of initiating and driving the transition to them. In putting together this paper and the ones that will follow from it we want to clarify how to assess the need for, invest in and act on the process of deliberate system change.

The Patterns of Possibility - How to Recast Relationships to Create Healthier Systems and Better Outcomes - Winhall, Jennie; Leadbeater, Charles (The Rockwool Foundation, 2022)
In Building Better Systems, we introduced four keys to unlock system innovation: purpose and power, relationships and resource flows. These four keys make up a set. Systems are often hard to change because power, relationships, and resource flows are locked together in a reinforcing pattern to serve the system’s current purpose. Systems start to change fundamentally when this pattern is disrupted and opened up so that a new configuration can emerge, serving a new purpose. In this article series we delve deeper into these four keys and provide practical advice on how they can be put to use. This article is about relationships. Systems are defined by the patterns of interactions between their parts: their relationships. Those interactions generate the outcomes of the system as a whole. Transforming the outcomes of a system requires remaking its relationships and then unlocking the other keys to system innovation: purpose, power and resources. This shift in relationships allows all those in the system to learn faster, to be more creative. System innovators redesign the relationships in the system to allow dramatically enhanced learning across the system, and thereby generate far better outcomes.

What is adaptive management? - Wilson, Gregory (ΔNTYLLES, 2016)
Recently there has been growing recognition that most development challenges are in actual fact ‘wicked’ problem that requires development actors to recognize that the optimal management approaches…

Test It and They Might Come: Improving the Uptake of Digital Tools in Transparency and Accountability Initiatives - Wilson, Christopher; de Lanerolle, Indra (IDS Bulletin, 2016)
Keywords: IMPORTANT

Outcome Harvesting - Wilson-Grau, Ricardo; Britt, Heather (Ford Foundation, 2012)
Outcome Harvesting was developed by Ricardo Wilson-Grau with colleagues Barbara Klugman, Claudia Fontes, David Wilson-Sánchez, Fe Briones Garcia, Gabriela Sánchez, Goele Scheers, Heather Britt, Jennifer Vincent, Julie Lafreniere, Juliette Majot, Marcie Mersky, Martha Nuñez, Mary Jane Real, Natalia Ortiz, and Wolfgang Richert. Over the past 8 years, Outcome Harvesting and has been used to monitor and evaluate the achievements of hundreds of networks, non-governmental organizations, research centers, think tanks, and communitybased organizations around the world. This brief is intended to introduce the concepts and approach used in Outcome Harvesting to grant makers, managers, and evaluators, with the hope that it may inspire them to learn more about the method and apply it to appropriate contexts. Thus, it is not a comprehensive guide to or explanation of the method, but an introduction to allow evaluators and decision makers to determine if the method is appropriate for their evaluation needs. Where possible, we have included examples to illustrate how Outcome Harvesting is applied to real situations. For each case story, organizations were asked to provide a description of the outcome and a summary of the role played by the organization. Sometimes they added other information such as the outcome’s significance. Some details and identifiers were redacted for confidentiality purposes. A draft of this brief was graciously commented on by Bob Williams, Fred Carden, Sarah Earl, Richard Hummelbrunner and Terry Smutylo. The final text is, of course, the sole responsibility of the authors and editor.

What is Outcome Harvesting? - Wilson-Grau, Ricardo (2014)
Ricardo Wilson-Grau, former Chair of the Board of Stewards of the Outcome Mapping Learning Community, introduces Outcome Harvesting, an approach for retrospective identification of outcomes based on the principles of Outcome Mapping. More details here: http://www.outcomemapping.ca/nuggets/outcome-harvesting

Outcome Harvesting - Wilson-Grau, R. (Better Evaluation, 2015)
Outcome Harvesting collects (“harvests”) evidence of what has changed (“outcomes”) and, then, working backwards, determines whether and how an intervention has contributed to these changes. Outcome Harvesting has proven to be especially useful in complex situations when it is not possible to define concretely most of what an intervention aims to achieve, or even, what specific actions will be taken over a multi-year period.

Guidelines and good practices for evidence-informed policy-making in a government department - Wills, Alf; Tshangela, Mapula; Shaxson, Louise; Datta, Ajoy; Matomela, Bongani (2016)

Change in challenging contexts - Williamson, Tim (ODI, 2015)
This report explores how genuine institutional change takes place in fragile and conflict-affected states reforms can be supported.

External validity and policy adaptation. From impact evaluation to policy design - Williams, Martin J. (BSG Working Paper Series, 2017)
With the growing number of rigorous impact evaluations worldwide, the question of how best to apply this evidence to policymaking processes has arguably become the main challenge for evidence-based policymaking. How can policymakers predict whether a policy will have the same impact in their context as it did elsewhere, and how should this influence the design and implementation of policy? This paper introduces a simple and flexible framework to address these questions of external validity and policy adaptation. I show that failures of external validity arise from an interaction between a policy’s theory of change and a dimension of the context in which it is being implemented, and develop a method of “mechanism mapping” that maps a policy’s theory of change against salient contextual assumptions to identify external validity problems and suggest appropriate policy adaptations. In deciding whether and how to adapt a policy in a new context, I show there is a fundamental informational trade-o↵ between the strength and relevance of evidence on the policy from other contexts and the policymaker’s knowledge of the local context. This trade-o↵ can guide policymakers’ judgments about whether policies should be copied exactly from elsewhere, adapted, or invented anew.

Thinking and Working Politically on Health Systems Resilience: Learning from the experience of Cameroon, Nepal and South Africa during COVID-19 - Williams, Gareth (TWP Community of Practice, 2022)

Adaptive Management: From More Talk to Real Action - Williams, Byron K.; Brown, Eleanor D. (Environmental Management, 2014)
The challenges currently facing resource managers are large-scale and complex, and demand new approaches to balance development and conservation goals. One approach that shows considerable promise for addressing these challenges is adaptive management, which by now is broadly seen as a natural, intuitive, and potentially effective way to address decision-making in the face of uncertainties. Yet the concept of adaptive management continues to evolve, and its record of success remains limited. In this article, we present an operational framework for adaptive decision-making, and describe the challenges and opportunities in applying it to real-world problems. We discuss the key elements required for adaptive decision-making, and their integration into an iterative process that highlights and distinguishes technical and social learning. We illustrate the elements and processes of the framework with some successful on-the-ground examples of natural resource management. Finally, we address some of the difficulties in applying learning-based management, and finish with a discussion of future directions and strategic challenges.
Keywords: IMPORTANT

Technical challenges in the application of adaptive management - Williams, Byron K.; Brown, Eleanor D. (Biological Conservation, 2016)
Adaptive management is an approach for simultaneously managing and learning about natural resources, by acknowledging uncertainty and seeking to reduce it through the process of management itself. Adaptive decision making can be applied to pressing issues in conservation biology such as species reintroduction, disease and invasive species control, and habitat restoration, as well as to management of natural resources in general. After briefly outlining a framework and process for adaptive management, we focus on an overview of the key technical issues related to problem framing and the ability of resource managers to learn from their experience. These technical issues include the treatment of uncertainty and its propagation over time; nonstationarity in long-term environmental trends; the applicability of adaptive management across scales; requirements for models and management alternatives that promote learning; the value of the information produced with adaptive management; the challenge to management of uncertainty and surprise; and institutional (social) learning. To accommodate these and other challenges that are now coming into focus, the learning-based approach of adaptive management will need to be adjusted and expanded in the future.
Keywords: Adaptive Management; Learning; Nonstationarity; Resilience; Uncertainty; Value of information

Adaptive management of natural resources—framework and issues - Williams, Byron K. (Journal of Environmental Management, 2011)
Adaptive management, an approach for simultaneously managing and learning about natural resources, has been around for several decades. Interest in adaptive decision making has grown steadily over that time, and by now many in natural resources conservation claim that adaptive management is the approach they use in meeting their resource management responsibilities. Yet there remains considerable ambiguity about what adaptive management actually is, and how it is to be implemented by practitioners. The objective of this paper is to present a framework and conditions for adaptive decision making, and discuss some important challenges in its application. Adaptive management is described as a two-phase process of deliberative and iterative phases, which are implemented sequentially over the timeframe of an application. Key elements, processes, and issues in adaptive decision making are highlighted in terms of this framework. Special emphasis is given to the question of geographic scale, the difﬁculties presented by non-stationarity, and organizational challenges in implementing adaptive management.

Passive and active adaptive management: Approaches and an example - Williams, Byron (Journal of Environmental Management, 2010)
Adaptive management is a framework for resource conservation that promotes iterative learning-based decision making. Yet there remains considerable confusion about what adaptive management entails, and how to actually make resource decisions adaptively. A key but somewhat ambiguous distinction in adaptive management is between active and passive forms of adaptive decision making. The objective of this paper is to illustrate some approaches to active and passive adaptive management with a simple example involving the drawdown of water impoundments on a wildlife refuge. The approaches are illustrated for the drawdown example, and contrasted in terms of objectives, costs, and potential learning rates. Some key challenges to the actual practice of AM are discussed, and tradeoffs between implementation costs and long-term benefits are highlighted.

Rewards and Risks Associated with Community Engagement in Anti-Poaching and Anti-Trafficking - Wilkie, David; Painter, Michael; Jacob, Anila (Measuring Impact, 2016)
This Biodiversity Technical Brief summarizes research on the roles communities do and should play in countering wildlife crime, motivations, and benefits for individuals and communities to engage in anti-poaching and anti-trafficking efforts, and risks and mitigation of risks associated with involvement. Peer-reviewed and grey literature, structured interviews, and case studies informed the research, with highlights from six case studies in Peru, Madagascar, Namibia, Philippines, Kenya, and Indonesia.

Adaptive Management in Habitat Conservation Plans - Wilhere, George F. (Conservation Biology, 2002)
Habitat conservation plans (HCPs) allow incidental take of threatened or endangered species in exchange for conservation measures that minimize and mitigate such taking. Habitat conservation plans entail a compromise between regulatory certainty and scientific uncertainty. This compromise is controversial because many HCPs are thought to inadequately address scientific uncertainty. Adaptive management is the systematic acquisition and application of reliable information to improve natural resource management over time. Ideally, under adaptive management, conservation strategies are implemented as a deliberate experiment. This approach can establish cause-and-effect relationships and point the way toward optimal strategies. Adaptive management has been promoted as essential to management under uncertainty, but few HCPs incorporate genuine adaptive management. Habitat conservation plans will continue to lack adaptive management until certain conditions are met, such as acknowledgment that an HCP is a management hypothesis, landowner interest in improving biological outcomes, and sufficient financial resources. Economic incentives would encourage adaptive management in HCPs. Habitat conservation plan permittees might receive direct payments or tax deductions for reliable information that benefits a species. “Mitigation credits” could be awarded for information produced through adaptive management. In effect, habitat would be exchanged for information that benefits a species. Successful use of mitigation credits would depend on correctly valuing information and enforcement of the U.S. Endangered Species Act. Under a “precautionary polluter pays principle,” an HCP permittee would put up an environmental assurance bond. Portions of the bond are returned with interest as adaptive management demonstrates that environmental damages are unlikely to occur. Funds spent on adaptive management are funds unavailable for habitat protection, so limited financial resources may force a compromise between protecting habitat and acquiring knowledge.

Building a global learning alliance on adaptive management - Wild, Leni; Ramalingam, Ben (ODI, 2018)

Putting theory into practice: how DFID is doing development differently - Wild, Leni; Booth, David; Valters, Craig (ODI, 2017)
Keywords: IMPORTANT

Adapting development: improving services to the poor - Wild, Leni; Booth, David; Cummings, Clare; Foresti, Marta; Wales, Joseph (ODI, 2015)
On current trends, it will take decades or longer to bring basic services to the world’s most disadvantaged people. Meeting this challenge means recognising the political conditions that enable or obstruct development progress - a radical departure from the approach of the Millennium Development Goals.
Keywords: Practice

Doing Development Differently - Wild, Leni (Stanford Social Innovation Review, 2021)
International aid must use different approaches to address the massive systemic problems it seeks to solve.
Keywords: ⛔ No DOI found

Uncertainty, Precaution, and Adaptive Management in Wildlife Trade - Wiersema, Annecoos (Michigan Journal of International Law, 2015)

Capacity Development in a Participatory Adaptive Programme: the Case of the Clarissa Consortium - Widmer, Mireille; Apgar, Marina; Afroze, Jiniya; Malla, Sudhir; Healey, Jill; Constant, Sendrine (CLARISSA Research and Evidence Paper, 2022)
Doing development differently rests on deliberate efforts to reflect and learn, not just about what programmes are doing and achieving, but about how they are working. This is particularly important for an action research programme like Child Labour: Action- Research-Innovation in South and South-Eastern Asia (CLARISSA), which is implemented by a consortium of organisations from across the research and development spectrum, during a rapidly changing global pandemic. Harnessing the potential of diverse skills and complementary strengths across partners in responding to the complex challenge of the worst forms of child labour, requires capacity to work together in novel ways. This Research and Evidence Paper documents how CLARISSA approached capacity development, and what we learnt from our challenges and successes. From the start, the programme incorporated a capacity development strategy resting on self-assessment of a wide range of behavioural and technical competencies that were deemed important for programme implementation, formal training activities, and periodic review of progress through an after-action review (AAR) process. An inventory of capacity development activities that took place during the first year of implementation reveals a wide range of additional, unplanned activities, enabled by the programme’s flexibility and adaptive management strategy. These are organised into eight modalities, according to the individual or collective nature of the activity, and its sequencing – namely, whether capacity development happens prior to, during, or after (from) implementation. We conclude with some reflections on the emergent nature of capacity development. Planning capacity development in an adaptive programme provides a scaffolding in terms of time, resources, and legitimacy that sustains adaptiveness. We also recognise the gaps that remain to be addressed, particularly on scaling up individual learning to collective capabilities, and widening the focus from implementation teams to individuals working at consortium level.

Wildlife laws monitoring as an adaptive management tool in protected area management in Ghana: a case of Kakum Conservation Area - Wiafe, Edward Debrah (SpringerPlus, 2016)
INTRODUCTION: The wildlife laws of Ghana alienated the rural communities from forests and material well-being depended upon for their livelihood and this manifests itself in the progressive conflict between the park patrol staff and poachers from the fringes of the protected areas. CASE DESCRIPTION: The main aim of this study was to determine the impact of quantification of patrol efforts on indicators of illegal hunting activities that occur in rainforest protected areas, as a result of monitoring patrol operations and modifying the original plan. The specific objectives were to determine the optimal patrol efforts necessary to reduce illegal wildlife use to minimal; and the influence of the rainfall and seasonal activities on illegal wildlife use. DISCUSSION AND EVALUATION: The results indicated that as the patrol efforts increased the encounter with illegal wildlife use also increased until a certain point that the encounter rates started decreasing. Neither rainfall nor seasonal activities influenced the illegal activities and the patrol efforts. The protection staff of rainforest protected areas would work effectively to bring down illegal wildlife off-take to the barest minimum if monitored, quantified and provide feed-back. CONCLUSIONS: Illegal wildlife off-take can also be reduced by the protection staff if the original plans are made flexible to be adjusted. Recommendations for further studies have been made.

How We Model Matters: A Manifesto for the Next Generation of Program Theorizing - Whynot, Jane; Lemire, Sebastian; Montague, Steve (Canadian Journal of Program Evaluation, 2019)
In this concluding article, grounded on the exemplary contributions contained in the preceding pages, the guest editors scale the proverbial soapbox and present a manifesto to guide the pursuit and advancement of the next generation of program theorizing. Formulating ten declarations for program theory development and examination, the modest hope of the authors is to motivate and inspire reflective evaluation practitioners to broaden their views, approaches, and techniques for future program theorizing.

A Guide to Digital Feedback Loops. An approach to strengthening program outcomes through data for decision making - Whittle, Dennis; Campbell, Megan (USAID, 2019)
Feedback is information about what happens as a result of what you do. Using that information to adapt what you do or how you do it creates what is called a feedback loop. A digital feedback loop uses digital technology at some stage of the feedback loop. Digital feedback loops help USAID missions improve the effectiveness and efficiency of their activities and can support partner countries on their journey to self-reliance through increased information sharing and improved government and civil society capacity. USAID missions and their partners have used digital feedback loops to improve HIV treatment targeting in Zimbabwe, engage three million young people in Nigeria in policy, and reduce field reporting times from one month to one day. Digital feedback loops provide access to information faster and more cheaply than ever before. As part of an adaptive management approach, digital feedback loops increase our ability to act on opportunities and respond to challenges. That in turn can improve outcomes and reduce the risk of waste and failure. This guide outlines the case for integrating digital feedback loops into your work, and provides tools, resources and guiding questions to support you as you get started. The guide includes examples of how USAID missions are using digital feedback loops to improve their programs in various sectors and provides worksheets to help you apply digital feedback loops to your own context.

Impact Evaluation of Development Interventions: A Practical Guide - White, Howard; Raitzer, David (Asian Development Bank, 2017)
This book offers guidance on the principles, methods, and practice of impact evaluation. It contains material for a range of audiences, from those who may use or manage impact evaluations to applied researchers.

Using Emergence to Take Social Innovation to Scale - Wheatley, Margaret; Frieze, Deborah (Fieldnotes, 2007)
In spite of current ads and slogans, the world doesn’t change one person at a time. It changes as networks of relationships form among people who discover they share a common cause and vision of what’s possible. This is good news for those of us intent on changing the world and creating a positive future. Rather than worry about critical mass, our work is to foster critical connections. We don’t need to convince large numbers of people to change; instead, we need to connect with kindred spirits. Through these relationships, we will develop the new knowledge, practices, courage, and commitment that lead to broad-based change. But networks aren’t the whole story. As networks grow and transform into active, working communities of practice, we discover how Life truly changes, which is through emergence. When separate, local efforts connect with each other as networks, then strengthen as communities of practice, suddenly and surprisingly a new system emerges at a greater level of scale. This system of influence possesses qualities and capacities that were unknown in the individuals. It isn’t that they were hidden; they simply don’t exist until the system emerges. They are properties of the system, not the individual, but once there, individuals possess them. And the system that emerges always possesses greater power and influence than is possible through planned, incremental change. Emergence is how Life creates radical change and takes things to scale. Emergence has a life-cycle. It begins with networks, shifts to intentional communities of practice and evolves into powerful systems capable of global influence. Since its inception in 1992, The Berkana Institute has striven to learn how living systems work, how they emerge from networks to communities to systems of influence. In our global work—primarily with economically poor communities in many different nations—we have experimented actively with emergence in many different contexts. We have demonstrated what’s possible when we connect people across difference and distance. By applying the lessons of living systems and working intentionally with emergence and its life-cycle, we have become confident that local social innovations can be taken to scale and provide solutions to many of the world’s most intractable issues.

Supporting Pioneering Leaders as Communities of Practice - How to Rapidly Develop New Leaders in Great Numbers - Wheatley, Margaret J (The Berkana Institute, 2002)
Do you ever stand back and try to see the big picture, the view from 50,000 feet of what's going on in organizations, communities, the world? From up there, how would you describe these times? Is it a time of increasing economic and political instability, of growing divisiveness and fear, of failing systems and dying dreams? Is it a time of new possibilities, of great examples of hope, of positive human evolution, of transformation? Are we succeeding in solving major problems, are we creating more? Is it any of these things, is it all of these things? It's important to think about how we answer this question, because that answer affects our choice of actions. If we think that, generally, things are working, that at present we're going through a difficult but temporary downturn, then we don't question current systems or their operating assumptions. Instead, we work hard to revive and improve them. We support initiatives and programs focused on process improvements, developing present systems to work more effectively and more efficiently. If we believe that the old system cannot be repaired, if we expect to see only more system failures, then the work is not to fix. Instead, support needs to be given to radically different processes and methods, new systems based on new assumptions. The work becomes not process improvement but process revolution.

Understanding Political Economy Analysis and Thinking and Working Politically - Whaites, Alan; Piron, Laure-Hélène; Menocal, Alina Rocha; Teskey, Graham (FCDO and TWP CoP, 2023)
This guide is adapted from work by the UK Foreign, Commonwealth and Development Office (FCDO) with inputs from members of the Thinking and Working Politically Community of Practice (TWP CoP). It outlines how to understand and use a set of analytical tools that are collectively known as Political Economy Analysis (PEA). The guide aims to equip practitioners to act in an informed manner, given that development objectives are invariably politically complex, and entail engaging with counterparts’ political incentives and preferences. The guide summarises different types of tools – from very light-touch to more in-depth approaches – and provides advice on how development professionals can decide what is most appropriate in a given context, with illustrations based on the experiences of teams working on these issues. This guide will help development professionals and others to make use of PEA and to apply it to their own specific needs. The first part of the guide offers a general picture of the approach. The second part provides more specific guidance for those who are tasked with deploying a PEA. Contents --> Main audience What is PEA, its role and purpose (Section 2) --> General information for all readers The main elements of PEA (Section 3) Thinking and Working Politically (Section 4) --> Core information for teams planning and using PEA How to ensure quality (Section 5) --> Essential reading for those directly responsible for a PEA Important concepts and terminology (Annex) --> General information for all readers

A Governance Practitioner’s Notebook: Alternative Ideas and Approaches - Whaites, Alan; Gonzalez, Eduardo; Fyson, Sara; Teskey, Graham (OECD, 2015)
The Governance Practitioner’s Notebook takes an unusual approach for the OECD-DAC Network on Governance (GovNet). It brings together a collection of specially written notes aimed at those who work as governance practitioners within development agencies. It does so, however, without attempting to offer definitive guidance – instead aiming to stimulate thinking and debate. To aid this process the book is centred on a fictional Governance Adviser. The Notebook’s format provides space for experts to speak on today’s governance issues: politics, public sector reform and stakeholder engagement. It encourages debate, charts the evolution of donor thinking, and highlights future challenges in the age of the Sustainable Development Goals. Each section introduces both technical issues and major areas of debate, providing ideas for future development support to institutional reform.
Keywords: IMPORTANT

The Beginner’s Guide to Political Economy Analysis (PEA) - Whaites, Alan (National School of Government International, 2017)
Over the last two decades aid agencies and academics have been on a journey of lesson learning and adaptation in relation to `politics.’ This journey has been driven by a determination to improve impact in all areas of development, but for some time it was particularly associated with work on public sector reform. Now, however, there is an increasing expectation that Political Economy Analysis (PEA) should be part and parcel of designing and implementing any programme or activity (and a brief history of the meandering journey of development actors on PEA can be found in The Policy Practice’s Briefing Paper 11 – see below). DFID in the UK is fairly typical among large development organisations in running an excellent course on political economy analysis, complete with 200 pages of resources and various online videos and case studies (and this type of course is recommended for those who want to take their exploration of PEA further). Even so, PEA is not just for those who have `done the course and bought the T-shirt,’ it is something that can be absorbed and implemented quickly by everybody. Indeed, the growth of interest in PEA is a reminder that this can look like a complex and daunting field and so this guide aims to offer an entry-point for all those who want to use PEA in their own work. In doing so, this guide borrows from the best materials that are available while also adapting some approaches by incorporating wider ideas on politics and institutions. This guide affirms that there should never be an official `orthodoxy’ for PEA and so the emphasis here is on questions, prompts and ideas to help thinking and practice. There is also an attempt to clarify jargon wherever needed, while recognising that The Policy Practice (TPP) and the Overseas Development Institute (ODI) have produced a more complete glossary of PEA terminology. The note will instead focus on `the essentials’ of PEA as they relate to the following questions:  Why do we do political economy analysis, and what is it?  What kinds of issues and ingredients are often included in a PEA?  How do we make sense of the different varieties of PEA?  What tools are out there to help us conduct a PEA?  What is thinking and working politically?

Governance and Service Delivery: Practical Applications of Social Accountability Across Sectors - Wetterberg, Anna; Brinkerhoff, Derick W.; Hertz, Jana C. (RTI Press, 2016)
The six case studies in Governance and Service Delivery: Practical Applications of Social Accountability Across Sectors illustrate the multiple ways citizen participation in accountability – called social accountability – can lead to positive effects on governance, citizen empowerment, and service delivery. Drawing on their extensive experience implementing international donor-funded programs and projects, the authors examine six recent RTI International projects in Africa and Asia. The analysis focuses on both intended and actual effects, and it unpacks the influence of context on implementation and outcomes achieved, as well as on prospects for sustainability. Donors, academics, policy makers, practitioners and others interested in sustainable development and governance will find this systematic review invaluable.

Realist Impact Evaluation: An introduction - Westhorp, Gill (Methods Lab, 2014)
• Realist evaluation is a member of a family of theory-based evaluation approaches which begin by clarifying the ‘programme theory’: the mechanisms that are likely to operate, the contexts in which they might operate and the outcomes that will be observed if they operate as expected. • Realist approaches assume that nothing works everywhere for everyone: context makes a big difference to programme outcomes. A realist evaluation asks not ‘what works?’ but ‘how or why does this work, for whom, in what circumstances?’ • Realist impact evaluation is most appropriate for evaluating new initiatives or programmes that seem to work but where ‘how and for whom’ is not yet understood; programmes that have previously demonstrated mixed patterns of outcomes; and those that will be scaled up, to understand how to adapt the intervention to new contexts.

Digital habitats: stewarding technology for communities - Wenger, Etienne; White, Nancy; Smith, John D. (CPsquare, 2009)

Digital habitats: stewarding technology for communities. Chapter 10 - Action Book - Wenger, Etienne; White, Nancy; Smith, John D. (Cpsquare, 2009)

Cultivating Communities of Practice: A Guide to Managing Knowledge - Wenger, Etienne; McDermott, Richard Arnold; Snyder, William (Harvard Business Press, 2002)
Today's marketplace is fueled by knowledge. Yet organizing systematically to leverage knowledge remains a challenge. Leading companies have discovered that technology is not enough, and that cultivating communities of practice is the keystone of an effective knowledge strategy.Communities of practice come together around common interests and expertise- whether they consist of first-line managers or customer service representatives, neurosurgeons or software programmers, city managers or home-improvement amateurs. They create, share, and apply knowledge within and across the boundaries of teams, business units, and even entire companies-providing a concrete path toward creating a true knowledge organization.In Cultivating Communities of Practice, Etienne Wenger, Richard McDermott, and William M. Snyder argue that while communities form naturally, organizations need to become more proactive and systematic about developing and integrating them into their strategy. This book provides practical models and methods for stewarding these communities to reach their full potential-without squelching the inner drive that makes them so valuable.Through in-depth cases from firms such as DaimlerChrysler, McKinsey & Company, Shell, and the World Bank, the authors demonstrate how communities of practice can be leveraged to drive overall company strategy, generate new business opportunities, tie personal development to corporate goals, transfer best practices, and recruit and retain top talent. They define the unique features of these communities and outline principles for nurturing their essential elements. They provide guidelines to support communities of practice through their major stages of development, address the potential downsides of communities, and discuss the specific challenges of distributed communities. And they show how to recognize the value created by communities of practice and how to build a corporate knowledge strategy around them.Essential reading for any leader in today's knowledge economy, this is the definitive guide to developing communities of practice for the benefit-and long-term success-of organizations and the individuals who work in them.Etienne Wenger is a renowned expert and consultant on knowledge management and communities of practice in San Juan, California. Richard McDermott is a leading expert of organization and community development in Boulder, Colorado. William M. Snyder is a founding partner of Social Capital Group, in Cambridge, Massachusetts.

System Change: A Guidebook for Adopting Portfolio Approaches - Wellsch, Brent (UNDP, 2022)
This guidebook codifies the principles and methods of applying systems change and portfolio approaches to complex development challenges with practical tools and examples. It is based on the empirical learning generated from the collaborative initiatives in UNDP Country Offices in Bhutan, Pakistan, the Philippines, and Viet Nam with support from Regional Innovation Centre for Asia and the Pacific.

Testing the Waters: A Qualitative Comparative Analysis of the Factors Affecting Success in Rendering Water Services Sustainable Based on ICT Reporting - Welle, Katharina; Williams, Jennifer; Pearce, Joseph; Befani, Barbara (MAVC, 2015)
This research conducted by WaterAid, Itad and IRC aims to understand the factors that facilitate and inhibit the success of ICT-based reporting to improve rural water supply sustainability.

Timely evaluation in international development - Webster, Jayne; Exley, Josephine; Copestake, James; Davies, Rick; Hargreaves, James (Journal of Development Effectiveness, 2018)
Impact and process evaluations are increasingly used in international development; however they are generally retrospective in outlook. A more timely approach to evaluation aims to identify necessary, feasible and effective changes during a programme or intervention’s lifetime. This paper aims to identify, categorise, describe and critically appraise methods to support more timely evaluation in international development. Potential methods were identified through scoping seminar, public symposium, targeted review of the literature, and the authors’ own experiences and opinions. Findings from the different data sources were reviewed collectively by the author group and triangulated to develop an analytical framework. We identified four purposes of timely evaluation for international development, and critiqued the use of approaches against four dimensions of timeliness and flexibility. Whilst we found significant interest in more timely approaches to evaluation in international development, there was a dearth of published empirical evidence upon which to base strong recommendations. There is significant potential for timely evaluation to improve international development outcomes. New approaches to mixing and adapting existing methods, together with new technologies offer increased potential. Research is needed to provide an empirical evidence base upon which to further develop the application, across sectors and contexts, of timely evaluation in international development.
Keywords: Adaptive learning; Outcome evaluation; Programme improvement; impact evaluation

Principles for Monitoring, Evaluation, and Adaptive Management of Environmental Water Regimes - Webb, J. Angus; Watts, Robyn J.; Allan, Catherine; Warner, Andrew T. (Water for the Environment, 2017)
Environmental water is often a contentious investment in the environment that must be delivered under uncertainty regarding the ecological and social benefits they deliver. Adaptive management can be used to facilitate decision making under uncertainty, and use new knowledge and understanding to improve management decisions and outcomes over time. However, there is a perception that adaptive management has failed to deliver improvements in decision making and learning. Monitoring and evaluation are an essential feature of adaptive management. However, past monitoring for river protection and restoration has too often been of insufficient quality to facilitate adaptive learning. Moreover, environmental water represents a class of protection and restoration that is inherently difficult to monitor, and some of the learnings are not evident in part because fragmented assessment, documentation, and reporting of adaptive management can obscure successes. We outline a set of general principles for improving monitoring, evaluation, and adaptive management of environmental water regimes, and provide several examples from Australia and North America. Successful adaptive management depends upon the formation and maintenance of strong partnerships and an appreciation of the importance of individuals in teams. Through such partnerships, it is possible to coordinate monitoring programs over large scales, create programs of requisite simplicity, and use innovative approaches to evaluation. Finally, we note that although adaptive management has occurred in the past, too often it has been poorly documented and reported. The inclusion of reflectors in monitoring and evaluation teams will better capture the lessons learned by individual programs, improving adaptive management into the future.
Keywords: Environmental water management; Knowledge exchange; Legitimacy; implementation challenge

Habits of a system thinker - WCST (Waters Center For Systems Thinking, 2020)
From the Waters Center For Systems Thinking. See: https://thinkingtoolsstudio.waterscenterst.org/courses/habits https://thinkingtoolsstudio.waterscenterst.org/cards

The Global Wildlife Program - Knowledge Platform - 2016-2018 - WB (The World Bank, 2018)

Policy-driven monitoring and evaluation: Does it support adaptive management of socio-ecological systems? - Waylen, Kerry A.; Blackstock, Kirsty L.; van Hulst, Freddy J.; Damian, Carmen; Horváth, Ferenc; Johnson, Richard K.; Kanka, Robert; Külvik, Mart; Macleod, Christopher J. A.; Meissner, Kristian; Oprina-Pavelescu, Mihaela M.; Pino, Joan; et al. (Science of The Total Environment, 2019)
Inadequate Monitoring and Evaluation (M&E) is often thought to hinder adaptive management of socio-ecological systems. A key influence on environmental management practices are environmental policies: however, their consequences for M&E practices have not been well-examined. We examine three policy areas - the Water Framework Directive, the Natura 2000 Directives, and the Agri-Environment Schemes of the Common Agricultural Policy - whose statutory requirements influence how the environment is managed and monitored across Europe. We use a comparative approach to examine what is monitored, how monitoring is carried out, and how results are used to update management, based on publicly available documentation across nine regional and national cases. The requirements and guidelines of these policies have provided significant impetus for monitoring: however, we find this policy-driven M&E usually does not match the ideals of what is needed to inform adaptive management. There is a tendency to focus on understanding state and trends rather than tracking the effect of interventions; a focus on specific biotic and abiotic indicators at the expense of understanding system functions and processes, especially social components; and limited attention to how context affects systems, though this is sometimes considered via secondary data. The resulting data are sometimes publicly-accessible, but it is rarely clear if and how these influence decisions at any level, whether this be in the original policy itself or at the level of measures such as site management plans. Adjustments to policy-driven M&E could better enable learning for adaptive management, by reconsidering what supports a balanced understanding of socio-ecological systems and decision-making. Useful strategies include making more use of secondary data, and more transparency in data-sharing and decision-making. Several countries and policy areas already offer useful examples. Such changes are essential given the influence of policy, and the urgency of enabling adaptive management to safeguard socio-ecological systems.
Keywords: Adaptive Management; Environmental governance; Monitoring and Evaluation; Policy making; Socio-ecological systems; Sustainability

The Soul in the Computer: The Story of a Corporate Revolutionary - Waugh, Barbara (Inner Ocean Publishing, 2001)
When Barbara Waugh joined the Hewlett-Packard Corporation in the mid-80's, this 60's radical encountered a company with a benign but topdown leadership. As she progressed from recruiting manager to world change manager, she used a set of radical tools to transform its corporate culture and to help realize the true potential of The HP Way.

From principle to practice: Implementing the Principles for Digital Development - Waugaman, Adele (2016)
The Principles for Digital Development (download PDF here) find their roots in the efforts of individuals, development organizations, and donors alike who have called for a more concerted effort by donors and implementing partners to institutionalize lessons learned in the use of information and communication technologies (ICTs) in development projects.
Keywords: IMPORTANT

Thinking Tools Studio - Waters Center (Waters Center for Systems Thinking, 2021)
Tools to help you Think

VANUATU GOVERNANCE FOR GROWTH PROGRAM – REVIEW - Warner, Robert; Gouy, Jonathan; Samson, Anthony (2017)

Monitoring Humanitarian Innovation - Warner, Alexandra; Obrecht, Alice (HIF-ALNAP working paper, 2017)

Understanding ‘the users’ in Technology for Transparency and Accountability Initiatives - Wanjiku Kelbert, Alexandra; McGee, Rosemary; Carlitz, Ruth (2013)

Bridging & Bonding: Improving Links between Transparency & Accountability Actors - Wanjiku Kelbert, Alexandra (2014)

Dynamic knowledge management strategy development in international non-governmental organisations - Walsh, John N.; Lannon, John (Knowledge Management Research & Practice, 2020)
Knowledge management strategies are important for firms’ competitive positioning. This paper examines how knowledge management codification and personalisation strategies are developed in response to environmental and organisational dynamics in an international non-governmental organisation. A longitudinal case study of the organisation’s strategic reformulation of its KM strategy over a 2.5 period is drawn upon. The research examines how pressures in the firm’s operating environment led to the organisation identifying the need to leverage the value of local contextual knowledge. Subsequent reformulation required the organisation to change its strategic mix of codification and personalisation over time. Although efforts were focused on increasing personalisation, developments were supported through codification demonstrating a symbiotic, mutually supporting relationship between the strategies. The strategic reformulation involved processes of reflection, repackaging and support activities.
Keywords: Knowledge management strategy; case study; international development; knowledge dynamics; non-governmental organisation

The Aid Chain: Coercion and Commitment in Development NGOs - Wallace, Tina; Bornstein, Lisa; Chapman, Jennifer (Practical Action, 2007)
Significant proportions of aid already flow through the non-governmental sector, but questions are increasingly being asked about the role of NGOs and whether they can deliver on their ambitious claims. This study examines conditionality and mutual commitment between international aid donors and recipient NGOs, North and South. Fieldwork and case study material from Uganda and South Africa are used to support the authors’ contention that the fast changing aid sector has--in the context of a dynamic policy environment--encouraged the mainstreaming of a managerial approach that does not admit of any analysis of power relations or cultural diversity. This increasing--essentially technical-- definition of the roles of NGOs has worked to limit the extent of the very development that the organizations were initially established to promote.

Using what we know: How to ensure tech projects meet the brief - Walker, Tom (Making All Voices Count, 2017)
Making tech effective - building on what we already know

Connecting the Dots: Systems Practice & Political Economy Analysis - Walker, Tjip; Jacobstein, David (Local Systems Community, 2017)
This slide deck, from a presentation to the Local Systems Community by Tjip Walker and David Jacobstein, shows various ways in which assessing the political economy of a context and understanding that context through the lens of systems thinking can reinforce each other. Effective systems practice should be grounded in the incentives and power dynamics of a particular local system; thinking and working politically requires understanding the systems dynamics of an issue or sector.

Resilience Thinking: Sustaining Ecosystems and People in a Changing World - Walker, Brian; Salt, David; Reid, Walter V. (Island Press, 2006)
Increasingly, cracks are appearing in the capacity of communities, ecosystems, and landscapes to provide the goods and services that sustain our planet's well-being. The response from most quarters has been for "more of the same" that created the situation in the first place: more control, more intensification, and greater efficiency. "Resilience thinking" offers a different way of understanding the world and a new approach to managing resources. It embraces human and natural systems as complex entities continually adapting through cycles of change and seeks to understand the qualities of a system that must be maintained or enhanced in order to achieve sustainability. It explains why greater efficiency by itself cannot solve resource problems and offers a constructive alternative that opens up options rather than closing them down. In "Resilience Thinking", scientist Brian Walker and science writer David Salt present an accessible introduction to the emerging paradigm of resilience. The book arose out of appeals from colleagues in science and industry for a plainly written account of what resilience is all about and how a resilience approach differs from current practices. Rather than complicated theory, the book offers a conceptual overview along with five case studies of resilience thinking in the real world. It is an engaging and important work for anyone interested in managing risk in a complex world.

Designing Regenerative Cultures - Wahl, Daniel Christian (Triarchy Press Ltd, 2016)
This is a 'Whole Earth Catalog' for the 21st century: an impressive and wide-ranging analysis of what's wrong with our societies, organizations, ideologies, worldviews and cultures - and how to put them right. The book covers the finance system, agriculture, design, ecology, economy, sustainability, organizations and society at large. In this remarkable book, Daniel Wahl explores ways in which we can reframe and understand the crises that we currently face, and he explores how we can live our way into the future. Moving from patterns of thinking and believing to our practice of education, design and community living, he systematically shows how we can stop chasing the mirage of certainty and control in a complex and unpredictable world. The book asks how can we collaborate in the creation of diverse regenerative cultures adapted to the unique biocultural conditions of place? How can we create conditions conducive to life? *** "This book is a valuable contribution to the important discussion of the worldview and value system we need to redesign our businesses, economies, and technologies - in fact, our entire culture - so as to make them regenerative rather than destructive." --Fritjof Capra, author of The Web of Life, co-author of The Systems View of Life: A Unifying Vision *** "This is an excellent addition to the literature on ecological design and it will certainly form a keystone in the foundations of the new MA in Ecological Design Thinking at Schumacher College, Devon. It not only contains a wealth of ideas on what Dr Wahl has termed 'Designing Regenerative Cultures' but what is probably more important, it provides some stimulating new ways of looking at persistent problems in our contemporary culture and hence opens up new ways of thinking and acting in the future." -- Seaton Baxter OBE, Prof. in Ecological Design Thinking, Schumacher College, UK [Subject: Systems Thinking, Education, Social Anthropology, Environmentalism, Ecology, Regenerative Culture, Sociology]

MSP Guide - Wageningen University (MSP Guide, 2015)
The MSP Guide Designing and facilitating effective multi-stakeholder partnerships Get started Download now The MSP Guide(EN, FR, ES) Explore the rationale, principles and process of designing and facilitating effective MSPs Get it now The MSP Tool Guide Detailed descriptions of 60 tried-and-tested tools to facilitate multi-stakeholder partnerships Get it now Reflection Methods: Practical Guide Proven…

Peer reviews - Guidance for facilitators and participants - Wadley, Ian (Mediation Practice Series, 2021)
Essential points for practitioners and donors • Mediation offers a cost-effective and proven method for resolving armed conflict. Between 1985 and 2015, 75 per cent of armed conflicts in the world were resolved through agreement rather than by force. In most cases these processes will have involved third party facilitation or support. • Professional mediators understand the high stakes involved in their work to prevent, mitigate and resolve armed conflict. In addition, they and their financial supporters are increasingly required to demonstrate ‘value-for-money’ to ensure continued funding. • However, traditional monitoring and evaluation (M&E) methods are not well suited to this task, typically imposing artificially linear project models on a dynamic conflict situation, as well as compliance reporting that moves attention away from real value. • Traditional M&E methods tend to focus on documenting the past and generating vast amounts of data, rather than enabling timely adaptation of the project in the present. • Traditional M&E approaches rely heavily on external evaluation consultants. Even in the best of cases this may interfere with the mediation process and impose a heavy time burden on the project team, leading to low acceptance of traditional M&E approaches by mediation practitioners. • In contrast, an ideal M&E approach for mediation should deliver useful insights in even the most dynamic and sensitive mediation environments, impose a light reporting burden, and be readily accepted by mediation teams. It should protect discretion and trust, enable rapid adaptation, and also provide some assurance that donor funds are being well spent.

Valuing peace: delivering and demonstrating mediation results - Dilemmas & options for mediators - Wadley, Ian (Mediation Practice Series, 2017)
Essential points for practitioners and donors • Mediation offers a cost-effective and proven method for resolving armed conflict. Between 1985 and 2015, 75 per cent of armed conflicts in the world were resolved through agreement rather than by force. In most cases these processes will have involved third party facilitation or support. • Professional mediators understand the high stakes involved in their work to prevent, mitigate and resolve armed conflict. In addition, they and their financial supporters are increasingly required to demonstrate ‘value-for-money’ to ensure continued funding. • However, traditional monitoring and evaluation (M&E) methods are not well suited to this task, typically imposing artificially linear project models on a dynamic conflict situation, as well as compliance reporting that moves attention away from real value. • Traditional M&E methods tend to focus on documenting the past and generating vast amounts of data, rather than enabling timely adaptation of the project in the present. • Traditional M&E approaches rely heavily on external evaluation consultants. Even in the best of cases this may interfere with the mediation process and impose a heavy time burden on the project team, leading to low acceptance of traditional M&E approaches by mediation practitioners. • In contrast, an ideal M&E approach for mediation should deliver useful insights in even the most dynamic and sensitive mediation environments, impose a light reporting burden, and be readily accepted by mediation teams. It should protect discretion and trust, enable rapid adaptation, and also provide some assurance that donor funds are being well spent.

Process Tracing as a Practical Evaluation Method: Comparative Learning from Six Evaluations - Wadeson, Alix; Monzani, Bernardo; Aston, Tom (2020)

Developing an effective adaptive monitoring network to support integrated coastal management in a multiuser nature reserve - Vugteveen, Pim; van Katwijk, Marieke M.; Rouwette, Etiënne; Lenders, H. J. Rob; Hanssen, Lucien (Ecology and Society, 2015)
ABSTRACT. We elaborate the necessary conceptual and strategic elements for developing an effective adaptive monitoring network to support Integrated Coastal Management (ICM) in a multiuser nature reserve in the Dutch Wadden Sea Region. We discuss quality criteria and enabling actions essential to accomplish and sustain monitoring excellence to support ICM. The Wadden Sea Long-Term Ecosystem Research project (WaLTER) was initiated to develop an adaptive monitoring network and online data portal to better understand and support ICM in the Dutch Wadden Sea Region. Our comprehensive approach integrates ecological and socioeconomic data and links research-driven and policy-driven monitoring for system analysis using indicators of pressures, state, benefits, and responses. The approach and concepts we elaborated are transferable to other coastal regions to accomplish ICM in complex social-ecological systems in which scientists, multisectoral stakeholders, resource managers, and governmental representatives seek to balance long-term ecological, economic, and social objectives within natural limits.
Keywords: IMPORTANT

Participatory approaches: a facilitator's guide - VSO (VSO, 2004)
This book provides a set of guidelines for people who will be involved in participatory processes and projects with specific design focus on VSO (Voluntary Services Overseas) volunteers. It looks at appropriate levels of participation; pitfalls of participatory approaches (PA); best practice in facilitation; and tools for participation. The guide is organised into three parts (I) Principles, (II) Methods and (III) Toolkit. Part I gives a background to PA with a comparative analysis of PA in relation to top-down approaches, and within the range of PAs; looks at the role PA in VSO; discusses how to facilitate participatory processes with multiple stakeholders; presents a framework for PA on different levels of participation; and examines key facilitation skills needed to support participatory activities. Part II collates a range of participatory methods that have been used successfully in the field by VSO volunteers. Methods are categorised according to this suitability for use at different stages of a project process. Examples are also given of methods that can be used for specific purposes, such as participatory organisational appraisal and gender/diversity analysis. Part III gives tips on how to choose the most appropriate tool and how to organise participatory workshops and small group activities. It also systematically records a range of tools used by development workers all over the world with reference to what tool is appropriate in what situation. A profile of each tool includes guidelines on its purpose, potential applications and variations, as well as possible pitfalls. Illustrative case studies taken form real experiences of development workers in the field are also included.

Adaptive development: great progress and some niggles - Vowles, Pete (Medium, 2016)
After 3 years in DFID headquarters, championing adaptive approaches to the delivery of aid and development programmes, I am back in an…

Adaptive programming - Vowles, Pete (DFID Bloggers, 2013)
Staff blogs from the UK Department for International Development. Get real-life perspectives from those on the ground to fight poverty and join in the debate.

What Are the Next Steps in Innovation for Good Governance? - von Sturmer, Lucy (Making All Voices Count, 2015)
What are the next steps in innovation for good governance? Lucy von Sturmer reflects on lessons learned at Buntwani 2015.

Applying Rigorous Impact Evaluation in GIZ Governance Programmes: Results of a GIZ Initiative on Impacts in Governance - von Schiller, Armin (GIZ GmbH, 2020)
Pressure is mounting on international development cooperation agencies to prove the impact of their work. Private and public commissioners as well as the general public are increasingly asking for robust evidence of impact. In this context, rigorous impact evaluation (RIE) methods are increasingly receiving attention within the broader German development system and in GIZ. Compared to other implementing agencies such as DFID or USAid, the Deutsche Gesellschaft für Internationale Zusammenarbeit (GIZ) GmbH has so far relatively little experience in systematically applying rigorous methods of impact evaluation. This is particularly true in the governance sector. In order to gain more experience and to understand which methods and formats are best suited for GIZ governance programmes, the Governance and Conflict division and the Africa department launched the ‘Impact Initiative Africa’ in 2016, a cooperative effort with several programmes in Africa. The Initiative set out to apply the experiences from GIZ governance programmes to design and conduct RIEs, and to use the results to steer programme implementation. Initially, the Initiative included three countries: Benin (Programme for Decentralisation and Local Development), Malawi (Support to Public Financial and Economic Management) and Mozambique (Good Financial Governance in Mozambique). During its implementation, the Initiative also benefitted from the experience of two additional governance programmes which had already undertaken RIEs, namely Peru (Citizen-oriented State Reform Programme) and Pakistan (Support to Local Governance Programme II). This report summarizes the insights gained from these experiences and discusses opportunities and limitations regarding the use and usability of RIEs in GIZ governance programmes as well as proposals on how to organise RIEs to maximise learning potential and benefits for the specific programmes and the GIZ Governance sector at large.

Rigorous Impact Evaluation in GIZ Governance Programmes - von Schiller, Armin (How-To-Note, 2020)
Why should I integrate an impact assessment in my programme? How will the programme benefit from it? Are the benefits worth the effort and expenses? How do I design and implement it in detail? Who is addressable for support? What should I pay attention to in order to get the most out of it? This note is meant to answer these questions. It addresses leaders and project staff of governance programmes who are interested in using this tool within their specific governance programme or project. This note provides a guideline and good practice recommendations on how to design and conduct an impact assessment and on how to fully utilise the benefits of the results for the programme and for communication with commissioners, partners and other donors. Additionally, this note will point to indirect benefits that can arise and that should not be ignored. Results of impact assessments are highly relevant for the GIZ as an institution. However, in this note we stress the benefits for the programme or project itself. In particular this note addresses the following aspects: • What are rigorous impact assessments and why should GIZ Governance programmes conduct them more often within their programmes? • Which phases does an impact assessment include? How do I set one up and which aspects deserve special attention in each phase to maximise the benefits for my programme? • What are the benefits I can expect from implementing and impact assessment? • Whom to ask at headquarters in case I need support? This note complements the policy brief “Strategic use of Rigorous Impact Evaluation” and the corporate strategic review on “Rigorous Impact Evaluation” written by the GIZ evaluation unit which focuses on the strategic use of rigorous impact evaluations (RIE) at GIZ. Based on the review findings, the policy brief presents recommendations for strategic planning and implementation of purpose-sensitive RIE using a number of key levers. By adopting central coordination and needs-based support mechanisms, the evaluation unit intends to promote the strategic use of RIE for evidence-based learning and decision-making within the organisation.

Appendix 3. ToC Examples - Vogel, Isabel; Stephenson, Zoe (DFID, 2012)

Innovation in development: Sharing learning, improving impact - Vogel, Isabel; Minkley, Gabrielle; Chowdhury, Sajid (Ideas to Impact / IMC Worldwide, 2017)

Laying the Foundations for Impact: Lessons from the GCRF Evaluation - Vogel, Isabel; Barnett, Chris (The European Journal of Development Research, 2023)
Research for development (R4D) aims to make a tangible difference to development challenges, but these effects typically take years to emerge. Evaluation (especially impact evaluation) often takes place before there is evidence of development impact. In this paper, we focus on opportunities for assessing the potential for impact at earlier stages in the research and innovation process. We argue that such a focus can help research programme managers and evaluators learn about the pre-conditions for impact and adjust accordingly. Using the Global Challenges Research Fund (GCRF) as a large-scale case of R4D evaluation, we identify and explore some of the building blocks that can increase impact potential. Guided by GCRF’s theory of change, we explore emerging evidence that highlights the importance of ways of working that supports positioning for impact. We conclude by drawing out a unifying construct around standards of development excellence; to sit alongside notions of scientific excellence for research intended to have an impact. Standards can help programme managers, researchers and evaluators learn and adapt to increase the likelihood of impact.

Innovating in Development Learning Event - Challenge brief - Vogel, Isabel (Ideas to Impact / iMC, 2017)

Review of the use of ‘Theory of Change’ in International development - Vogel, Isabel (DFID, 2012)
A new report on Theory of Change (ToC) and its use in International development has just been produced.

Impact Evaluation: Taking stock and looking ahead - Conference report - Visser, Irene; Guijt, Irene; Kusters, Cecile (Centre for Development Innovation, 2013)
This report summarises the presentations and discussions of the Conference ‘Impact evaluation. Taking stock and looking ahead’, which took place in Wageningen on March 25 and 26, 2013. The Conference was organised and funded by the Centre for

What exactly do we mean by systems? - Vexler, Dan (Stanford Social Innovation Review, 2017)
Everyone is talking about systems. Or at least, that's how it seems in my wonkish corner of the philanthropic world. You can't attend a conference or even have a meeting without hearing about systems, whether it's people trying to disrupt them, map them, learn from them, or catalyze them.

Innovative M&E from the Sandbox and beyond - Vester, Søren; Tran, Samuel (Medium, 2023)
In this blog we are sharing a digest of some of the many useful and innovative monitoring, evaluation and learning resources and efforts that have come through the M&E Sandbox in 2022. A lot of these resources have been shared by our community in response to the overwhelmingly positive feedback from the launch of the Sandbox (please keep them coming!). We hope you find it useful. We have grouped these efforts and resources under six broad questions: - How do we measure systems transformation? - How do we know if we are on track? - How do we rethink complexity and independence in evaluation? - Why, how and for whom do we measure? - How do we generate insights and learn? - How do we make decisions and adapt?

Re-Perceiving Results: Aid Programs and Change in Fragile Societies - van Veen, Erwin; Rijper, Alies (Policy Brief, 2017)
Aid programmes need to be able to adapt their objectives and operations to changes in their political environment, since development processes are subject to political contestation. Change takes time and is often a matter of seizing the opportunity.
Keywords: IMPORTANT

Critical System Heuristics : a very, very short introduction - van 't Hof, Sjon (CSL4D/SystemicAgency, 2021)

Learning to adapt & adapting to learn - Using elements of outcome mapping in the ‘Resilient Adolescents in the Syria Crisis’ programme - van Ongevalle, Jan; Kvintradze, Ana; Rennesson, Gaël; Miller, David (Save the Children, 2021)
This learning paper highlights how elements of outcome mapping were used by Save the Children Sweden in a project (2018-2020) that supports adolescents, affected by the Syria crisis, to become more resilient. The paper first outlines how the spheres of influence framework has been applied to develop an actor focused theory of change. It then describes how progress markers, as an alternative to SMART indicators, were formulated to monitor the programme’s results. The paper also outlines how long lists of progress markers were categorised in a more realistic and practical results framework. The paper then continues to elaborate how outcome journals, qualitative data analysis techniques and regular review meetings and reflection workshops were utilised for data collection, for collective learning among programme stakeholders and for informing planning and programme adjustment. Various practical guidelines and tips on how to implement elements of outcome mapping are provided. The final part of the paper explores to what extent outcome mapping was able to foster several key enablers of adaptive programme management and highlights some of the challenges that programme stakeholders faced. Practical recommendations towards the use of outcome mapping in future programmes are also proposed.

Understanding and governing learning in sustainability transitions: A review - van Mierlo, Barbara; Beers, Pieter J. (Environmental Innovation and Societal Transitions, 2018)
Many transitions scholars underscore the importance of learning in sustainability transitions, but the associated learning processes have hardly been conceptualised. The diverse, well-established research fields related to learning are broadly ignored or loosely applied. In this paper, we systematically explore four interesting learning traditions in terms of their value for gaining an in-depth understanding of learning in sustainability transitions and their relevance for fostering learning, by connecting them to key features of transitions. The selected learning traditions from different disciplinary backgrounds provide valuable insights. None of them sufficiently addresses the complexity of transitions. They include, however, a diversity of relevant learning contexts. We conclude that they have value for investigating new areas such as learning in socio-technological regimes and in later phases of a transition, while enlightening forms of learning that have not yet been fully recognised in transition studies, such as superficial learning, unlearning, and learning to resist change.
Keywords: Collaborative learning; Interactive learning; Organizational learning; Social learning; Sustainability transition

Balancing Inclusiveness, Rigour and Feasibility: Insights from Participatory Impact Evaluations in Ghana and Vietnam - van Hemelrijck, Adinda; Guijt, Irene (Practice Paper, 2016)
This paper by Adinda Van Hemelrijck and Irene Guijt explores how impact evaluation can live up to standards broader than statistical rigour in ways that address challenges of complexity and enable stakeholders to engage meaningfully. A Participatory Impact Assessment and Learning Approach (PIALA) was piloted to assess and debate the impacts on rural poverty of two government programmes in Vietnam and Ghana funded by the International Fund for Agricultural Development (IFAD). We discuss the trade-offs between rigour, inclusiveness and feasibility encountered in these two pilots. Trade-offs occur in every impact evaluation aiming for more than reductionist rigour, but the pilots suggest that they can be reduced by building sufficient research and learning capacity.

Theory of Change Thinking in Practice - van Es, Marjan; Guijt, Irene; Vogel, Isabel (2015)
Want to know better how your interventions can contribute to change? A Theory of Change (ToC) approach helps in deepening your understanding - and that of your partners - of how you collectively think change happens and what the effect will be of your intervention. Not only does it show what political, social, economic, and/or cultural factors are in play, it also clarifies your assumptions. Once a ToC has been developed, it can be used to continually reflect on it in ways that allow for adaptation and checking of assumptions of your intervention. This user friendly guideline helps you to use a Theory of Change approach. Theories of change are the ideas and beliefs people have – consciously or not – about why and how the world and people change. How people perceive and understand change and the world around them is infused by their underlying beliefs about life, human nature and society. They are deep drivers of people’s behaviour and of the choices they make. Social change processes are complex and characterised by non-linear feedback loops: our own actions interact with those of others and a myriad of influencing factors. This triggers reactions that cannot be foreseen and makes outcomes of change interventions unpredictable. Given these uncertainties, how can we plan strategically and sensibly? How can social change initiatives move forward in emerging change processes in a flexible way, while remaining focused on the goal? In this context of complexity, Hivos has found a theory of change (ToC) approach useful in guiding its strategic thinking and action, as well as its collaborative efforts with others. As it fosters critical questioning of all aspects of change interventions and supports adaptive planning and management in response to diverse and quickly changing contexts. It contributes to the quality and transparency of strategic thinking, and therefore to personal, organisational and social learning. This guide builds on the experiences of Hivos working with a ToC approach. It is a practical guide for Hivos staff in applying a Theory of Change approach, but is also very useful for others working on social change such as social entrepreneurs and innovators. This guide builds on the work of a ToC Learning Group initiated by Hivos and comprising of staff of the Centre for Development Innovation (CDI) of Wageningen University and Research Centre and of experts Iñigo Retolaza Eguren, Isabel Vogel and Irene Guijt. For current thinking and work on the use of Theory of Change thinking in complex change processes, see http://www.theoryofchange.nl

A conceptual map of political participation - van Deth, Jan W. (Acta Politica, 2014)
How would you recognize a mode of participation if you see one? Owing to the rapid expansion of political activities in the last decades this question has become increasingly difficult to answer. Neither the development of all-embracing nominal definitions, nor deductive analyses of existing modes of participation seem to be helpful. In addition, the spread of expressive modes of participation makes it hard to avoid purely subjective definitions. The aim of this discussion paper is to develop an operational definition of political participation, which allows us to cover distinct conceptualizations systematically, efficiently and consistently. This goal can only be arrived at if the conventional approach of presenting nominal definitions to solve conceptual problems is left behind. Instead, available definitions are included in a set of decision rules to distinguish three main variants of political participation. A fourth variant is distinguished for non-political activities used for political purposes. Together, the four variants of political participation cover the whole range of political participation systematically without excluding any mode of political participation unknown yet. At the same time, the endless expansion of the modes of political participation in modern democracies does not result in an endless conceptual expansion. Implications for research and various examples are discussed.

Making Sense of Complexity: Using SenseMaker as a Research Tool - Van der Merwe, Susara E.; Biggs, Reinette; Preiser, Rika; Cunningham, Charmaine; Snowden, David J.; O’Brien, Karen; Jenal, Marcus; Vosloo, Marietjie; Blignaut, Sonja; Goh, Zhen (Systems, 2019)
There is growing interest in studying processes of human sensemaking, as this strongly influences human and organizational behavior as well as complex system dynamics due to the diverse lenses people use to interpret and act in the world. The Cognitive Edge SenseMaker® tool is one method for capturing and making sense of people’s attitudes, perceptions, and experiences. It is used for monitoring and evaluation; mapping ideas, mind-sets, and attitudes; and detecting trends and weak signals. However, academic literature describing the tool-set and method is lacking. This introduction aims to guide researchers in choosing when to use SenseMaker and to facilitate understanding of its execution and limitations. SenseMaker can provide nuanced insight into system-level patterns of human sensemaking that can provide insight to nudge systems towards more desirable futures, and enable researchers to measure beyond what they know.
Keywords: Mixed methods; SenseMaker tool; complex systems; sensemaking; social complexity

Cherry-Picked Intelligence. The Weapons of Mass Destruction Dispositive as a Legitimation for National Security in the Post 9/11 Age - van der Heide, Liesbeth (Historical Social Research / Historische Sozialforschung, 2013)
»Geheimdienste mit Scheuklappen: Das Dispositiv der Massenvernichtungswaffen als Legitimation für Nationale Sicherheit nach 9/11«. The claim that Iraq possessed Weapons of Mass Destruction (WMD) led to the invasion of Iraq in 2003 by the US army. For the George W. Bush administration, the likely presence of WMD in Iraq was the major justification for going to war. However, Bush' opponents suspected he used the WMD-dispositive as a legitimation for an invasion that was already set in motion for different reasons. The Iraq invasion and the underlying ideas about the presence of WMD thus provide a tangible case for the analysis of theories of conspiracy and security. The development of the WMD-dispositive will be contextualized using the toolkit of securitization theory. The article explores the notions of security and conspiracy that were used to build the dispositive and shows how it ultimately failed and turned into a counter-narrative in which the Bush administration itself became the Great Conspirator.

The guiding logics and principles for designing emergent transdisciplinary research processes: learning experiences and reflections from a transdisciplinary urban case study in Enkanini informal settlement, South Africa - van Breda, John; Swilling, Mark (Sustainability Science, 2019)
Transdisciplinarity is not a new science per se, but a new methodology for doing science with society. A particular challenge in doing science with society is the engagement with non-academic actors to enable joint problem formulation, analysis and transformation. How this is achieved differs between contexts. The premise of this paper is that transdisciplinary research (TDR) methodologies designed for developed world contexts cannot merely be replicated and transferred to developing world contexts. Thus a new approach is needed for conducting TDR in contexts characterised by high levels of complexity, conflict and social fluidity. To that end, this paper introduces a new approach to TDR titled emergent transdisciplinary design research (ETDR). A core element of this approach is that the research process is designed as it unfolds, that is, it transforms as it emerges from and within the fluid context. The ETDR outlined in this paper emerged through a case study in the informal settlement (slum) of Enkanini in Stellenbosch, South Africa. This case study demonstrates the context from and within which the ETDR approach and identifies a set of guiding logics that can be used to guide ETDR approaches in other contexts. The study demonstrates that the new logics and guiding principles were not simply derived from the TDR literature, but rather emerged from constant interacting dynamics between theory and practice. Learning how to co-design the research process through co-producing transformative knowledge and then implementing strategic interventions to bring about incremental social change is key to theory development in ways that are informed by local contextual dynamics. There are, however, risks when undertaking such TDR processes such as under-valuing disciplinary knowledge, transferring risks onto a society, and suppressing ‘truth-to-power’.

Top Tips: How to design and manage adaptive programmes - Valters, Craig; Wild, Leni (Overseas Development Institute, 2019)
Within DFID, there is now a commitment to more flexible and adaptive programming. This recognises that: • DFID works in contexts that continuously evolve and change, sometimes in unpredictable ways. To respond to this, the agency needs to remain flexible – to expect change and have a good understanding of context, with resources that can be adjusted and scope to change direction if needed. All DFID programmes should be able to do this. • Some DFID programmes aim to support change in complex systems, behaviours and incentives. Efforts to address women’s empowerment, improve sanitation or build more sustainable health systems, for instance, all require engagement with the way in which complex systems operate and the people and behaviours within them. Trying to deliver reforms in these circumstances is challenging because the pathway to reform itself will be unclear: as a reform is rolled out, the system itself will react and respond. These types of programmes therefore need to build in from the start deliberate processes of learning and testing, to allow for adaptations as more information is gathered for what works over time. These top tips are concerned with programmes that aim to be flexible and adaptive – which work in dynamic contexts and are trying to address complex problems. While there is growing commitment to these approaches, feedback suggests staff still have questions about how to do this well. This document highlights some of the commonly reported issues related to adaptive programming and a set of tips, strategies and examples to help in addressing them. It is aimed at programme managers and advisors who may be Senior Responsible Owners (SROs) or those managing and supporting adaptive programming in a range of ways. We have collated these lessons from discussions with country offices and SROs, feedback from surveys and the wider evidence. We have looked specifically at the adaptive programmes we can find in the DFID portfolio, but this is not an exhaustive list. It should be noted that there is as yet no wide variety of case law to review, but there is a growing set of examples within DFID that can provide continuous learning for the organisation.

Putting learning at the centre: Adaptive development programming in practice - Valters, Craig; Cummings, Clare; Nixon, Hamish (ODI, 2016)
Adaptive programming suggests, at a minimum, that development actors react and respond to changes in the political and socio-economic operating environment. It emphasises learning and the development practitioner is encouraged to adjust their actions to find workable solutions to problems that they may face. Being prepared to react to change may seem like common sense – and indeed it is. However much development thinking and practice remains stuck in a linear planning model which discourages learning and adaptation, in part because projects are seen as ‘closed, controllable and unchanging systems’ (Mosse, 1998: 5). This paper critically engages with this problem and makes clear why and how learning needs to be at the centre of adaptive development programming. It begins by clarifying why and what kind of learning matters for adaptive programming. The paper then turns its focus to how strategies and approaches applied throughout a programme’s conception, design, management and M&E can enable it to continually learn and adapt.
Keywords: IMPORTANT

Building justice and peace from below? Supporting community dispute resolution in Asia - Valters, Craig (Working Politically in Practice Series: Case Study, 2016)

Learning and adaptation: 6 pitfalls to avoid - Valters, Craig (Devex, 2016)

Learning from adaptive programmes - 10 lessons and 10 case studies - Valters, Craig (Overseas Development Institute, 2018)
Internal DFID document from the DevAdapt Programme. Based on another previous document (also internal).

Theories of change in international development: Communication, learning, or accountability? - Valters, Craig (JSRP Paper 17, 2014)

Theories of Change: time for a radical approach to learning in development - Valters, Craig (ODI, 2015)
The Theory of Change approach demands a radical shift towards more and better learning in development thinking and practice, creating a productive and much-needed space for critical reflection.

Learning from Results-Based Management evaluations and reviews - Vähämäki, Janet; Verger, Chantal (Development Co-operation Working Papers, 2019)
What have we learned from implementing results-based management in development co-operation organisations? What progress and benefits can be seen? What are the main challenges and unintended consequences? Are there good practices to address these challenges? To respond to these questions this paper reviews and analyses the findings from various evaluations and reviews of results-based management systems conducted by members of the Development Assistance Committee (DAC), the OECD/DAC Results Community Secretariat and other bodies in the past four years (2015-2018). It also draws on emerging lessons from new methods for managing development co-operation results. This analytical work aims to: identify recent trends in results-based management, explore challenges faced by providers when developing their results approaches and systems, select good practices in responding to these challenges that can be useful for the OECD/DACResults Community, considering new approaches, new technologies and evolving contexts. This body of evidence will inform the development of a core set of generic guiding principles for results-based management in development co-operation.

Matrixing Aid: The Rise and Fall of 'Results Initiatives' in Swedish Development Aid - Vähämäki, Janet (Stockholm Business School, Stockholm University, 2017)

Evaluation of International Development Interventions: An Overview of Approaches and Methods - Vaessen, Jos; Lemire, Sebastian; Befani, Barbara (World Bank, 2020)
This guide provides an introductory overview of a range of methods that have been selected for their actual and potential use in the field of international development evaluation. For each method, a detailed guidance note presents the method’s main features and procedural steps, key advantages and disadvantages, as well as its applicability. Each guidance note includes references for relevant background readings (basic and advanced) as well as references to other additional resources of interest. Both the choice of approaches and methods and the associated guidance are by no means definitive. IEG plans to periodically update the guide as evaluation practices evolve.

Using Systemic M&E Tools in Feed The Future Uganda: Sensemaker ® - Webinar - USAID Learning Lab (2015)
This webinar is part of a series organized by SEEP's Market Facilitation Initiative (MaFI), USAID's LEO project and the BEAM Exchange as a part of the "Learning with the Toolmakers" webinar series. As part of a new cycle of webinars focusing on inclusive market facilitation tools. In this series, Eric Derks and Leanne Rasmussen will share learning from this five-year, multi-million dollar USAID program, the objective of which is to increase Ugandan farmers’ use of good quality agro-inputs by fostering more inclusive systemic changes in the agro-inputs industry. Using Systemic M&E Tools in Feed The Future Uganda: Sensemaker ® This third webinar in the series, titled “,” will look at how the project team used this tool and method to monitor systemic change. Leanne and Eric will share their experiences with using the tool and give recommendations to other practitioners who are interested in applying it. What is Sensemaker ®? SenseMaker combines an innovative research methodology with patented software to collect and analyze large quantities of narratives in order to understand complex change. It brings together insights from complexity sciences, anthropology, and cognitive science. It uses participants’ narratives to uncover foundational attitudes and norms that inform and influence behavior. This webinar is organized by SEEP's Market Facilitation Initiative (MaFI), USAID's LEO project and the BEAM Exchange as a part of the "Learning with the Toolmakers" webinar series.

CLA Maturity Tool - Card Deck (Implementing Partners version 1) - USAID LEARN (USAID, 2022)
USAID’s Bureau for Policy, Planning and Learning (PPL) and its support mechanism, LEARN, have developed a Collaborating, Learning, and Adapting (CLA) Framework and Maturity tool to help USAID missions think more deliberately about how to plan for and implement CLA approaches that fit the mission’s context and assist them in achieving their development objectives. While the tool is intended primarily for USAID audiences to be used in participatory self-assessment workshops, the CLA Framework and maturity spectrum are relevant to a wider audience. USAID’s CLA Framework identifies key components and subcomponents of daily work that may be opportunities for intentional, systematic, and resourced CLA. The framework recognizes the diversity of what CLA can look like in various organizations and programs while also giving CLA structure, clarity, and coherence across two key dimensions: • CLA in the Program Cycle: how CLA is incorporated throughout Program Cycle processes, including strategy, project, and activity design and implementation; and • Enabling Conditions: how an organization’s culture, business processes, and resource allocation support CLA integration. Recognizing that CLA is not binary—it’s not an issue of “doing it or not doing it”—PPL and LEARN have developed a spectrum of practice for each of the 16 subcomponents in the framework. The spectrum offers examples of what integration might look like at different stages: Not Yet Present, Emergent, Expanding, Advanced and Institutionalized. The maturity stage descriptions are only illustrative and are intended to spark reflection on current practice and opportunities for improvements. In this resource, each CLA subcomponent page describes the key concepts for that topic and includes a description of the maturity stages. Although the descriptions were originally developed for USAID, the majority of the concepts easily transfer or have equivalents in the partner community. For example, although organizations outside of USAID may not hold “Portfolio Reviews” (part of the Pause & Reflect subcomponent), the majority hold some type of meeting to review programmatic progress. This is the seventh version of the CLA Framework and maturity spectrum. PPL and LEARN will continue reviewed and periodically update them based on user feedback, so if you have comments about the content, please email learning@usaid.gov. We would also love to hear how you’ve used this content with your team or organization.

Collaborating, Learning and Adapting (CLA) Maturity Spectrum (v7) - USAID LEARN (USAID, 2022)
USAID’s Bureau for Policy, Planning and Learning (PPL) and its support mechanism, LEARN, have developed a Collaborating, Learning, and Adapting (CLA) Framework and Maturity tool to help USAID missions think more deliberately about how to plan for and implement CLA approaches that fit the mission’s context and assist them in achieving their development objectives. While the tool is intended primarily for USAID audiences to be used in participatory self-assessment workshops, the CLA Framework and maturity spectrum are relevant to a wider audience. USAID’s CLA Framework identifies key components and subcomponents of daily work that may be opportunities for intentional, systematic, and resourced CLA. The framework recognizes the diversity of what CLA can look like in various organizations and programs while also giving CLA structure, clarity, and coherence across two key dimensions: • CLA in the Program Cycle: how CLA is incorporated throughout Program Cycle processes, including strategy, project, and activity design and implementation; and • Enabling Conditions: how an organization’s culture, business processes, and resource allocation support CLA integration. Recognizing that CLA is not binary—it’s not an issue of “doing it or not doing it”—PPL and LEARN have developed a spectrum of practice for each of the 16 subcomponents in the framework. The spectrum offers examples of what integration might look like at different stages: Not Yet Present, Emergent, Expanding, Advanced and Institutionalized. The maturity stage descriptions are only illustrative and are intended to spark reflection on current practice and opportunities for improvements. In this resource, each CLA subcomponent page describes the key concepts for that topic and includes a description of the maturity stages. Although the descriptions were originally developed for USAID, the majority of the concepts easily transfer or have equivalents in the partner community. For example, although organizations outside of USAID may not hold “Portfolio Reviews” (part of the Pause & Reflect subcomponent), the majority hold some type of meeting to review programmatic progress. This is the seventh version of the CLA Framework and maturity spectrum. PPL and LEARN will continue reviewed and periodically update them based on user feedback, so if you have comments about the content, please email learning@usaid.gov. We would also love to hear how you’ve used this content with your team or organization.

Collaborating, Learning, and Adapting Framework & Key Concepts (Implementing Partner Version 1) - USAID LEARN (USAID, 2022)
Although collaborating, learning, and adapting (CLA) are not new to USAID and its implementing partners, they often do not happen regularly or systematically and are not intentionally resourced. The CLA Framework above identifies components and subcomponents to help you think more deliberately about what approach to CLA might be best tailored to your organizational or project context. The framework recognizes the diversity of what CLA can look like in various organizations and projects while also giving CLA structure, clarity, and coherence across two key dimensions: - CLA in the Program Cycle: how CLA is incorporated into planning and design processes throughout the Program Cycle in order to improve their effectiveness; and - Enabling Conditions: how an organization’s culture, daily operating processes, and resource allocation support CLA integration. Organizations need both integrated CLA practices appropriate for their context and conducive enabling conditions to become stronger learning organizations capable of managing adaptively. The framework stresses the holistic and integrated nature of the various components of CLA to reinforce the principle that CLA is not a separate workstream—it should be integrated into existing processes to strengthen the discipline of development and improve aid effectiveness.

Course: Introduction to Collaborating, Learning and Adapting (CLA) in the Program Cycle - USAID LEARN (USAID Learning Lab, 2018)

Acquisition and Assistance Strategy - USAID (USAID, 2018)
The U.S. Agency for International Development (USAID) of tomorrow is one in which the Agency enables partner countries to plan, resource, and manage their own development through strengthened capacity and commitment — this is the essence of the Journey to Self-Reliance. In Fiscal Year (FY) 2018, the Agency obligated over 80 percent of its programmatic funding — a total of $17 billion — through acquisition and assistance (A&A) mechanisms. Partnering and procurement are central to how we do our work. Therefore, we must think holistically about with whom we work and how we work, if we are to achieve our ambitious goals for the USAID of tomorrow. The purpose of this document is to provide a transparent strategy that guides changes to Agency policy and practice for both our staff and implementing partners in how we approach our core A&A work. This strategy, which builds on previous and current reform efforts, outlines the shifts we will make to embrace a self-reliance model for A&A and move concretely toward the goal of ending the need for foreign assistance. GUIDING PRINCIPLES OF PARTNERING AT USAID USAID’s partnering approaches have evolved significantly over time as we have tested new and innovative ways of working. The principles outlined below support the Journey to Self-Reliance by empowering our staff and partners to produce results-driven solutions responsive to partner country needs and priorities — building capacity and commitment based on creative and entrepreneurial approaches that have already begun to develop. The Agency must move beyond our traditional approaches to embrace greater collaboration, co-design, and co-financing to improve upon our models of partnering.

ADS 201 Additional Help- Whole-of-Project Evaluation - USAID (USAID, 2017)

Biodiversity How-To Guide 1: Developing Situation Models in USAID Biodiversity Programming - USAID (USAID, 2016)
The Biodiversity How-To Guide 1: Developing Situation Models in USAID Biodiversity Programming is the first in a series of three guides that provide in-depth guidance on key tools and practices to support design teams as they design and manage biodiversity programs within the Program Cycle and in accordance with the USAID Biodiversity Policy. It focuses on how to develop situation models to map out the biodiversity conservation problem context to be addressed.

Biodiversity How-To Guide 2: Using Results Chains to Depict Theories of Change in USAID Biodiversity Programming - USAID (USAID, 2016)
The Biodiversity How-To Guide 2: Using Results Chains to Depict Theories of Change in USAID Biodiversity Programming is the second in a series of three guides that provide in-depth guidance on key tools and practices to support design teams as they design and manage biodiversity programs within the Program Cycle and in accordance with the USAID Biodiversity Policy. It builds off the situation model guide to help design teams clearly state the expected results and assumptions behind the proposed strategic approaches that make up the program’s theory of change. The situation model provides the foundation to identify necessary results to reduce threats; brainstorm and prioritize strategic approaches; and then develop results chains to clarify assumptions behind selected approaches.

Biodiversity How-To Guide 3: Defining Outcomes & Indicators for Monitoring, Evaluation, and Learning in USAID Biodiversity Programming - USAID (USAID, 2016)
The Biodiversity How-To Guide 3: Defining Outcomes and Indicators for Monitoring, Evaluation, and Learning in USAID Biodiversity Programming is the third in a series of three guides that provide in-depth guidance on key tools and practices to support design teams as they design and manage biodiversity programs within the Program Cycle and in accordance with the USAID Biodiversity Policy. It uses the results chains developed in the second guide and provides help identifying key results for developing outcome statements and indicators, as required by USAID’s Biodiversity Code, updated in the Biodiversity Policy.

Broad Agency Announcement for Locally Led Development Innovation - USAID (USAID, 2019)
This BAA allows USAID Operating Units (OUs) to co-create, co-design, co-invest, and collaborate in the research, development, piloting, testing, and scaling of innovative, practical, and cost-effective interventions to catalyze locally led development. The BAA aligns with a number of Agency priorities and policies, including the Journey to Self-Reliance, resilience, procurement innovation, and expanding and diversifying the partner base - as well as the New Partnerships Initiative and Acquisition and Assistance Strategy. E3/LS has put up the BAA’s first Addendum, "New Directions in Advancing Locally Led Development," which seeks Expressions of Interest on the following: - Engaging the Local Private Sector - Conflict, Post-Conflict, and Non-Permissive Environments - Effective Partnerships - The Changing Role of the Donor - Local Giving, Philanthropy, and Other Private Resources

CLA for More Effective Development Programs (video) - USAID (USAID LearningLab, 2014)
This presentation features USAID/PPL's Stacey Young discussing the history of learning at USAID; how collaborating, learning, and adapting (CLA) can enhance development outcomes, and the role of implementing partners in supporting this vision.

CLA Maturity Tool: Example Spectrum Cards - USAID (2016)

Collaborate Learn Adapt - Case Competition - USAID (USAID Learning Lab, 2015)

Collaborating, Learning and Adapting (CLA) - Maturity Matrix Overview - USAID (ADS 201 Additional Help, 2016)
Collaborating, Learning and Adapting (CLA) involves strategic collaboration, continuous learning, and adaptive management. CLA approaches to development include collaborating intentionally with stakeholders to share knowledge and reduce duplication of effort, learning systematically by drawing on evidence from a variety of sources and taking time to reflect on implementation, and applying learning by adapting intentionally. The purpose of the maturity matrix tool is to help USAID Missions think more deliberately about how to plan for and implement CLA approaches that fit the Mission’s context and needs. Through a set of easy-to-use cards, the CLA maturity matrix offers examples of what CLA looks like at different stages of maturity. USAID staff can use the decks of cards to both assess current practice and plan for the future. The maturity matrix consists of two decks of cards corresponding to the dimensions of CLA: ● CLA in the Program Cycle: By using the matrix cards to guide a conversation, teams can explore how well CLA is incorporated into the planning and implementation processes of the Program Cycle. ● Enabling Conditions: The cards encourage discussion of the conditions that can significantly influence how CLA and the Program Cycle are implemented at a Mission, including operating processes, organizational culture, and resourcing. Within these decks, the matrix covers 16 subcomponents of CLA. Each has one key concept card and five matrix cards that describe the component in practice, along a spectrum ranging from Not Yet Present to Institutionalized.

Collaborating, Learning, and Adapting Framework & Key Concepts - USAID (USAID, 2016)
Although collaborating, learning, and adapting (CLA) are not new to USAID, they often do not happen regularly or systematically and are not intentionally resourced. The CLA framework above identifies components and subcomponents to help you think more deliberately about what approach to CLA might be best tailored to your organizational or project context. The framework recognizes the diversity of what CLA can look like in various organizations and projects while also giving CLA structure, clarity, and coherence across two key dimensions: CLA in the Program Cycle: how CLA is incorporated throughout Program Cycle processes, including strategy, project, and activity design and implementation; and Enabling Conditions: how an organization’s culture, business processes, and resource allocation support CLA integration. Organizations need both integrated CLA practices appropriate for their context and conducive enabling conditions to become stronger learning organizations capable of managing adaptively. The framework stresses the holistic and integrated nature of the various components of CLA to reinforce the principle that CLA is not a separate workstream—it should be integrated into existing processes to strengthen the discipline of development and improve aid effectiveness.

Combating Wildlife Trafficking Case Study Compilation: Capacity Building for Enforcement and Prosecution - USAID (2018)

Developmental Evaluation in Practice: Tips, Tools, and Templates - USAID (USAID, 2017)

Discussion Note: Complexity-Aware Monitoring - USAID (Program Cycle, 2018)
This Discussion Note complements ADS 201 and outlines general principles and promising approaches for monitoring complex aspects of USAID development assistance. Complexity-aware monitoring is a type of complementary monitoring that is useful when results are difficult to predict due to dynamic contexts or unclear cause and-effect relationships. Key principles of the Program Cycle include applying analytic rigor, managing adaptively, and utilizing a range of approaches to achieve results. ADS 201.3.5.5 identifies three types of program monitoring – performance, context, and complementary. All USAID programming incorporates performance monitoring and should include context monitoring. Performance monitoring “is the ongoing and systematic collection of performance indicator data and other quantitative or qualitative information to reveal whether implementation is on track and whether expected results are being achieved.” Context monitoring is “[t]he systematic collection of information about conditions and external factors relevant to the implementation and performance of an operating units (OU’s) strategy, projects, and activities.” As stated in ADS 201.3.5.5, complementary monitoring may be used by missions and Washington OUs to complement performance and context monitoring in situations where results are difficult to predict due to dynamic contexts or unclear cause-andeffect relationships. This Discussion Note provides an explanation of when to use complementary monitoring approaches that are complexity-aware and summarizes the three principles of complexity-aware monitoring.

Guide for Adopting Remote Monitoring Approaches During COVID-19 - USAID (USAID, 2020)
USAID maintains staff and operations in more than 80 countries around the world, all of which the COVID-19 pandemic will disrupt. USAID remains committed to protecting the health and safety of our staff, while continuing appropriate oversight of our programs and ensuring the accountable and effective use of U.S. taxpayer funds. In the current operating environment, USAID and implementing partners face new challenges in implementing activities, monitoring progress, collecting data, and tracking indicators. As we adapt our approaches, we will work with implementing partners to find innovative, responsible, and safe ways to monitor and evaluate programming. Digital tools can support novel approaches to remote monitoring. Responsible use of digital tools also supports operating unit alignment with the Digital Data Collection mandate in the Agency’s new Digital Strategy. This guide provides information for Agency staff and implementing partners on remote monitoring techniques and when they can be employed. We encourage use of this guide to identify and pursue appropriate remote-monitoring approaches for your needs. CORs/AORs and Activity Managers should work with implementing partners to document updated approaches in each agreement’s plans for monitoring, evaluation, and learning (MEL), and should upload these amended Activity MEL Plans into the Agency Secure Image and Storage Tracking System (ASIST) as soon as possible. This can be done in collaboration with Mission and Washington Bureau M&E Specialists, as appropriate

How-To Note: Conduct a Data Quality Assessment - USAID (2017)

How-To Note: Developing a Project Logic Model (and its Associated Theory of Change) - USAID (Program Cycle, 2017)
This How-To Note describes considerations for developing a project logic model, as well as steps for thinking through a more complete theory of change (TOC). A logic model is a graphic or visual depiction that summarizes key elements of a TOC, and it is often used as a facilitation tool during the design process. There are many types of logic models, including but not limited to logical frameworks (logframes), results chains, results frameworks, and local actor-oriented models, among others. The project logic model and its associated TOC are included in the Project Appraisal Document (PAD) that approves a project design (see ADS 201.3.3.13). While this How-To Note focuses on logic models at the project level, logic models are also used at the strategy level (specifically, results frameworks – see Box 1), and often at the activity level. The concepts and steps presented here are generally applicable to the process of developing logic models and TOCs throughout the Program Cycle.

Knowing When to Adapt - A Decision Tree - USAID (USAID LEARN, 2019)
This job aid will help you determine how best to resolve a challenge/issue in programming. It is intended for use by USAID, partners, and others working in international development.

Knowledge Retention and Transfer (KRT) - Model and Matsurity Matrix - USAID (CLA Toolkit, 2019)
The Maturity Matrix is not a standalone tool. It is one of the three major components of the KRT Model. The Maturity Matrix is meant to be used in conjunction with the KRT Toolkit and Implementation Plan.

Knowledge Retention and Transfer (KRT) - Model overview presentation - USAID (USAID, 2023)
What problem is the KRT model trying to solve? For workforces that experience continuous staff turnover, the lack of systematic knowledge transfer can often lead to: - Loss of programmatic momentum, - Duplication of efforts and frustration, and - Wasted time and resources. The Knowledge Retention and Transfer (KRT) model provides tools, processes, and practices to individuals, teams, offices, and organizations to improve knowledge handover, which in turn improves efficiency and programmatic and operational learning.

Local Capacity Strengthening Policy - USAID (USAID, 2022)

Local Systems: a framework for supporting sustained development - USAID (USAID, 2014)

Local Works Guidance - Round 4 - USAID (USAID, 2019)

Managing staff transistions through CLA: preserving institutional memory as staff come and go - USAID (CLA Toolkit, 2019)
This document is relevant for any position or hiring mechanism. While this document does not explicitly address what happens when someone arrives into a newly-created position, which has its own set of challenges, many of the principles, actions, and resources can be applied in that context. Section 1 offers guidance for how to set up and implement systems at Mission/OU or office to ensure that all staff help preserve institutional memory and enable continuity of relationships. This section is for any staff in a Mission/OU working to build systems that improve handovers and knowledge capture and sharing across the Mission or OU. If a Mission or OU is just beginning to develop a system to manage staff transitions, you may want to start by developing a system within an office or team and then expand it out to other offices or to the Mission/OU writ large. Section 2 includes key principles and actions individuals could consider following when arriving or departing a position, regardless of hiring mechanism and position. This includes staff new to the Mission/OU and staff who are currently in the Mission/OU and are moving into a new position. Section 3 provides a select number of resources to help you take a systematic and comprehensive approach to manage staff transitions as effectively and efficiently as possible at the individual or organizational level.

Measuring Efforts to Combat Wildlife Crime - A toolkit for Improving Actiona and Accountability (v.1.3) - USAID (Measuring Impact, 2017)

Program Cycle - Discussion Note: Adaptive Management - USAID (USAID, 2018)
This Discussion Note complements ADS 201.3.1.2 Program Cycle Principles by elaborating on Principle 2: Manage Adaptively through Continuous Learning. This Discussion Note is intended for USAID staff interested in learning about recent and promising practices in adaptive management across the Program Cycle.

Program Cycle - Discussion Note: Designing Monitoring, Evaluation, and Learning Platforms - USAID (USAID, 2018)
This Discussion Note complements ADS 201.3.1.2 Program Cycle Principles by elaborating on Principle 2: Manage Adaptively through Continuous Learning. This Discussion Note is intended for USAID staff interested in learning about recent and promising practices in adaptive management across the Program Cycle.

Program Cycle - Discussion Note: Managing Monitoring, Evaluation, and Learning Platforms - USAID (USAID, 2018)
This Discussion Note complements ADS 201.3.1.2 Program Cycle Principles by elaborating on Principle 2: Manage Adaptively through Continuous Learning. This Discussion Note is intended for USAID staff interested in learning about recent and promising practices in adaptive management across the Program Cycle.

Program Cycle - How-To Note: Strategy-Level Portfolio Review - USAID (USAID PPL, 2017)

Program Cycle Road Map - USAID (2016)

RFI-521-17-000015 - Improving Governance in Haiti Program (IGHI) - USAID (USAID, 2017)

SPACES MERL: Systems and Complexity White Paper - USAID (2016)
The Strategic Program for Analyzing Complexity and Evaluating Systems (SPACES MERL) project is an activity funded by USAID’s Global Development Lab and the Bureau for Policy, Planning and Learning (PPL). This three-year activity aims to bring a variety of tools and methodologies that decision-makers can use (alone or in combination) to provide comprehensive systems analysis. The activity is being implemented from 2015 to 2018 by a consortium of organizations expert in systems and complexity, including the Global Obesity Prevention Center (GOPC) at Johns Hopkins University (Prime), Global Knowledge Initiative (GKI), LINC and ResilientAfrica Network (RAN). This Systems and Complexity White Paper is a collaborative effort of the SPACES MERL team, designed to frame the international development landscape, with particular reference to USAID-funded activities, for application of systems and complexity approaches to design, monitoring and evaluation. Customized to the systems and complexity layperson with in-depth knowledge of international development practice, the objectives of this white paper are three-fold:  Provide an overview of systems and complexity practice, its current state of application and relevance to international development practice;  Establish a taxonomy of systems and complexity tools, highlighting the fit of those offered by SPACES MERL within the wider landscape; and  Review and provide information on application of SPACES MERL tools, their purpose and construction, required data, and their applicability to specific contexts.

Stakeholder Engagement for Biodiversity Conservation Goals: Assessing the Status of the Evidence - USAID (Measuring Impact, 2016)
Learning how to communicate and work with different groups builds constituencies to support conservation aims. An analysis by USAID's research partner the American Museum of Natural History evaluated 148 references, selected through a rigorous systematic review process, to identify key lessons for stakeholder engagement. This briefer describes the analysis, which will soon be published in full.

The 5 Rs Framework in the program cycle - USAID (Program Cycle, 2016)
USAID’s Program Cycle Operational Policy (ADS 201) provides guidance to missions and other operating units on how to implement the Program Cycle. A key principle of the Program Cycle is to “Promote Sustainability through Local Ownership.” The purpose of this Technical Note is to describe the “5Rs Framework”, a practical methodology for supporting sustainability and local ownership in projects and activities through ongoing attention to local actors and local systems. This Note is rooted in USAID’s 2014 Local Systems Framework paper, which establishes that achieving sustained improvement in development results depends on the contributions of multiple and interconnected local actors. That document also states that USAID needs to improve its systems practice if it is to engage local actors and strengthen local systems more effectively and thus realize sustained results more consistently. The 5Rs Framework, also introduced in the Local Systems Framework, is intended as a simple and practical tool to promote good systems practice. The 5Rs Framework highlights five key dimensions of systems: Results, Roles, Relationships, Rules and Resources. Collectively these 5Rs can serve as a lens for assessing local systems and a guide for identifying and monitoring interventions designed to strengthen them. This Technical Note is divided in two parts. The first part provides an introduction to the 5Rs Framework and the systems practice from which it emerges. The second part demonstrates how systems practice can be embedded in the Program Cycle by continuously applying the 5Rs, especially to the design, implementation, and monitoring of USAID projects and their accompanying activities.

USAID ADS 201 - Operational Policy for the Program Cycle (Update 01/23/2021) - USAID (USAID, 2021)
The Program Cycle is USAID’s operational model for planning, delivering, assessing, and adapting development programming in a given region or country to advance U.S. foreign policy. It encompasses guidance and procedures for: 1) Making strategic decisions at the regional or country level about programmatic areas of focus and associated resources; 2) Designing projects and supportive activities to implement strategic plans; and 3) Learning from performance monitoring, evaluations, and other relevant sources of information to make course corrections as needed and inform future programming.

USAID ADS 201 - Operational Policy for the Program Cycle (Update 05/22/2023) - USAID (USAID, 2023)
The Program Cycle is USAID’s operational model for planning, delivering, assessing, and adapting development programming in a given region or country to advance U.S. foreign policy. It encompasses guidance and procedures for: 1) Making strategic decisions at the regional or country level about programmatic areas of focus and associated resources; 2) Designing supportive projects and/or activities to implement these strategic plans; and 3) Learning from performance monitoring, evaluations, and other relevant sources of information to make course corrections as needed and inform future programming.

USAID ADS 201 - Program Cycle Operational Policy (Update 1/23/2017) - USAID (USAID, 2017)
The Program Cycle is USAID’s operational model for planning, delivering, assessing, and adapting development programming in a given region or country to advance U.S. foreign policy. It encompasses guidance and procedures for: 1) Making strategic decisions at the regional or country level about programmatic areas of focus and associated resources; 2) Designing projects and supportive activities to implement strategic plans; and 3) Learning from performance monitoring, evaluations, and other relevant sources of information to make course corrections as needed and inform future programming.

USAID ADS 201 - Program Cycle Operational Policy (Update 10/29/2018) - USAID (USAID, 2018)

USAID Risk Appetite Statement - USAID (USAID, 2018)

USAID’s Data Quality Standards & Conducting a DQA - USAID (2014)

Webimar - Adaptive Management at the Strategy Level: Portfolio Reviews and Mid Course Stocktaking - USAID (USAID, 2017)
On Wednesday, November 15, 2017 at 8:00 a.m. EST, the Bureau for Policy, Planning and Learning (PPL) held a one-hour webinar exploring Strategy-level Portfolio Reviews and Mid Course Stocktaking.

An Upside Down View of Governance - Unsworth, Sue (Institute of Development Studies, 2010)
Informal institutions and personalised relationships are usually seen as governance problems. However the research presented in this synthesis paper suggests that they can also be part of the solution...

It's the politics! Can donors rise to the challenge? - Unsworth, Sue (A Governance Practitioner’s Notebook: Alternative Ideas and Approaches, 2015)
The Governance Practitioner’s Notebook takes an unusual approach for the OECD-DAC Network on Governance (GovNet). It brings together a collection of specially written notes aimed at those who work as governance practitioners within development agencies. It does so, however, without attempting to offer definitive guidance – instead aiming to stimulate thinking and debate. To aid this process the book is centred on a fictional Governance Adviser. The Notebook’s format provides space for experts to speak on today’s governance issues: politics, public sector reform and stakeholder engagement. It encourages debate, charts the evolution of donor thinking, and highlights future challenges in the age of the Sustainable Development Goals. Each section introduces both technical issues and major areas of debate, providing ideas for future development support to institutional reform.
Keywords: IMPORTANT

The UNDP Accelerator Labs enter a year of maturity: let a thousand flowers bloom! - Annual report 2022 - UNDP accelerator labs (UNDP, 2022)
The UNDP Accelerator Labs were designed as an agile and dynamic Network to allow communication and information transfer between 91 Accelerator Labs in 115 countries, and with the global innovation ecosystem, UNDP as a whole, and thousands of partners including grassroots innovators and their communities. The accumulated knowledge of this Network creates new pathways to the solutions that hold the key to sustainable development problems. What it looked like and how it unfolded last year will be illuminated in this annual report: The UNDP Accelerator Labs enter a year of maturity: let a thousand flowers bloom!

Project Cycle Hacker's Toolkit - changing the conversation around your projects - UNDP (UNDP, 2017)
The Istanbul Regional Hub partnered with Nesta and 4 country offices (fYR Macedonia, Moldova, Georgia and Kyrgyzstan) to develop a program that can help us bring innovation closer to the center of the organization by embedding it in the key project management business processes - The Project Cycle Hackers Toolkit.

Scaling Up Development Programmes - UNDP (UNDP, 2013)
Many development organizations, national and local governments and civil society organizations are faced with the issue of scaling up development interventions — the main questions raised time and again are: a) what should be scaled up, and how it can be scaled up; b) is there a strong reason for a particular initiative to be scaled up; and c) what should be the value-added of the scaling up efforts, and how can practitioners ensure that technological and other innovations are being integrated for improved development effectiveness? Answers to these questions depend on a host of complex realities—the relative strengths and weaknesses in national and local systems, political and economic situations, including vulnerability of country systems to shocks, commitments from development partners, power dynamics between various groups and stakeholders in a given country, regional and global environments. For the United Nations Country Teams (UNCT) and UNDP offices, another factor to consider is how to maximize our respective comparative advantages. This guidance note summarizes the main conceptual thinking available from development institutions and academia, and presents a simplified conceptual framework and roadmap for scaling up processes. It also provides UNDP programme staff and UNCTs with practical actions and checklists to consider when designing and implementing programmes that support national scaling up initiatives, and proposes actions that can be undertaken at the regional and global levels. Although intended for internal audiences, the scaling up concept and recommendations can also be used by the public and private sector, civil society and social entrepreneurs. This note benefited from a wide consultative process held in 2011, including practical recommendations and proposals from colleagues at country offices and regional/global centres. Their insights have been instrumental in the distillation of main recommendations presented in this note, and the individuals who provided substantive contributions are gratefully noted in the Acknowledgements section.

Sensemaking Workshop Preparation Guide and Facilitator Guide and Sensemaking Training - UNDP (UNDP, 2022)
Based on experience from running Sensemaking workshops for UNDP offices and government partners, the Asia-Pacific Regional Innovation Centre developed the Sensemaking Preparation Guide and Facilitator Guide to share its knowledge with teams and organization that are interested in using the Sensemaking process.

UNDP Digital Leadership Learning Modules - UNDP (United Nations Development Programme, 2022)
In order to support the digital transformation of government operations Digital Learning Modules for Civil Servants are available, an off-the-shelf package of capacity development in form of replicable training modules to empower public servants at both the local and central government level to be leaders of digital transformation for delivering better public services. The modules cover a multitude of fundamental areas: comprehending digital government and services, human-centered design for inclusivity and agile learning cycles; feature the importance of security and privacy, the value of data and how to manage data and technology related risks; spotlight the key role of supportive leadership and offer practical tools for assessing and overcoming main barriers to ensure a successful digital transformation journey.

Feasibility Study: Crowdsourcing High-frequency food price data in rural Indonesia - UN Global Pulse (2015)

A Practical Guide for Collective Action against Corruption - UN Global Compact (UN Global Compact, 2015)
All anti-corruption professionals looking to better understand the relationship between collective action and corruption should refer to these two comprehensive guides that the UN Global Compact published. The 2015 guide has modules that cover theoretical concepts, practical recommendations for undertaking CA initiatives, and multiple in-depth case studies of CA projects around the world.

Uniting Against Corruption - A Playbook on Anti-Corruption Collective Action - UN Global Compact (UN Global Compact, 2021)
The 2021 guide provides an easy-to-follow six-step approach on how to develop, implement, and sustain a CA, with respect to the reader’s local corruption landscape and potential stakeholders. The adaptive framework proposed can be used to address corruption challenges, mitigate possible business risks, and achieve optimal results.

A Framework for Freshwater Ecosystem Management - UN Enviornment (2018)

Critical systems heuristics - Ulrich, Werner; Reynolds, Martin (Systems Approaches to Manging Change: A Practical Guide, 2010)
This chapter provides a detailed introduction to critical systems heuristics and the use of its central tool boundary critique. What Is CSH? Two Studies in Applying CSH Using CSH as an Intervention Tool: Some Basic Concepts A Core Concept of CSH: Systematic Boundary Critique Boundary Critique Applied to NRUA-Botswana Boundary Critique Applied to ECOSENSUS-Guyana Boundary Critique and Personal Competence Recognising Boundary Judgements – and Keeping Them Fluid Towards a New Ethos of Professional Responsibility

A Brief Introduction to Critical Systems Heuristics (CSH) - Ulrich, Werner (2005)

Werner Ulrich's Homepage - Ulrich, Werne (2018)
This website provides a variety of resources from Werner Ulrich's work in a range of philosophical and research areas with a particular focus on his own work in critical systems thinking and practice or, Critical Systems Heuristics (CSH).

A mini-primer of critical systems heuristics. - Ulrich, W. (2005)
"Critical Systems Heuristics," also just called "Critical Heuristics" or "CSH," is a framework for reflective practice based on practical philosophy and systems thinking. The basic idea of CSH is to support boundary critique – a systematic effort of handling boundary judgments critically. Boundary judgments determine which empirical observations and value considerations count as relevant and which others are left out or are considered less important. Because they condition both "facts" and "values," boundary judgments play an essential role when it comes to assessing the meaning and merits of a claim. Their systematic discussion can help bridge differences of perspectives across disciplines and between experts and non-experts. They also lend themselves to a specific critical employment, called emancipatory boundary critique, against claims that do not uncover their underlying boundary assumptions. CSH can thus serve as a tool for coproducing knowledge as well as for critical and emancipatory purposes on the part of people concerned by, but not necessarily involved in, the definition of relevant facts and values.

An introductory systems thinking toolkit for civil servants - UK Government (UK Government Office for Science, 2022)
This document is a an Introductory Toolkit for for civil servants. It is one component of a suite of documents that aims to act as a springboard into systems thinking for civil servants unfamiliar with this approach. These documents introduce a small sample of systems thinking concepts and tools, chosen due to their accessibility and alignment to civil service policy development, but which is by no means comprehensive. They are intended to act as a first step towards using systems thinking approaches to solve complex problems and the reader is encourage to explore the wider systems thinking field further.

Case study - Lepra - Adaptive programming in “reaching the unreached” - UK Aid Direct (DFID, 2017)

Beneficiary feedback mechanisms - UK Aid (FCDO, 2022)
This guidance seeks to ensure that UK Aid Direct applicants and grant holders understand what the Foreign, Commonwealth & Development Office (FCDO) means by beneficiary feedback mechanisms, and more specifically, that they: • Understand the terms used that relate to beneficiary feedback mechanisms in UK Aid Direct guidance and templates • Understand beneficiary feedback mechanisms and why they are a useful tool for project monitoring and learning • Learn how to use beneficiary feedback mechanisms during project implementation. • Can demonstrate that using beneficiary feedback mechanisms can lead to greater accountability.

Managing Facilities: a stock-take from the first 12 months - Tyrrel, Lavinia; Teskey, Graham; de Lacy, Jacqui (Governance Working Paper 3, 2017)

Uncertainty and COVID-19: A turning point for Monitoring Evaluation, Research and Learning? - A discussion note for aid actors, policymakers and practitioners - Tyrrel, Lavinia; Roche, Chris; Jackson, Elisabeth (Abt Associates, 2020)
The COVID-19 pandemic has significantly shifted the context in which aid and development is being delivered. The global scale of the pandemic and the speed at which it is spreading mean that the ‘normal’ economic, ideological and organisational influences which shape (if not determine) aid delivery are in flux. This means that – for a relatively short-period – there is scope for aid actors to work collectively to embed more locally-led, politically-informed and adaptive forms of MERL in aid and development practice. These forms of Monitoring Evaluation Research and Learning (MERL) are not only well-suited to the current global pandemic. They also offer ways for aid program decision makers and practitioners to make sense of the complex and uncertain contexts in which much development work takes place. Applying locally-led, politically-informed and adaptive forms of MERL in the COVID-19 context and beyond requires a shift in mindset and approaches. Situations of complexity, in which it is difficult to predict the relationships between cause and effect, do not lend themselves to linear approaches and fixed indicators. Instead, they require ‘navigation by judgement’, ongoing learning and adaptation and greater privileging of local knowledge, and of the perspectives of those who are often excluded. Rather than being focused on upwards accountability, simple numbers and good news stories, the core function of MERL in this context is to support a better understanding – in real-time – of the changing operating context, to generate learning about the immediate impact of policy and program responses and their longer-term effects, and to inform decision making by front line staff. Whether the opportunities afforded by this ‘critical juncture’ are realised will depend on the degree to which those in the aid and development sector use this opportunity to promote a shift in the deep incentive structures within which development agencies are embedded. On the one hand, the pandemic underscores the limits of the linear understandings of change which underpin many orthodox approaches to planning, design and associated MERL. On the other hand, there is a vested interest in the status quo amongst many organisations, consultants, researchers and MERL practitioners. This is because approaches which promote locally-led development inevitably require those in power to relinquish control. While a range of factors make this shift difficult, there is more scope to change internal ways of working in development agencies than is commonly acknowledged. There is no time like the present to advocate for a ‘new normal’ for MERL.

Dealing with uncertainty: Reflections on donor preferences for pre-planned project models - Tyrrel, Lavinia; Cole, William (Working Politically in Practice, 2016)
Keywords: IMPORTANT

Getting Past the Rhetoric #1: Tendering for ‘Thinking and Working Politically’ in Large Facilities - Tyrrel, Lavinia (Abt Associates - Governance Soapbox, 2018)

Getting Past the Rhetoric #2: Managing for ‘Thinking and Working Politically’ in Large Facilities - Tyrrel, Lavinia (Abt Associates - Governance Soapbox, 2018)

What's in a name? Unpacking “participatory” environmental monitoring - Turreira-García, Nerea; Lund, Jens; Domínguez, Pablo; Carrillo-Anglés, Elena; Brummer, Mathias; Duenn, Priya; Reyes-García, Victoria (Ecology and Society, 2018)
Turreira-García, N., J. F. Lund, P. Domínguez, E. Carrillo-Anglés, M. C. Brummer, P. Duenn, and V. Reyes-García. 2018. What's in a name? Unpacking “participatory” environmental monitoring. Ecology and Society 23(2):24. https://doi.org/10.5751/ES-10144-230224

Doing weeknotes - What weeknotes are, how weeknotes work, and how to start writing weeknotes of your own - Turnbull, Giles (Use the Human Voice, 2024)
Doing weeknotes brings together various things I’ve written about weeknotes in different places. This text expands on things I wrote in The agile comms handbook, as well as various blog posts. Quite a lot of it is brand new. - Weeknotes for beginners - Why write weeknotes - The weeknotes rules - Weeknotes within the corporate environment - What weeknotes can bring about - Examples of good weeknotes - How to write weeknotes - Weeknotes tips and tricks - Further reading