Learning about Theories of Change for the Monitoring and Evaluation of Research Uptake - Barnett, Chris; Gregorowski, Robbie (IDS Practice Paper In Brief, 2013)
This paper captures lessons from recent experiences on using ‘theories of change’ amongst organisations involved in the research–policy interface. The literature in this area highlights much of the complexity inherent in the policymaking process, as well as the challenges around finding meaningful ways to measure research uptake. As a tool, ‘theories of change’ offers much, but the paper argues that the very complexity and dynamism of the research-to-policy process means that any theory of change will be inadequate in this context. Therefore, rather than overcomplicating a static depiction of change at the start (to be evaluated at the end), incentives need to be in place to regularly collect evidence around the theory, test it periodically, and then reflect and reconsider its relevance and assumptions.

Adaptive Peacebuilding Programming Through Effective Feedback Loops: Promising Practices - Barnard-Webster, Kiely; Jean, Isabella (CDA, 2018)
This case study report was developed by CDA’s part of Peacebuilding Evaluation Consortium collaborative research on the use of feedback loops in support of adaptive programming. Feedback loops are just one element of adaptive programming and are defined and used in different ways by organizations working on social change, peace and development issues. This document includes key findings from our desk review of reports and grey literature, key informant interviews, observations documented during CDA’s advisory and learning engagements with peacebuilding partners as well as the case study on SFCG’s feedback processes in Burundi. The findings are most relevant to program and M&E staff but have implications for senior management and policymakers in light of growing interest in strengthening adaptive management.

Complexity, Adaptation, and Results - Barder, Owen; Ramalingam, Ben (Center For Global Development, 2012)
In the last of a series of three blog posts looking at the implications of complexity theory for development, Owen Barder and Ben Ramalingam look at the implications of complexity for the trend towards results-based management in development cooperation. They argue that is a common mistake to see a contradiction between recognising complexity and focusing on results: on the contrary, complexity provides a powerful reason for pursuing the results agenda, but it has to be done in ways which reflect the context. In the 2012 Kapuscinski lecture Owen argued that economic and political systems can best be thought of as complex adaptive systems, and that development should be understood as an emergent property of those systems. As explained in detail in Ben’s forthcoming book, these interactive systems are made up of adaptive actors, whose actions are a self-organised search for fitness on a shifting landscape. Systems like this undergo change in dynamic, non-linear ways; characterised by explosive surprises and tipping points as well as periods of relative stability. If development arises from the interactions of a dynamic and unpredictable system, you might draw the conclusion that it makes no sense to try to assess or measure the results of particular development interventions. That would be the wrong conclusion to reach. While the complexity of development implies a different way of thinking about evaluation, accountability and results, it also means that the ‘results agenda’ is more important than ever.
Keywords: IMPORTANT

The Implications of Complexity for Development - Barder, Owen (2012)
In this lecture, adapted from his May 2012 Kapuściński Lecture, Owen Barder explores the implications of complexity theory for development policy. He explains how traditional economic models have tried and failed to understand why some countries have managed to improve living standards while other countries have not.
Keywords: IMPORTANT

Qualitative Comparative Analysis - A Rigorous Qualitative Method for Assessing Impact - Baptist, Carrie; Befani, Barbara (Coffey, 2015)

Biased Policy Professionals - Banuri, Sheheryar; Dercon, Stefan; Gauri, Varun (The World Bank Economic Review, 2019)
Although the decisions of policy professionals are often more consequential than those of individuals in their private capacity, there is a dearth of studies on the biases of policy professionals: those who prepare and implement policy on behalf of elected politicians. Experiments conducted on a novel subject pool of development policy professionals (public servants of the World Bank and the Department for International Development in the UK) show that policy professionals are indeed subject to decision-making traps, including the effects of framing outcomes as losses or gains, and, most strikingly, confirmation bias driven by ideological predisposition, despite having an explicit mission to promote evidence-informed and impartial decision making. These findings should worry policy professionals and their principals in governments and large organizations, as well as citizens themselves. A further experiment, in which policy professionals engage in discussion, shows that deliberation may be able to mitigate the effects of some of these biases.
Keywords: bias

Increasing Participation in Evaluation - Baker, A; Bruner, B (Effectiveness Initiatives, 2014)
The Increasing Participation in Evaluation bulletin was developed by Anita Baker with Beth Bruner to help organizations integrate evaluative thinking into their organizational practice. This three page bulletin discusses how Organization Staff, Evaluators, and Funders are typically involved in participatory evaluation. The guide also defines the term "Ripple", and how to accomplish Ripple as well as examining what it looks like when Executive Leaders and Management Staff use Evaluative Thinking. Contents How are Organization Staff, Evaluators, and Funders typically involved in Participatory Evaluation? Organization Managers/Staff Roles Evaluators/Roles Funder Roles Shared Roles What is Ripple Anyway? How do you accomplish Ripple? What does it look like when Executive Leaders and Management Staff Use Evaluative Thinking? How to use these Bulletins

Participatory Budgeting as if Emancipation Mattered - Baiocchi, Gianpaolo; Ganuza, Ernesto (Politics & Society, 2014)
Participatory Budgeting has by now been widely discussed, often celebrated, and is now instituted in at least 1,500 cities worldwide. Some of its central features—its structure of open meetings, its yearly cycle, and its combination of deliberation and representation—are by now well known. In this article, however, we critically reflect on its global travel and argue for more careful consideration of some of its less well-known features, namely the coupling of the budgeting meetings with the exercise of power. We disaggregate PB into its communicative and empowerment dimensions and argue that its empowerment dimensions have usually not been part of its global expansion—and this is cause for concern from the point of view of emancipation. We thus discuss the specific institutional reforms associated with empowerment in the original version as well as its analytic dimensions. We also address some of the specific dangers of a communication-only version of PB as well as some suggestions for reintroducing empowerment.

Institutional Change, Political Economy, and State Capabilities : Learning from Edo State, Nigeria - Bain, Katherine; Porter, Doug; Watts, Michael (Justice, security, and development series, 2015)
This paper is one of a series aimed at deepening the World Bank’s capacity to follow through on commitments made in response to the World Development Report (WDR) 2011, which gave renewed prominence to the nexus between conflict, security, and development. Nigeria is a remarkable illustration of how deeply intractable the cycle of poverty, conflict, and fragility can become when tied to the ferocious battles associated with the political economy of oil. This paper places the corpus of analytic and programmatic work concerning institutional reform in conversation with a now substantial body of work on resource politics and most especially, the debate over the politico-institutional character (sometimes called political settlements or pacting arrangements associated with the order of power) and reform landscape of the petro-state. Recent institution reform policy writing appears to have little to say about the political and economic conditions in which crises and institutional disjunctures may authorize, and thereby enable, agents to embark on institutional reforms. The authors focus on Edo state for two reasons. First, it does not on its face appear to be an obvious location in which to explore a reform experience, given its entanglement in the Niger Delta conflict and the maladies typically associated with state fragility. Second, Edo is of interest also because of the changes that its experience is contributing to the World Bank country team’s effort to engage operationally across all its instruments with the political economy of institutional reform in Nigeria, its largest client country in Africa.

Doing Development Differently at the World Bank: updating the plumbing to fit the architecture - Bain, Katherine A.; Booth, David; Wild, Leni (ODI, 2016)
Keywords: IMPORTANT

Digital lives in Ghana, Kenya and Uganda - Bailur, Savita; Donner, Jonathan; Locke, Chris; Schoemaker, Emrys; Smart, Charlotte (Caribou Digital, 2015)
Keywords: IMPORTANT

The Art of Community: Building the New Age of Participation - Bacon, Jono (O'Reilly Media, 2012)
Online communities provide a wide range of opportunities for supporting a cause, marketing a product or service, or building open source software. The Art of Community helps you recruit members, motivate them, and manage them as active participants. Author Jono Bacon offers experiences and observations from his 14-year effort to build and manage communities, including his current position as manager for Ubuntu.Discover how your community can become a reliable support network, a valuable source of new ideas, and a powerful marketing force. This expanded edition shows you how to keep community projects on track, make use of social media, and organize collaborative events. Interviews with 12 community management leaders, including Linus Torvalds, Tim O’Reilly, and Mike Shinoda, provide useful insights.Develop specific objectives and goals for building your communityBuild processes to help contributors perform tasks, work together, and share successesProvide tools and infrastructure that enable members to work quicklyCreate buzz around your community to get more people involvedHarness social media to broadcast information, collaborate, and get feedbackUse several techniques to track progress on community goalsIdentify and manage conflict, such as dealing with divisive personalities

The role of research and learning in adaptive programming - Babon, Andrea; Denney, Lisa (Devpolicy Blog from the Development Policy Centre, 2018)
Andrea Babon and Lisa Denney explore how learning partners - a common feature of aid programs - can operate and feed into programs.

Global mapping of technology for transparency and accountability - Avila, Renata; Feigenblatt, Hazel; Heacock, Rebekah; Heller, Nathaniel (Transparency & Accountability Initiative, 2010)

Building Bridges to an Uncertain Future Lived Now: Lessons from the Use of Participatory Action Research and Theory of Change Towards A Realistic Community-Based Participatory Monitoring and Evaluation System - Avila, Enrique M.; Tolentino, Lutgarda L.; Binondo, Claudia B.; Perez, Maripaz L.; Apgar, J. Marina (International Journal of Agriculture System, 2016)
Building on experience from the CGIAR Research Program on Aquatic Agricultural Systems implemented by WorldFish in the Visayas and Mindanao regions of the Philippines, known as the VisMin Hub, we describe the development and evolution of a monitoring and evaluation (M&E) system emerging from the facilitated action-reflection cycles of testing and adopting theories of change carried out with community partners through participatory action research (PAR). The former guides our community partners and us, as members of the potentially emergent PAR groups, towards the realization of the community’s vision; the latter facilitates learning to understand what, how and why change is unfolding. Unlike the conventional M&E system where indicators are pre-set at the beginning of program implementation, these processes result in an organically-evolved, communitybased participatory M&E system that is continuously revised according to contexts to guide communities towards realizing their visions. Its ultimate outcome is enhanced people’s capacity to own the product and process, giving rise to an internally-driven change. Towards the end, the paper offers an iterative discussion of learnings from implementing such an approach.
Keywords: Aquatic agricultural systems; CGIAR; Community-based participatory MEL; Learning; Participatory action research; research in development; theory of change

One step back, two steps forward: CARE’s journey towards doing development differently - Aston, Tom (Care Insights, 2017)
The Doing Development Different (DDD) community emerged in August 2014 and advocates that (a) the barriers to development are as much political as tec...

Sounding clever or being smart? – How to do more with less in evaluating governance programmes - Aston, Tom (Care Insights, 2018)
About a decade ago, the development sector fell into the same trap the financial services industry did in the mid-1990s. We were all seduced by clever...

Tools for supporting sustainable natural resource management and livelihoods - Ashley, Holly; Kenton, Nicole; Milligan, Angela (2013)

Bringing adaptive management to life: Insights from practice - Arora, Anmol; Gogoi, Elizabeth; Joy, Divya; Kumar, Pankaj; Luthra, Rajni; Pal, Uma; Pervaiz, Arif (Oxford Policy Management, 2019)

Mission-Based Monitoring, Evaluation and Learning (MEL) Platforms Assessment Report - Arntson, Laura; Giannoni, Tonya; Peek, Nancy; Saarlas, Kristin (USAID, 2017)
In 2016, the Bureau of Policy, Planning and Learning (PPL) commissioned an internal stocktaking of USAID's mission-based MEL (Monitoring, Evaluation, and Learning) Platforms.

Twinning “Practices of Change” With “Theory of Change”: Room for Emergence in Advocacy Evaluation - Arensman, Bodille; van Waegeningh, Cornelie; van Wessel, Margit (American Journal of Evaluation, 2018)
Theory of change (ToC) is currently the approach for the evaluation and planning of international development programs. This approach is considered especially suitable for complex interventions. We question this assumption and argue that ToC’s focus on cause–effect logic and intended outcomes does not do justice to the recursive nature of complex interventions such as advocacy. Supported by our work as evaluators, and specifically our case study of an advocacy program on child rights, we illustrate how advocacy evolves through recursive interactions, with outcomes that are emergent rather than predictable. We propose putting “practices of change” at the center by emphasizing human interactions, using the analytical lenses of strategies as practice and recursiveness. This provides room for emergent outcomes and implies a different use of ToC. In this article, we make a clear distinction between theoretical reality models and the real world of practices.

Using adaptive processes and adverse outcome pathways to develop meaningful, robust, and actionable environmental monitoring programs - Arciszewski, Tim J.; Munkittrick, Kelly R.; Scrimgeour, Garry J.; Dubé, Monique G.; Wrona, Fred J.; Hazewinkel, Rod R. (Integrated Environmental Assessment and Management, 2017)
The primary goals of environmental monitoring are to indicate whether unexpected changes related to development are occurring in the physical, chemical, and biological attributes of ecosystems and to inform meaningful management intervention. Although achieving these objectives is conceptually simple, varying scientific and social challenges often result in their breakdown. Conceptualizing, designing, and operating programs that better delineate monitoring, management, and risk assessment processes supported by hypothesis-driven approaches, strong inference, and adverse outcome pathways can overcome many of the challenges. Generally, a robust monitoring program is characterized by hypothesis-driven questions associated with potential adverse outcomes and feedback loops informed by data. Specifically, key and basic features are predictions of future observations (triggers) and mechanisms to respond to success or failure of those predictions (tiers). The adaptive processes accelerate or decelerate the effort to highlight and overcome ignorance while preventing the potentially unnecessary escalation of unguided monitoring and management. The deployment of the mutually reinforcing components can allow for more meaningful and actionable monitoring programs that better associate activities with consequences. Integr Environ Assess Manag 2017;13:877–891. © 2017 The Authors. Integrated Environmental Assessment and Management Published by Wiley Periodicals, Inc. on behalf of Society of Environmental Toxicology & Chemistry (SETAC)
Keywords: Adaptive monitoring; Environmental management; Environmental monitoring; Tier; Trigger

Assumptions, conjectures, and other miracles: The application of evaluative thinking to theory of change models in community development - Archibald, Thomas; Sharrock, Guy; Buckley, Jane; Cook, Natalie (Evaluation and Program Planning, 2016)
Unexamined and unjustified assumptions are the Achilles’ heel of development programs. In this paper, we describe an evaluation capacity building (ECB) approach designed to help community development practitioners work more effectively with assumptions through the intentional infusion of evaluative thinking (ET) into the program planning, monitoring, and evaluation process. We focus specifically on one component of our ET promotion approach involving the creation and analysis of theory of change (ToC) models. We describe our recent efforts to pilot this ET ECB approach with Catholic Relief Services (CRS) in Ethiopia and Zambia. The use of ToC models, plus the addition of ET, is a way to encourage individual and organizational learning and adaptive management that supports more reflective and responsive programming.
Keywords: Assumptions; Community development; Critical thinking; Evaluation capacity building; Evaluative thinking; International Development; Theory of change

Appreciative Inquiry Australia - Appreciative Inquiry Australia (2015)
The site provides news and information on AI events and learning opportunities in Australia and also provides an archive of articles and discussions centred on the use of AI.

Bringing Participation into Complexity-Aware MEL: What is the evidence? - Apgar, Marina; Higdon, Grace Lyn (2018)

Getting beneath the surface in program planning, monitoring and evaluation: Learning from use of participatory action research and theory of change in the CGIAR Research Program on Aquatic Agricultural Systems - Apgar, J Marina; Allen, Will; Albert, Joelle; Douthwaite, Boru; Paz Ybarnegaray, Rodrigo; Lunda, Jeston (Action Research, 2017)
Many rural poor and marginalized people strive to make a living in social-ecological systems that are characterized by multiple and often inequitable interactions across agents, scale and space. Uncertainty and inequality in such systems require research and development interventions to be adaptive, support learning and to engage with underlying drivers of poverty. Such complexity-aware approaches to planning, monitoring and evaluating development interventions are gaining strength, yet, there is still little empirical evidence of what it takes to implement them in practice. In this paper, we share learning from an agricultural research program that used participatory action research and theory of change to foster learning and support transformative change in aquatic agricultural systems. We reflect on our use of critical reflection within participatory agricultural research interventions, and our use of theory of change to collectively surface and revisit assumptions about how change happens. We share learning on the importance of being strengths-based in engaging stakeholders across scales and building a common goal as a starting point, and then staging a more critical practice as capacity is built and opportunities for digging deeper emerge.

3 fallacies of embracing complexity - Ang, Yuen Yuen (UNDP Blog, 2018)
Global development is seeing an exciting paradigm shift. Increasingly, leaders and practitioners recognize that development is not a “complicated” challenge that can be neatly parsed out into separate problems and siloed departments, like assembling a car. Rather, the various tasks of development—poverty eradication, improving governance, climate action, gender equality, and so on—are all connected, making development a “complex” challenge.

Combining Big Data and Thick Data to Improve Services Delivery - Ang, Yuen Yuen (IBM Center for The Business of Government, 2018)

Complexity & Development 2.0: From Agreeing We Should Adapt to Creating the Conditions that Enable Adaptation - Ang, Yuen Yuen (2018)
The field of global development has reached a critical turning point. Almost gone is the mechanical, one-size-fits-all “good governance” paradigm of the past. In its place is a growing embrace of complexity and systems thinking. While this is an encouraging shift in the right direction, the discussion mostly ends by concluding that we should adapt. Yuen Yuen Ang urges that it’s time to take our conversation on “complexity & development” to the next level: how to enable adaptation. Effective adaptation doesn’t automatically arise just because we desire it—rather, it requires certain enabling conditions. In How China Escaped the Poverty Trap (2016), she introduces a generic complexity 2.0 framework that she terms “directed improvisation.” Effective adaptation, Yuen Yuen argues, requires a paradoxical blend of top-down direction and bottom-up improvisation. Such a system can be created—and produce dramatic results and adaptive solutions—even within a closed political regime like China.

Going Local 2.0: How to Reform Development Agencies to Make Localized Aid More Than Talk - Ang, Yuen Yuen (Stanford Social Innovation Review, 2018)
More and more global aid agencies believe they should replace one-size-fits-all best practices with locally tailored solutions, but they must shift from just agreeing to “go local” to preparing development experts for the task by taking on three major problems with their internal practices.

How China Escaped the Poverty Trap - Ang, Yuen Yuen (Cornell University Press, 2016)
Before markets opened in 1978, China was an impoverished planned economy governed by a Maoist bureaucracy. In just three decades it evolved into the world's second-largest economy and is today guided by highly entrepreneurial bureaucrats. In How China Escaped the Poverty Trap, Yuen Yuen Ang explains this astonishing metamorphosis. Rather than insist that either strong institutions of good governance foster markets or that growth enables good governance, Ang lays out a new, dynamic framework for understanding development broadly. Successful development, she contends, is a coevolutionary process in which markets and governments mutually adapt.By mapping this coevolution, Ang reveals a startling conclusion: poor and weak countries can escape the poverty trap by first harnessing weak institutions―features that defy norms of good governance―to build markets. Further, she stresses that adaptive processes, though essential for development, do not automatically occur. Highlighting three universal roadblocks to adaptation, Ang identifies how Chinese reformers crafted enabling conditions for effective improvisation.How China Escaped the Poverty Trap offers the most complete synthesis to date of the numerous interacting forces that have shaped China’s dramatic makeover and the problems it faces today. Looking beyond China, Ang also traces the coevolutionary sequence of development in late medieval Europe, antebellum United States, and contemporary Nigeria, and finds surprising parallels among these otherwise disparate cases. Indispensable to all who care about development, this groundbreaking book challenges the convention of linear thinking and points to an alternative path out of poverty traps.
Keywords: IMPORTANT

Integrating Big Data and Thick Data to Transform Public Services Delivery - Ang, Yuen Yuen (IBM Center for The Business of Government, 2019)
This report explains how public organizations can combine big data and thick data to transform public services delivery—a strategy called mixed analytics. Governments can greatly enhance the value of big data by combining it with “thick” data—rich qualitative information about users, such as their values, goals, and consumption behavior, obtained by observing or interacting with them in their daily lives. Big data holds great promise for improving public services delivery and innovation in government, but they are not a panacea . Having lots of data can be overwhelming or have little utility if the data are “thin”—that is, they lack meaning for users or fail to capture issues that matter most . By yielding insights into what citizens really care about and how they consume services, thick data can inform both the collection and analysis of big data. This report introduces the concept of “mixed analytics,” integrating big data and thick data to transform government decision making, public services delivery, and communication. The report presents three case studies of organizations that employ mixed analytics at the international, federal, and city level, respectively. Together, this research offers a set of transferable lessons for agencies at all levels of government: Lesson 1: Big data is a means to an end, rather than an end. Lesson 2: Thick data can identify unexpected problems or previously unexpressed needs. Lesson 3: Thick data can inform the analysis of big data. Lesson 4: Mixed analytics can offer both scale and depth. Lesson 5: Applying technology is a social activity, not an isolated technical task. Lesson 6: The best solutions are not always high-tech. The report concludes with five actionable recommendations for public managers. Make data and technology relevant to the people who use it. Leverage thick data at appropriate stages of the problem-solving process. Build an interdisciplinary team of quantitative and qualitative experts who work closely with stakeholders. Combine big and thick data to improve communication Improve government agencies’ knowledge of mixed research methods. This report builds on multiple past IBM Center reports about how government can leverage data and analytics to improve decisions, including Data-Driven Government: The Role of Chief Data Officers, by Jane Wiseman; Ten Actions to Implement Big Data Initiatives: A Study of 65 Cities, by Alfred Ho and Bo McCall, and Realizing The Promise of Big Data, by Kevin DeSouza. The report also complements several chapters that assess the past and potential use of data across the public sector in our recent book, Government For The Future: Reflection and Vision For Tomorrow’s Leaders. At a time when the US and other governments continue to enhance their use of data as a strategic asset for transformation, we hope that Professor Ang’s report provides useful insights for government managers and stakeholders.

Making Details Matter: How to Reform Aid Agencies to Generate Contextual Knowledge - Ang, Yuen Yuen (SSRN, 2014)
My essay considers a central problem of reinventing foreign aid in the twenty-first century: how to reform aid agencies to enable a “best-fit” approach to development assistance. For the past decades, the aid community has tried to transplant best practices from the developed world to the developing world. Increasingly, however, it is recognized that copying best practices does not work and may even backfire; rather, aid programs work best when they are tailored to local contexts. Yet while the idea of a best-fit approach is widely embraced in principle, actualizing it is easier said than done. For meaningful changes to take root in practice, we must first identify the obstacles to localizing development assistance and suggest ways to address these problems. To this end, I propose a three-pronged strategy to promote the generation of contextual knowledge among aid professionals, a necessary condition for crafting solutions that can fit various local contexts, namely: (1) build a bank of knowledge about unorthodox practices that work, (2) diversify expertise within aid agencies; and (3) carve experimental pockets. My proposal does not fit neatly into any one of the six themes specified in the GDN competition; rather, it concerns all of the themes. Whether it is to use aid to improve governance, apply information technology, or design financial instruments, the overarching challenge is to empower and incentivize aid professionals to learn and apply contextual knowledge to creatively solve problems in developing societies.
Keywords: Contextual knowledge; Localization; Public policy; foreign aid; international development

Building State Capability: Evidence, Analysis, Action - Andrews, Matt; Pritchett, Lant; Woolcock, Michael (Oxford University Press, 2017)
This is an open access title available under the terms of a CC BY-NC-ND 4.0 licence. It is free to read at Oxford Scholarship Online and offered as a free PDF download from OUP and selected open access locations. Governments play a major role in the development process, and constantly introduce reforms and policies to achieve developmental objectives. Many of these interventions have limited impact, however; schools get built but children don't learn, IT systems areintroduced but not used, plans are written but not implemented. These achievement deficiencies reveal gaps in capabilities, and weaknesses in the process of building state capability.This book addresses these weaknesses and gaps. It starts by providing evidence of the capability shortfalls that currently exist in many countries, showing that many governments lack basic capacities even after decades of reforms and capacity building efforts. The book then analyses this evidence, identifying capability traps that hold many governments back - particularly related to isomorphic mimicry (where governments copy best practice solutions from other countries that make them look morecapable even if they are not more capable) and premature load bearing (where governments adopt new mechanisms that they cannot actually make work, given weak extant capacities). The book then describes a process that governments can use to escape these capability traps. Called PDIA (problem driveniterative adaptation), this process empowers people working in governments to find and fit solutions to the problems they face. The discussion about this process is structured in a practical manner so that readers can actually apply tools and ideas to the capability challenges they face in their own contexts. These applications will help readers devise policies and reforms that have more impact than those of the past.
Keywords: IMPORTANT

Doing Iterative and Adaptive Work - Andrews, Matt; Pritchett, Lant; Woolcock, Michael (CID Working Paper, 2016)
Many of the challenges in international development are complex in nature. They involve many actors in uncertain contexts and with unclear solutions. Our work has proposed an approach to addressing such challenges, called Problem Driven Iterative Adaptation (PDIA). This paper is the most recent in a series intended to show how one can do PDIA, building on the first paper, "Doing Problem Driven Work.” The current paper addresses a key part of the approach one moves to once a problem has been identified, performing real-time experimental iterations. This is intended as a practical paper that builds on experience and embeds exercises for readers who are actually involved in this kind of work.
Keywords: IMPORTANT

Doing Problem Driven Work - Andrews, Matt; Pritchett, Lant; Woolcock, Michael (HKS Working Paper, 2015)
Keywords: IMPORTANT

Escaping Capability Traps through Problem-Driven Iterative Adaptation (PDIA) - Andrews, Matt; Pritchett, Lant; Woolcock, Michael (HKS Working Paper, 2012)
Keywords: IMPORTANT

Building capability by delivering results: Putting Problem-Driven Iterative Adaptation (PDIA) principles into practice - Andrews, Matt; Pritchett, Lant; Samji, Salimah; Woolcock, Michael (A Governance Practitioner’s Notebook: Alternative Ideas and Approaches, 2015)
The Governance Practitioner’s Notebook takes an unusual approach for the OECD-DAC Network on Governance (GovNet). It brings together a collection of specially written notes aimed at those who work as governance practitioners within development agencies. It does so, however, without attempting to offer definitive guidance – instead aiming to stimulate thinking and debate. To aid this process the book is centred on a fictional Governance Adviser. The Notebook’s format provides space for experts to speak on today’s governance issues: politics, public sector reform and stakeholder engagement. It encourages debate, charts the evolution of donor thinking, and highlights future challenges in the age of the Sustainable Development Goals. Each section introduces both technical issues and major areas of debate, providing ideas for future development support to institutional reform.
Keywords: IMPORTANT

A Guide to Peer-to-Peer Learning - Andrews, Matt; Manning, Nick (Effective Institutions Platform, 2016)

Learning to Target for Economic Diversification: PDIA in Sri Lanka - Andrews, Matt; Ariyasinghe, Duminda; Batuwanthudawa, Thamari; Darmasiri, Shivanthika; Silva, Nilupul; Harrington, Peter; Jayasinghe, Prasanna; Jayasinghe, Upul; Jayathilake, Gamini; Karunaratne, Jayani; Katugampala, Lalit; Liyanapathirane, Jeewani; et al. (CID Faculty Working Paper 332, 2017)

Explaining positive deviance in public sector reforms in development - Andrews, Matt (World Institute for Development Economic Research (UNU-WIDER), 2013)
Public sector reforms are commonplace in developing countries. Much of the literature about these reforms reflects on their failures. This paper asks about the successes and investigates which of two competing theories best explain why some reforms exhibi
Keywords: Development; Innovation; Reform; case survey; governance; growth; leadership

The Limits of Institutional Reform in Development: Changing Rules for Realistic Solutions - Andrews, Matt (Cambridge University Press, 2013)
Institutional reforms are common across the globe. Think of efforts to build new governments in Afghanistan and Iraq; or decades worth of interventions intended to improve fiscal management, reduce corruption or introduce efficient public sector service delivery in African countries.These reforms often have limited results, however. They lead to new laws that are not properly implemented, and new organizations that have poor capacities and fail to function as needed. In this book, Matt Andrews explains why reform results are frequently limited and suggests ways to overcome these limits. In the first half of the book, Andrews argues that reforms fail to make governments better when they are introduced as signals to gain short-term support--from donors and others. Reforms as signals introduce unrealistic best practices that do not fit developing country contexts and are not considered relevant by implementing agents. The result is a set of new forms that do not function properly. Andrews uses examples to prove this point, ranging from efforts to introduce fiscal rules in Argentina to reforms aimed at international accounting standard adoption in many African countries, and anti corruption interventions in Malawi and Uganda. In the second half of the book, Andrews notes that there are instances where reforms are not being introduced as signals, and are having more of an impact on government effectiveness. Examples include local government reforms in Rwanda, anti corruption initiatives in Indonesia, and a variety of initiatives ranging from results based management to civil service modernization and internal control regime adoption in governments like Kenya, Kosovo and Afghanistan. Andrews uses these examples to discuss ways in which reforms can actually provide realistic solutions to governance challenges in developing countries. Lessons from these experiences suggest that reform limits can be overcome by focusing interventions on problem solving, and promoting incremental and localized processes to find solutions, involving multiple agents who can authorize and implement reforms.

LAC MEL Specialists Use Peer Network to Improve Quality and Use of Evidence - Anderson, Todd M; Prevatt, Amy (CLA Case Competition, 2019)
Based on consultations and stocktakings with LAC Missions, the LAC Bureau identified Mission Monitoring, Evaluation and Learning (MEL) Specialists as influential decision makers regarding data and evidence use. The LAC Bureau aims to improve the quality and use of evidence for decision making, and ultimately to improve development outcomes.

Time to Listen: Hearing People on the Receiving End of International Aid - Anderson, Mary B.; Brown, Dayna; Jean, Isabella (CDA Collaborative Learning Projects, 2012)
Does the way international assistance is organized make sense? Is it working as we mean it to? This book approaches these questions through the experiences of people living on the receiving side of international assistance. It reports on the ideas, insights, and analyses of almost 6,000 people across 20 countries where international aid has been provided. From such a range of locations and people, one might expect vastly different ideas and opinions. However, remarkably consistent patterns and common judgments emerged. In the midst of difference, there was striking unanimity and consistency about the processes and the effects of the international aid system. Using their words, their experiences, and their ideas, this book describes why the cumulative impacts of international aid efforts have not met expectations. It describes a way forward to make changes that, according to those on the receiving end, will lead to more effective and lasting results.

Doing Accountability Differently: A 'Vertically Integrated' Approach - Amakom, Uzochukwu; Fashola, Temitope; Gay, Charles; Shutt, Cathy (Christian Aid, 2018)

Making humanitarian response more flexible: bibliography - ALNAP (2018)

An introduction to systems thinking and tools for systems thinking - Allen, Will; Kilvington, Margaret (Learning fro Sustainability, 2018)
Systems thinking is an approach to integration that is based on the belief that the component parts of a system will act differently when isolated from the system’s environment or other parts of the system. THIS PAGE PROVIDES PLENTY OF RESOURCES ON SYSTEM THINKING

Navigating complexity: adaptive management and organizational learning in a development project in Northern Uganda - Allana, Amir; Sparkman, Timothy (Knowledge Management for Development Journal, 2014)
Adaptive management is a management paradigm for intervening in complex, unpredictable systems where continual learning and adaptation is vital for success. This management approach requires a fundamentally different set of tools, processes, and most importantly, staff behaviors and organizational culture than ‘traditional’ management. A facilitative approach to development, where the goal is creating systemic change that spreads in networks of local businesses, government, and civil society organizations, necessitates an adaptive approach. Mercy Corps Uganda’s Northern Karamoja Growth, Health, and Governance Program (GHG) has been an ongoing experiment in applying the principles of adaptive management and facilitation. This article details two examples from GHG where adaptation has occurred, and provides an analysis of the tools, processes, and organizational culture that exists. Three salient takeaways for effective facilitation are elaborated on: the importance of staff behaviors and underlying beliefs, particularly with regards to ‘failure’, flexibility to experiment, dissent & debate, and curiosity with the subject matter of their work; importance of consistent messaging from senior management with regards to the same; and tools and processes playing a support function to these behaviours, rather than being their source. Lessons and implications are drawn out from the GHG experience for funders and implementing organizations wanting to apply adaptive management in the context of development programming. These include building flexibility into budgeting and contracts, rethinking the structure and content of reports, and utilizing alternative hiring criteria to attract senior managers who are more likely to succeed at adaptive management.

Navigating complexity: Adaptive management in the Northern Karamoja Growth, Health & Governance program - Allana, Amir (MercyCorps, 2014)
Development actors increasing agree that managing programs adaptively – especially complex interventions – can improve their effectiveness. But what does adaptive management look like in practice?
Keywords: IMPORTANT; Practice

Where have we got to on adaptive learning, thinking and working politically, doing development differently etc? Getting beyond the People’s Front of Judea - Algoso, Dave; Hudson, Alan (From Poverty to Power, 2016)
Dave Algoso and Alan Hudson at Global Integrity compare and contrast 9 different initiatives that are all heading in roughly the right direction in aid reform
Keywords: IMPORTANT

Adapting Aid: Lessons from six case studies - Algoso, Dave; Beloe, Jon; Hemberger, Alison; Hill, Philippa; Proud, Emma (MercyCorps & IRC, 2016)
Keywords: IMPORTANT

Eight Practices for Strategic Agility - Algoso, Dave (Stanford Social Innovation Review, 2019)
Rather than a glossy brochure that no one reads, your strategy should be an ongoing practice that informs your decisions and adapts as circumstances change. A Viewpoint from the Summer 2019 issue.

How INGOs are Doing Development Differently - Algoso, Dave (Care, IRC, Mercy Corps, Oxfam & World Vision, 2017)

More books on complexity than you can shake a stick at—plus a mea culpa - Algoso, Dave (Praxis, 2016)
Last month, Duncan Green was kind enough to post my overly ambitious multi-book review on complexity thinking in development on his From Poverty to Power blog. It covered three books: Ben Ramalingam’s Aid on the Edge of Chaos; Jean Boulton, Peter Allen, and Cliff Bowman’s Embracing Complexity; and Danny Burns and Stuart Worsley’s Navigating Complexity in International Development. It...

Positive deviance, big data, and development: A systematic literature review - Albanna, Basma; Heeks, Richard (The Electronic Journal of Information Systems in Developing Countries, 2019)
Positive deviance is a growing approach in international development that identifies those within a population who are outperforming their peers in some way, eg, children in low-income families who are well nourished when those around them are not. Analysing and then disseminating the behaviours and other factors underpinning positive deviance are demonstrably effective in delivering development results. However, positive deviance faces a number of challenges that are restricting its diffusion. In this paper, using a systematic literature review, we analyse the current state of positive deviance and the potential for big data to address the challenges facing positive deviance. From this, we evaluate the promise of “big data-based positive deviance”: This would analyse typical sources of big data in developing countries—mobile phone records, social media, remote sensing data, etc—to identify both positive deviants and the factors underpinning their superior performance. While big data cannot solve all the challenges facing positive deviance as a development tool, they could reduce time, cost, and effort; identify positive deviants in new or better ways; and enable positive deviance to break out of its current preoccupation with public health into domains such as agriculture, education, and urban planning. In turn, positive deviance could provide a new and systematic basis for extracting real-world development impacts from big data.
Keywords: big data; developing countries; machine learning; mobile data; positive deviance; systematic literature review

Politics matters, so what? Time for bigger bets (and more learning) on adaptive programming - Alan (www.globalintegrity.org, 2016)
By Alan Hudson, Executive Director, Global Integrity, July 26, 2016 Politics matters. Context too. And blueprints have limited value. Our strategy is based on these insights, so we’re totally on board. A World Development Report (WDR) that puts power and politics...

Try, learn, adapt, repeat: T/A Learn reflections - Alan (www.globalintegrity.org, 2015)
By Alan Hudson — November 16, 2015. I spent last week in Rio de Janeiro (tough assignment, I know), participating in the Transparency and Accountability Initiative’s third T/A Learn Annual Workshop. As the report of the second Annual Workshop, held...

Afterword: Alabert Hirschman observed - Alacevich, Michele (Development Projects Observed, 2015)
Albert Hirschman considered Development Projects Observed a natural sequel to his earlier work. As Hirschman put it to an old acquaintance at the World Bank, his previous books identified the inner and often hidden mechanisms of development sequences: “Having worked out a few basic hypotheses …, I could perhaps test them (and hit on some new ones) by looking at Bank-financed projects that have had enough time to give rise to such sequences.”¹ But as much as it appeared as the logical evolution of Hirschman’s intellectual trajectory, this new research was also the offshoot of an important crisis in development

Is (Web) Science Ready for Empowerment? - Akkermans, Hans; Gyan, Nana Baah; Bon, Anna; Tuyp, Wendelien; Grewal, Aman; Boyera, Stéphane; Allen, Mary (ACM WebSci'11, 2011)

Beyond Tweets and Screams: Action for Empowerment and Accountability in Nigeria – The Case of the #BBOG Movement - Aina, Tade Akin; Atela, Martin; Ojebode, Ayo; Dayil, Plangsat; Aremu, Fatai (IDS, 2019)
This paper explores the nature, role and dynamics of new forms of social and political action as pathways to empowerment and accountability in fragile conflict- and violence-affected settings in Africa. Through an in-depth analysis of the case of the Bring Back Our Girls (#BBOG) movement in Nigeria and a multi-methods approach, the paper provides new knowledge that addresses evidence gaps in the following areas: (1) the multiple ways through which social and political action play out in fragile, conflict- and violence-affected settings; (2) whether the conditions in which new forms of social and political action applicable to fragile, conflict- and violence-affected settings – the settings most dominant in African countries – emerge as currently projected in the literature; and (3) whether these social and political actions necessarily produce accountability and empowerment in fragile, conflict- and violence-affected settings. Although often expressed as contentious and/or unruly politics, experiences from the BBOG movement suggest that the new forms of social and political action possess a wide range of implications for citizen action and governance, including leading to multiple forms of empowerment in fragile settings.

’Adaptive Authoritarianism’ in Contemporary China: Identifying Zones of Legitimacy Building - Ahlers, Anna Lisa; Schubert, Gunter (Reviving Legitimacy: Lessons for and from China, 2011)
Anna L. Ahlers and Gunter Schubert, ’Adaptive Authoritarianism’ in Contemporary China: Identifying Zones of Legitimacy Building, in: Deng Zhenglai and Guo Sujian (eds), Reviving Legitimacy: Lessons for and from China, Lanham: Lexington Books (2011),

Agile Glossary and Terminology - Agile Alliance (Agile Alliance, 2015)
Learn the unique terminology used in Agile development from the experts at Agile Alliance.

Subway Map to Agile Practices - Agile Alliance (Agile Alliance, 2015)
AGILE 101

Can policy-makers think like designers? - Adler, Natalia (A workshop on Doing Development Differently, 2014)
This is the poster used by Natalia Adler, from Unicef, at the Doing Development Differently workshop in 2014. Can policy-makers think like designers? Since 2012, UNICEF has been supporting two autonomous governments in Nicaragua to develop empathy-driven policies for children. While policies are the final product, the goal of the Designing for Children Initiative is to make government officials think like designers. A good policy or service is nothing without active and engaged frontline workers. From the onset, our objective was to transform public employees into ‘public entrepreneurs.’ In doing so, we have disrupted the traditional assumptions about participatory governance, where it normally suffices to hold a meeting and have people talk about their needs and aspirations. For participation to work, government officials have to relearn how to listen. They have to let go of pre-determined assumptions and their expert status, and put themselves in the shoes of the people they are trying to reach. This is the process of empathy-driven policy-making. For the past two years, we have used a Human Centered Design (HCD) approach to make this happen. While HCD is often seen as an innovation in itself, for us, it’s just a means to an end: common-sense development.

+Acumen MOOC: Adaptive Leadership: Mobilizing for Change and Disrupting the Status Quo - +Acumen (NovoEd, 2017)
This four-module course by +Acumen and Adaptive Change Advisors on adaptive leadership, which is a practical leadership framework developed at Harvard Kennedy School for driving systemic change, particularly during times of uncertainty or when there are no easy answers.

Lean Data Approaches to Measure Social Impact - ACUMEN (NovoEd, 2018)

Alps - Accountability, Learning and Planning System - 2011 Update - ActionAid (ActionAid International, 2011)
Alps is ActionAid’s over arching accountability framework, containing within it our programme planning system. Alps is distinctive in that it is strongly driven by principles, and sets out necessary personal attitudes and behaviours alongside organisational processes for planning, strategy formulation, learning, monitoring reviews/evaluations and audit. Alps defines our standards, not only about what we do but also how we do it. Alps is part of ActionAid’s human rights-based work.

Getting strategic: vertically integrated approaches - Aceron, Joy; Isaac, Francis (Institute of Development Studies, 2016)
Holding power to account requires understanding where power lies and how it is exercised. It entails understanding how decisions are made, who makes them and what decision criteria are used to make them. Vertically integrated civil society action takes into account how power is exercised and how decisions are made in a given policy, programme or process.

Going vertical: citizen-led reform campaigns in the Philippines - Aceron, Joy; Isaac, Francis (Making All Voices Count Research Report, 2016)
The Philippines has a long history of state–society engagement to introduce reforms in government and politics. Forces from civil society and social movements have interfaced with reform-oriented leaders in government on a range of social accountability initiatives – to make governance more responsive, to introduce policy reforms, and to make government more accountable. Several theoretical propositions on which strategic approaches work best for social accountability initiatives have been put forward – including the idea of vertically integrated civil society monitoring and advocacy. This multi-authored research report uses vertical integration as a framework for examining seven successful civil society social accountability initiatives in the Philippines, looking at what made them successful, and how the gains they realised can be deepened and sustained.

Thinking big: How to use theory of change for systems change - Abercrombie, Rob; Boswell, Katie; Thomasoo, Rosanna (NPC, 2018)
We think that, applied well, theory of change can support charities and funders to take a systemic approach to their work. This report identifies five common pitfalls that organisations fall into when using theory of change, and walks through five rules of thumb that will help organisations to use the approach to tackle complex problems. We think that, applied well, theory of change can support charities and funders to take a systemic approach to their work. This report identifies five common pitfalls that organisations fall into when using theory of change, and walks through five rules of thumb that will help organisations to use the approach to tackle complex problems.

18F Method Cards - A collection of tools to bring human-centered design into your project - 18F-GSA (18F-GSA, US Governement, 2017)
18F’s method cards describe how our organization puts human-centered design into practice.

About the BEAM Exchange (BEAM Exchange, 2018)

Adaptive management in aid programmes (BEAM Exchange, 2014)
Aid works better – especially in complex and conflict-affected environments – when there is scope to trial and adapt programme strategies. So argues Amir Allana in Navigating Complexity, a case study of Northern Karamoja's Growth, Health and Governance (GHG) Programme that is adopting an 'adaptive management' approach. This BEAM Exchange webinar invited Amir, Tim Sparkman and Peter Roggekamp to discuss lessons from their work in Uganda and Cambodia. What does adaptive management look like in practice? And what does it require of managers and funders to make it happen? Tim and Peter share their experiences of applying adaptive management techniques to the Growth, Health and Governance Programme and the Cambodia Agricultural Value Chain Program (CAVAC). Amir highlight insights from the Navigating Complexity report, including the importance of office culture, consistent management signals, and the role of supportive tools and processes.

Book Review: Branko Milanovic's brilliant take on Global Inequality (From Poverty to Power, 2016)
Some of my favourite development economists are nomads, people with feet in different regions, which seems to make them better able to identify interesting patterns and similarities/differences bet...

Busan Outcome Document (2011)

Collaboration for Impact (2018)
The site seeks to accelerate the adoption of system collaboration and Collective Impact in Australia. It does so by creating knowledge, network and tools; shining a light on successful Collective Impact initiatives; and building an Australian community of practice.

Conflict and Human Rights Shareweb (2019)

Donor Committee for Enterprise Development (DCED, 2018)
The DCED is a forum for learning about the most effective ways to fulfil the SDGs – based on practical experience in Private Sector Development (PSD).

Feedback Commons (Keystone Accountability, 2018)

Fostering Learning and Effectiveness through Collaboration: A New Collaborative on Transparency, Accountability, and Participation (TAI, 2018)
What are the features, values, and practices of effective learning organizations? How do learning practices contribute to more effective programming? And, how can collaborations between academics, researchers and practitioners better support learning organizations in the global South?  These are just a few of the questions that a new global learning collaborative seeks to explore. In …

Making All Voices Count's Research & Evidence Strategy (2014)
Building an evidence-base on what works in technology for voice, transparency and accountability, how, and why.

mHero and the Principles for Digital Development: Development Done Right | iHRIS (2016)

MIRADI: Adaptive Management Software for Conservation Projects (2018)

Monitoring and Measuring Results in Private Sector Development (Swiss Agency for Development and Cooperation SDC, 2016)

Nairobi Outcome Document (2016)

NetworkWeaver (NetworkWeaver, 2018)
Weaving Smart Networks

Positive Deviance Initiative (2016)

Swiss Programme for Research on Global Issues for Development (r4d programme) - NFP [Nr.] (2018)
Research and innovation are decisive factors for sustainable global development. With their knowledge, skills, experience and worldwide network of partners, researchers can make an important contribution towards the solution of global problems.

Transparency, Participation, & Accountability Grantmaking Strategy (THE WILLIAM AND FLORA HEWLETT FOUNDATION, 2015)

TWP CoP (Thinking and Working Politically - Community of Practice, 2016)

Women and power: overcoming barriers to leadership ODI women and power coverand influence (ODI, 2016)
ODI have just wrapped up an excellent two year project on ‘Women and power: overcoming barriers to leadership and influence’ with a final synthesis report that is well worth reading. It’s an intell...