Linking the prepositions: using power analysis to inform strategies for social action - Gaventa, John (Journal of Political Power, 2021)
This article reviews longstanding debates about the relationship between power over and power to – often posed as the tension between domination and emancipation. It then turns to several frameworks which integrate these approaches to inform strategies for social action. In particular, it focuses on recent empirical studies which apply one such framework, the ‘powercube’, to glean insights into how social actors navigate across multiple forms, spaces and levels of power. In so doing, we gain clues into how relatively powerless groups develop the capacities for agency and action which challenge domination and in turn give new possibilities for emancipation.
Keywords: Lukes; Power; agency; collective action; empowerment

Participation Makes A Difference: But Not Always How And Where We Might Expect - Gaventa, John (Development Outreach, 2011)

Mapping the Outcomes of Citizen Engagement - Gaventa, John; Barrett, Gregory (World Development, 2012)
Summary Despite the normative beliefs that underpin the concept of participation, its impact on improved democratic, and developmental outcomes has proven difficult to assess. Using a meta-case study analysis of a sample of 100 cases, we inductively create a typology of four democratic and developmental outcomes, including (a) the construction of citizenship, (b) the strengthening of practices of participation, (c) the strengthening of responsive and accountable states, and (d) the development of inclusive and cohesive societies. We find that citizen participation produces positive effects across these outcome types, though in each category there are also important types of negative outcomes as well.
Keywords: Citizen engagement; Citizenship; Development outcomes; Participation; democratic outcomes; meta-analysis

Challenging the Boundaries of the Possible: Participation, Knowledge and Power - Gaventa, John; Cornwall, Andrea (IDS Bulletin, 2006)

Introduction: Making change happen–citizen action and national policy reform - Gaventa, John; McGee, Rosemary (Citizen Action and National Policy Reform. London: Zed Books, 2010)

The Impact of Transparency and Accountability Initiatives - Gaventa, John; McGee, Rosemary (Development Policy Review, 2013)

Empowerment and Accountability in difficult settings: What are we learning? - Gaventa, John; Oswald, Katy (Key messages emerging from theAction for Empowerment and Accountability Programme, 2019)
Empowerment and Accountability in Difficult Settings: What Are We Learning? Key Messages Emerging from the Action for Empowerment and Accountability Programme Empowerment and accountability have long been part of the international development vocabulary and a core part of governance, social development and civil society programmes. Yet, much of what has been learnt about these approaches has been drawn from studies in somewhat stable, open and middle-income places around the world. Less is known about how empowerment and accountability are achieved through social and political action in more difficult settings – those faced by institutional fragility, conflict, violence, and closing civic space. This document highlights key messages emerging from the work of the Action for Empowerment and Accountability Research Programme (A4EA), and the implications for how donors, policy makers and practitioners support strategies for empowerment and accountability in fragile, conflict and violence affected settings (FCVAS). Our eight key messages have strong implications for the theories of change used for effective programming in the field.

Everyday patterns for shifting systems - GCSI (GCSI, 2022)

Everyday patterns for shifting systems - Right scaling - GCSI (GCSI, 2022)

Global Delivery at the World Bank Group - GDI (Global Delivery Initiative, 2014)

Typology of sociotechnical transition pathways - Geels, Frank W.; Schot, Johan (Research Policy, 2007)
Contributing to debates about transitions and system changes, this article has two aims. First, it uses criticisms on the multi-level perspective as stepping stones for further conceptual reﬁnements. Second, it develops a typology of four transition pathways: transformation, reconﬁguration, technological substitution, and de-alignment and re-alignment. These pathways differ in combinations of timing and nature of multi-level interactions. They are illustrated with historical examples.

Advancing urban adaptation where it counts: reshaping unequal knowledge and resource diffusion in networked Indonesian cities - Geldin, Samuel (Environment and Urbanization, 2019)
Climate adaptation literature vocalizes the need for transnational municipal networks (TMNs) to expand activities in vulnerable medium-sized cities, but little work has examined the granular extent of city participation and processes constraining TMN growth. This study explores the effectiveness of TMNs in reaching adaptation outcomes and how financial, material, and knowledge exchanges of TMNs tend to exclude adaptation in high-priority intermediary cities. Nearly 40 semi-structured interviews with Indonesian city actors and a preliminary catalogue of cities participating in TMNs reveal that risk-averse selection criteria, insufficient impact assessments, and duplicative institutional efforts reinforce disparities between primary and intermediary cities. To effectively build adaptive capacity in the most vulnerable regions, TMNs should remove participation barriers for intermediary cities, improve incentives for institutional collaboration, and adopt more rigorous evaluative metrics. These results directly inform the governance, resource allocation, and operational goals of TMN stakeholders to advance distributive climate justice.

The Social Realities of Knowledge for Development - Georgalakis, James; Jessani, Nasreen; Oronje, Rose; Ramalingam, Ben (IDS, 2017)
This edited collection of peer-reviewed papers explores critical challenges faced by organisations and individuals involved in evidence-informed development through a diverse set of case studies and t...

Coping with Complexity - Gerard Seijts; Niels Billou; Mary Crossan (Ivey Business Journal, 2010)

Meant to Keep Malaria Out, Mosquito Nets Are Used to Haul Fish In - Gettleman, Jeffrey (The New York Times, 2015)
Nets like his are widely considered a magic bullet against malaria — one of the cheapest and most effective ways to stop a disease that kills at least half a million Africans each year. But Mr. Ndefi and countless others are not using their mosquito nets as global health experts have intended. Nobody in his hut, including his seven children, sleeps under a net at night. Instead, Mr. Ndefi has taken his family’s supply of anti-malaria nets and sewn them together into a gigantic sieve that he uses to drag the bottom of the swamp ponds, sweeping up all sorts of life: baby catfish, banded tilapia, tiny mouthbrooders, orange fish eggs, water bugs and the occasional green frog

Producer-led value chain analysis: The missing link in value chain development - Ghore, Yogesh (Coady, 2015)
Introduction and rationale The concept of a value chain is increasingly being applied in the design and implementation of development programs aimed at poverty reduction. As an analytical tool, it provides a useful framework for understanding key activities, relationships, and mechanisms that allow producers, processors, buyers, sellers, and consumers—separated by time and space—to gradually add value to products and services as they pass from one link of the chain to another, making it a “value chain” (UNIDO, 2009). While it has been popular in the private sector ever since it was conceptualized by Michael Porter in the 1980s, more recently various donors and governments have shown interest in its use and have applied it to a range of development interventions, particularly in the area of sector development, livelihoods promotion, small and medium enterprise (SME) development, and rural and economic development. Academics and development organizations have designed numerous instruments for value chain analysis (VCA) and implementation. Beginning in the early 2000s, international organizations and donor agencies have sponsored the development of these tools, including the United States Agency for International Development (USAID) Deutsche Gesellschaft für Internationale Zusammenarbeit (GIZ), the World Bank, the International Finance Corporation (IFC), and the International Labour Organization (ILO). These guides and tools have been used in many development programs to address organizational, donor and local and regional priorities for development. While VCA guides and tools are important in understanding markets and relationships among key stakeholders, a recent comparative review of popularly used guides by Donovan et al. (2013) suggests that most of these guides are designed to be implemented independently of the local context and do not sufficiently focus on mutual learning, whether related to tool design or to the outcomes and impacts of the designed project activities. VCA is often conducted by external experts and the knowledge generated in the process is often confined to reports. Without the capacity building and effective participation of women and men producers—smallholder farmers who hold critical knowledge about the local context—an important link is missing in the entire process of VCA. The insufficient attention to the human, social, and other contextual factors undermines the full potential of the value chain systems approach, not only for economic outcomes, but for the long term sustainability of the intended benefits. So what are the ways to effectively engage producers in the community in the VCA? The producer-led process described here tries to address this basic question. It introduces simplified tools and an approach that ensures farmer participation in data collection, analysis, and identification of opportunities and constraints, and design of value chain interventions. This participation requires: a) an environment in which they feel comfortable to share their knowledge and insights, which is often not the case when extractive surveys and questionnaires are administered to collect information; and b) the use of simple and participatory tools that will allow them to provide inputs into the process of VCA and also help them to understand complex value chain systems and use this understanding for making livelihood decisions. The key steps and tools described are aimed at empowering the women and men farmers to make informed decisions about their own enterprises and how they relate to the value chain, thereby directly contributing to, and influencing, the overall process of value chain development. The process of VCA involving these steps and tools was developed at Coady Institute and first tested with Oxfam Canada and its local partners in Ethiopia in August 2012.

The Samaritan's Dilemma: The Political Economy of Development Aid - Gibson, Clark C.; Andersson, Krister; Ostrom, Elinor; Shivakumar, Sujai (Oxford University Press, 2005)
What's wrong with foreign aid? Many policymakers, aid practitioners, and scholars have called into question its ability to increase economic growth, alleviate poverty, or promote social development. At the macro level, only tenuous links between development aid and improved living conditions have been found. At the micro level, only a few programs outlast donor support and even fewer appear to achieve lasting improvements. The authors of this book argue that much of aid's failure is related to the institutions that structure its delivery. These institutions govern the complex relationships between the main actors in the aid delivery system and often generate a series of perverse incentives that promote inefficient and unsustainable outcomes. In their analysis, the authors apply the theoretical insights of the new institutional economics to several settings. First, they investigate the institutions of Sida, the Swedish aid agency, to analyze how that aid agency's institutions can produce incentives inimical to desired outcomes, contrary to the desires of its own staff. Second, the authors use cases from India, a country with low aid dependence, and Zambia, a country with high aid dependence, to explore how institutions on the ground in recipient countries also mediate the effectiveness of aid. Throughout the book, the authors offer suggestions about how to improve aid's effectiveness. These suggestions include how to structure evaluations in order to improve outcomes, how to employ agency staff to gain from their on-the-ground experience, and how to engage stakeholders as "owners" in the design, resource mobilization, learning, and evaluation processes of development assistance programs.

One step to a thousand miles: building accountability in Liberia - Gilberds, Heather (Accountability Lab Liberia, 2017)
Lessons for enabling accountability and integrity in Liberia

Exploring the potential for interactive radio to improve accountability and responsiveness to small-scale farmers in Tanzania - Gilberds, Heather; Handforth, Calum; Leclair, Mark (IDS, 2016)
What are the possibilities of using new digital technologies alongside radio to help ensure that agricultural development projects are farmer-centred, and meet the needs of the rural citizens they intend to serve? This research assesses Farm Radio International’s Listening Post – a model that combines radio and digital technologies with the aim of collecting and aggregating farmer feedback to aid decision-making and adaptive project implementation. The research shows that linking a mobile-based crowd-sourcing tool with radio is effective at ensuring engagement from a large number of farmers, who felt it was a useful way of raising their questions and concerns to NGOs, policy-makers and experts. The model has clear potential to strengthen the chain of relationships between citizens, extension services suppliers, projects and policymakers. It has also demonstrated its potential to collect real-time feedback from farmers that could be used to aid decision-making and improve accountability in agricultural development initiatives, helping to ensure they are more responsive to farmers. However, it also concludes that ‘closing the feedback loop’ – ensuring that farmer’s comments, questions and concerns are responded to – is a challenge for the Listening Post. Sometimes, the interactive radio programmes close the loop by disseminating answers to questions or concerns raised by farmers, or by connecting farmers to extension services – but only when a solid and systematic process had been developed for this to happen. Therefore is critical to identify and incentivise actors who are capable of responding during the design phase of a Listening Post, and to ensure that they are involved in every stage of the process. It also important that project partners who are interested in beneficiary feedback identify the flexible or actionable points in their project frameworks from the outset, rather than collecting data that they are not able to use to adapt their programmes

Capacity WORKS - GIZ (2014)
One of GIZ’s core competencies is understanding how cooperation works in societies, and supporting that process. The kind of process we mean is cooperation between state, civil society and private-sector actors who wish to jointly shape societal changes. GIZ has systematised its knowledge on how to achieve this, and made it available in the Capacity WORKS management model. The model describes how to manage cooperation systems.

Capacity Works - online training - GIZ (GIZ, 2014)

Cooperation Management for Practitioners: Managing Social Change With Capacity Works - GIZ (Springer Gabler, 2015)

GIZ's Evaluation Policy: Principles, guidelines and requirements of our evaluation practice - GIZ (GIZ, 2018)

Toolkit Digitalisation: Tools to Support Strategic Planning and Implementation - GIZ (2017)

GIZ’s evaluation system - GIZ Evaluation Unit (GIZ GmbH, 2018)
This provides a general description of GIZ’s new evaluation system, which is designed to implement our new policy and to achieve the objectives of evaluation reform. This general description is accompanied by two other documents that provide supplementary details of GIZ’s key evaluation instruments: project evaluations for BMZ business and corporate strategic evaluations. They are geared in particular to GIZ staff members who commission, implement or support evaluations and to evaluators who conduct evaluations on behalf of GIZ as well as repre-sentatives of commissioning parties, clients and cooperation partners. The current versions are very much a ‘work in progress’ in many regards. We are well aware that we will only be able to meet our own high expectations of our evaluations if we continue to develop their quality in dialogue with project staff, our commissioning parties, clients and partners and with evaluation practitioners and experts from the academic, scientific and research community. We look forward to taking on this task and to sharing our experience with all interested parties.

GLAM Monitoring and Learning Strategy (internal document) - GLAM (ODI, 2018)

Making All Voices Count: A Slam Dunk? - Global Integrity (Global Integrity, 2012)
Yesterday, I had the chance to attend the public launch of a new government transparency and accountability funding mechanism – Making All Voices Count. Held at USAID headquarters, the discussion featured a veritable who’s who of open government and transparency practitioners in the Washington area; probably 250 people were packed into the room. (Announcing a …

Assessing Innovation Potential for Social Impact: Overview for Social Sector Leaders - Global Knowledge Initiative (2016)

Overview for Social Sector Leaders: Assessing Innovation Potential for Social Impact - Global Knowledge Initiative (2016)
We aim to provide decision makers with greater insight and confidence into the process of assessing innovation impact potential. Rather than considering the role innovation plays after an investment is made, or based on historic evaluations of how innovation has or has not delivered solutions to a problem, this approach is forward-looking. This customizable toolset assesses the future impact that innovation can deliver in a system to tackle particularly complex problems

Appreciative Inquiry: Three decades of generative impact - Godwin, Lindsey N. (AI Practitioner, 2016)

NGO-IDEAs Impact Toolbox - Participatory Monitoring of Outcome and Impact - Gohl, Eberhard; Causemann, Bernward (Impact+, 2011)
NGO-IDEAs (NGO – Impact on Development, Empowerment and Actions) NGO-IDEAs is a cooperation of about 40 non-govermental organisations (NGOs) from South Asia, East Africa and the Philippines and 14 German NGOs working in the field of development cooperation. It identifies and develops jointly with all partners, concepts and tools for NGOs in the areas of Outcome and Impact Assessment and Monitoring & Evaluation (M&E). NGO-IDEAs is further being supported by VENRO, the umbrella organisation of development non-governmental organisations in Germany as well as PARITÄT, the legal holder of the project. The Federal Ministry for Economic Cooperation and Development (BMZ) has cofinanced the project. NGO-IDEAs is not just another study evaluating the impact of NGOs’ work – it combines research & development, knowledge management, learning & training as well as advice & coaching to initiate a collective learning process for all partners involved. Additionally, NGOIDEAs intends to create a valuable resource base for use by NGOs. NGO-IDEAs aims at: • Empowering community based organisations or groups and the poor among the rural communities to use and practice impact monitoring for project management • Empowering NGOs to further improve the effectiveness, impact and sustainability of their efforts • Making social changes more visible for implementing and funding NGOs, thus improving development practice • Improving public recognition of NGOs and CBOs and their contribution to development. The NGO-IDEAs “Impact Toolbox” is to enable NGOs and grassroots organisations to monitor projects together with the so called target groups involved, in a manner that will enhance positive outcomes and impacts, and reduce negative ones. It focuses on joint setting of goals, on monitoring them and finally on taking joint decisions about the further design and direction of interventions. The instruments of the NGO-IDEAs “Impact Toolbox” are simple and participatory. Simple means: setting out from people’s knowledge and know-how, therefore, easy to understand and apply. Application can easily be fitted into the “normal” activities of the NGOs or grassroots organisations. The participatory character emerges through democratic elements promoting a “Culture of Learning” that the people can assimilate

Adaptive Evaluation: A Complexity-based approach to Systematic Learning for Innovation and Scaling in Development - Gokhale, Siddhant; Walton, Michael (CID Faculty Working Paper, 2023)
Nearly all challenges in international development tend to be complex because they depend on constantly evolving human behaviour, systems, and contexts, involving multiple actors, entities, and processes. As a result, both the discovery and scaling of innovations to address challenges in development often involve changes in system behaviour or even system-level transformation. This is rarely a linear process over time and can result in unexpected outcomes. Existing evaluation techniques commonly used in international development, including Randomized Control Trials (RCT) and quasi-experimental methods, are good at assessing specific effects of interventions but are not designed for the change processes inherent to innovation and scaling within a system. There is a need to reconstruct how we use existing measurement tools, techniques, and methodologies so that they capture the complexity of the environment in which an intervention or change occurs. We introduce Adaptive Evaluation, designed to learn at various levels of complexity while supporting the transformation needed to foster sustainable change. An Adaptive Evaluation uses three main approaches to work with complex questions—systems diagnosis, theorybased assessment of change processes, and iterative designs. An Adaptive Evaluation typically builds hypotheses from field-based interactions, emphasizes learning over testing, advocates open-mindedness with techniques, and appreciates the value of dialogue and participation in navigating complex processes. It can use RCT or similar techniques to analyse specific processes within a system or a development cycle, but these are embedded in a broader approach to assessment and interpretation. It is designed to be flexible and adjust to shifting contexts. Finally, an Adaptive Evaluation can be applied at any stage in a complex intervention's lifecycle, from the interpretation of the system and change processes to rapid experimentation, prototyping, and testing of select interventions, and then adaptation to different settings for impact at scale. This paper provides the theoretical basis for an Adaptive Evaluation—the main approaches, core ideology, process, and applications.

The Innovative Power of Positive Deviance - Goldstein, Jeffrey; Hazy, James; Lichtenstein, Benyamin B. (Complexity and the Nexus of Leadership, 2010)

Monitoring and Evaluating Conflict Sensitivity: Methodological Challenges and Practical Solutions - Goldwyn, Rachel; Chigas, Diana (DFID, 2013)

Integrating Livelihood and Conservation Goals: A retrospective Analysis of World Bank Projects - Gómez, Andrés; Price, Claire (Measuring Impact, 2017)
Using a database of World Bank project evaluations, E3/FAB’s new study called “Integrating Livelihood and Conservation Goals: A Retrospective Analysis of World Bank Projects” aims to contribute to building the evidence base around the integration of biodiversity conservation and livelihood goals.

Operationalizing the Science of Delivery Agenda to Enhance Development Results - Gonzalez Asis, Maria; Woolcock, Michael (Washington, DC: World Bank, 2015)
The clear development gains achieved in recent decades should not deflect attention from the scale and type of challenges that remain. The strategies largely responsible for these initial gains have been technical reforms promoting economic growth and logistical systems supplying basic inputs. Today, strategies are needed that focus on enhancing the quality of implementation— for example, ensuring learning and not just building schools and enrolling students. This concern now spans numerous domains of professional practice (especially health) and has entered World Bank discussions framed as the “science of delivery.” At the World Bank, the Global Delivery Initiative (GDI) is an operational manifestation and extension of these ideas. To date, the GDI has prepared a number of different case studies across numerous sectors on ways in which innovative teams solve particular problems during project implementation. On the basis of the initial case studies, the authors outline five key principles of how high-quality implementation occurs and invite others to add to this growing storehouse of knowledge. Specifically, task teams are encouraged to develop “live” case studies by and for their staff, documenting how, in real time, implementation challenges are being met. Projects must “learn” more rapidly and systematically how to solve the myriad range of complex implementation challenges they inevitably encounter, since most of these (by definition) cannot be anticipated ex ante. Delivery challenges of this kind will only intensify in the coming years as citizens demand effective responses to ever-more complex—and contentious—policy domains, such as justice, regulation, and taxation.

Opportunities for Using Complexity-Aware Approaches to Theory of Change - Goodier, Sarah; Apgar, Marina (SDC-IDS Briefing Note 8, 2018)
The purpose of this briefing note is to review opportunities for using complexity aware approaches to Theory of Change (ToC) to inform the SDC approach. It provides an overview of complexity-aware approaches and then focuses on demonstrating how complexity thinking can support programming by building on the frameworks currently being used in the project/programme cycle management (PCM) processes. It is aimed at SDC staff, in particular Programme Officers and staff of partner organisations involved in the management of SDC interventions.

State of the Art on Use of Theory of Change in the Development Sector - Goodier, Sarah; Apgar, Marina; Clark, Louise (SDC-IDS Briefing Note 7, 2018)
The purpose of this briefing note is to add to SDC’s understanding of Theory of Change (ToC), drawing on the literature and practice to sketch out the current state of the art approach. This involves expanding on ToC beyond SDC’s current practice of using Impact Hypotheses (IH) to bridge it to operational practice and use ToC more explicitly in the project/programme cycle management (PCM) processes. Sharing the state of the art on use of ToC in the development sector, this briefing note outlines what a ToC is, what it is used for and why it is needed in the development sector. It discusses ToC as both a process and a product, providing step by step guidance on how to facilitate a ToC process. The differences between a ToC and a logframe are highlighted. Some key criteria for recognising when you have a ‘good’ ToC are also included. This brief is aimed at SDC staff, in particular Programme Officers, and staff of partner organisations involved in the management of SDC interventions.

Fostering Systems Change - Gopal, Srik; Kania, John (2015)
Five simple rules for foundations seeking to create lasting social change.

Reflection methods: Tools to make learning more meaningful - Gordijn, Femke; Eernstman, Natalia; Helder, Jan; Brouwer, Herman (Wageningen Centre for Development Innovation, 2018)
This handbook summarises methods that can be used to facilitate the process of reflection on the knowledge and experiences people acquire during a capacity development trajectory or training event. We believe that by explicitly integrating reflection in the learning process the learning will become clearer and better articulated and will contribute more strongly to meaningful change. Therefore we advise facilitators to deliberately include reflective learning sessions in their process design and implementation. This handbook can inspire you to do so and provides many methods which help to facilitate this.

Agile vs Lean vs Design Thinking - Gothelf, Jeff (Medium, 2016)
There’s an unforgettable scene in my favorite movie, Goodfellas, where Joe Pesci, Robert DeNiro and Ray Liotta pay a late night visit to…

Lean vs Agile vs Design Thinking: What you really need to know to build high-performing digital product teams - Gothelf, Jeff (CreateSpace, 2017)
As companies evolve to adopt, integrate and leverage software as the defining element of their success in the 21st century, a rash of processes and methodologies are vying for their product teams' attention. In the worst of cases, each discipline on these teams -- product management, design and software engineering -- learn a different model. This short, tactical book reconciles the perceived differences in Lean Startup, Design Thinking and Agile software development by focusing not on rituals and practices but on the values that underpin all 3 methods.

Sense and Respond: How Successful Organizations Listen to Customers and Create New Products Continuously - Gothelf, Jeff; Seiden, Josh (Harvard Business Review Press, 2017)
The End of Assembly Line ManagementWe’re in the midst of a revolution. Quantum leaps in technology are enabling organizations to observe and measure people’s behavior in real time, communicate internally at extraordinary speed, and innovate continuously. These new, software-driven technologies are transforming the way companies interact with their customers, employees, and other stakeholders.This is no mere tech issue. The transformation requires a complete rethinking of the way we organize and manage work. And, as software becomes ever more integrated into every product and service, making this big shift is quickly becoming the key operational challenge for businesses of all kinds. We need a management model that doesn’t merely account for, but actually embraces, continuous change. Yet the truth is, most organizations continue to rely on outmoded, industrial-era operational models. They structure their teams, manage their people, and evolve their organizational cultures the way they always have.Now, organizations are emerging, and thriving, based on their capacity to sense and respond instantly to customer and employee behaviors. In Sense and Respond, Jeff Gothelf and Josh Seiden, leading tech experts and founders of the global Lean UX movement, vividly show how these companies operate, highlighting the new mindset and skills needed to lead and manage them—and to continuously innovate within them.In illuminating and instructive business examples, you’ll see organizations with distinctively new operating principles: shifting from managing outputs to what the authors call “outcome-focused management”; forming self-guided teams that can read and react to a fast-changing environment; creating a learning-all-the-time culture that can understand and respond to new customer behaviors and the data they generate; and finally, developing in everyone at the company the new universal skills of customer listening, assessment, and response.This engaging and practical book provides the crucial new operational and management model to help you and your organization win in a world of continuous change.

Open Policy Making toolkit - gov.uk (2016)
This manual includes information about Open Policy Making as well as the tools and techniques policy makers can use to create more open and user led policy.

Open Policy Making toolkit - Low cost tools - GOV.UK (2016)

Building a team culture for Adaptive Management in MSD: 5 Strategies MEL Managers Say Work - Gover, Dun; Nasution, Zulka; Okutu, David; Bolder, Meghan; Henao, Lina (MSD in MEL Brief, 2023)
Keywords: Bolder Meghan; Gover Dun; Henao Lina; Nasution Zulka; Okutu David

Enhancing partner and system-level learning: 8 Tips from MEL Managers - Gover, Dun; Nasution, Zulka; Okutu, David; Bolder, Meghan; Henao, Lina (MSD in MEL Brief, 2023)
Effective learning is a key driver of market systems change, with the potential to enhance system competitiveness, resilience, and inclusiveness. Shifting the Locus of Learning: Catalyzing Private Sector Learning to Drive Systemic Change recently outlined a rationale for enhancing the scale and quality of learning in a system and identifying 10 strategies programs can contextualize to catalyze learning. These strategies are also backed with robust examples from 13 programs doing this work across 11 countries. To deepen insights on what MSD Monitoring, Evaluation and Learning (MEL) Managers have experienced in putting several of those strategies into practice, the Feed the Future Market Systems and Partnerships (MSP) Activity convened a series of peer discussions as part of a larger initiative (see Figure 1). This brief shares the collective learning and experience on this topic of three senior MEL Managers who were interested in and had experience with this topic. The group represented full-time, program-based MEL Leads working on MSD programs funded by USAID and DFAT, based in Fiji, Albania, and Kosovo working for Adam Smith International, SwissContact, and DT Global, respectively. From those discussions, this paper synthesizes eight tips from MEL Managers for practically enhancing partner and system-level learning: 1. Identify the right decision-maker(s) at potential partners. 2. Use a co-creation process to identify learning opportunities. 3. Use diagnostics and assessments to strengthen partner and system capacity for actionable learning. 4. Use a phased capacity strengthening process tied to behavior change. 5. Measure partners’ continued investment in and use of learning—not the continuation of specific learning activities. 6. Work with sector-level institutions for scale but be aware of risks. 7. Leverage informal communities of practice to share learning. 8. Use the right terminology to talk about partner and system-focused learning.

Practioners Guide to Assessing Systems Change - Gover, Dun; Nasution, Zulka; Okutu, David; Bolder, Meghan; Henao, Lina (MSD in MEL Brief, 2023)

Practitioners Guidance to Assessing Systems Change: Co-Authors Preview - Gover, Dun; Nasution, Zulka; Okutu, David; Bolder, Meghan; Henao, Lina (USAID, 2023)
Check out this video to see what’s inside our new resource: Practitioners' Guidance to Assessing Systems Change, developed by MEL Managers for MEL Managers. (Check out the Guidance here https://bit.ly/MSPMELClinics.) Hear from the authors about which parts they love the most and how this guide challenges MEL managers to assess systems change as an ongoing aspect of implementation, generating feedback that teams need to better understand and catalyze change, for more impact.

Evaluating with the Success Case Method - Gram, Tom (Performance X Design, 2011)
The method was developed by Robert Brinkerhoff as an alternative (or supplement) to the Kirkpatrick approach and its derivatives. It is very simple and fast (which is part of it’s appeal) and goes something like this: Step 1. Identify targeted business goals and impact expectations Step 2. Survey a large representative sample of all participants in a program to identify high impact and low impact cases Step 3. Analyze the survey data to identify: a small group of successful participants a small group unsuccessful participants Step 4. Conduct in-depth interviews with the two selected groups to: document the nature and business value of their application of learning identify the performance factors that supported learning application and obstacles that prevented it. Step 5. Document and disseminate the story report impact applaud successes use data to educate managers and organization The process produces two key outputs In-depth stories of documented business effect that can be disseminated to a variety of audiences Knowledge of factors that enhance or impede the effect of training on business results. Factors that are associated with successful application of new skills are compared and contrasted with those that impede training. It answers practical and common questions we have about training and other initiatives: What is really happening? Who’s using what, and how well? Who’s not using things as planned? What’s getting used, and what isn’t? Which people and how many are having success? Which people and how many are not? What results are being achieved? What value, if any, is being realized? What goals are being met? What goals are not? Is the intervention delivering the promised and hoped for results? What unintended results are happening? What is the value of the results? What sort of dollar or other value can be placed on the results? Does the program appear to be worthwhile? Is it producing results worth more than its costs? What is its return on investment? How much more value could it produce if it were working better? How can it be improved? What’s helping? What’s getting in the way? What could be done to get more people to use it? How can everyone be more like those few who are most successful?

Opening the black box: the contextual drivers of social accountability - Grandvoinnet, Helene; Aslam, Ghazia; Raha, Shomikho (New frontiers of social policy, 2015)
Keywords: Social accounting

Hidden Potential: The Science of Achieving Greater Things - Grant, Adam (WH Allen, 2023)
#1 New York Times bestseller "This brilliant book will shatter your assumptions about what it takes to improve and succeed. I wish I could go back in time and gift it to my younger self. It would've helped me find a more joyful path to progress."-Serena Williams, 23-time Grand Slam singles tennis championThe #1 New York Times bestselling author of Think Again illuminates how we can elevate ourselves and others to unexpected heights.We live in a world that’s obsessed with talent. We celebrate gifted students in school, natural athletes in sports, and child prodigies in music. But admiring people who start out with innate advantages leads us to overlook the distances we ourselves can travel. We can all improve at improving. And when opportunity doesn’t knock, there are ways to build a door.Hidden Potential offers a new framework for raising aspirations and exceeding expectations. Adam Grant weaves together groundbreaking evidence, surprising insights, and vivid storytelling that takes us from the classroom to the boardroom, the playground to the Olympics, and underground to outer space. He shows that progress depends less on how hard you work than how well you learn. Growth is not about the genius you possess – it’s about the character you develop. Grant explores how to build the character skills and motivational structures to realize our own potential, and how to design systems that create opportunities for those who have been underrated and overlooked.This book reveals how anyone can rise to achieve greater things. The true measure of your potential is not the height of the peak you’ve reached, but how far you’ve climbed to get there.

Innovation lifecycle and the missing middle - Gray, Ian; McClure, Dan (Managing Humanitarian Innovation - The cutting edge of aid, 2018)

Calibrating adaptive programming across multi-country, multi-partner programs - Gray, Stephen; Carl, Andy (Adapt Peacebuilding, 2019)
This past week Andy Carl and I joined the international NGO Christian Aid Ireland team in Dublin to reflect on progress in a key feature of their work - adaptive programming. This post presents what Andy Carl and I learned in reflecting this week with the international NGO Christian Aid Ireland r

The Difference Learning Makes - Factors that enable and inhibit adaptive programming - Gray, Stephen; Carl, Andy (Christian Aid, 2022)
Executive Summary When Christian Aid (CA) Ireland devised its multi-country and multi-year Irish Aid funded Programme Grant II (2017-2022), they opted to move away from a linear programme management approach and to explore an adaptive one. Across seven countries: Angola, Colombia, El Salvador, Guatemala, Israel and the occupied Palestinian territory, Sierra Leone, and Zimbabwe, CA and partner organisations support marginalised communities to realise their rights, reduce violence and address gender inequality. Since 2019, Adapt Peacebuilding has accompanied CA Ireland, CA country teams and partner organisations as they experimented with using a deliberate adaptive approach. The authors were also asked to follow up on an initial study by CA Ireland and Overseas Development Institute in 2018, which described the rationale for adopting this new approach and included early lessons from its first year of implementation. The aim of this study is to help deepen CA Ireland, CA country teams’ and partners’ understanding of (a) whether their application of adaptive programming has resulted in better development outcomes, and (b) how they can better understand the factors that enabled or inhibited the effectiveness of using this approach. Over the past three years, this study has found evidence and multiple examples that show adaptive programming contributed to better development outcomes. The main reasons cited were that these were made possible both from improvements to programming strategies based on proactive reflection and learning, as well as those that stem from the reactive capacity of adaptive programmes to change course in response to unanticipated changes in operating conditions. This study found that adaptive programming has enabled better development practice where organisations are enhancing their skills to better respond and be flexible to contextual challenges. 72% of partners surveyed described adaptive programming as the most useful approach to programme management that they have used. The programme approach has meant that CA and partner staff were better able to explore the significance of change in the context and their contributions to them. It also enabled spaces for meaningful engagement with communities in learning and programme planning processes and encouraged opportunities for experimentation in programming. The study also found that adaptive programming has supported flexible delivery. This led to better outcomes that would not have been possible were the programme not able to make flexible adjustments. The main focus has been the analysis of nine factors that can determine the effectiveness and impact (or otherwise) of using an adaptive approach, flagging important issues for understanding. These factors are identified as: 1) Leadership; 2) Organisational culture; 3) Conceptual understanding; 4) Staff capacities; 5) Partnership approaches; 6) Participation; 7) Methods and tools; 8) Administrative procedures; and 9) The operating context. Together these can provide an analytical framework for assessing an organisation’s ‘adaptive scope’, which can be used as a tool for better understanding an organisation’s potential to generate improved development outcomes via adaptive programming and how to strengthen them. The study concludes with several recommendations for CA Ireland, all of which have relevance for a broader community of donors and implementing organisations interested in the potential of adaptive programming.

A top Toolkit on Adaptive Management. But is that a good idea? – FP2P - Green, Duncan (From Poverty to Power, 2021)

Adaptive Management looks like it’s here to stay. Here’s why that matters. - Green, Duncan (From Poverty to Power, 2016)
First installment of reflections on my US trip. This is on the rise of adaptive management approaches in USAID, and some of the questions it raises
Keywords: IMPORTANT

Book Review: ‘Thinking and Working Politically in Development’ - Green, Duncan (From Poverty to Power, 2020)
Review of a new book that explores the 'secret sauce' of Coalitions for Change - an unusually successful governance programme in the Philippines

From poverty to power: how active citizens and effective states can change the world - Green, Duncan (2012)
From Poverty to Power From Poverty to Power argues that a radical redistribution of power, opportunities, and assets rather than traditional models of charitable or government aid is required to break the cycle of poverty and inequality. The forces driving this transformation are active citizens and effective states. Published in association with Oxfam GB. Full description

How Change Happens - Green, Duncan (OUP Oxford, 2016)
Human society is full of would-be 'change agents', a restless mix of campaigners, lobbyists, and officials, both individuals and organizations, set on transforming the world. They want to improve public services, reform laws and regulations, guarantee human rights, get a fairer deal for those on the sharp end, achieve greater recognition for any number of issues, or simply be treated with respect. Striking then, that not many universities have a Department of Change Studies, to which social activists can turn for advice and inspiration. Instead, scholarly discussions of change are fragmented with few conversations crossing disciplinary boundaries, rarely making it onto the radars of those actively seeking change. This book bridges the gap between academia and practice, bringing together the best research from a range of academic disciplines and the evolving practical understanding of activists to explore the topic of social and political change. Drawing on many first-hand examples from the global experience of Oxfam, one of the world's largest social justice NGOs, as well as the author's insights from studying and working on international development, it tests ideas on How Change Happens and offers the latest thinking on what works to achieve progressive change. This is an open access title available under the terms of a CC BY-NC-ND 4.0 International licence. It is available as a free PDF download from OUP and selected open access locations.

How to do Adaptive Management in 15 easy steps – from a top new toolkit – FP2P - Green, Duncan (From Poverty to Power, 2021)

NGOs Doing Development Differently - Green, Duncan (From Poverty to Power, 2017)
Report back from a meeting of international NGOs to set up a research and practice network on 'Doing Development Differently' that can complement other actors

Old Wine in New Bottles? 6 ways to tell if a programme is really ‘doing development differently’ - Green, Duncan (From Poverty to Power, 2018)
Some top adaptive management exponents give their top tips on how to distinguish spin from reality, when looking at an avowedly AM programme

Rules of Thumb – good idea or double-edged sword? - Green, Duncan (From Poverty to Power, 2020)
A recent conversation with a good governance programme in Myanmar tried to identify its underlying rules of thumb. Was that a good idea?

Second (and Third) Thoughts on Adaptive Management and Thinking and Working Politically - Green, Duncan (From Poverty to Power, 2022)
Self-Critical reflections on AM and TWP. Linking it with The Hype Cycle - "it feels like we are heading downward to the ‘trough of disillusionment’ form the initial peak of ‘inflated expectations’, but we will bounce back to something more sustained, that becomes a permanent feature of the aid landscape".

The context v intervention 2x2 | From Poverty to Power - Green, Duncan (From Poverty to Power, 2016)
Lots of discussion on my US trip around the strengths/weaknesses of the context v intervention 2x2 that suggests particular theories of change acc to situation
Keywords: IMPORTANT

The Hidden Life of Theories of Change - Green, Duncan (From Poverty to Power, 2020)
A smart new report explores how a good idea - theories of change - is distorted by the way it is implemented in the aid sector

The NGO-Academia Interface: Realising the shared potential - Green, Duncan (The Social Realities of Knowledge for Development, 2017)
This edited collection of peer-reviewed papers explores critical challenges faced by organisations and individuals involved in evidence-informed development through a diverse set of case studies and t...

Theories of Change for Promoting Empowerment and Accountability in Fragile and Conflict-Affected Settings - Green, Duncan (IDS Working Paper, 2017)
This paper explores the current state of thinking among a range of aid actors (multilaterals, bilateral, applied scholars and international non-governmental organisations) on how to promote empowerment and accountability in fragile, conflict and violence affected settings. It seeks to identify trends, gaps and weaknesses in that thinking, and propose research questions and hypotheses to test. Three underlying sources of confusion are identified that are hindering progress on both understanding empowerment and accountability in fragile, conflict and violence affected settings, and taking helpful action to promote it. They are: Theory of endogenous change (e.g. on how empowerment and accountability arise in situ) versus the theory of action of an external intervention; Fragility versus conflict: there is no clear justification for combining these different aspects into a single category; and Empowerment versus accountability: donor analysis and practice has been overwhelmingly weighted towards accountability, exhibiting limited understanding or interest in the nature of power.
Keywords: A4EA; External actors; Participation

Thinking and working politically: What have we learned since 2013? – FP2P - Green, Duncan (From Poverty to Power, 2022)
It’s always a red letter day when a new paper from Graham Teskey drops. His most recent is Thinking and working politically: What have we learned since 2013? For those that don’t know him, Graham is a consummate insider-outsider within the aid sector – long stints at DFID (UK), DFAT (Australia) and now Abt (Management Consultants). From this vantage point he has been one of the leading proponents of ‘thinking and working politically’, always ready to call out the hand-wavey academics and demand some practical lessons, please. This paper is part biography of an idea, setting out the timeline, moments and key documents and policy wins in the evolution of TWP (which seems to have involved a lot of seminars that I missed due to Oxfam’s meagre travel budget). The other part is, to be honest, a bit of a lament – a study in ‘Why Change Hasn’t Happened’, because TWP has ‘got lost in the maelstrom’ of the wider, largely negative, changes in the aid sector. Overall, it’s a brilliant summary, and one I’ll be recommending to my increasingly long-suffering activism students….

What are the Simple Rules that guide our Strategies? - Green, Duncan (From Poverty to Power, 2019)
A management treatise on 'simplre rules' for companies facing unpredictable situations could provide useful guidance to aid organizations

What is different about how INGOs do Adaptive Management/Doing Development Differently? - Green, Duncan (From Poverty to Power, 2019)

Where have we got to on Thinking and Working Politically? Update and a Mildly Heretical Thought. – FP2P - Green, Duncan (From Poverty to Power, 2022)
Headed off recently to discuss the state of Thinking and Working Politically within the aid sector. This is a loose network of aid wonks that came together to try and move aid from a pure focus on technical issues, towards taking account of power and politics and why they can facilitate/frustrate attempts to make change happen in any given context. It was great to be in a room with others (50/50 in person and online) – the neurons fire in a way that just doesn’t happen online (but I also need to brush up on my meeting skills, as when I accidentally clicked on a random video in my timeline and it started playing at full volume. Super awkward). On to the content (Chatham House rule, so no names or institutions). Some observations about the evolution of a movement I’ve been connected with for over a decade. I got a lot of pushback from participants on an earlier draft and have made quite a few changes, but they should definitely feel free to set me straight in comments!

Adaptive Programming in Fragile, Conflict and Violence-Affected Settings, What Works and Under What Conditions?: The Case of Institutions for Inclusive Development, Tanzania - Green, Duncan; Guijt, Irene (Action for Empowerment and Accountability Research Programme, 2019)
Adaptive Management involves a dynamic interaction between three elements: delivery, programming and governance. This case study focuses on a large DfID governance project, the Institutions for Inclusive Development (I4ID), a five-year initiative in Tanzania. The study forms part of a research project to examine whether and how adaptive approaches can strengthen aid projects promoting empowerment and accountability in fragile, conflict and violence-affected settings (FCVAS). The research examines some of the assertions around the adaptive management approach and explores if and how adaptive approaches, including rapid learning and planning responses (fast feedback loops and agile programming) are particularly relevant and useful for citizen empowerment and government accountability (E&A) in FCVAS.
Keywords: A4EA; Adaptive Development; Economy; Fishery

Summer School Course - Adaptive Management - Working Effectively in the Complexity of International Development (weekplan) - Green, Duncan; Guijt, Irene (Oxfam, 2018)

From Thinking Politically To Working Politically: Are We Really Doing Development Any Differently? (2019)
In June it will be seven years since the Center for Global Development published the Problem Driven Iterative Adaptation (PDIA) paper. Few academic papers have had such an impact on development thinking. The question is whether the paper – and subsequent debate and experimentation - have had a demonstrable and beneficial impact on development practice. Experience to date suggests thinking politically is easier than working politically. Practitioners in development agencies, governments and contractors find that convincing theoretical literature can be tricky to operationalize in a fractious environment. Still, a growing number of programs have set out to be politically informed in design and politically savvy in implementation. Engaging the reality of politics creates the potential to promote change. To consider these issues, Abt Associates is hosting a workshop for international development practitioners in Washington, DC, on June 18th, 2019, to discuss the successes and failures in operationalizing this agenda. The session will allow practitioners to discuss experiences in TWP and propose solutions or approaches. Please join us in this Innovations in Governance series finale where four internationally renowned thinkers and doers will join us to offer their perspectives.

The Radical How - Greenway, Andrew; Loosemore, Tom (Nesta and Public Digital, 2024)
Any mission-focused government should be well equipped to define, from day one, what outcomes it wants to bring about. But radically changing what the government does is only part of the challenge. We also need to change how government does things. The usual methods, we argue in this paper, are too prone to failure and delay. There’s a different approach to public service organisation, one based on multidisciplinary teams, starting with citizen needs, and scaling iteratively by testing assumptions. We’ve been arguing in favour of it for years now, and the more it gets used, the more we see success and timely delivery. We think taking a new approach makes it possible to shift government from an organisation of programmes and projects, to one of missions and services. It offers even constrained administrations an opportunity to improve their chances of delivering outcomes, reducing risk, saving money, and rebuilding public trust. The Radical How in a nutshell The struggles and shortcomings of delivering in government are well rehearsed. Many of the root causes that make it tough have been restated several times over several decades. But what to do? We believe the government can and should change how it delivers, by: organising around multidisciplinary teams embracing incremental, feedback-driven iteration focusing more on outcomes. The Radical How is a change of mindset as much as a change in organisation. It promotes methods and processes that have been shown to work, multiple times, at scale. They are the default ways of working for many of the world’s most successful companies. However, the occasions where they have been deployed are rare in government. These occasions have come about thanks to exceptional leaders, exceptional circumstances, or both. We think they’d make a big difference if they became the norm, rather than the exception. We also think that without them, mission oriented government will not become a reality. New policy ideas will remain just that, rather than translating into profound improvements to society. Central to this approach is the widespread adoption of internet-era ways of working. This paper explains both those and our thinking in more detail, with reference to real examples.

Deconstructing Adaptive Management: Criteria for Applications to Environmental Management - Gregory, R.; Ohlson, D.; Arvai, J. (Ecological Applications, 2006)
[The concept of adaptive management has, for many ecologists, become a foundation of effective environmental management for initiatives characterized by high levels of ecological uncertainty. Yet problems associated with its application are legendary, and many of the initiatives promoted as examples of adaptive management appear to lack essential characteristics of the approach. In this paper we propose explicit criteria for helping managers and decision makers to determine the appropriateness of either passive or active adaptive-management strategies as a response to ecological uncertainty in environmental management. Four categories of criteria--dealing with spatial and temporal scale, dimensions of uncertainty, the evaluation of costs and benefits, and institutional and stakeholder support--are defined and applied using hypothetical yet realistic case-study scenarios that illustrate a range of environmental management problems. We conclude that many of the issues facing adaptive management may have less to do with the approach itself than with the indiscriminate choice of contexts within which it is now applied.]

Good Enough Governance Revisited - Grindle, Merilee S. (Development Policy Review, 2007)
The concept of good enough governance provides a platform for questioning the long menu of institutional changes and capacity-building initiatives currently deemed important (or essential) for development. Nevertheless, it falls short of being a tool to explore what, specifically, needs to be done in any real world context. Thus, as argued by the author in 2004, given the limited resources of money, time, knowledge, and human and organisational capacities, practitioners are correct in searching for the best ways to move towards better governance in a particular country context. This article suggests that the feasibility of particular interventions can be assessed by analysing the context for change and the implications of the content of the intervention being considered.

Citizens’ Use of New Media in Authoritarian Regimes: A Case Study of Uganda - Grönlund, Åke; Wakabi, Wairagala (The Electronic Journal of Information Systems in Developing Countries, 2015)
By subsidizing the costs of civic participation, the use of the Internet use is believed to stimulate participation but there are fears that intensive Internet use causes withdrawal from public life. This paper investigates the connection between the way individuals participate online and offline in authoritarian, low-income regimes, and the nature of eParticipation among citizens in authoritarian regimes such as Uganda. Based on personal interviews with 116 Internet users, the study found that common drivers of eParticipation, such as low cost, security and anonymity are hard to transplant into the offline world for citizens of authoritarian states such as Uganda. Perceived risks of retribution and intimidation for expressing a particular opinion or supporting a political cause mean that citizen-to-citizen participation is the predominant form but still at low levels, while citizen-to-government participation is negligible.

Information Technology and Political Engagement: Mixed Evidence from Uganda - Grossman, Guy (2016)

Can SMS-Mobilization Increase Citizen Reporting of Public Service Deficiencies to Politicians? - Grossman, Guy; Michelitch, Kristin; Santamaria, Marta (2015)

Beyond the log frame: a new tool for examining health and peacebuilding initiatives - Grove, Natalie J.; Zwi, Anthony B. (Development in Practice, 2008)
How do we move from identifying ethical principles to enhancing development practice? How can donors and NGOs move beyond the reporting of technical outputs to explore less tangible aspects of their health projects: contributions to rebuilding trust, promoting social cohesion, and enhancing good governance at community level? This article considers these questions in relation to health and peace-building activities in conflicted settings. It describes difficulties facing practitioners and donors seeking to undertake health and peace work, in particular focusing on the lack of appropriate tools for screening, monitoring, and evaluating projects. It critiques the logical framework, a tool commonly used in project planning, monitoring, and evaluation, and considers it alongside a new tool, the Health and Peace Building Filter, which has been designed to reflect on health programming in fragile or conflicted settings. The authors argue that such tools can help to move us beyond focusing on inputs and outputs to examining processes, relationships, and the indirect consequences of aid programmes.

Scaling Up Development Impact - Guerrero, Isabel; Gokhale, Siddhant; Fahsbender, Jossie (Bowker, 2023)
Today, nearly one billion people lack electricity, over three billion lack clean water, and 750 million lack basic literacy skills. Many of these challenges could be solved with existing solutions, and technology enables us to reach the last mile like never before. Yet, few solutions attain the necessary scale to match the size of these challenges. Scaling Up Development Impact offers an analytical framework, a set of practical tools, and adaptive evaluation techniques to accompany the scaling process. It presents rich organizational experiences that showcase real-world journeys toward increased impact. From the people from IMAGOgg.org

Adaptive Management Across Project Cycles: Look into Coherence in Time - Guerzovich, Florencia (Medium, 2023)
This is a series about Monitoring, Evaluating and Learning (MEL) whether sets of interventions/portfolios are adding more together than each one would produce on their own. In post 1, I pointed to coherence, the new OECD-DAC evaluation criteria as a way to bridge the ambition of bringing bigger change with the MEL world. In post 2, I shared 3 of 4 practical lessons I’ve learned in experimenting with MEL systems and exercises that focus explicitly on interactions of interventions/portfolios. In the third post, I bring Paul Pierson’s groundbreaking argument for social science to MEL. Paraphrasing, most contemporary MEL takes a “snapshot” view of interventions and portfolios, distorting their effects and meaning by ripping them from their temporal context. Instead, we should place in time interventions/portfolios with the ambition to add more than the sum of the part by constructing MEL systems looking at “moving pictures” rather than taking snapshots. ...

Addressing the evaluation-of-sustainability-paradox: A relational rubric for evidencing… - Guerzovich, Florencia (Medium, 2024)
Florencia Guerzovich and Alix Wadeson

Does the whole add more than the SUM of its parts? - Guerzovich, Florencia (Medium, 2023)
This is a series about Monitoring, Evaluating and Learning (MEL) whether sets of interventions/portfolios are adding more together than each one would produce on their own. In post 1, I pointed to coherence, the new OECD-DAC evaluation criteria as a way to bridge the ambition of bringing bigger change with the MEL world. In post 2, I shared 3 of 4 practical lessons I’ve learned in experimenting with MEL systems and exercises that focus explicitly on interactions of interventions/portfolios. In the third post, I bring Paul Pierson’s groundbreaking argument for social science to MEL. Paraphrasing, most contemporary MEL takes a “snapshot” view of interventions and portfolios, distorting their effects and meaning by ripping them from their temporal context. Instead, we should place in time interventions/portfolios with the ambition to add more than the sum of the part by constructing MEL systems looking at “moving pictures” rather than taking snapshots.

Storytelling and Evaluation: Can Pathways be the Theory Missing in Explanatory Narratives? - Guerzovich, Florencia (Medium, 2023)
Florencia Guerzovich

Three out of Four Practical Lessons about right fitting MEL for Additive Effects - Guerzovich, Florencia (Medium, 2023)
This is a series about Monitoring, Evaluating and Learning (MEL) whether sets of interventions/portfolios are adding more together than each one would produce on their own. In post 1, I pointed to coherence, the new OECD-DAC evaluation criteria as a way to bridge the ambition of bringing bigger change with the MEL world. In post 2, I shared 3 of 4 practical lessons I’ve learned in experimenting with MEL systems and exercises that focus explicitly on interactions of interventions/portfolios. In the third post, I bring Paul Pierson’s groundbreaking argument for social science to MEL. Paraphrasing, most contemporary MEL takes a “snapshot” view of interventions and portfolios, distorting their effects and meaning by ripping them from their temporal context. Instead, we should place in time interventions/portfolios with the ambition to add more than the sum of the part by constructing MEL systems looking at “moving pictures” rather than taking snapshots.

Anatomy of Social Accountability 3.0: A working framework for moving forward - Guerzovich, Florencia; Aston, Tom (Florianopolis, SC, Brazil: Grupo Politeia, ESAG, University of the State of Santa Catarina and act4delivery, 2023)
This note presents the key components of a potential next generation of social accountability programming. The key aspects outlined emerge from a review of over 150 cases as well as tacit knowledge from dozens of practitioners (Guerzovich and Aston, 2023).The main thread of social accountability 3.0 and what distinguishes it from previous generations is a focus on its contribution towards more responsive systems and accountable social contracts. In particular, social accountability should be considered as an operational means to rework social contracts.Over the last two decades, the field has been defined by a supposed dichotomy between minimalist (i.e., “widget” based) approach or a maximalist (“go big or go home”) approaches. Yet, most practice sits somewhere between these extremes. The paper presents various options derived from practice; practitioners should choose whichever best fits their own particular circumstances.Those hoping to improve and expand their practice with a systems-lens may benefit from considering how they approach the key elements of social accountability 3.0.
Keywords: accountability; development; responsiveness; social accountability; social contract; systems

Social Accountability 3.0: Engaging Citizens to Increase Systemic Responsiveness. Background Paper - Guerzovich, Florencia; Aston, Tom (Grupo Politeia, ESAG, UDESC and act4delivery, 2024)
Social accountability has become increasingly important to development programming since the 2004 World Development Report spearheaded its first generation of programming. In 2016, Thomas Carothers synthesized global experts’ perspectives on what became known as the second generation. With an evolving global context, new research and evaluations and tacit knowledge on how those assumptions played out in practice, the time is ripe for asking what the evidence since 2004 can tell us about what the social accountability’s next generation might look like? This meta-analysis takes stock of that knowledge through a systems lens and presents a possible “social accountability 3.0.” It draws on a database of 157 cases of initiatives implemented by different civil society organizations around the world and the lived experience of practitioners. Scholarly debates present practitioners’ options in binary terms, contrasting 1.0 minimalist and 2.0 maximalist options. The review finds that this binary is inaccurate and misleading; it masks practitioners’ diverse interpretations of 2.0 and range of feasibility considerations. Instead, work in the last decade looks more like a mosaic. By opening the analysis to the broader range of evidence, and reflecting more critically on scholars’ assumptions, a more adaptable middle path for social accountability 3.0 emerges. Its key outcomes include contributing to systemic responsiveness and an operational mechanism to rework the social contract.

Adaptive Learning - Guerzovich, Florencia; Poli, Maria (GPSA Note 5, 2014)

Scaling up Social Accountability in Complex Governance Systems: A Relational Approach for Evidencing Sustainability - Guerzovich, Florencia; Wadeson, Alix (World Bank, 2024)
When social accountability interventions scale up and their sustainability depends on the interactions of many agents and system components, related results are rarely observable at the end of an intervention. The 2019 OECD Development Assistance Committee’s (OECD DAC) revamped evaluations criteria for assessing sustainability acknowledges that such results are often emergent, and should be monitored and evaluated with this in mind. It therefore emphasizes a turn towards assessing complex processes prospectively. It also asks evaluations to consider how likely it is that these results are evident at the time they are monitored or evaluated. However,the social accountability field continues to have gaps regarding doing this effectively in practice. This paper presents and provides evidence from testing an innovative operational approach that has promising potential to support this aim - a sequential, relational rubric. This approach can support practitioners to monitor, evaluate and learn about the causal processes of scale up of social accountability interventions with an eye towards sustainability i.e., considering prospective sustainability. It is grounded in systems thinking, co-production and social learning theory, as well as links with collective governance and social contract theory for development. Evidence yielded from the authors’ testing of this approach on a sample of diverse projects from the Global Partnership for Social Accountability (GPSA) program revealed that the alleged ‘absence of evidence’ dilemma of social accountability scale up is due to ill-fitting concepts and methods for assessment. It challenges existing assumptions and findings that claim that social accountabilityprocesses do not scale and are unsustainable. The authors propose that by using fit-for-purpose concepts and methods with a focus on social learning and compromise – also called a ‘resonance pathway to scale’ which this paper discusses in detail – it is possible to observe loosely coordinated scale up processes at work in many (but not all) social accountability interventions and identify tangible evidence of prospective sustainability. An important caveat is that these processes, the outcomes they generate, and the corresponding evidence often look qualitatively different than the original intervention design and predictions for scale-up at that point in time. This is because the process of deliberation and compromise inherent to social accountability work in dynamic local systems introduces changes and new conditions for uptake by diverse actors in the public sector, civil society, and donor institutions. The paper concludes that even relatively small-scale localized projects of three to five years with budgets of less than one million USD, across different contexts and sectors can produce processes and outcomes which contribute to many forms of sustainability, including via scaleup.Furthermore, the cross-fertilization of learning and aggregation of results for scale-up across projects within and beyond the GPSA (and other programs) can help monitoring evaluation and learning (MEL) and social accountability practitioners alike to deliver on a program’s mandate. Doing so can also create new knowledge for the wider social accountability field that siloed interventions, lacking suitable concepts and methods for assessing scale-up and prospective sustainability, often fail to produce. The paper ends with recommendations for taking forward this approach and the associated benefits, implications and required investments.

Accountability and Learning: Exploding the Myth of Incompatibility between Accountability and Learning - Guijt, Irene (NGO Management - The Earthscan Compendium, 2010)
When accountability is understood as reporting on pre-deined deliverables, it is often considered to be irreconcilable with learning. This conventional wisdom inhibits an appreciation of their connection. In this chapter, Irene Guijt exposes the laws and traps in reasoning that keep accountability and learning apart. She provides practitioners with principles and basic good ideas that open up prospects for accountability and learning to complement each other.

ALPS in action: a review of the shift in ActionAid towards a new Accountability, Learning and Planning System | Participatory Methods - Guijt, Irene (2004)

Participatory Approaches - Guijt, Irene (Methodological Briefs: Impact Evaluation, 2014)
This guide, written by Irene Guijt for UNICEF, looks at the use of participatory approaches in impact evaluation. Using participatory approaches means involving stakeholders, particularly those affected by intervention, in the evaluation process. This includes involvement in the design, data collection, analysis, reporting, and management of the study. Excerpt "By asking the question, ‘Who should be involved, why and how?’ for each step of an impact evaluation, an appropriate and context-specific participatory approach can be developed. Managers of UNICEF evaluations must recognize that being clear about the purpose of participatory approaches in an impact evaluation is an essential first step towards managing expectations and guiding implementation. Is the purpose to ensure that the voices of those whose lives should have been improved by the programme or policy are central to the findings? Is it to ensure a relevant evaluation focus? Is it to hear people’s own versions of change rather than obtain an external evaluator’s set of indicators? Is it to build ownership of the UNICEF programme? These, and other considerations, would lead to different forms of participation by different combinations of stakeholders in the impact evaluation."

The Learning Power of Listening - Guijt, Irene; Gottret, Maria Veronica; Hanchar, Anna; Deprez, Steff; Muckenhirn, Rita (PRACTICAL ACTION PUBLISHING, 2022)
Steff had the pleasure to co-author the first SenseMaker Practitioner Guide with a group of friends and colleagues supported and published by Oxfam and CRS. This practical guide is for those who wish to use SenseMaker to conduct assessments, monitor progress, and undertake evaluations or research. Drawing on more than a decade of experience, the authors share dozens of examples from international development, providing practical tips and ideas for context-specific adaptations. They show how the method can be used to for difficult-to-measure outcomes related to poverty reduction, social justice, peacebuilding, resilience, gender norms, behavior change, governance and environmental management. ​ SenseMaker is a unique participatory method of inquiry that encourages and enables novel insights not obtained from conventional quantitative and quantitative and qualitative methods. It is action-oriented and, therefore, well-suited for people needing data- informed insights for adaptive management. "Writing this guide together with Irene, Veronica, Anna and Rita was an enormous learning process in itself and has further shaped our thinking and practice. We hope it will support first-time and experienced users to enhance their practice and that it will inspire people to explore and innovate further with the method."

Inspiring Radically Better Futures - Evidence and Hope for Impact at Scale in a Time of Crisis - Guijt, Irene; Mayne, Ruth (Oxfam, 2020)
The world faces converging crises of health, climate, gender and racial injustice and extreme economic inequality. The calls are mounting to ‘build back better’ to create more inclusive, caring and environmentally sustainable futures. But what evidence exists that this is possible? The Inspiring Better Futures case study series investigates whether radical change at scale is possible and how it was achieved. This paper synthesises 18 cases which show that people are already successfully building better futures, benefitting millions of people, even against the odds in some of the world’s toughest contexts in lower-income countries. Together they offer hope that transformative change and radically better futures after the pandemic are within reach.

Reforming donors in fragile states: using public management theory more strategically - Gulrajani, Nilima; Honig, Dan (Overseas Development Institute, 2016)
This paper identifies ways in which donors can be more effective in fragile and conflict-affected states by exploiting theories and concepts drawn from public management.

Open data: Empowering the empowered or effective data use for everyone? - Gurstein, Michael B. (First Monday, 2011)
Keywords: e-Government; open access; open data

Voice or chatter? Making ICTs work for transformative engagement: research report summary - Gurumurthy, Anita; Bharthur, Deepti; Chami, Nandini (MAVC Research Report, 2017)
What are the conditions in democratic governance that make information and communication technology (ICT)-mediated citizen engagement transformative? While substantial scholarship exists on the role of the Internet and digital technologies in triggering moments of political disruption and cascading upheavals, academic interest in the sort of deep change that transforms institutional cultures of democratic governance, occurring in ‘slow time’, has been relatively muted. This study attempts to fill this gap. It is inspired by the idea of participation in everyday democracy and seeks to explore how ICT-mediated citizen engagement can promote democratic governance and amplify citizen voice. The study involved empirical explorations of citizen engagement initiatives in eight sites – two in Asia (India and Philippines), one in Africa (South Africa), three in South America (Brazil, Colombia, Uruguay) and two in Europe (Netherlands and Spain).

Civil society organizations and managerialism: On the depoliticization of the adaptive management agenda - Gutheil, Lena; Koch, Dirk-Jan (Development Policy Review, 2022)
Motivation In the last decade, a movement formed around making aid delivery more adaptive, relying on principles such as context sensitivity, flexibility, and ownership. The approaches seem promising for civil society organizations (CSOs) to fulfil their mission of fostering social transformation. While several donor agencies have started engaging with such approaches, the authors hardly see their political implications in practice. Purpose The article aims to provide evidence on an adaptive project and demonstrate how the social transformative and political nature of adaptive development management is rendered technical and depoliticized in practice. Methods and approach We use a case study of a development programme based on a social transformative policy framework that is implemented through CSOs in Uganda and Vietnam. Data were collected by means of interviews, participant observation and document analysis. Findings We find that, in practice, the social transformative policy framework is competing with managerial logics. We compare this process with the depoliticization of the Paris Declaration on Aid Effectiveness, finding striking similarities. By using practice theory, we show how managerialism remains the dominant paradigm in the civil society aid sector, fuelling the “anti-politics machine.” Policy implications The article shows that policy frameworks do not always work as intended. Donors should therefore not only change policy frameworks, but also start addressing institutional and operational requirements.

Governance for Growth in Vanuatu: Review of a decade of thinking and working politically - Hadley, Sierd; Tilley, Helen (ODI, 2017)

Understanding Indicators And Monitoring For Sustainability In The Context Of Complex Social-Ecological Systems - Haider, L. Jamila; Iribarrem, Alvaro; Gardner, Toby; Latawiec, Agnieszka E; Alves-Pinto, Helena; Strassburg, Bernardo (De Gruyter Open, 2015)

Reflections on applying iterative and incremental software development methodologies to aid and development work in developing countries - Haikin, Matt (MattHaikin.com, 2013)

Development is Going Digital: What is the role of INGOs? ICT for Development programmes in the Horn, East and Central Africa - Haikin, Matt; Flatters, George (Policy & Practice, 2017)
Development is going digital and INGOs like Oxfam have a vital convening role to play. This paper draws on ICT for Development in Oxfam’s programmes in the Horn, East and Central Africa to consider what this role is.  In order to realise the opportunities

We have experimented with different approaches to systems transformation — here are five insights - Haldrup, Søren Vester (UNDP Innovation, 2021)
At UNDP Innovation we are on a journey to shift our approach to innovation to help tackle complex development challenges. In short, we are moving away from single point solutions, and instead we are…

Accountability ecosystems: directions of accountability and points of engagement - Halloran, Brendan (IDS, 2016)
Accountability, the obligation of those in power to take responsibility for their actions, is a process involving relationships between different actors (in state and society) and mechanisms, and is influenced by many contextual factors. Using the lens of an accountability ‘ecosystem’ focuses our attention on the complexity of accountability processes. An ‘ecosystem’ perspective suggests that simple ideas about accountability – such as citizen feedback reaching decision-makers ensures more accountability, or that greater transparency equals greater accountability – are often actually much more complex.

Getting Strategic about Technology and Accountability: MAVC Learning and Inspiration Event in Manila - Halloran, Brendan (Politics, Governance and Development, 2016)
Two years ago, Jonathan Fox released a widely read and well-received paper looking at the evidence for citizen-led accountability.  In it, he noted the proliferation of short-term, isolated interve…

Navigating the Evidence on Transparency, Participation and Accountability: What Insights Have Emerged? What Gaps Remain? - Terms of reference for the Consultant Author(s) - Halloran, Brendan (2015)
Example of Terms of Reference for a Report on TAP.

Opening Governance: What have we learned and how do we translate into better practice? - Halloran, Brendan (Politics, Governance and Development, 2016)
In their introductory essay to the recent IDS Bulletin on Opening Governance (the entire issue is open access), Duncan Edwards and Rosie McGee critically appraise developments in the open governmen…

Politics, Political Change and International Development - Halloran, Brendan (Politics, Governance and Development, 2013)
As my inaugural post on my new blog, I thought I would re-post a short piece I wrote earlier this year about foreign aid (original here). Thomas Carothers and Diane de Garamont address this issue i…

Strengthening Accountability Ecosystems: a Discussion Paper - Halloran, Brendan (Think Piece, 2015)

Thinking and working politically in the transparency and accountability field - Halloran, Brendan (Transparency & Accountability Initiative, 2014)

Why Learning & Adaptation are Central to Making All Voices Count - Halloran, Brendan (Making All Voices Count, 2014)

Mobilizing Accountability: Citizens, Movements and the State - Halloran, Brendan; Flores, Walter (Transparency & Accountability Initiative, 2015)

System Stewardship. The future of policy making? - Hallsworth, Michael (Institute for Government, 2011)
This working paper looks at the future of policy making in a world of decentralisation and more complex problems. It argues that policy makers need to see themselves less as sitting on top of a delivery chain, but as stewards of systems with multiple actors and decision makers – whose choices will determine how policy is realised. We are keen to open up a debate on what this means.

Guide to Innovation Partnerships – A United Nations Ghide - Halse, Michelle; Ginsberg-Fletcher, Gabriella; Balbi, Luisa (UN Global Pulse, 2023)

Democratic governance and evaluation. - Hanberger, A. (Sixth EES (European Evaluation Society) conference, Berlin, Germany, 2004)
This paper, presented by Anders Hanberger at the Sixth EES (European Evaluation Society) Conference in Berlin, Germany (September 30-October 2, 2004) argues that governance, democracy and evaluation impact each in different ways. It offers a discussion centred around the evaluation of three general democratic governance models and the implications of leading democratic evaluations. "Since governance and democracy are changing phenomena, and evaluation is embedded in these structures, there is a need to illuminate and discuss the role of evaluation. Evaluation is to most people thought of as a democratic tool, but what do we mean with democratic evaluation? Furthermore, what is the role of evaluation in times when democracy and governance are changing? The many meanings of democracy and the shifting role of evaluation in various democratic governance settings are also a motive for discussing governance and democratic evaluation. Governance issues can be discussed in relation to different political systems. However, in this paper the discussion is confined to democratic governance systems. The premise of this paper is that governance, democracy and evaluation affect one another in different ways. Governance is intertwined with democracy, and democracy and governance can be maintained or strengthened by evaluation, for example. Because different models of governance and democracy presuppose one another, evaluating governance models, or programme processes/outcomes where a specific governance model sets up the context, have implications for the model under scrutiny and subsequently for democracy. Accordingly, the implications of democratic evaluations need to be discussed in various ways. For analytical purpose democracy and governance are sometimes kept apart." (Hanberger, 2004)​

Adapt: Why Success Always Starts with Failure - Harford, Tim (Abacus, 2012)
Everything we know about solving the world's problems is wrong. Out: Plans, experts and above all, leaders. In: Adapting - improvise rather than plan; fail, learn, and try againIn this groundbreaking new book, Tim Harford shows how the world's most complex and important problems - including terrorism, climate change, poverty, innovation, and the financial crisis - can only be solved from the bottom up by rapid experimenting and adapting.From a spaceport in the Mojave Desert to the street battles of Iraq, from a blazing offshore drilling rig to everyday decisions in our business and personal lives, this is a handbook for surviving - and prospering - in our complex and ever-shifting world.

The Last 10 Per Cent: Why the World Needs a Leaner, More Innovative and Pragmatic Development Sector, Today - Harper, Erica (Routledge India, 2023)
Criticism that the development sector has not delivered in terms of eliminating extreme poverty, fast-tracking growth and preventing conflict, is neither new nor surprising. In fact, it may be the one thing that scholars, donors and practitioners agree on. While many of these concerns are valid, this book makes a case that the sector is closer to unlocking the gates to more effective and efficient development outcomes than is popularly believed. Specifically, it argues that by overturning a few myths, making better use of evidence and employing some different rules, practitioners, policy specialists and donors can foster the changes in the development architecture that are needed to reach the 10 percent of the world’s population still living in extreme poverty.Engaging, provocative and clear sighted, the book provides insight into interventions around democratic governance, refugee response, counterterrorism, gender mainstreaming, environmental protection and private sector engagement. It is instructive reading for professionals across the development sector, think tanks and NGOs.

Can PDIA become a regular part of how a government works? - Building State Capability - Harrington, Peter (Building State Capability, 2022)
Institutional change is part of the theory of change of PDIA – scaling through the diffusion of new ways of thinking and greater problem-solving know-how. And once a community of practice reaches critical mass across an eco-system, a tipping point can happen where the eco-system becomes generally more open to novelty, where success is a more effective route to legitimacy, and where leadership is oriented towards value creation.

Leather sector reform in Bangladesh - Harris, Dan (Working Politically in Practice, 2016)
This paper examines the Asia Foundation’s efforts to support change in Bangladesh’s leather sector. Working closely with local partners, the Asia Foundation team has specifically supported efforts to move tanneries out of a dangerously polluted location to a modern industrial park that will improve compliance with health and environmental protection standards, and potentially lead to growth in the sector. At the time of release, this critical relocation has already begun. This case study lays out the real-time decisions and processes which drove the strategy and implementation of this project, providing useful insights into how politically astute and flexible programs can be successfully implemented. This case has emerged from an action research process, which was led by a researcher from the Overseas Development Institute (ODI) and conducted over the course of almost two years. By capturing and analyzing the experiences of the program team in Bangladesh, the paper intends to provide practical insights for others in the development community aiming to implement similar kinds of programming. This is the seventh paper in the DFAT-TAF Partnership Working Politically in Practice Paper Series released under the Australian Government Department of Foreign Affairs and Trade (DFAT) and The Asia Foundation (TAF) Partnership.

PFM Reform Through PDIA: What Works and When it Works - Harris, Jamelia; Lawson, Andrew (Public Financial Management Blog - IMF, 2022)

How ICT4D Research Fails the Poor - Harris, Roger W. (Information Technology for Development, 2016)
Research can improve development policies and practices and funders increasingly require evidence of such socioeconomic impact from their investments. This article questions whether information and communication technologies for development (ICT4D) research conforms to the requirements for achieving socioeconomic impact. We report on a literature review of the impact of research in international development and a survey of ICT4D researchers who assessed the extent to which they follow practices for achieving socioeconomic impact. The findings suggest that while ICT4D researchers are interested in influencing both practice and policy, they are less inclined toward the activities that would make this happen, especially engaging with users of their research and communicating their findings to a wider audience. Their institutions do not provide incentives for researchers to adopt these practices. ICT4D researchers and their institutions should engage more closely with the users of their research through more and better communications with the public, especially through the use of information and communication technologies.
Keywords: ICT4D; Policy making; Practice; Research; impact

Fostering Learning in Large Programmes and Portfolios: Emerging Lessons from Climate Change and Sustainable Development - Harvey, Blane; Pasanen, Tiina; Pollard, Alison; Raybould, Julia (Sustainability, 2017)
In ﬁelds like climate and development, where the challenges being addressed can be described as “wicked”, learning is key to successful programming. Useful practical and theoretical work is being undertaken to better understand the role of reﬂexive learning in bringing together different knowledge to address complex problems like climate change. Through a review of practical cases and learning theories commonly used in the areas of resilience, climate change adaptation and environmental management, this article: (i) reviews the theories that have shaped approaches to reﬂexive learning in large, highly-distributed climate change and resilience-building programmes for development; and (ii) conducts a comparative learning review of key challenges and lessons emerging from early efforts to promote and integrate reﬂexive learning processes in programmes of this nature. Using a case study approach, the authors focus on early efforts made in four large, inter-related (or nested) programmes to establish, integrate and sustain learning processes and systems. Eight themes emerged from the review and are considered from the perspective of learning programmes as emergent communities of practice. By investigating how these themes play out in the nested programming, the paper contributes to the limited existing body of evidence on learning in large climate change programmes and identiﬁes areas where future efforts might focus.

Salt River Ecosystem Restoration Project - Adaptive Management Plan - Harvey, H.T. (Humboldt County Resource Conservation District, 2011)

The Dynamics of Team Learning: Harmony and Rhythm in Teamwork Arrangements for Innovation - Harvey, Jean-François; Cromwell, Johnathan R.; Johnson, Kevin J.; Edmondson, Amy C. (Administrative Science Quarterly, 2023)
Innovation teams must navigate inherent tensions between different learning activities to produce high levels of performance. Yet, we know little about how teams combine these activities—notably reflexive, experimental, vicarious, and contextual learning—most effectively over time. In this article, we integrate research on teamwork episodes with insights from music theory to develop a new theoretical perspective on team dynamics, which explains how team activities can produce harmony, dissonance, or rhythm in teamwork arrangements that lead to either positive or negative effects on overall performance. We first tested our theory in a field study using longitudinal data from 102 innovation teams at a Fortune Global 500 company; then, we replicated and elaborated our theory in a study of 61 MBA project teams at an elite North American university. Results show that some learning activities can occur within the same teamwork episode to have harmonious positive effects on team performance, while other activities combine to have dissonant negative effects when occurring in the same episode. We argue that dissonant activities must be spread across teamwork episodes to help teams achieve a positive rhythm of team learning over time. Our findings contribute to theory on team dynamics, team learning, and ambidexterity.

Adaptive management intentions with a reality of evaluation: Getting science back into policy - Hasselman, Lyndal (Environmental Science & Policy, 2017)
In Australia’s Murray-Darling Basin water reform has been contentious as government attempts to reconcile historical over allocation of water to irrigation with the use of water for environmental outcomes. However, in many aspects, scientific knowledge of the environment is either imperfect, incomplete or environmental responses are unpredictable, with this uncertainty preventing definitive policy and closure of political arguments. In response to uncertainty and knowledge gaps, adaptive management has been written into the legislation, along with provisions for periodic evaluation. This research ascertains how adaptive management is understood by policy makers, with this indicative of future implementation of adaptive management. The way in which adaptive management is constructed by policy makers is determined through legislation, public speeches, government reports and semi-structured interviews. The findings demonstrate that adaptive management has been subsumed by evaluation. The loss of adaptive management as a distinct concept is seen as a loss of science and discovery from the policy process, with the dominance of evaluation discussed as limiting innovation and reinforcing a ‘muddling through’ of policy.
Keywords: Accountability; Adaptive Management; Evaluation; Science; Uncertainty

Digitally-induced change in the public sector: a systematic review and research agenda - Haug, Nathalie; Dan, Sorin; Mergel, Ines (Public Management Review, 2023)
Digital transformation has become a buzzword that is permeating multiple fields, including public administration and management. However, it is unclear what is transformational and how incremental and transformational change processes are linked. Using the PRISMA method, we conduct a systematic literature review to structure this growing body of evidence. We identified 164 studies on digitally-induced change and provide evidence for their drivers, implementation processes, and outcomes. We derive a theoretical framework that shows which incremental changes happen in public administrations that are implementing digital technologies and what their cumulative, transformative effects are on society as a whole.

Program Logic Foundations: Putting the Logic Back into Program Logic - Hawkins, Andrew J. (Journal of MultiDisciplinary Evaluation, 2020)
Abstract Background: Program logic is one of the most used tools by the public policy evaluator. There is, however, little explanation in the evaluation literature about the logical foundations of program logic or discussion of how it may be determined if a program is logical. This paper was born on a long journey that started with program logic and ended with the logic of evaluation. Consistent throughout was the idea that the discipline of program evaluation is a pragmatic one, concerned with applied social science and effective action in complex, adaptive systems. It gradually became the central claim of this paper that evidence-based policy requires sound reasoning more urgently than further development and testing of scientific theory. This was difficult to reconcile with the observation that much evaluation was conducted within a scientific paradigm, concerned with the development and testing of various types of theory. Purpose: This paper demonstrates the benefits of considering the core essence of a program to be a proposition about the value of a course of action. This contrasts with a research-based paradigm in which programs are considered to be a type of theory, and in which experimental and theory-driven evaluations are conducted. Experimental approaches focus on internal validity of knowledge claims about programs and on discovering stable cause and effect relationships—or, colloquially, ‘what works?’. Theory-driven approaches tend to focus on external validity and in the case of the realist approach, the search for transfactual causal mechanisms—extending the ‘what works’ mantra to include ‘for whom and in what circumstances’. On both approaches, evaluation aspires to be a scientific pursuit for obtaining knowledge of general laws of phenomena, or in the case of realists, replicable context-mechanism-outcome configurations. This paper presents and seeks to justify an approach rooted in logic, and that supports anyone to engage in a reasonable and democratic deliberation about the value of a course of action. It is consistent with systems thinking, complexity and the associated limits to certainty for determining the value of a proposed, or actual, course of action in the social world. It suggests that evaluation should learn from the past and have an eye toward the future, but that it would be most beneficial if concerned with evaluating in the present, in addressing the question ‘is this a good idea here and now? Findings: In seeking foundations of program logic, this paper exposes roots that extend far deeper than the post-enlightenment, positivist and post-positivist social science search for stable cause and effect relationships. These roots lie in the 4th century BCE with Aristotle’s ‘enthymeme’. The exploration leads to conclusions about the need for a greater focus on logic and reasoning in the design and evaluation of programs and interventions for the public good. Science and research are shown to play a crucial role in providing reasons or warrants to support a claim about the value of a course of action; however, one subordinate to the alpha-discipline of logical evaluation and decision making that must consider what is feasible given the context, capability and capacity available, not to mention values and ethics. Program Design Logic (PDL) is presented as an accessible and incremental innovation that may be used to determine if a program makes sense ‘on paper’ in the design stage as well as ‘in reality’ during delivery. It is based on a configurationalist theory of causality and the concepts of ‘necessary’ and ‘sufficient’ conditions. It is intended to guide deliberation and decision making across the life cycle of any intervention intended for the public good.

Beneficiary Feedback Mechanisms - Hayman, Rachel (INTRAC, 2020)
There is a growing emphasis among NGOs and donors on ensuring that the voices of beneficiaries are heard. A range of beneficiary feedback mechanisms (BFMs) exist which enable beneficiary perspectives and suggestions to be gathered and used within an M&E system. BFMs are tools designed to enable a continuous cycle of interaction between those receiving and those delivering aid-funded interventions.

Digital Principles into practice - HDIF (HDIF, 2018)

Large language model applications for evaluation: Opportunities and ethical implications - Head, Cari Beth; Jasper, Paul; McConnachie, Matthew; Raftree, Linda; Higdon, Grace (New Directions for Evaluation, 2023)
Large language models (LLMs) are a type of generative artificial intelligence (AI) designed to produce text-based content. LLMs use deep learning techniques and massively large data sets to understand, summarize, generate, and predict new text. LLMs caught the public eye in early 2023 when ChatGPT (the first consumer facing LLM) was released. LLM technologies are driven by recent advances in deep-learning AI techniques, where language models are trained on extremely large text data from the internet and then re-used for downstream tasks with limited fine-tuning required. They offer exciting opportunities for evaluators to automate and accelerate time-consuming tasks involving text analytics and text generation. We estimate that over two-thirds of evaluation tasks will be affected by LLMs in the next 5 years. Use-case examples include summarizing text data, extracting key information from text, analyzing and classifying text content, writing text, and translation. Despite the advances, the technologies pose significant challenges and risks. Because LLM technologies are generally trained on text from the internet, they tend to perpetuate biases (racism, sexism, ethnocentrism, and more) and exclusion of non-majority languages. Current tools like ChatGPT have not been specifically developed for monitoring, evaluation, research, and learning (MERL) purposes, possibly limiting their accuracy and usefulness for evaluation. In addition, technical limitations and challenges with bias can lead to real world harm. To overcome these technical challenges and ethical risks, the evaluation community will need to work collaboratively with the data science community to co-develop tools and processes and to ensure the application of quality and ethical standards.

What we’ve learned about adaptive management from our Somalia education programming - Heales, Charlotte (Care Insights, 2020)
Adaptive management in its various incarnations has long been a focus of a development community that is more and more frequently bumping up against the barriers of complexity, and looking for ways to overcome its challenges. In a field where we consistently have to deal with multifaceted problems,...

What is impact? - Hearn, Simon; Anne, Buffardi (2016)
• Impact is a multi-dimensional concept. Some definitions focus on very precise understandings of impact, while others cast a much broader net. • How impact is defined and used has a significant effect on the design, management and evaluation of development programmes. • Development programmes should hold explicit conversations with different stakeholders about how impact is used and understood, in order to come to a shared understanding. • There are six dimensions of impact that may help development programmes be clearer about what they mean.

Not everything that connects is a network - Hearn, Simon; Mendizabal, Enrique (ODI, 2011)
In the public policy context, networks have been predominant factors in policy formulation, innovation and global governance. While not a new phenomenon in the development landscape, networks are becoming increasingly prevalent. Donors, for example, are turning to networks to deliver aid interventions, NGOs are working through networks for collective advocacy and researchers collaborate across networks for greater policy influence. But are networks always the most appropriate vehicle? Where they are appropriate, how can we make the best use of them? This Background Note argues for a more rigorous understanding of their nature, particularly their value (and costs), and presents a revised Network Functions Approach as a model for rationalised investment in networks.

Making Outcome Mapping Work - Volume 2 - Innovations in Participatory Planning, Monitoring and Evaluation - Hearn, Simon; Schaeffer, Heidi; van Ongevalle, JAn (Outcome Mapping Learning Community, 2009)

Communities of practice: Linking knowledge, policy and practice - Hearn, Simon; White, Nancy (Working and discussion papers, 2009)
This paper describes the basic characteristics of CoPs and provides a rationale for their growing importance in international development. It also suggests some ways in which CoPs can be supported by development agencies, research institutes and donors to strengthen the linkages between knowledge, policy and practice.

Failure, Success and Improvisation of Information Systems Projects in Developing Countries - Heeks, Richard (Development Informatics Working Paper Series, No.11/2002, 2002)
Research on the Information Society, the Digital Divide and Information and Communication Technologies for development

Information Systems and Developing Countries: Failure, Success, and Local Improvisations - Heeks, Richard (The Information Society, 2002)
This article presents evidence that–alongside the successes– many information systems in developing countries can be categorized as failing either totally or partially. It then develops a new model that seeks to explain the high rates of failure. The model draws on contingency theory in order to advance the notion of design-actuality gaps: the match or mismatch between IS designs and local user actuality. This helps identify two high-risk archetypes that affect IS in developing countries: country context gaps and hard-soft gaps. The model is also of value in explaining the constraints that exist to local IS improvisations in developing countries. Overall, the article shows how model and theory help understand IS cases in developing countries, and equally, how those cases provide valuable data to help develop IS models and theories.
Keywords: Developing Country; Evaluation; Failure; Information System; implementation

Most egovernment-for-development projects fail: how can risks be reduced? - Heeks, Richard (Development Informatics Working Paper Series, No.14, 2003)
Research on the Information Society, the Digital Divide and Information and Communication Technologies for development

Success and Failure in eGovernment Projects - Heeks, Richard (eGovernment for Development, 2008)

Impact assessment of ICT-for-development projects a compendium of approaches - Heeks, Richard; Molla, Alemayehu (University of Manchester. Institute for development policy and management (IDPM), 2009)

Deliberation and Development: Rethinking the Role of Voice and Collective Action in Unequal Societies - Heller, Patrick; Rao, Vijayendra (World Bank Publications, 2015)
This book marries two fields that rarely converse with one another:deliberative democracy and development studies. The study ofdeliberation―which explores normative and practical questions aroundgroup-based decision making through discussion or debate―has emergedas a critical area of study over the past two decades. Concurrently, the fieldof development has seen a spurt of interest in community-led developmentand participation premised on the ability of groups to arrive at decisionsand manage resources via a process of discussion and debate. Despite thegrowing interest in both fields, they have rarely engaged with one another.This book, which brings together new essays by some of the leading scholarsin the field, deepens our understanding of participatory decision makingin developing countries while initiating a new field of study for scholarsof deliberation. In the process, it sheds light on how to best design andimplement policies to strengthen the role of participation in development.

RAND Program Evaluation Toolkit for Countering Violent Extremism - Helmus, Todd; Matthews, Miriam; Ramchand, Rajeev; Beaghley, Sina; Stebbins, David; Kadlec, Amanda; Brown, Michael; Kofner, Aaron; Acosta, Joie (RAND Corporation, 2017)

HERMES (Highly Extensible Resource for Modeling Event-Driven Supply Chains) - HERMES (hermes.psc.edu, 2015)
Systems modeling is a tool for policymakers and program managers to capture all the direct and indirect effects of changes to a supply chain, identify sustainable solutions to the root causes of issues and save time, effort and resources in costly trial and error. Without systems modeling, evaluating the supply chain, identifying gaps, and implementing solutions can be insufficient, unsustainable and costly. Stock2 Vaccine supply chains are complex systems, comprising all the equipment, personnel, policies and processes needed to deliver a vaccine from its point of origin to the population. Understanding how the various components of a vaccine supply chain interact with each other is critical to evaluating supply chain function, identifying the root causes of issues and formulating sustainable solutions. HERMES (Highly Extensible Resource for Modeling Event-Driven Supply Chains) is a software program that allows users to generate a detailed computer simulation model of a supply chain

Towards evidence-informed adaptive management - Hernandez, Kevin; Ramalingam, Ben; Wild, Leni (Working Paper, 2019)
Development and humanitarian organisations seeking to be adaptive have emphasised the need to be transparent about complexity and uncertainty; to be honest about their inability to control what happens; and to design programmes that change over time to become more appropriate and relevant. At their heart, adaptive management approaches emphasise the ability to lean, 'unlearn' and adapt programming accordingly. The cornerstone of effective learning is the creation, gathering, accumulation, interpretation and use of data and evidence. This working paper provides development professionals with tools, strategies and ideas to help them use evidence for adaptive management in practical and evidence-informed ways. Key messages - Evidence is central to effective and rigorous adaptive management. However, despite this central importance, exactly how evidence has been used to inform decisions for adapting development and humanitarian programmes in the past remains unobservable to many. - There is a need to strengthen and document evidence-informed adaptive management. This working paper proposes a roadmap to do this. - Those seeking to use evidence for adaptive management will need to manage trade-offs between ensuring a rigorous, documented (and auditable) trail of evidence-informed actions, being pragmatic about the time and resources allocated to documentation and recognising that it may be necessary to proceed without rigorous evidence when it is unavailable.

Leaving No One Behind in a Digital World - Hernandez, Kevin; Roberts, Tony (K4D Emerging Issues Report, 2018)

Are you doing what’s needed to get the state to respond to its citizens? Or are you part of the problem? - Herringshaw, Vanessa (Making All Voices Count, 2017)
Three challenges from Making All Voices Count research on responsive governance

Mentoring Programmes: Supporting Effective Technology Use in Transparency and Accountability Organisations - Herringshaw, Vanessa; Faith, Becky (Transparency & Accountability Initiative, 2015)
The global movement to hold governments and companies accountable is growing rapidly, and technology can play a vital role. Some actors harness this potential to brilliant effect. But many others waste precious money and staff time on technology that isn’t a good fit for their aims or capacity. Mentorships can be a really effective way …

Complexity-Aware Monitoring and Evaluation - Hertz, Tilman; Brattander, Eva; Rose, Loretta (Journal of MultiDisciplinary Evaluation, 2021)
Background: Addressing today’s sustainability challenges requires adopting a systemic approach where social and ecological systems are treated as integrated social-ecological systems. Such systems are complex, and the international development sector increasingly recognises the need to account for the complexity of the systems that they seek to transform. Purpose: This paper sketches out the elements of a complexity-aware monitoring and evaluation (M&E) system for international development programmes in the area of sustainable development.  Setting: Not applicable. Data Collection and Analysis: The authors draw on existing literature on complexity and evaluation and on their own experience from working in the field of M&E. Findings: An M&E system should not be seen simply as a tool to track compliance against a pre-determined theory of change. Instead, it is most useful as a real-time approach, constantly defining and re-defining narratives for change that help push systems along trajectories of interest. Dealing with complexity involves embracing uncertainty; and this challenges established notions of accountability—something which funders and implementers must begin to redefine together. Keywords: monitoring; evaluation; complexity; social-ecological systems; international development programmes; narratives for change; theory of change

Monitoreo ciudadano en México: Participación ciudadana para mejorar la provisión de servicios públicos - Hevia, Felipe (CIDE, 2016)
Existe un amplio consenso sobre la importancia de los sistemas de monitoreo y evaluación para la mejora de políticas, programas y servicios públicos. Sin embargo, en la literatura especializada sorprende el escaso interés por analizar un tipo

Understanding Social Accountability: Politics, Power and Building New Social Contracts - Hickey, Sam; King, Sophie (The Journal of Development Studies, 2016)
Calls to deepen levels of social accountability within social protection interventions need to be informed by the now extensive experience of promoting social accountability in developing countries. Drawing on a systematic review of over 90 social accountability interventions, including some involving social protection, this paper shows that politics and context are critical to shaping their success. We argue that the politics of social protection and of social accountability resonate strongly with the broader project of transforming state-society relations in developing countries. This requires a reconceptualisation of social accountability and social protection in terms of the broader development of ‘social contracts’, and that the current emphasis on promoting bottom-up forms of accountability needs to be balanced by efforts to strengthen and legitimise public authority in developing countries.

Incorporating Geospatial Analysis into USAID Biodiversity Program Design - Higgins, Mark; Salafsky, Nick; Adeney, Marion; Petrova, Silvia (Measuring Impact, 2017)
Incorporating Geospatial Analysis into USAID Biodiversity Program Design is a biodiversity case example that describes the use of geospatial data and analysis for program design, including examples and lessons learned.

Adaptive Leadership: Accelerating Enterprise Agility - Highsmith, Jim (Addison-Wesley Professional., 2013)

Adaptive Software Development: A Collaborative Approach to Managing Complex Systems - Highsmith, Jim (Dorset House Publishing, 2000)
In today's turbulent e-business world, software project teams that survive and thrive won't be those that continue their traditions of optimization, efficiency, and control, but those that exhibit adaptability, speed, and collaboration. Adaptive Software Development is targeted at software teams where competition creates extreme pressure on the delivery process. Four goals of the book are * to support an adaptive culture in which change and uncertainty are assumed to be the natural state * to introduce frameworks to guide the iterative process of managing change * to institute collaboration, the interaction of people on interpersonal, cultural, and structural levels * to add rigor and discipline to the RAD approach, making it scalable to the uncertainty and complexity of real-world undertakings. This innovative text, grounded in the science of complex adaptive systems theory, offers a practical, realistic approach to managing the high-speed, high-change projects characteristic of our highly uncertain economy.

Agile Project Management: Creating Innovative Products - Highsmith, Jim (Addison Wesley, 2009)
Best practices for managing projects in agile environments―now updated with new techniques for larger projectsToday, the pace of project management moves faster. Project management needs to become more flexible and far more responsive to customers. Using Agile Project Management (APM), project managers can achieve all these goals without compromising value, quality, or business discipline. In Agile Project Management, Second Edition, renowned agile pioneer Jim Highsmith thoroughly updates his classic guide to APM, extending and refining it to support even the largest projects and organizations.  Writing for project leaders, managers, and executives at all levels, Highsmith integrates the best project management, product management, and software development practices into an overall framework designed to support unprecedented speed and mobility. The many topics added in this new edition include incorporating agile values, scaling agile projects, release planning, portfolio governance, and enhancing organizational agility. Project and business leaders will especially appreciate Highsmith’s new coverage of promoting agility through performance measurements based on value, quality, and constraints. This edition’s coverage includes: Understanding the agile revolution’s impact on product development Recognizing when agile methods will work in project management, and when they won’t Setting realistic business objectives for Agile Project Management Promoting agile values and principles across the organizationUtilizing a proven Agile Enterprise Framework that encompasses governance, project and iteration management, and technical practicesOptimizing all five stages of the agile project: Envision, Speculate, Explore, Adapt, and CloseOrganizational and product-related processes for scaling agile to the largest projects and teamsAgile project governance solutions for executives and management  The “Agile Triangle”: measuring performance in ways that encourage agility instead of discouraging itThe changing role of the agile project leader

Knowledge Networks: Innovation Through Communities of Practice - Hildreth, Paul; Kimble, Chris (IGI Global, 2003)
Knowledge Networks: Innovations Through Communities of Practice draws on the experience of people who have worked with CoPs in the real world and to present their combined wisdom in a form that is accessible to a wide audience. CoPs are examined from a practical, rather than a purely academic point of view. The book also examines the benefits that CoPs can bring to an organization, provides a number of case studies, lessons learned and sets of guidelines. It also looks at virtual CoPs and to the future by asking 'what next?' This book is a resource for all people who work with CoPs - both in academia and in the real world.

Sensing and Cyberinfrastructure for Smarter Water Management: The Promise and Challenge of Ubiquity - Hill David; Kerkez Branko; Rasekh Amin; Ostfeld Avi; Minsker Barbara; Banks M. Katherine (Journal of Water Resources Planning and Management, 2014)

Engaging Farmers in Climate Change Adaptation Planning: Assessing Intercropping as a Means to Support Farm Adaptive Capacity - Himanen, Sari J.; Mäkinen, Hanna; Rimhanen, Karoliina; Savikko, Riitta (Agriculture, 2016)
Agriculture is one of the most vulnerable and adaptation-prone sources of livelihood facing climate change. Joint adaptation planning by farmers and researchers can help develop practically feasible and environmentally and economically sound adaptation actions as well as encourage the proactive building of farm adaptive capacity. Here, the perceptions of Finnish farmers and rural stakeholders regarding intercropping, the cultivation of two or more crop genotypes together in time and space, as a means to prepare for climate change, were collected in an open workshop. Our aim was to identify the potentials and challenges associated with intercropping, its role as an adaptation strategy, and in farm adaptive capacity. Qualitative analysis revealed better yield security, increased nutrient and protein self-sufficiency, soil conservation and maintenance, reduced pathogen pressure and regulation of water dynamics as the main perceived potentials of intercropping. Potentials relating to the farm economy and environment were also recognized. The main challenges associated with intercropping were related to the lack of information on crop variety performance and optimal yielding in mixtures, industry and policy requirements for seed purity, more complicated crop management and harvesting, and the economic risks associated with experimenting with novel mixtures. Nitrogen-fixing legumes; deep-rooted species, such as lucerne (Medicago sativa L.); special crops, such as herbs in forage mixtures; and autumn-sown winter oilseeds and cereals were highlighted as the most promising intercrops. Because the recognized potentials relate to the safeguarding of field cropping from anticipated climate change and the associated weather variability, we conclude that intercropping can serve as one adaptation strategy to strengthen the adaptive capacity of Finnish farms. However, assuring markets and policies that allow the development of intercropping, performing experiments to assess the benefits and implement options in practice, and providing farmers and farm advisors with more knowledge on the method represent the critical prerequisites for the broader adoption of intercropping.
Keywords: Adaptation planning; Adaptive capacity; Climate change; ecological intensification; intercropping; yield security

An introduction to the Learning Architecture - HLA (Humanitarian Leadership Academy, 2017)
The Learning Architecture is built on a flexible Learning Methodology and provides principles, technologies and behaviours of Scaffolded Social Learning.

Adaptive Management Practice in the UK Government - The MUVA case - HLS (Human Learning Systems, 2019)
Case study about the MUVA programme in Mozambique. (Maybe it "misses the point of AP which is not learning for learning. Is learning for impact. The word impact doesn’t even come up once!") Adaptive Management programming within the Foreign & Commonwealth Development Office demonstrates that the UK Government has examples of optimising for learning within its existing management practice. However, currently, the adaptive management practices are unhelpfully framed by an approach which limits both their effectiveness and scope. The programmes use learning as their metastrategy. They succeed in connecting practicebased learning with strategic learning, and make a shift towards reframing accountability as accountability for learning. This learning strategy is enabled by funding and contract management arrangements which prioritise learning. Both at macro-level of programme management and micro level of de-risking experimentation and enabling necessary failure by decoupling people’s job security from potential failure.

The Futures Toolkit: Tools for Futures Thinking and Foresight Across UK Government - HM Government (2017)

The Hidden Life of Theories of Change - Ho, Wenny; Tamas, Peter; van Wessel, Margit (Hivos, 2020)
Theory of Change is thought to be very useful for learning and adaptive management of complex interventions such as advocacy. Nevertheless, the use of Theory of Change is also under critique. One common criticism is that Theory of Change is often used as a framework that fixes agreements rather than as a living, guiding tool that helps reflection and adaptation. However, while such criticism stresses forms of control, little research has looked at the way Theory of Change and advocacy practice relate. This is a pertinent issue considering that formally agreed Theories of Change and realities on the ground can be very different. This raises questions: Do advocates work in ways different from what Theory of Change states, and if so, how, and why? How does the way they strategize relate to formal Theories of Change? With what implications? In this brief, we explore these more hidden aspects of the life of Theories of Change.

Stop Trying to Save the World: Big ideas are destroying international development - Hobbes, Michael (New Republic, 2014)
Gives some examples of poorly designed interventions (e.g. The play pump)

Network Weaver Handbook - Holley, June (Network Weaver, 2012)

Transformative Networks Are Multiscalar - Holley, June (NetworkWeaver, 2018)
This is an excerpt from the Network Weaver Handbook. I’ll be writing an update with my latest research and thinking on the topic in the coming weeks so stay tuned!     June Holley How does tr…

Measuring Up: Evaluating the Impact of P/CVE Programs - Holmer, Georgia; Bauman, Peter (United States Institute of Peace, 2018)
This report considers the various conceptual and practical challenges in measuring the impact and value of programs designed to prevent and counter violent extremism (P/CVE). It examines potential solutions and emphasizes the significance of efforts to assess changes in attitudes, behaviors, and relationships. The report was developed in tandem with “Taking Stock: Analytic Tools for Understanding and Designing P/CVE Programs” and seeks to help advance more rigor and sustainability in P/CVE programming.

How AI Fits into Lean Six Sigma - Holweg, Matthias; Davenport, Thomas H.; Snyder, Ken (Harvard Business Review, 2023)
AI already is being used in some areas of process improvement, and the usage of this technology — including generative AI — promises to grow. That’s because it can perform tasks faster and much less expensively than humans alone. But it will never fully replace people — and that poses management challenges.

Actually Navigating by Judgment: Towards a new paradigm of donor accountability where the current system doesn’t work - Honig, Dan (Center for Global Development, 2020)
This working paper explores how donors can move towards greater Navigation by Judgment, highlighting the actions people inside and outside aid agencies can work to make change—encouraging more Navigation by Judgment on the margin, starting today.

Managing Better: What All of Us Can Do to Encourage Aid Success - Honig, Dan (Center for Global Development, 2020)
Management by way of top-down controls and targets sometimes gets in the way of aid donors’ aims, undermining project success. These unhelpful controls often stem from a need to account for performance; legislatures or executive boards induce agencies to exercise tight process controls and orient projects towards what is measurable and reportable.

Navigation by Judgment: Why and when Top Down Management of Foreign Aid Doesn't Work - Honig, Dan (Oxford University Press, 2018)
Foreign aid organizations collectively spend hundreds of billions of dollars annually, with mixed results. Part of the problem in these endeavors lies in their execution. When should foreign aid organizations empower actors on the front lines of delivery to guide aid interventions, and when should distant headquarters lead? In Navigation by Judgment, Dan Honig argues that high-quality implementation of foreign aid programs often requires contextual information that cannot be seen by those in distant headquarters. Tight controls and a focus on reaching pre-set measurable targets often prevent front-line workers from using skill, local knowledge, and creativity to solve problems in ways that maximize the impact of foreign aid. Drawing on a novel database of over 14,000 discrete development projects across nine aid agencies and eight paired case studies of development projects, Honig concludes that aid agencies will often benefit from giving field agents the authority to use their own judgments to guide aid delivery. This "navigation by judgment" is particularly valuable when environments are unpredictable and when accomplishing an aid program's goals is hard to accurately measure. Highlighting a crucial obstacle for effective global aid, Navigation by Judgment shows that the management of aid projects matters for aid effectiveness.

Making good on donors’ desire to Do Development Differently - Honig, Dan; Gulrajani, Nilima (Third World Quarterly, 2018)
AbstractForeign aid donors are increasingly focused on changing the way their development agencies function. This discourse has focused on desired qualities, including greater knowledge of local contextual realities, appropriate adaptation to context and greater flexibility to respond to changing circumstances. We argue that more attention needs to be devoted to the achievement of these qualities and turn to contingency theory to identify some under-exploited ways to ?do development differently?. The qualities sought by donors are emergent properties of complex organisational systems and will only be achieved through a micro-level and interlinked focus on the fundamentals of organisation.

The Limits of Accounting-Based Accountability in Education (and Far Beyond): Why More Accounting Will Rarely Solve Accountability Problems - Honig, Dan; Pritchett, Lant (Center for Global Development, 2019)
Accountability is rightly at the center of the conversation regarding how to improve governance systems, particularly health and education systems. But efforts to address accountability deficits often focus primarily on improving what can be counted and verified—what we term “accountingbased accountability.” We argue that introducing greater accounting-based accountability will only very rarely be the appropriate solution for addressing accountability problems. We illustrate this by exploring the role of Accountability ICT in (not) improving education system performance. Strengthening “real” accountability is not the same as improving data systems for observation and verification, and often attempts at the latter undermine the former. The development discourse’s frequent semantic misunderstanding of the term “accountability” has pernicious effects on system reform efforts and ultimately global welfare.

Navigating by Judgment: Organizational Structure, Autonomy, and Country Context in Delivering Foreign Aid - Honig, Daniel (Harvard University, Graduate School of Arts & Sciences, 2015)
This dissertation examines when initiatives by International Development Organizations (IDOs) are more, and less, successful. The core argument is that allowing field-level agents to drive initiatives – what I call organizational Navigation by Judgment – will often be the most effective way to deliver aid. This inverts what a classical application of the principal agent model – the workhorse of studies of public management and bureaucracy – would predict, with better performance resulting from less control. In the delivery of foreign aid the costs of monitoring to the principal are often overshadowed by the deleterious effects of the monitoring itself. The core of the argument is that development implementation requires soft information, tacit knowledge, and flexibility that are crowded out by tight controls or an organizational navigation strategy focused on short term measurement and targets. As a result there are increasing returns to Navigation by Judgment in environments that are uncertain or difficult to understand from the outside and tasks where outputs are difficult to observe and/or poorly correlated with long term intervention goals. Insecure political authorizing environments which constrain the autonomy of IDOs prevent these organizations from Navigating by Judgment in situations where this is the best strategy. Empirically, this dissertation examines a cross-IDO dataset of projects (including over 14,000 projects over 50 years over 9 organizations), which I have assembled. It also examines eight cases of development interventions in Liberia and South Africa. These cases are matched pairs comparing the performance and navigation strategies of the US Agency for International Development (a low autonomy IDO) and the UK’s Department for International Development (a higher autonomy IDO) in capacity building and health sector interventions.
Keywords: IMPORTANT

When Reporting Undermines Performance: The Costs of Politically Constrained Organizational Autonomy in Foreign Aid Implementation - Honig, Daniel (International Organization, 2019)

Signal Left, Turn Right: Central Rhetoric and Local Reform in China - Huang, Haifeng (Political Research Quarterly, 2013)
How have local governments in China been able to break through central policy restrictions in a unitary and authoritarian political system? Why is China's official discourse in the reform era often so conservative and unfavorable to reform? The author argues the two issues are components of a signaling game between China's central government and local officials, in which local officials know that the center may be reformist, but the reformist center imitates the rhetoric of a conservative center to control the pace of local liberalization. The result is a gradualist reform of "signaling left, turning right," with glaring incongruity of speech and actions in the process.

Mind the gaps: What's missing in political economy analysis and why it matters - Hudson, David; Marquette, Heather (A Governance Practitioner’s Notebook: Alternative Ideas and Approaches, 2015)
The Governance Practitioner’s Notebook takes an unusual approach for the OECD-DAC Network on Governance (GovNet). It brings together a collection of specially written notes aimed at those who work as governance practitioners within development agencies. It does so, however, without attempting to offer definitive guidance – instead aiming to stimulate thinking and debate. To aid this process the book is centred on a fictional Governance Adviser. The Notebook’s format provides space for experts to speak on today’s governance issues: politics, public sector reform and stakeholder engagement. It encourages debate, charts the evolution of donor thinking, and highlights future challenges in the age of the Sustainable Development Goals. Each section introduces both technical issues and major areas of debate, providing ideas for future development support to institutional reform.
Keywords: IMPORTANT

Everyday Political Analysis - Hudson, David; Marquette, Heather; Waldock, Sam (Developmental Leadership Program, 2016)
This short note introduces a stripped-back political analysis framework designed to help frontline development practitioners make quick but politically-informed decisions. It aims to complement more in-depth political analysis by helping programming staff to develop the 'craft' of political thinking in a way that fits their everyday working practices. Everyday Political Analysis involves two steps: Understanding interests: What makes people tick? Understanding change: What space and capacity do people have to effect change? For each step five questions, accompanied by prompts, aim to help staff to conduct quick political analysis. The EPA framework can be used at any stage of the aid management cycle, and can help users to respond rapidly to unexpected change. We are keen to hear back from people on their experience of using EPA to help us adapt the framework. Was it useful (or not)? Do people tend to use just one or both steps? Are there missing statements or prompts that would improve the analysis? Please email us at info [at] dlprog.org.

What works for Social Accountability? Findings from DFID’s Macro Evaluation - Hughes, Claire (Policy Briefing, 2017)

Beyond Logframe: Critique, Variations and Alternatives - Hummelbrunner, Richard (Beyond Logframe; Using Systems Concepts in Evaluation, 2010)
Over the last decades, the Logical Framework Approach (LFA) has become universally known and has assumed a key role for planning and managing development interventions. LFA, however, is not uncontroversial and the approach has been subject to criticism, concerning both its theoretical foundations and practical use. Despite these criticisms LFA’s position has not been fundamentally weakened and while many donors acknowledge its limits and weaknesses, they maintain (some would say impose) its use as a planning and monitoring tool. This chapter reviews some of the experience gained with LFA and outlines major attempts to develop variations – or move beyond it altogether. The first section briefly describes the LFA concept and summarizes the main points of critique. Section 2 then explores some variations which have been developed in response to this critique and to improve LFA as a management tool. Section 3 proposes a systemic alternative to logframe and Section 4 outlines alternatives to LFA which have recently been introduced in German development aid.

Process Monitoring of Impacts: Towards a new approach to monitor the implementation of Structural Fund Programmes - Hummelbrunner, Richard; Huber, Wolf; Arbter, Roland (ÖAR Regionalberatung, 2005)

What can Australia learn from the public service revolution taking hold in Europe? - Hurcombe, Sarah (Centre For Public Impact (CPI), 2021)
Sarah Hurcombe shares what she's learning from the public service revolution building momentum in Europe.

Logics of Government Innovation and Reform Management in China - Husain, Lewis (STEPS centre, 2015)
Since the beginning of reforms in the late 1970s, China has developed rapidly, transforming itself into a middle-income country, raising hundreds of millions out of poverty and, latterly, developing broad-based social protection systems. The country’s approach to reform has been unorthodox, leading many to talk of a specific Chinese model of development. This paper analyses the role of innovation (chuangxin) and experimentation in the Chinese government repertoire and their contribution to management of change during the rapid, complex and interconnected reforms that China is undergoing. ‘Innovation’ is understood as the process of generation, putting into use, and spread, of new ideas. This contemporary focus on innovation is an extension of an older Chinese government attachment to sub-national initiative in policy formulation and development that goes back to the beginnings of the PRC and before. Central government backing for, and endorsement of, proactive agency on the part of sub-national governments responds to a belief that China is too large, and conditions around the country too diverse, to allow adoption of ‘one size fits all’ policy. Sub-national governments are expected to show initiative in adapting policy locally, and creating locally-useable policy solutions within the overall scope of central policy mandates/ paradigms. The paper argues that innovation by sub-national government is systemically embedded: while central government sets the policy agenda, local governments are frontline managers, and develop a range of policy practices. Differences in conditions between localities mean that multiple variant policy practices are often in circulation at any one time. While innovation is not quantifiable in the aggregate, there is much controlled experimentation, freewheeling innovation, and trial and error, all of which are part of a search for new policy fixes and institutional solutions. Many forms of policy transfer and learning are in evidence, including much central learning from sub-national models, as well as sub-national circulation of a range of innovative policy practices. While much government innovation is not ‘original’, and may be ‘inefficient’ or of little systemic usefulness, overall, the churn of government innovation remains valuable in underpinning systemic adaptation and reform. The paper situates the analysis of government innovation within a larger framework on the functioning of Chinese government and international literature on policy transfer, and outlines an agenda for future research on the structural bases of Chinese government innovation and its contribution to adaptive management.

Policy experimentation and innovation as a response to complexity in China’s management of health reforms - Husain, Lewis (Globalization and Health, 2017)
There are increasing criticisms of dominant models for scaling up health systems in developing countries and a recognition that approaches are needed that better take into account the complexity of health interventions. Since Reform and Opening in the late 1970s, Chinese government has managed complex, rapid and intersecting reforms across many policy areas. As with reforms in other policy areas, reform of the health system has been through a process of trial and error. There is increasing understanding of the importance of policy experimentation and innovation in many of China’s reforms; this article argues that these processes have been important in rebuilding China’s health system. While China’s current system still has many problems, progress is being made in developing a functioning system able to ensure broad population access. The article analyses Chinese thinking on policy experimentation and innovation and their use in management of complex reforms. It argues that China’s management of reform allows space for policy tailoring and innovation by sub-national governments under a broad agreement over the ends of reform, and that shared understandings of policy innovation, alongside informational infrastructures for the systemic propagation and codification of useful practices, provide a framework for managing change in complex environments and under conditions of uncertainty in which ‘what works’ is not knowable in advance. The article situates China’s use of experimentation and innovation in management of health system reform in relation to recent literature which applies complex systems thinking to global health, and concludes that there are lessons to be learnt from China’s approaches to managing complexity in development of health systems for the benefit of the poor.

Participatory design with marginalized people in developing countries: Challenges and opportunities experienced in a field study in Cambodia - Hussain, Sofia; Sanders, Elizabeth B.-N.; Steinert, Martin (International Journal of Design, 2012)

International Public Participation Models 1969-2020 - Hussey, Sally (Bang The Table, 2020)
International Public Participation Models 1969 – 2020 provides an essential resource of 60 different models to better map public participation in practice and theory.

Shoulder to Shoulder - Hymowitz, Dan (Art of Delivery series, 2016)

"That's how the light gets in": Making Change in Closing Political Environments - IBP (International Budget Partnership, 2017)
This collection of learning and reflection essays from the International Budget Partnership's 2016 Annual Report illustrate the multidimensional nature of budget work and democratic engagement. Read more.

Achieving value for money through procurement. Part 2: DFID’s approach to value for money through tendering and contract management - A performance review - ICAI (Independent Commission for Aid Impact (ICAI), 2018)
Summary In 2016-17, the Department for International Development (DFID) spent £1.4 billion, or 14% of its budget, through commercial suppliers on contracts ranging from school construction to family planning services and the delivery of humanitarian aid. The quality of its procurement and contract management – how it engages and manages commercial firms to support the delivery of aid programmes on time, to budget and at the appropriate quality – is a key driver of value for money for UK aid. It is also a subject of considerable Parliamentary and public interest. In recent years, DFID has implemented a range of initiatives to strengthen its procurement practices and embed commercial capability across the department – including its 2017 Supplier Review, undertaken to address concerns about excessive profit-making by DFID suppliers. This review on achieving value for money through procurement (part 2) was published in September 2018. We awarded a green-amber score and made three recommendations to government. The follow-up to this review was published in July 2020, and rated the government’s response as ‘inadequate’, as a result of DFID’s failure to put in place a formal contract management regime, despite the risks this entails for programme results. We will return to this in the next follow-up in summer 2021. Review Our review found an appropriate overall approach to procurement with good performance in most areas of tendering, but significant weaknesses in contract management. We therefore awarded a green-amber score and made three recommendations to government. Findings Since 2015, DFID has progressed towards a more mature procurement approach. DFID’s tender process follows current EU legislation and UK government guidelines and cross-government peer reviews confirm improvements in DFID’s procurement approach. The Supplier Review lent momentum to the reforms but risks having unintended consequences. Poor consultation with suppliers has heightened the risk of unintended consequences. DFID is reviewing its business processes to protect aid recipients from sexual abuse and exploitation. DFID does not always choose the most appropriate procurement process but a new strategic sourcing process has resulted in stronger procurement planning. DFID has built up its commercial capability, but this will need to be an ongoing process. The contract management function in DFID is poorly defined and contracts are frequently amended or extended beyond their advertised length and values. Inception phases are often too short for adequate preparation and planning. Progress on flexible and adaptive programming will require more innovative approaches to tendering and contract management. Recommendations Before the next major revision of its supplier code and contracting terms, or future changes that may materially affect suppliers, DFID should conduct an effective consultation process with its supplier market, to ensure informed decisions and minimise the risks of unintended consequences. DFID should accelerate its timetable for acquiring a suitable management information system for procurement, to ensure that its commercial decisions are informed by data. DFID should instigate a formal contract management regime, underpinned by appropriate training and guidance and supported by a senior official responsible for contract management across the department. The new regime should include appropriate adaptive contract management techniques, to ensure that supplier accountability is balanced with the need for innovation and adaptive management in pursuit of development results.

DFID’s approach to delivering impact - ICAI (Independent Commission for Aid Impact (ICAI), 2015)
UK aid, at its best, makes a real and positive difference to the lives and livelihoods of poor people around the world. Ensuring the best possible performance across a large and multifaceted aid programme is, however, a complex management challenge. This report reviews ICAI’s previous 44 reports and looks at how well DFID ensures positive, long-term, transformative impact across its work.

DFID’s approach to value for money in programme and portfolio management - ICAI (Independent Commission for Aid Impact (ICAI), 2018)
ICAI published this review on DFID’s approach to value for money in February 2018, and as value for money is both a process and an outcome and cuts across all aspects of DFID’s operations, did not score this review. We made five recommendations and published a follow-up to this review in July 2019. All UK government departments are required to achieve value for money in their use of public funds. In recent years, DFID has been working to build value for money considerations further into its management processes and its relationships with implementers and multilateral partners, establishing itself as a global champion on value for money. Review This review was published in February. Though it was not rated, ICAI made five recommendations, and found that the Department for International Development’s approach to value for money was helping to make UK aid spending go further, but improvements are still needed. Findings This review found that DFID has strengthened its processes and systems for ensuring it gains maximum value for each pound spent, has taken swift remedial action to tackle under-performing programmes, and has become a strong global champion on value for money. The review also found that DFID has been diligent in its efforts to cut waste, detect fraud, and improve efficiency, and that this work is improving the return on the UK investment in aid. However, the review found that DFID’s approach was not adequately reporting and capturing results and value for money at the country portfolio level, or how programmes work together to deliver lasting impact, including reducing future dependency on aid. It also found that weakness in the annual review process could undermine DFID’s approach to value for money. It found that targets were frequently revised, and that there could be pressures for optimistic scoring of programmes. Recommendations Based on this review, we made the following five recommendations to help DFID improve its approach to value for money still further: DFID country offices should articulate cross-cutting value for money objectives at the country portfolio level, and should report periodically on progress at that level. Drawing on its experience with introducing adaptive programming, DFID should encourage programmes to experiment with different ways of delivering results more cost-effectively, particularly for more complex programming. DFID should ensure that principles of development effectiveness – such as ensuring partner country leadership, building national capacity and empowering beneficiaries – are more explicit in its value for money approach. Programmes should reflect these principles in their value for money frameworks, and where appropriate incorporate qualitative indicators of progress at that level. DFID should be more explicit about the assumptions underlying the economic case in its business cases, and ensure that these are taken into account in programme monitoring. Delivery plans should specify points in the programme cycle when the economic case should be fully reassessed. Senior responsible owners should also determine whether a reassessment is needed following material changes in the programme, results targets or context. Annual review scores should include an assessment of whether programmes are likely to achieve their intended outcomes in a cost-effective way. DFID should consider introducing further quality assurance into the setting and adjustment of logframe targets.

DFID’s partnerships with civil society organisations - A performance review - ICAI (Independent Commission for Aid Impact (ICAI), 2019)
DFID values civil society organisations (CSOs), but its funding and partnership practices do not fully support the long-term health of the civil society sector.

How DFID learns - ICAI (Independent Commission for Aid Impact (ICAI), 2014)
Excellent learning is essential for UK aid to achieve maximum impact and value for money. We take learning to mean the extent to which DFID uses information and experience to influence its decisions. Each ICAI review assesses how well learning takes place. Our reports to date indicate a mixed performance. This review seeks to identify the way DFID learns and what inhibits it from doing so consistently. We drew on our reviews, assessed data from DFID’s own surveys and carried out interviews inside and outside the department. Review DFID generates considerable volumes of information, much of which, such as funded research, is publicly available. DFID does not clearly or consistently link this investment to how it can deliver better impact. We made five recommendations and gave an amber-red score. Findings DFID does not clearly identify how its investment in learning links to its performance and delivering better impact. DFID has the potential to be excellent at organisational learning if its best practices become common. DFID staff learn well as individuals. They are highly motivated and DFID provides opportunities and resources for them to learn. DFID is not yet, however, managing all the elements that contribute to how it learns as a single, integrated system. DFID does not review the costs, benefits and impact of learning. Insufficient priority is placed on learning during implementation. The emphasis on results can lead to a bias to the positive. Learning from both success and failure should be systematically encouraged. Recommendations DFID needs to focus on consistent and continuous organisational learning based on the experience of DFID, its partners and contractors and the measurement of its impact, in particular during the implementation phase of its activities. All DFID managers should be held accountable for conducting continuous reviews from which lessons are drawn about what works and where impact is actually being achieved for intended beneficiaries. All information commissioned and collected (such as annual reviews and evaluations) should be synthesised so that the relevant lessons are accessible and readily usable across the organisation. The focus must be on practical and easy-to-use information. Knowhow should be valued as much as knowledge. Staff need to be given more time to acquire experience in the field and share lessons about what works and does not work on the ground. DFID needs to continue to encourage a culture of free and full communication about what does and does not work. Staff should be encouraged always to base their decisions on evidence, without any bias to the positive.

Building a Learning Community in Liberia Through Partnerships at iCampus - iCampus (2018 CLA Case Competition, 2018)
Accountability Lab Liberia (ALab) and iLab Liberia established iCampus- a shared innovation, co-working and community space for organizations focusing on the intersection of technology, accountability and social change in Liberia. ALab implements DAI’s Learning activities at iCampus, and it is referred to as the Strategic Learning Partner of USAID Liberia Accountability and Voice Initiative (LAVI). LAVI is a five year USAID program with an overarching goal to strengthen multi-stakeholder partnerships to advocate for and monitor policy and accountability reforms. The iCampus CLA case scenario captures joint efforts of Accountability Lab Liberia, iLab Liberia and USAID LAVI, to ensure that learning and methodologies are shared and applied by development actors; address common capacity challenges that particularly hinder civil society from engaging in evidence-based and issues-based advocacy using various approaches, including network-building, creation of a cross-sectoral learning space- through collaboration and learning. In 2017, iCampus conducted a learning mapping survey to understand how organizations are learning in Liberia. The result indicated that frameworks and concepts of organizational learning are not yet well understood or used to guide strategic or programmatic practices, by either Liberian or international organizations. ALab, iLab and USAID LAVI then adopted a set of strategies, that led to the implementation of a CLA, enabling the establishment of the first multi-sectoral learning community in Liberia.

Enabling citizen-driven improvement of public services. Leveraging Technology to Strengthen Accountability in Nigerian Healthcare - ICT4SA (World Bank and Reboot, 2015)
Effective public service delivery begins with knowing whether the services offered are working as intended. We launched a system to enable citizen input on the delivery of public services.

How to Apply Agile Principles to International Development M&E - ICTworks (ICT Works, 2016)
We all want to be good at our jobs. We want to accomplish the things we set out to do. If we aren’t accomplishing them, we want to figure out why or try new solutions. The trend toward Adaptive M&E is exactly that: a desire to be better at our jobs. Similar trends exist in the software world (agile) and in manufacturing and start-ups (lean). But by any name, this process of seeking to improve is about speeding up decision-making and solution delivery by focusing on incremental, iterative planning and execution. I presented on agile with Monalisa Salib of USAID Learning Lab at MERL Tech 2016. I talked about creating software; she talked about creating evaluation tools for USAID missions, but we were describing the same basic process: Develop, Release, Reflect, and Adapt. You can see our slide deck for more details, but here are the key discussion takeaways that won’t show up there: 1: Eyes on the Prize Any effort should start with a clear definition of what you’re hoping to achieve.
Keywords: IMPORTANT

Should the Principles of Digital Development be Agile? - ICTworks (ICT Works, 2016)
Recently, Abt Associates endorsed the “Principles of Digital Development.” These nine principles have been widely adopted by international development funders and practitioners to absorb and disseminate technology best practices in the field of international development. More than 50 organizations ranging from various offices in the United Nations and the U.S. Agency for International Development (USAID) down to niche technology providers have endorsed the principles. The principles are aimed at moving the field away from a disconnected series of technology pilots, each of which are largely ad hoc and duplicate many of the same lessons, and toward scalability and sustainability. Ann Mei Chang, the Executive Director of the U.S. Global Development Lab at USAID in her foreword to the recent report “From Principle to Practice: Implementing the Principles for Digital Development”, characterized the problem that these principles are trying to address as follows: “Pilots have failed to move

The field guide to human-centered design: design kit - IDEO (IDEO, 2015)
Keywords: Design; IMPORTANT; Methode