Innovation History - Better Evaluation (Better Evaluation)
Preparing an ‘innovation history’ is an option for recording and reflecting on an innovation process. People who have been involved in the innovation jointly construct a detailed written account (sometimes referred to as a ‘learning history’) based on their recollections and on available documents. The process of preparing this history stimulates discussion, reflection and learning among stakeholders. Subsequent planning drawing on the innovation history, can (i) build on the lessons learned, (ii) inform a shared vision, (iii) act as a catalyst for change and (iv) improve future performance. An innovation history is developed in stages. Based on the initial detailed account of the innovation process, more concise informational products can be prepared that summarize the innovation process for internal use. Products designed for wider dissemination of findings can help external parties build upon and expand their knowledge and understanding about how innovations are brought about. Such products may include public awareness materials, policy briefs and articles in professional journals. They may be based on the study of an individual case or on reviews that compare and contrast experiences across several cases. Innovation histories are underpinned by two sets of concepts that guide data gathering and analysis. The first set comes from a model of the innovation process called the ‘learning selection’ model (Douthwaite, 2002). The second set is derived from social network analysis (e.g. Cross and Parker, 2004). These concepts help participants involved in creating the innovation history to appreciate innovation as an evolutionary process driven by experiential learning cycles. The experimentation and learning involved in this evolutionary process leads to the generation of novelty, followed by its selection and promulgation. During this process, technologies become ‘fitter’, in the sense that they perform better. Stakeholder group’s social networks influence how this evolutionary learning selection process evolves hence the importance of identifying, tracking and documenting them.

Institutional Histories - Better Evaluation (Better Evaluation)
An institutional history (IH) is a narrative that records key points about how institutional arrangements – new ways of working – have evolved over time and have created and contributed to more effective ways to achieve project or programme goals. An IH is generated and recorded in a collaborative way by scientists, farmers and other stakeholders. A key intention behind institutional histories is to introduce institutional factors into the legitimate narrative of success and failure in research organizations. Histories can be written by using interviews and ‘writeshops’ to construct a timeline, gain a clear understanding of roles and relationships, enquire into what triggers successful innovations and reflect on failures. Lessons drawn from this analysis can be used to improve performance. The dialogue that is promoted between the actors during the preparation of institutional histories can promote learning and capacity building.

Most Significant Change - Better Evaluation (Better Evaluation)
The Most Significant Change (MSC) approach involves generating and analysing personal accounts of change and deciding which of these accounts is the most significant – and why. The are three basic steps in using MSC: Deciding the types of stories that should be collected (stories about what - for example, about practice change or health outcomes or empowerment) Collecting the stories and determining which stories are the most significant Sharing the stories and discussion of values with stakeholders and contributors so that learning happens about what is valued. MSC is not just about collecting and reporting stories but about having processes to learn from these stories – in particular, to learn about the similarities and differences in what different groups and individuals value. It provides some information about impact and unintended impact but is primarily about clarifying the values held by different stakeholders. By itself it is not sufficient for impact evaluation as it does not provide information about the usual experience but about the extremes. If you imagine a normal distribution of outcomes for individuals then the stories often come from the extremity of positive change. It can be useful to explicitly add a process to generate and collect stories from the extremity of little or negative change. MSC can be very helpful in explaining HOW change comes about (processes and causal mechanisms) and WHEN (in what situations and contexts). It can therefore be useful to support the development of programme theory (theory of change, logic models).

Outcome Mapping - Better Evaluation (Better Evaluation, 2013)
Outcome mapping (OM) is a methodology for planning, monitoring and evaluating development initiatives in order to bring about sustainable social change. As the name suggests, its niche is understanding outcomes; the so-called ‘missing-middle’ or ‘black box’ of results that emerge downstream from the initiative’s activities but upstream from longer-term economic, environmental, political or demographic changes. At the planning stage, the process of outcome mapping helps a project team or program be specific about the actors it intends to target, the changes it hopes to see and the strategies appropriate to achieve these. For ongoing monitoring, OM provides a set of tools to design and gather information on the results of the change process, measured in terms of the changes in behaviour, actions or relationships that can be influenced by the team or program.

Participatory Evaluation - Better Evaluation (Better Wvaluation)
Participatory evaluation is an approach that involves the stakeholders of a programme or policy in the evaluation process. This involvement can occur at any stage of the evaluation process, from the evaluation design to the data collection and analysis and the reporting of the study. A participatory approach can be taken with any impact evaluation design, and with quantitative and qualitative data. However, the type and level of stakeholder involvement will necessarily vary between different types, for example between a local level impact evaluation and an evaluation of policy changes (Gujit 2014, p.1). It is important to consider the purpose of involving stakeholders, and which stakeholders should be involved how, in order to maximise the effectiveness of the approach.

Participatory Rural Appraisal - Better Evaluation (Better Evaluation)
"Participatory Rural Appraisal (PRA) recently renamed Participatory Learning for Action (PLA), is a methodological approach that is used to enable farmers to analyse their own situation and to develop a common perspective on natural resource management and agriculture at village level. PRA is an assessment and learning process that empowers farmers to create the information base they need for participatory planning and action. Outsiders contribute facilitation skills and external information and opinions. Many different tools have been developed for use in PRA. There are four main classes: tools used in group and team dynamics; tools for sampling; options for interviews and dialogue; and options for visualisation and preparing diagrams. Most countries have had some experience with PRA and local publications are available. IIED regularly reports on new developments in its PLA notes (Pretty et al 1995)." (Bie, 1998)

Positive Deviance - Better Evaluation (Better Evaluation)
Positive Deviance (PD) refers to a behavioral and social change approach which is premised on the observation that in any context, certain individuals confronting similar challenges, constraints, and resource deprivations to their peers, will nonetheless employ uncommon but successful behaviors or strategies which enable them to find better solutions. Through the study of these individuals– subjects referred to as “positive deviants” - the PD approach suggests that innovative solutions to such challenges may be identified and refined from their outlying behavior.

Qualitative Impact Assessment Protocol (QUIP) - Better Evaluation (Better Evaluation)
The QUIP sets out to generate differentiated evidence of impact based on narrative causal statements elicited directly from intended project beneficiaries without use of a control group. Evidence of attribution is sought through respondents’ own accounts of causal mechanisms linking X to Y alongside Z rather than by relying on statistical inference based on variable exposure to X. This narrative data is intended to complement quantitative evidence on changes in X, Y and Z obtained through routine project monitoring.

Randomised Controlled Trial - Better Evaluation (Better Evaluation)
Randomised controlled trials (RCTs), or randomised impact evaluations, are a type of impact evaluation which uses randomised access to social programmes as a means of limiting bias and generating an internally valid impact estimate.

Realist Evaluation - Better Evaluation (Better Evaluation)
Realist evaluation is a form of theory-driven evaluation, but is set apart by its explicit philosophical underpinnings. Pawson and Tilley (1997) developed the first realist evaluation approach, although other interpretations have been developed since. Pawson and Tilley argued that in order to be useful for decision makers, evaluations need to identify ‘what works in which circumstances and for whom?’, rather than merely ‘does it work?. The complete realist question is: “What works, for whom, in what respects, to what extent, in what contexts, and how?”. In order to answer that question, realist evaluators aim to identify the underlying generative mechanisms that explain ‘how’ the outcomes were caused and the influence of context.

Social Return on Investment - Better evaluation (Better Evaluation, 2016)
Social Return on Investment (SROI) is a systematic way of incorporating social, environmental, economic and other values into decision-making processes. By helping reveal the economic value of social and environmental outcomes it creates a holistic perspective on whether a development project or social business or enterprise is beneficial and profitable. This perspective opens up new opportunities and forms the basis for innovative initiatives that genuinely contribute to positive social change and poverty reduction for all. SROI balances proving and improving or addresses the paradox between accountability and learning by placing the perspectives of the different stakeholders at the center of the valuation process. ​SROI originated in the USA from social enterprises interested in new ways to value the contributions they were making to society. It later arrived in Europe, where there is an increasing interest in the methodology as noted by recent publications by Context international cooperation in the Netherlands, the New Economics Foundation in the UK and the SROI Network head-quartered in the UK. SROI is used for planning purposes in terms of designing a Theory of Change, or Business Plan, and for assessing to what extent impact is realised or changes need to occur in the Business Plan. Although the SROI approach supports the thinking along the lines of a result chain, it does not support the idea of the components being connected in a linear fashion. The SROI approach is embedded in the acceptance of development taking place in situations of complexities. Here is a simple, illustrative example: A project aims to uplift the standard of living of people in a certain area and a beekeeping initiative is set up. As a result a beekeeper now enjoys regular meals whereas before this was not the case. In traditional Cost Benefit analyses, the value of the lunch would be measured in market prices. However after interviewing the beneficiaries, and applying some valuation tools, it turns out that the ‘real’ value is much higher than the market price; social value has been created above the market / economic value which is now being accounted for. Like traditional cost-benefit analysis, SROI includes a ratio; in this case a Social Return on Investment ratio. Where in traditional cost benefit analyses the ratios would be used to compare different projects, the SROI ratio is much more seen as one element in explaining and communicating general progress of certain developments. The number itself is not seen as the end goal. It can be interpreted as aiding the narrative of this particular initiative. The aspect of stakeholder perspectives is essential in the SROI approach. It is precisely the value perspectives of the stakeholders (and most importantly the key beneficiaries), assessed, not by assuming these values, but by thoughtfully and intellectually engaging the stakeholders themselves, which is at the heart of this innovative (e)valuation approach.

Success Case Method - Better Evaluation (Better Evaluation)
The Success Case Method (SCM) involves identifying the most and least successful cases in a program and examining them in detail. This approach was developed by Robert Brinkerhoff to assess the impact of organizational interventions, such as training and coaching, though the use of SCM is not limited to this context. It is a useful approach to document stories of impact and to develop an understanding of the factors that enhance or impede impact. The Success Case Method deliberately looks at the most, and least, successful participants of a program. The purpose is not to examine the average performance - rather, by identifying and examining the extreme cases, it asks: 'When the program works, how well does it work? What is working, and what is not?'.

Theory of change software - Better Evaluation (2023)
There are a number of options when it comes to using software to help create a logic model. These range from generic word processing tools (Word, Powerpoint, or their Google Doc or Mac equivalents), to software that has been specifically tailored for visualising Theories of Change, like TOCO or Miradi. You should consider what resources you have to invest in software, both in terms of cost and in time to learn and use the features. If you only have a short timeframe and have simple needs, then a basic tool may suit you better than some of the more complex software available. It's important to investigate a few options and see what is going to be best for you.

Utilization-Focused Evaluation - Better Evaluation (Better Evaluation)
Utilization-Focused Evaluation (UFE), developed by Michael Quinn Patton, is an approach based on the principle that an evaluation should be judged on its usefulness to its intended users. Therefore evaluations should be planned and conducted in ways that enhance the likely utilization of both the findings and of the process itself to inform decisions and improve performance.

Big Data to Data Science - Moving from “What” to “How” in the MERL Tech Space - Bertermann, Kecia; Robinson, Alexandra; Bamberger, Michael; Higdon, Grace Lyn; Raftre, Linda (MERL Tech, 2020)
This paper probes trends in the use of big data by a community of early adopters working in monitoring, evaluation, research, and learning (MERL) in the development and humanitarian sectors. Qualitative analysis was conducted on data from MERL Tech conference records and key informant interviews. Findings indicate that MERL practitioners are in a fragmented, experimental phase, with use and application of big data varying widely, accompanied by shifting terminologies. We take an in-depth look at barriers to and enablers of use of big data within MERL, as well as benefits and drawbacks. Concerns about bias, privacy, and the potential for big data to magnify existing inequalities arose frequently. The research surfaced a need for more systematic and broader sharing of big data use cases and case studies in the development sector.

Leading Change Through Adaptive Design - Berstein, Maya; Linsky, Marty (Stanford Social Innovation Review, 2016)
By integrating two practices—design thinking and adaptive leadership—social innovators can manage projects in a way that’s both creatively confident and relentlessly realistic.

Rediscovery of Traditional Ecological Knowledge as Adaptive Management - Berkes, Fikret; Colding, Johan; Folke, Carl (Ecological Applications, 2000)
Indigenous groups offer alternative knowledge and perspectives based on their own locally developed practices of resource use. We surveyed the international literature to focus on the role of Traditional Ecological Knowledge in monitoring, responding to, and managing ecosystem processes and functions, with special attention to ecological resilience. Case studies revealed that there exists a diversity of local or traditional practices for ecosystem management. These include multiple species management, resource rotation, succession management, landscape patchiness management, and other ways of responding to and managing pulses and ecological surprises. Social mechanisms behind these traditional practices include a number of adaptations for the generation, accumulation, and transmission of knowledge; the use of local institutions to provide leaders/stewards and rules for social regulation; mechanisms for cultural internalization of traditional practices; and the development of appropriate world views and cultural values. Some traditional knowledge and management systems were characterized by the use of local ecological knowledge to interpret and respond to feedbacks from the environment to guide the direction of resource management. These traditional systems had certain similarities to adaptive management with its emphasis on feedback learning, and its treatment of uncertainty and unpredictability intrinsic to all ecosystems.
Keywords: Adaptive Management; Resource management; Social learning; Traditional Ecological Knowledge; human ecology; resilience

Simple Habits for Complex Times: Powerful Practices for Leaders - Berger, Jennifer Garvey; Johnston, Keith (Stanford University Press, 2016)

ALL - The question of inclusiveness in ICT-mediated citizen engagement - Berdou, Evangelia (Think Piece, 2014)

Learning about New Technologies and the Changing Evidence Base for Social Science Research and Decision Making in International Development - Berdou, Evangelia (IDS Practice Paper in Brief, 2011)

Mediating Voices, Communicating Realities: Using Information Crowdsourcing Tools, Open Data Initiatives and Digital Media to Support and Protect the Vulnerable - Berdou, Evangelia (IDS, 2011)
This is the final report from a research project, supported by the UK Department for International Development, examining whether and how open ICT projects designed to support the poor can make a diff...

Lessons for PFM Reform - the PDIA Approach in Africa - Bento, Joana (Public Financial Management Blog - IMF, 2021)
Some important lessons are as follows: First, identifying a policy problem is critical for reforms to get traction both at a political and administrative level. Second, appropriate solutions can emerge from a process of experimentation, iteration, and adaptation. Third, building teams and institutional capabilities is a critical part of solving complex problems in a sustainable way.

Cognitive bias cheat sheet. An organized list of cognitive biases because thinking is hard - Benson, Buster (Better Humans, 2016)

Time to let go: remaking humanitarian action for the modern era - Bennett, Christina (ODI, 2016)
As the international humanitarian system faces a crisis of legitimacy, the Humanitarian Policy Group’s landmark report proposes a new model of humanitarian action.

Blurring the Boundaries: Citizen Action Across States and Societies - Benequista, Nicholas; Gaventa, John (Brighton: Citizenship DRC, 11., 2011)

Journalism from the ‘Silicon Savannah’: The Vexed Relationship Between Nairobi’s Newsmakers and its ICT4D Community - Benequista, Nicholas (Stability: International Journal of Security and Development, 2015)
During the course of a year-long knowledge exchange initiative called the Networked News Lab, a small group of Kenyan journalists and a PhD researcher from the London School of Economics and Political Science sought to identify opportunities for collaboration between newsmakers and practitioners from the field of information and communication technologies for development (ICT4D). In almost every instance, the project failed to promote cooperation between the two groups, though it succeeded in highlighting the fundamental issues that separate them. Drawing from interviews, project documents and participant observation, this chapter describes the incompatibilities between the two communities and what they suggest about current efforts to strengthen journalism in Africa through the application of ICTs.
Keywords: ICT4D; ICTs; IMPORTANT; Journalism; Kenya; state-building

Kenya’s tech community will not save journalism - Benequista, Nicholas (Briefing Note, 1, 2014)

Bridging Real-Time Data and Adaptive Management: Ten Lessons for Policy Makers and Practitioners - Ben Ramalingam; Inka Barnett; Kevin Hernandez; Panthea Lee; Anna Levy; Carrie Oppenheimer; Dennis Whittle; Craig Valters; Leni Wild (USAID, 2017)
The ongoing digital revolution has seen rapid growth in the deployment of technology enabled real-time data initiatives in international development and humanitarian work, developed with the goal of accelerating decision-making processes and enhancing aid effectiveness and efficiency.

Bridging the Gap: How Real-Time Data Can Contribute to Adaptive Management in International Development - Ben Ramalingam; Inka Barnett; Kevin Hernandez; Panthea Lee; Anna Levy; Carrie Oppenheimer; Dennis Whittle; Craig Valters; Leni Wild (USAID, 2017)
The ongoing digital revolution has seen rapid growth in the deployment of technology enabled real-time data initiatives in international development and humanitarian work, developed with the goal of accelerating decision-making processes and enhancing aid effectiveness and efficiency.

Evaluating interventions that prevent or counter violent extremism: A practical guide - Hofman, Joanna; Sutherland, Alex (RAND Corporation, 2018)

The Success Case Method: A Simple Evaluation Tool to Identify Critical Success Factors and Program Impa - Bell, Catherine; McDonalds, Diane (2006 Australian Evaluation Society International Conference, 2006)
Identification and reporting of critical success factors and program impact in an efficient, yet comprehensive manner is an inherent difficulty facing many evaluators of large-scale evaluations. This paper details how two evaluators encountered such problems in the initial review of a large-scale initiative and then successfully addressed these issues through the application of the Success Case Method in a subsequent evaluation of the same program. The Success Case Method is a down-to-earth evaluation tool that can be used for: finding out what is working and what is not, which also provides accurate and trustworthy information that can be used to make timely decisions (Brinkerhoff, 2003, p.3). This paper reports on the advantages of applying this approach to the evaluation of the Primary Welfare Officer Initiative (PWOI), a program that aims to improve the capacity of schools to support students at risk of disengagement and who are not achieving their educational potential.

An Introduction to the DCED Standard for Results Measurement - Bekkers, Nabanita Sen (2018)
Introduction used in the DCED-BEAM seminar

Guidelines for good market development program design. A managers’ perspective - Bekkers, Harald; Roggekamp, Peter (BEAM Exchange, 2014)
Good programme design is a key factor contributing to the success of market development programmes. Unfortunately, too many current designs have major flaws which prevent programmes becoming successful even before they get started. This document considers what makes market development programmes consistently successful and how to prevent programme design from being a hurdle to sustainable and efficient impact as scale.

Smart Implementation of complex change processes - Beier, Christoph; Kirsch, Renate (Global Solutions Journal, 2020)
Cooperation management facilitates the recoupling of progress toward sustainable development

Building Capacity for Strategic Innovation: an Emerging Competency Framework for Portfolio Work - Begovic, Milica; Colville, Jenniffer; Quaggiotto, Giulio; Vester, Søren; Naatujuna, Deborah; Oprunenco, Alex; Sadiku; Lejla; Uusikyla, Ida; Uriartt, Simone; Zorouali, Kawtar (Medium, 2023)
In a previous post we reflected on how a key learning from our “project to portfolio” journey to date is that it is ultimately about mustering the organizational will to transform. If the early days of our innovation work were about demonstrating results quickly and creating space for experimentation, now the challenge is of a different order. Eventually this means helping UNDP transition to a different value proposition and business model, as eloquently articulated by Gerd Trogemann: “No matter how well conceived and relevant in their own right, projects tend to pursue single point rather than systemic solutions, limit strategic space and the ability to adapt continuously and to connect the dots systemically. Systemic solutions need adaptive ways of working, strategic space, iterative learning, and radical collaboration.”

3 things that are enabling the UNDP’s shift to portfolios - Begovic, Milica (Medium, 2023)
By Millie Begovic, Head of the UNDP Strategic Innovation Unit

Introduction - Towards Systemic Approaches to Evaluation and Impact - Befani, Barbara; Ramalingam, Ben; Stern, Elliot (IDS Bulletin, 2015)

Clearing the fog: new tools for improving the credibility of impact claims - Befani, Barbara; D'Errico, Stefano; Booker, Francesca; Giuliani, Alessandra (iied Briefing Papers, 2016)
Development actors facing pressure to provide more rigorous assessments of their impact on policy and practice need new methods to deliver them. There is now a broad consensus that the traditional counterfactual analysis leading to the assessment of the net effect of an intervention is incapable of capturing the complexity of factors at play in any particular policy change. We suggest that evaluations focus instead on establishing whether a clearly-defined process of change has taken place, and improve the validity and credibility of qualitative impact statements. IIED research in Uganda shows that the methods of process tracing and Bayesian updating facilitate a dialogue between theory and evidence that allows us to assess our degree of confidence in ‘contribution claims’ in a transparent and replicable way.

Letting Evidence Speak for Itself: Measuring Confidence in Mechanisms - Befani, Barbara; D'Errico, Stefano (New Directions for Evaluation, 2020)
This chapter argues that the credibility of causal mechanisms can be greatly increased by formulating them as statements that are both empirically falsifiable and empirically confirmable. Whether statements can be so depends on the potential availability of the relevant evidence (e.g., no evidence exists that can prove or disprove the existence of God, but good quality evidence is potentially available in many other cases). The Bayes formula can be used to measure the extent to which a given set of empirical observations supports or weakens the belief that a causal mechanism exists. With this approach, confidence in the existence of a mechanism is increased or decreased through a process that can be open, transparent, and shared with the public or groups of stakeholders, reducing cognitive biases, and improving internal validity and consensus around the existence of given mechanisms. The approach is showcased in the evaluation of a learning partnership whereby a knowledge product released by a research organization influenced policy at the municipal level.

MSD competencies - BEAM Exchange (BEAM Exchange)
A catalogue of the full range of knowledge, skills and aptitudes found in high-performing teams that use the market systems approach. Useful for: Practitioners - identify personal training needs and continue your professional development Trainers - diversify your courses and refine your training / capacity-building offers Managers - strengthen your recruitment and induction processes

MSD Competency Framework - BEAM Exchange (BEAM Exchange, 2023)
MSD Competency Framework A catalogue of the full range of knowledge, skills and aptitudes found in high-performing teams that use the market systems approach. Useful for: Practitioners - identify personal training needs and continue your professional development Trainers - diversify your courses and refine your training / capacity-building offers Managers - strengthen your recruitment and induction processes MSD competencies Competency is a mix of knowledge, skills and attitudes demonstrated through concrete behaviours. This section explores the 17 competencies used by high-performing MSD teams. Arranged in three groups they relate to how we understand the world, make decisions and interact with others. Each competency is defined and has links to useful resources that explain or illustrate the knowledge or skills that feed it. Where available, sources of teaching and learning for that competency are included. Teaching and learning modes An exploration of the most common modes of teaching and learning that support practitioners to acquire knowledge, skills and attitudes. Some modes of learning are more relevant to certain types of competency - as indicated by the box colours. Each page has a summary of the teaching mode, guidance for team leaders and trainers, and examples of how to use this mode for developing specific competencies. Assessment modes Assessing competency is important for recruitment, performance appraisal and continued professional development These pages describe distinct approaches to evaluating individuals. They include a summary of the evaluation mode, guidance for assessors, and examples tailored to specific competencies.

Short Course - Adaptive Management, an overview - Bazaz Smith, Komal (American University, 2019)

A study of the sustained uptake developmental evaluation: How does developmental evaluation work in the USAID context, what factors help and hinder its success, and what is its value to stakeholders? - Baylor, Rebecca; Fatehi, Y. K.; Esper, H. (USAID, 2019)

Developmental Evaluation: How Barriers & Enablers Emerge Over Time - Baylor, Rebecca; Fatehi, Y. K.; Esper, H. (USAID, 2019)

The Logical Categories of Learning and Communication - Bateson, Gregory (Steps to an Ecology of Mind, 1973)

The Generalizability Puzzle - Bates, Mary Ann; Glennerster, Rachel (Stanford Social Innovation Review, 2017)
The practice of using rigorous scientific evaluations to study solutions to global poverty is relatively young. Although researchers continue to advance our knowledge of the mechanisms at work, confusion about their role and value persists. Having evidence from specific studies is fine and good, but for policy makers, the point is not simply to understand poverty, but to eliminate it. Do decisions always need to be informed by evidence from the local context? What potential and limits do randomized controlled trials have for improving our knowledge and finding effective answers? Two leaders in anti-poverty research—J-PAL and IPA—dispel some of the myths about their field. In this article, authors from J-PAL argue that rigorous impact evaluations tell us a lot about the world, not just the particular contexts in which they are conducted. Access to this article made possible by MIT.

Christian Practical Wisdom: What It Is, Why It Matters - Bass, Dorothy C.; Cahalan, Kathleen A.; McLemore, Bonnie J.; Nieman, James R.; Scharen, Christian B. (Wm. B. Eerdmans Publishing Company, 2016)
Why is the very kind of knowledge that people need to live well ― practical wisdom ― often the least understood, the hardest to learn, and the most devalued kind of knowledge?In this book five distinguished practical theologians examine the wisdom that is basic for faithful Christian living, question why it has been largely devalued, and advocate for its renewal. After first showing several concrete situations in which this kind of wisdom is visible ― marriage, church, community, culture, and more ― the authors then delve into the reasons for the decline of practical wisdom and set forth constructive cases for its renewal through biblical imagination and spiritual practice.

The Secret Sauce of Development Professionals: Tools for Assessing TOR Potential to Source Scalable Learning Interventions - Barton, Adam (Research on Improving Systems of Education (RISE), 2023)
Terms of reference (TORs) play an outsized role in driving scalable educational programming. These procurement documents shape, constrain, and signal programme priorities and possibilities. Successful funders and implementers across the globe hold rich processual knowledge about this documentation, which they use to draft and assess TORs. This project explores such best-practice knowledge around TOR review, seeking to support the design and implementation of educational programmes that can improve learning at scale in developing contexts.

Unpacking Mechanisms in Climate Resilient Agriculture Interventions - Barrett, Sam; D'Errico, Stefano; Anderson, Simon; Nebsu, Bayu (New Directions for Evaluation, 2020)
The investigation of causal mechanisms has the capacity to provide donors and implementing institutions with a greater understanding of people's reasoning and reactions as they work with interventions. This chapter contributes to the literature by identifying behavioral mechanisms generated through engagement with climate-resilient agriculture interventions within a larger livelihood project in Ethiopia. It works through the steps that enabled the study to unpack the black box between the climate-smart interventions and the outcome of crop production. The first step was a matching-based sampling design, following households over 3 years with six biannual surveys, and setting up a quasi-experimental evaluation setting. The second step used the data generated from the surveys in difference-in-difference models to assess the impact of the intervention on crop production. Third, to gain insight into the mechanisms at work, Focus Group Discussions (FGD) were conducted with smallholder farmer project beneficiaries. The FGDs revealed what the beneficiaries themselves considered the key mechanisms generated from the intervention, thus forming the bridge between the interventions and outcome. The result was an evaluation design enabling deeper insight into attribution claims. The findings offered novel insights for policymakers about how the climate-resilient interventions worked for the people themselves and shedding light on the inner workings of the climate-smart technologies. Finally, they provided key stakeholders (commissioning agency and implementing organizations) with a powerful means by which to learn about the last project, so to better plan for the next iteration and improve the climate resilience of smallholder farmers.

Learning about Theories of Change for the Monitoring and Evaluation of Research Uptake - Barnett, Chris; Gregorowski, Robbie (IDS Practice Paper In Brief, 2013)
This paper captures lessons from recent experiences on using ‘theories of change’ amongst organisations involved in the research–policy interface. The literature in this area highlights much of the complexity inherent in the policymaking process, as well as the challenges around finding meaningful ways to measure research uptake. As a tool, ‘theories of change’ offers much, but the paper argues that the very complexity and dynamism of the research-to-policy process means that any theory of change will be inadequate in this context. Therefore, rather than overcomplicating a static depiction of change at the start (to be evaluated at the end), incentives need to be in place to regularly collect evidence around the theory, test it periodically, and then reflect and reconsider its relevance and assumptions.

Celebrating Adaptive Delivery: A View from the Frontline in Myanmar - Barnes, Katrina; Lonsdale, Jane (IDS Working Paper, 2023)
The conversation on adaptive management has grown fast amongst development actors. These conversations often focus on designing, commissioning, and managing large-scale development programmes. Exactly how this impacts the frontline, the implementers, and day-to-day project delivery is still being debated. Yet, perspectives drawn directly from practice are often largely missing within these debates. This paper is written by two development practitioners. Through this paper, we reflect on the difference between adaptive management and adaptive delivery, and how this interacts with risk and aid accountability, particularly in contexts of fragility. Drawing on examples of Oxfam in Myanmar work and our personal insights in relation to delivering programming across humanitarian, peace-building, and development, we suggest that in complex, conflict-affected, and highly political environments adaptive delivery already happens far more regularly than is currently recognised, as a necessity to get activities delivered. However, it happens despite the system, not because of it, and is therefore often hidden and carried out ‘under the radar’ rather than celebrated as a success in difficult environments. This paper was written in 2019, before the military seized control of Myanmar in February 2021. Whilst it draws on examples from pre-2021 Myanmar to illustrate real life cases, it is a contribution to a broader global debate on adaptive management in practice, specifically in fragile contexts. This is not specifically aimed at practitioners working in Myanmar at present, who are now working in a protracted crisis. This paper makes tangible recommendations on steps that donors, international non-governmental organisations, local staff, and partners could take to promote a system of encouraging and celebrating adaptability in programme delivery in fragile contexts.

Understanding Governance from the Margins: What Does It Mean In Practice? - Barnes, Katrina, Barnes, Katrina; Anderson, Colin; de Chassy, Stephanie; Ahmed, Affaf; Ali, Mudabbir; Aung, Myo Min; Chaimite, Egidio; Joshi, Anuradha; Khan, Danyal; Loureiro, Miguel; Posse, Lucio; Rowlands, Jo; et al. (Institute of Development Studies (IDS), 2021)
What does governance look like ‘from below’ – from the perspectives of poor and marginalised households? How do patterns of conflict affect that? These were the questions at the heart of the Governance at the Margins research project. Over three years from 2017-2020 we worked to explore this through in-depth study in conflict-affected areas of Mozambique, Myanmar, and Pakistan. Our research teams interviewed the same people regularly over that time, finding out how they resolved problems and interacted with authorities. In this paper we connect what we found to the realities and complexities of development practice, drawing on the input of 20 experienced practitioners working in bilateral and multilateral development agencies and international NGOs, who generously gave their time to help us think through the practical implications of our wealth of findings.

Adaptive Peacebuilding Programming Through Effective Feedback Loops: Promising Practices - Barnard-Webster, Kiely; Jean, Isabella (CDA, 2018)
This case study report was developed by CDA’s part of Peacebuilding Evaluation Consortium collaborative research on the use of feedback loops in support of adaptive programming. Feedback loops are just one element of adaptive programming and are defined and used in different ways by organizations working on social change, peace and development issues. This document includes key findings from our desk review of reports and grey literature, key informant interviews, observations documented during CDA’s advisory and learning engagements with peacebuilding partners as well as the case study on SFCG’s feedback processes in Burundi. The findings are most relevant to program and M&E staff but have implications for senior management and policymakers in light of growing interest in strengthening adaptive management.

Complexity, Adaptation, and Results - Barder, Owen; Ramalingam, Ben (Center For Global Development, 2012)
In the last of a series of three blog posts looking at the implications of complexity theory for development, Owen Barder and Ben Ramalingam look at the implications of complexity for the trend towards results-based management in development cooperation. They argue that is a common mistake to see a contradiction between recognising complexity and focusing on results: on the contrary, complexity provides a powerful reason for pursuing the results agenda, but it has to be done in ways which reflect the context. In the 2012 Kapuscinski lecture Owen argued that economic and political systems can best be thought of as complex adaptive systems, and that development should be understood as an emergent property of those systems. As explained in detail in Ben’s forthcoming book, these interactive systems are made up of adaptive actors, whose actions are a self-organised search for fitness on a shifting landscape. Systems like this undergo change in dynamic, non-linear ways; characterised by explosive surprises and tipping points as well as periods of relative stability. If development arises from the interactions of a dynamic and unpredictable system, you might draw the conclusion that it makes no sense to try to assess or measure the results of particular development interventions. That would be the wrong conclusion to reach. While the complexity of development implies a different way of thinking about evaluation, accountability and results, it also means that the ‘results agenda’ is more important than ever.
Keywords: IMPORTANT

The Implications of Complexity for Development - Barder, Owen (2012)
In this lecture, adapted from his May 2012 Kapuściński Lecture, Owen Barder explores the implications of complexity theory for development policy. He explains how traditional economic models have tried and failed to understand why some countries have managed to improve living standards while other countries have not.
Keywords: IMPORTANT

Complexity Rising: From Human Beings to Human Civilization, a Complexity Proﬁle - Bar-Yam, Yaneer (New England Complex Systems Institute, 1997)
It is generally recognized that life is becoming more complex. This article analyzes the human social environment using the "complexity proﬁle," a mathematical tool for characterizing the collective behavior of a system. The analysis is used to justify the qualitative observation that complexity of existence has increased and is increasing. The increase in complexity is directly related to sweeping changes in the structure and dynamics of human civilization—the increasing interdependence of the global economic and social system, and the instabilities of dictatorships, communism and corporate hierarchies. Our complex social environment is consistent with identifying global human civilization as an organism capable of complex behavior that protects its components (us) and which should be capable of responding eﬀectively to complex environmental demands.

Qualitative Comparative Analysis - A Rigorous Qualitative Method for Assessing Impact - Baptist, Carrie; Befani, Barbara (Coffey, 2015)

Biased Policy Professionals - Banuri, Sheheryar; Dercon, Stefan; Gauri, Varun (The World Bank Economic Review, 2019)
Although the decisions of policy professionals are often more consequential than those of individuals in their private capacity, there is a dearth of studies on the biases of policy professionals: those who prepare and implement policy on behalf of elected politicians. Experiments conducted on a novel subject pool of development policy professionals (public servants of the World Bank and the Department for International Development in the UK) show that policy professionals are indeed subject to decision-making traps, including the effects of framing outcomes as losses or gains, and, most strikingly, confirmation bias driven by ideological predisposition, despite having an explicit mission to promote evidence-informed and impartial decision making. These findings should worry policy professionals and their principals in governments and large organizations, as well as citizens themselves. A further experiment, in which policy professionals engage in discussion, shows that deliberation may be able to mitigate the effects of some of these biases.
Keywords: bias

Using case studies to do program evaluation - Balbach, Edith D. (California Department of Health Services, 1999)
This paper, authored by Edith D. Balbach for California Department of Health Services is designed to allow evaluators to decide weather to use a case study evaluation approach. It also offers guidance on how to conduct a case study evaluation. Contents Using a Case Study as an Evaluation Tool 3 When to Use a Case Study 4 How to Do a Case Study 6 Unit Selection 6 Data Collection 7 Data Analysis and Interpretation 12

The Use and Abuse of the Logical Framework Approach: A Review of International Development NGOs' Experiences - Bakewell, Oliver; Garbutt, Anne (Sida, 2005)
The logical framework approach (LFA) has come to play a central role in the planning and management of development interventions over the last twenty years. Although the logical framework has become universally known, it is far from universally liked. It has been the subject of much criticism over the years, concerning both the theoretical basis of the approach, and the way it is applied in practice. In this review, we have attempted to take stock of the current views of international development NGOs on the LFA and the ways in which they use it.

Increasing Participation in Evaluation - Baker, A; Bruner, B (Effectiveness Initiatives, 2014)
The Increasing Participation in Evaluation bulletin was developed by Anita Baker with Beth Bruner to help organizations integrate evaluative thinking into their organizational practice. This three page bulletin discusses how Organization Staff, Evaluators, and Funders are typically involved in participatory evaluation. The guide also defines the term "Ripple", and how to accomplish Ripple as well as examining what it looks like when Executive Leaders and Management Staff use Evaluative Thinking. Contents How are Organization Staff, Evaluators, and Funders typically involved in Participatory Evaluation? Organization Managers/Staff Roles Evaluators/Roles Funder Roles Shared Roles What is Ripple Anyway? How do you accomplish Ripple? What does it look like when Executive Leaders and Management Staff Use Evaluative Thinking? How to use these Bulletins

Participatory Budgeting as if Emancipation Mattered - Baiocchi, Gianpaolo; Ganuza, Ernesto (Politics & Society, 2014)
Participatory Budgeting has by now been widely discussed, often celebrated, and is now instituted in at least 1,500 cities worldwide. Some of its central features—its structure of open meetings, its yearly cycle, and its combination of deliberation and representation—are by now well known. In this article, however, we critically reflect on its global travel and argue for more careful consideration of some of its less well-known features, namely the coupling of the budgeting meetings with the exercise of power. We disaggregate PB into its communicative and empowerment dimensions and argue that its empowerment dimensions have usually not been part of its global expansion—and this is cause for concern from the point of view of emancipation. We thus discuss the specific institutional reforms associated with empowerment in the original version as well as its analytic dimensions. We also address some of the specific dangers of a communication-only version of PB as well as some suggestions for reintroducing empowerment.

Institutional Change, Political Economy, and State Capabilities : Learning from Edo State, Nigeria - Bain, Katherine; Porter, Doug; Watts, Michael (Justice, security, and development series, 2015)
This paper is one of a series aimed at deepening the World Bank’s capacity to follow through on commitments made in response to the World Development Report (WDR) 2011, which gave renewed prominence to the nexus between conflict, security, and development. Nigeria is a remarkable illustration of how deeply intractable the cycle of poverty, conflict, and fragility can become when tied to the ferocious battles associated with the political economy of oil. This paper places the corpus of analytic and programmatic work concerning institutional reform in conversation with a now substantial body of work on resource politics and most especially, the debate over the politico-institutional character (sometimes called political settlements or pacting arrangements associated with the order of power) and reform landscape of the petro-state. Recent institution reform policy writing appears to have little to say about the political and economic conditions in which crises and institutional disjunctures may authorize, and thereby enable, agents to embark on institutional reforms. The authors focus on Edo state for two reasons. First, it does not on its face appear to be an obvious location in which to explore a reform experience, given its entanglement in the Niger Delta conflict and the maladies typically associated with state fragility. Second, Edo is of interest also because of the changes that its experience is contributing to the World Bank country team’s effort to engage operationally across all its instruments with the political economy of institutional reform in Nigeria, its largest client country in Africa.

Doing Development Differently at the World Bank: updating the plumbing to fit the architecture - Bain, Katherine A.; Booth, David; Wild, Leni (ODI, 2016)
Keywords: IMPORTANT

Digital lives in Ghana, Kenya and Uganda - Bailur, Savita; Donner, Jonathan; Locke, Chris; Schoemaker, Emrys; Smart, Charlotte (Caribou Digital, 2015)
Keywords: IMPORTANT

Are we there yet? Localisation as the journey towards locally led practice: models, approaches and challenges - Baguios, Arbie; King, Maia; Martins, Alex; Pinnington, Rose (ODI, 2021)
Localisation and locally led international development practice has long been discussed, but has still not been delivered. Systemic barriers have posed challenges, and the term itself is contested. Now, the last tumultuous 18 months could provide a critical juncture to finally move forward with this crucial agenda. The pandemic has highlighted structural inequalities in the global system, and disrupted ways of working in the international development sector. The Black Lives Matter movement has brought conversations about racism and colonialism to the fore. And the climate crisis has highlighted the need for global action on humanity’s challenges that remain rooted in local realities. The emerging analysis in this review aims to set out the key issues in this agenda, building on a wealth of existing knowledge. It aims to span sectors, highlighting many new and existing models and approaches in the humanitarian, development, philanthropic and private sectors. It reviews the barriers and challenges to localisation and locally led practice, with a view to informing a campaign for systemic change to move forward with this agenda. The review is based on: a rapid review of the literature and evidence on localisation and locally led practice; two consultations with over 100 total participants, targeted at Global South actors; and analysis of 28 existing models and approaches.

Looking for a way out of aid’s pre-pandemic mess? A model based on cake - Baguios, Arbie (From Poverty to Power, 2020)
Arbie Baguios presents his ideas on how to reform the aid system

The Art of Community: Building the New Age of Participation - Bacon, Jono (O'Reilly Media, 2012)
Online communities provide a wide range of opportunities for supporting a cause, marketing a product or service, or building open source software. The Art of Community helps you recruit members, motivate them, and manage them as active participants. Author Jono Bacon offers experiences and observations from his 14-year effort to build and manage communities, including his current position as manager for Ubuntu.Discover how your community can become a reliable support network, a valuable source of new ideas, and a powerful marketing force. This expanded edition shows you how to keep community projects on track, make use of social media, and organize collaborative events. Interviews with 12 community management leaders, including Linus Torvalds, Tim O’Reilly, and Mike Shinoda, provide useful insights.Develop specific objectives and goals for building your communityBuild processes to help contributors perform tasks, work together, and share successesProvide tools and infrastructure that enable members to work quicklyCreate buzz around your community to get more people involvedHarness social media to broadcast information, collaborate, and get feedbackUse several techniques to track progress on community goalsIdentify and manage conflict, such as dealing with divisive personalities

The role of research and learning in adaptive programming - Babon, Andrea; Denney, Lisa (Devpolicy Blog from the Development Policy Centre, 2018)
Andrea Babon and Lisa Denney explore how learning partners - a common feature of aid programs - can operate and feed into programs.

Harnessing Complexity: Organizational Implications of a Scientific Frontier - Axelrod, Robert M.; Cohen, Michael D. (Free Press, 1999)
Recent advances in the study of complexity have given scientists profound new insights into how natural innovation occurs and how its power can be exploited. Now two pioneers in the field, Robert Axelrod and Michael D. Cohen, provide leaders in business and government with a guide to complexity that will help them make effective decisions in a world of rapid change. Building on evolutionary biology, computer science, and social design, Axelrod and Cohen have constructed a unique framework for improving the way people work together. Their approach to management is based on the concept of the Complex Adaptive System, which can describe everything from rain forests to the human gene pool, and from automated software agents to multinational companies. The authors' framework reveals three qualities that all kinds of managers must cultivate in their organization: "Variation" What is the best way to manage the development of software? Should the problem be broken up into small pieces for programmers working independently, thus enhancing variation, or should there be a centralized hierarchy of programmers ruled by a chain of command? The authors show how the decentralized creation of variation combined with the centralized maintenance of standards was the key to the success of the Linux "open source software" project, which brought together thousands of volunteers in cyberspace to produce an operating system that can outperform Microsoft's. "Interaction" Why did northern Italy prosper while southern Italy remained poor? Recognizing the internal interactions of a Complex Adaptive System -- be it a national region, a company, or a nonprofit group -- reveals vitalnetworks of trust. Axelrod and Cohen explain that in successful adaptive systems, rich networks of horizontal linkages foster cooperation and provide an advantage over other less cooperatively networked groups. In the case of Italy, voluntary associations created networks of trust in the Middle Ages that became northern Italy's critical advantage over the south. "Selection" Is a Pulitzer Prize better than a National Book Award? How can foundations and corporations design competitions that have a positive effect on the evolution of excellence? The authors' framework makes clear that the worst selection processes are mired in orthodox standards that have not adapted to a new environment. The best selection processes, on the other hand, are created and run by leaders who understand how the standards they use can transform their organization and its environment. This simple, paradigm-shifting analysis of how people work together will transform the way we think about getting things done in a group. "Harnessing Complexity" is the essential guide to creating wealth, power, and knowledge in the 21st century.
Keywords: Organizational Behavior

Global mapping of technology for transparency and accountability - Avila, Renata; Feigenblatt, Hazel; Heacock, Rebekah; Heller, Nathaniel (Transparency & Accountability Initiative, 2010)

Building Bridges to an Uncertain Future Lived Now: Lessons from the Use of Participatory Action Research and Theory of Change Towards A Realistic Community-Based Participatory Monitoring and Evaluation System - Avila, Enrique M.; Tolentino, Lutgarda L.; Binondo, Claudia B.; Perez, Maripaz L.; Apgar, J. Marina (International Journal of Agriculture System, 2016)
Building on experience from the CGIAR Research Program on Aquatic Agricultural Systems implemented by WorldFish in the Visayas and Mindanao regions of the Philippines, known as the VisMin Hub, we describe the development and evolution of a monitoring and evaluation (M&E) system emerging from the facilitated action-reflection cycles of testing and adopting theories of change carried out with community partners through participatory action research (PAR). The former guides our community partners and us, as members of the potentially emergent PAR groups, towards the realization of the community’s vision; the latter facilitates learning to understand what, how and why change is unfolding. Unlike the conventional M&E system where indicators are pre-set at the beginning of program implementation, these processes result in an organically-evolved, communitybased participatory M&E system that is continuously revised according to contexts to guide communities towards realizing their visions. Its ultimate outcome is enhanced people’s capacity to own the product and process, giving rise to an internally-driven change. Towards the end, the paper offers an iterative discussion of learnings from implementing such an approach.
Keywords: Aquatic agricultural systems; CGIAR; Community-based participatory MEL; Learning; Participatory action research; research in development; theory of change

Social Accountability and Service Delivery Effectiveness: What is the Evidence for the Role of Sanctions? Background Paper - Aston, Tom; Zimmer Santos, Grazielli (2022)
Executive Summary Understanding how civil society can get government to respond to their needs, preferences and demands, and deliver goods and services to citizens is a central concern in social accountability initiatives. It is widely argued that sanctions make a key difference to service delivery outcomes, and that without them, transparency and accountability interventions are less likely to be effective and less likely to be sustainable (Anderson et al., 2020; Arugay, 2016; Fox, 2020; Goetz and Jenkins, 2005; Grandvoinnet et al., 2015; McGee and Gaventa, 2011; Molina et al., 2017; Joshi, 2010; Joshi, 2017; Tsai et al. 2019). In this paper, sanctions refer to the threat or imposition of a punishment for transgressing a rule or norm. Yet, what evidence is there to support the claim that sanctions are king? How much do we actually know about social and formal sanctions and their effectiveness in improving service delivery? Looking at 11 meta-reviews and 35 cases, this background paper sheds light on these questions and the conditions under which sanctions promoted within social accountability interventions may have contributed to improved service delivery. Sanctions, both social and formal, feature very prominently in the scholarly literature related to accountability, so much so that some scholars have argued that it has become synonymous with punishment (Mansbridge, 2014; Schedler, 1999). While it may not be true that these concepts are fully synonymous, this trend in scholarship significantly influenced the thinking of donors such as the Department for International Development (DFID) and the World Bank over the last two decades (Grandvoinnet et al., 2015; Malena and McNeil, 2010; Moore and Teskey, 2006; World Bank, 2003). As a result, sanctions also feature in the dominant models, or theories of change, in the social accountability sector. Yet, in reviewing 11 meta-reviews in the transparency and accountability sector we found that while there is plenty of theoretical argument asserting the potential effectiveness of social and formal sanctions, there was limited empirical evidence to support the claim that sanctions were key. Our review of 35 cases revealed five mechanisms of change related to social and formal sanctions. These were: (i) “sticks” – response to punishment; (ii) “big brother is watching” – response to threat of formal punishment; (iii) diagonal accountability – response to threat of formal punishment by horizontal accountability agencies; (iv) litigation – response to legal investigations or lawsuits supported by community paralegals and legal aid organizations and; (v) response to “naming and shaming” by civil society and/or media. We challenge the dominant view in scholarship that harder social sanctions and enforcement of stronger formal sanctions are either necessary to the achievement of higher-level service delivery outcomes or that they will deliver better and more sustainable outcomes. We find that both social and formal sanctions can contribute to improving service delivery outcomes across a variety of country contexts. In half of the 35 cases reviewed, we were able to establish a likely link between social or formal sanctions and intermediate effects over the short term with some degree of confidence. These effects ranged from increased service provider awareness and motivation, increased availability of funding, staff, and materials, to improved infrastructure quality, and in a minority of cases impact level changes such as improved test scores. However, the role of sanctions in delivering outcomes was often unclear, outcomes were almost never sustainable, and in close to half of the 35 cases reviewed there were substantial negative effects. These effects ranged from reducing transparency and funding, to discrediting, relocating, and reprisals for advocates and whistle-blowers, threats of violence to collaborating government actors, damaging staff morale, reducing attendance, and generating conflict among staff and between staff and community members and between staff and patients, damaging trust. There are therefore some serious ethical dilemmas associated with sanctioning efforts which need to be carefully considered. We argue that imposing sanctions without building relationships or systems to promote good behavior is unlikely to improve service delivery outcomes in a sustainable way. Another, perhaps surprising, finding was that there are some actors that are regular targets of sanctions, and in many cases, these actors are a lot weaker than is commonly assumed. Three broad types of actors were the most common targets of punishment: (i) absentee nurses or teachers who had their pay or allowances reduced; (ii) offending officials who were either suspended, relocated, or fired; and (iii) contractors who had to cover the cost of rejected materials or faced lawsuits, alongside civil servants involved in contracting. We suggest that closer relationships may perhaps reduce stakeholders’ appetite to impose sanctions. Particularly in the health sector, we found that more proximate relationships created disincentives for confrontation, and in such circumstances, a “policing” approach to monitoring was also deemed inappropriate and counterproductive. Conversely, it seems that actors generally prefer to sanction “others,” i.e., when an actor/organization was outside the group. Short-term consultants, contractors, and suppliers were easy (and quite vulnerable) targets for harder sanctions. It has also been argued that there may be productive combinations of collaborative and confrontational tactics — i.e., hybrids. We found that many supposed confrontational and collaborative hybrids were, in fact, dislocated in time and space. Many so-called “inside-outside” strategies, therefore, seem to be a potential mischaracterization. We also found, as Fung and Wright (2003) argued nearly two decades ago, that adversarial forms of engagement cannot easily be redeployed for collaborative purposes. It is widely argued that supposedly ‘weaker’ forms of citizen engagement are less effective than those with ‘strong enforceability (McGee and Gaventa, 2011).’ We found no compelling evidence to support this contention. A quarter of cases reviewed were collaborative for certain periods or in certain locations. On average, these were slightly more successful when compared with periods or locations when imposing sanctions was a key strategic emphasis. So, soft power can also be powerful. Rather than one approach necessarily being superior (confrontational, collaborative, or hybrid) however, we argue that the best approach is likely to be the one most appropriate to the context. While we were unable to identify strong trends of contextual factors which enabled social and formal sanctions to play a role in enhancing service delivery, we were able to identify several conditions which we believe offer the greatest promise. These conditions were: (i) supportive leaders who played a role as champions; opening doors or accompanying civil society efforts; (ii) high capacity and legal authority of oversight agencies; (iii) competitive elections, which provided windows of opportunity for CSOs to combine political and social accountability efforts, and; (iv) vulnerable public servants and service providers already in relatively precarious situations and are thus easy targets. Overall, our study finds that sometimes sanctions can be effective, but punishment is not the answer to all the world’s problems. Given these limitations, we recommend that it is time to reconsider “carrots” and enquire further into the enabling conditions for bureaucrats and service providers. Relatedly, scholars, evaluators, and program teams should look more closely at service providers’ or civil servants’ motivations and take context into account more seriously. To uncover these contextual and motivational features, we believe that scholars, evaluators, and prprogrameams should also make better use of theory-based and participatory methods. And perhaps most importantly, donors and practitioners should carefully consider and mitigate the potential for backlash from sanctions-based approaches.

Contribution Rubrics: A simple way to assess influence - Aston, Tom (Independent Consultant, 2020)
This document explores how organisations can measure the level of influence that they had over an outcome. It summarises influence in terms of the significance of the outcome, the level of contribution and the strength of evidence.

One step back, two steps forward: CARE’s journey towards doing development differently - Aston, Tom (Care Insights, 2017)
The Doing Development Different (DDD) community emerged in August 2014 and advocates that (a) the barriers to development are as much political as tec...

Sounding clever or being smart? – How to do more with less in evaluating governance programmes - Aston, Tom (Care Insights, 2018)
About a decade ago, the development sector fell into the same trap the financial services industry did in the mid-1990s. We were all seduced by clever...

W(h)ither sanctions? - Aston, Tom (Thomas Aston LinkedIn pulse, 2020)
Considerations on where and when sanctions fit into the conception of social accountability is a nudge toward better and more granular descriptions of what work is being undertaken effectively in which situations.

Process Tracing Innovations in Practice: Finding the Middle Path - Aston, Thomas; Wadeson, Alix (CDI Practice Paper, 2023)
Evaluation practitioners in the international development sector have given considerable attention in recent years to process tracing as a method for evaluating impact, including discussion of how to assess the relative importance of causal factors. Despite the increasing interest, there is a relative dearth of examples of practical learning and evidence of applying process tracing in practice. This CDI Practice Paper draws on comparative learning from applying three different types of process tracing in international development initiatives. It argues in favour of a ‘middle path’ of applying evidence tests and rubrics to structure evaluative judgements rather than formal Bayesian updating or looser forms of process tracing. It also calls attention to the potential added value of taking a participatory approach, offering practical recommendations for how to do this effectively.

Monitoring and evaluation for thinking and working politically - Aston, Thomas; Roche, Chris; Schaaf, Marta; Cant, Sue (Evaluation, 2022)
This article explores the challenges of monitoring and evaluating politically informed and adaptive programmes in the international development field. We assess the strengths and weaknesses of some specific evaluation methodologies which have been suggested as particularly appropriate for these kinds of programmes based on scholarly literature and the practical experience of the authors in using them. We suggest that those methods which assume generative causality are particularly well suited to the task. We also conclude that factoring in the politics of uncertainty and evidence generation and use is particularly important in order to recognize and value diverse experiential knowledge, integrate understandings of the local context, accommodate adaptation and realistically grapple with the power relations which are inherent in evaluation processes.

The Art and Craft of Bricolage in Evaluation - Aston, Thomas; Apgar, Marina (CDI Practice Paper, 2022)
This CDI Practice Paper by Tom Aston and Marina Apgar makes the case for ‘bricolage’ in complexity-aware and qualitative evaluation methods. It provides a framework based on a review of 33 methods to support evaluators to be more intentional about bricolage and to combine the component parts of relevant methods more effectively. It discusses two cases from practice to illustrate the value added of taking a more intentional approach. It further argues that navigating different forms of power is a critical skill for bricolage, and that doing so can help to ensure rigour.

Assumptions and triple loop learning - Aston, Thomas (Medium, 2021)
Triple loop learning

Changing theories - Aston, Thomas (Medium, 2023)
You may, or may not, be surprised to hear that many theories of change lack what we might generally understand as a theory.

Evaluating complexity, simplistically - Aston, Thomas (Medium, 2022)
A critical appraisal's of CEDIL papers on Evaluating Complex Interventions... A study was recently published by the Centre of Excellence for Development Impact and Learning (CEDIL) entitled Evaluating complex interventions in international development. This is the sort of title that raises great expectations. Complexity is a hugely popular theme and many of us are keen to know more about how to evaluate efforts that seek to achieve results amid complexity. In April 2021, CEDIL conducted a webinar on the paper, and in July 2021 CEDIL published a blog. In September 2021, I wrote a blog expressing some reservations regarding the focus of the study; its apparent over-emphasis on interventions, under-emphasis on context, as well as its choice of some supposedly under-used methods. These methods were: (1) factorial designs; (2) adaptive trials; (3) Qualitative Comparative Analysis (QCA); (4) synthetic control; (5) agent-based modelling, and system dynamics.

How to make theories of change more useful? - Aston, Thomas (Medium, 2021)
A fair amount has been written recently questioning the value added of theories of change. Have we gone through a hype cycle? Are they…

(Re)making the case for adaptive management - Aston, Thomas (Medium, 2022)
Great overview of what to read on adaptive management. It’s a long one, so I’ve split it into two – second installment tomorrow. Christian Aid Ireland’s recent publication The Difference Learning Makes by Stephen Gray and Andy Carl made a bit of a splash. The study found that Christian Aid Ireland’s application of adaptive programming contributed to better development outcomes and supported more flexible delivery. The much vaunted MUVA programme in Mozambique is also coming to a close and presenting its results from using and adaptive approach. So, it struck me that we might be at a critical juncture in the conversation on adaptive management. We’ve had the crashing to earth of inflated expectations in recent misanthropic reflections, [misanthropic, moi? – Duncan] alongside a fragile institutionalisation of adaptive management in donor agencies, NGOs, and private sector organisations. However, I’d argue that we’ve reached the point where adaptive management has passed the proof-of-concept stage.

“Real”​ process tracing: part 1 — context - Aston, Thomas (Medium, 2020)
When asserting the value of theory-based methods, you often here words like “black boxes” and “causal mechanisms.” These are commonly…

Rubrics as a harness for complexity - Aston, Thomas (Medium, 2020)
In this final blog in the series, I want to look at the potential value of rubrics. While evaluability assessments can help us to…

What, so what, now what? - Aston, Thomas (Medium, 2022)
Getting serious about systems change

Windows on the world: The power of assumptions in uncertain times - Aston, Thomas (Medium, 2020)
In my last blog on theory-based Monitoring Evaluation and Learning (MEL), I explained why relationships matter, and how to assess change…

A Guide To Agile Project Management Methodology & Tools - Aston, Ben (The Digital Project Manager, 2020)
Looking to brush-up on agile? Here's your complete guide to agile project management, agile principles and key components, and the best agile tools for 2020.

Embracing complexity - Towards a shared understanding of funding systems change - Ashoka (Ashoka, 2020)

Tools for supporting sustainable natural resource management and livelihoods - Ashley, Holly; Kenton, Nicole; Milligan, Angela (2013)

Participation by the poor in Luang Prabang tourism economy: current earnings and opportunities for expansion - Ashley, Caroline (ODI, 2006)

Requisite Variety and Its Implications for the Control of Complex Systems - Ashby, W. Ross (Cybernetica, 1958)

Bringing adaptive management to life: Insights from practice - Arora, Anmol; Gogoi, Elizabeth; Joy, Divya; Kumar, Pankaj; Luthra, Rajni; Pal, Uma; Pervaiz, Arif (Oxford Policy Management, 2019)

Mission-Based Monitoring, Evaluation and Learning (MEL) Platforms Assessment Report - Arntson, Laura; Giannoni, Tonya; Peek, Nancy; Saarlas, Kristin (USAID, 2017)
In 2016, the Bureau of Policy, Planning and Learning (PPL) commissioned an internal stocktaking of USAID's mission-based MEL (Monitoring, Evaluation, and Learning) Platforms.

Organizational Learning: A Theory of Action Perspective - Argyris, Chris; Schon, Donald A. (Addison Wesley, 1978)

Twinning “Practices of Change” With “Theory of Change”: Room for Emergence in Advocacy Evaluation - Arensman, Bodille; van Waegeningh, Cornelie; van Wessel, Margit (American Journal of Evaluation, 2018)
Theory of change (ToC) is currently the approach for the evaluation and planning of international development programs. This approach is considered especially suitable for complex interventions. We question this assumption and argue that ToC’s focus on cause–effect logic and intended outcomes does not do justice to the recursive nature of complex interventions such as advocacy. Supported by our work as evaluators, and specifically our case study of an advocacy program on child rights, we illustrate how advocacy evolves through recursive interactions, with outcomes that are emergent rather than predictable. We propose putting “practices of change” at the center by emphasizing human interactions, using the analytical lenses of strategies as practice and recursiveness. This provides room for emergent outcomes and implies a different use of ToC. In this article, we make a clear distinction between theoretical reality models and the real world of practices.

Using adaptive processes and adverse outcome pathways to develop meaningful, robust, and actionable environmental monitoring programs - Arciszewski, Tim J.; Munkittrick, Kelly R.; Scrimgeour, Garry J.; Dubé, Monique G.; Wrona, Fred J.; Hazewinkel, Rod R. (Integrated Environmental Assessment and Management, 2017)
The primary goals of environmental monitoring are to indicate whether unexpected changes related to development are occurring in the physical, chemical, and biological attributes of ecosystems and to inform meaningful management intervention. Although achieving these objectives is conceptually simple, varying scientific and social challenges often result in their breakdown. Conceptualizing, designing, and operating programs that better delineate monitoring, management, and risk assessment processes supported by hypothesis-driven approaches, strong inference, and adverse outcome pathways can overcome many of the challenges. Generally, a robust monitoring program is characterized by hypothesis-driven questions associated with potential adverse outcomes and feedback loops informed by data. Specifically, key and basic features are predictions of future observations (triggers) and mechanisms to respond to success or failure of those predictions (tiers). The adaptive processes accelerate or decelerate the effort to highlight and overcome ignorance while preventing the potentially unnecessary escalation of unguided monitoring and management. The deployment of the mutually reinforcing components can allow for more meaningful and actionable monitoring programs that better associate activities with consequences. Integr Environ Assess Manag 2017;13:877–891. © 2017 The Authors. Integrated Environmental Assessment and Management Published by Wiley Periodicals, Inc. on behalf of Society of Environmental Toxicology & Chemistry (SETAC)
Keywords: Adaptive monitoring; Environmental management; Environmental monitoring; Tier; Trigger

Assumptions, conjectures, and other miracles: The application of evaluative thinking to theory of change models in community development - Archibald, Thomas; Sharrock, Guy; Buckley, Jane; Cook, Natalie (Evaluation and Program Planning, 2016)
Unexamined and unjustified assumptions are the Achilles’ heel of development programs. In this paper, we describe an evaluation capacity building (ECB) approach designed to help community development practitioners work more effectively with assumptions through the intentional infusion of evaluative thinking (ET) into the program planning, monitoring, and evaluation process. We focus specifically on one component of our ET promotion approach involving the creation and analysis of theory of change (ToC) models. We describe our recent efforts to pilot this ET ECB approach with Catholic Relief Services (CRS) in Ethiopia and Zambia. The use of ToC models, plus the addition of ET, is a way to encourage individual and organizational learning and adaptive management that supports more reflective and responsive programming.
Keywords: Assumptions; Community development; Critical thinking; Evaluation capacity building; Evaluative thinking; International Development; Theory of change

Appreciative Inquiry Australia - Appreciative Inquiry Australia (2015)
The site provides news and information on AI events and learning opportunities in Australia and also provides an archive of articles and discussions centred on the use of AI.

Learning through and about Contribution Analysis for impact evaluation - Apgar, Marina; Ton, Giel (Institute of Development Studies, 2021)
Over the past five years, colleagues from the Centre for Development Impact (CDI) – a joint initiative between the Institute of Development Studies, Itad and University of East Anglia – have been innovating with and learning how to use Contribution Analysis as an overarching approach to impact evaluation. In this blog series, we share our learning and insights, some of them in raw emergent form, highlight the complexities, nuances, excitements, and challenges of embracing new ways of doing impact evaluation. We begin this series by sharing ideas about ‘causal hotspots’ that first surfaced as ‘aha’ moments in our collaboration on the CDI short course Contribution Analysis for Impact Evaluation – the moments when we found the ways and words to better articulate our ideas and help people navigate the messy realities of theory-based evaluation.

Designing Contribution Analysis of Participatory Programming to Tackle the Worst Forms of Child Labour - Apgar, Marina; Snijder, Mieke; Prieto Martin, Pedro; Ton, Giel; Macleod, Shona; Kakri, Shanta; Paul, Sukanta (CLARISSA Research and Evidence Paper, 2022)
This Research and Evidence Paper presents the theory-based and participatory evaluation design of the Child Labour: Action-Research- Innovation in South and South-Eastern Asia (CLARISSA) programme. The evaluation is embedded in emergent Participatory Action Research with children and other stakeholders to address the drivers of the worst forms of child labour (WFCL). The report describes the use of contribution analysis as an overarching approach, with its emphasis on crafting, nesting and iteratively reflecting on causal theories of change. It illustrates how hierarchically-nested impact pathways lead to specific evaluation questions and mixing different evaluation methods in response to these questions, critical assumptions, and agreement on causal mechanisms to be examined in depth. It also illustrates how realist evaluation can be combined with contribution analysis to deeply investigate specific causal links in the theory of change. It reflects on learning from the use of causal hotspots as a vehicle for mixing methods. It offers considerations on how to navigate relationships and operational trade-offs in making methodological choices to build robust and credible evidence on how, for whom, and under what conditions participatory programming can work to address complex problems such as child labour.

Breaking free from the theory of change straight jacket - Apgar, Marina; Snijder, Mieke; Paul, Sukanta (CLARISSA, 2024)
The CLARISSA Social Protection Intervention was set us as an innovative social policy intervention for tackling social ills, with a...

Evaluating CLARISSA: Innovation Driven by a Participatory Learning Agenda - Apgar, Marina; Snijder, Mieke; Kakri, Shanta; Macleod, Shona; Paul, Sukanta; Sambo, Anna; Ton, Giel (CLARISSA working paper, 2020)
Children end up in child labour as a result of many, often unknown or hidden, interactions between multiple actors and multiple factors within households, communities, and labour systems, leading to unpredictable outcomes for children and other sector stakeholders and sometimes resulting in the worst forms of child labour (WFCL). It is a complex problem, and interventions aimed at tackling it are also, inevitably, complex and challenging. The way they influence change is non-linear, causality is uncertain, and unintended consequences may result. Programmes such as the Child Labour: Action-Research-Innovation in South and South-Eastern Asia (CLARISSA) that are engaging with such intractable challenges and aim to reach the most left behind (children in WFCL) are operating in conditions of complexity. This complexity poses significant challenges to the way programmes are designed, planned, implemented, and evaluated, and requires a move away from linear and predetermined models. In this Working Paper, we share our experience and early learning about how to design and implement monitoring, evaluation and learning that intentionally embraces the challenge of complexity.

Evaluating Research for Development: Innovation to Navigate Complexity - Apgar, Marina; Snijder, Mieke; Higdon, Grace Lyn; Szabo, Sylvia (The European Journal of Development Research, 2023)
Large publicly funded programmes of research continue to receive increased investment as interventions aiming to produce impact for the world’s poorest and most marginalized populations. At this intersection of research and development, research is expected to contribute to complex processes of societal change. Embracing a co-produced view of impact as emerging along uncertain causal pathways often without predefined outcomes calls for innovation in the use of complexity-aware approaches to evaluation. The papers in this special issue present rich experiences of authors working across sectors and geographies, employing methodological innovation and navigating power as they reconcile tensions. They illustrate the challenges with (i) evaluating performance to meet accountability demands while fostering learning for adaptation; (ii) evaluating prospective theories of change while capturing emergent change; (iii) evaluating internal relational dimensions while measuring external development outcomes; (iv) evaluating across scales: from measuring local level end impact to understanding contributions to systems level change. Taken as a whole, the issue illustrates how the research for development evaluation field is maturing through the experiences of a growing and diverse group of researchers and evaluators as they shift from using narrow accountability instruments to appreciating emergent causal pathways within research for development.

Extending epistemology for programme evaluation – can After Action Reviews become spaces for critical reflection? - Apgar, Marina; Snijder, Mieke (Methodspace - Sage, 2020)
The term After Action Review (AAR) is becoming more common in the world of development evaluation, particularly in programmes that focus on evaluation as a formative learning process. As structured and facilitated learning moments, AARs take many shapes always built to support a specific team in a specific moment on their journey. Across such a diversity of practice how do we understand the evolution of the AAR as a method with an action research orientation? Being mindful always of the risk of instrumentalising and co-opting participatory methods, there is danger that AARs become an empty programme ritual, remaining at the surface and failing to achieve critical reflection and so falling short of their intention of pushing for deeper change in our practice. As the evaluation and learning team of two large complex, multi-partner projects using research to address development challenges, we have been applying the AAR method while adapting to virtual COVID working with partners across the world. As action researchers we have been thinking about what critical reflection means within a programme AAR process and whether and how they can open up a second person inquiry space.

Finding and using causal hotspots: a practice in the making - Apgar, Marina; Snijder, Mieke (Institute of Development Studies, 2021)
This is the second blog in our series ‘Lessons on using Contribution Analysis for impact evaluation’. In our first blog, we introduced Contribution Analysis (CA) as an overarching approach to theory-based evaluation and the idea of causal hotpots as a way to zoom in, unpack and make the hard choices of where to focus evaluation research. Identifying specific links in the theory of change (ToC) with specific evaluation questions enables you to then choose appropriate methods. We have applied the approach in diverse settings, testing how robust Contribution Analysis can really be. Here we walk you through an example.

Bringing Participation into Complexity-Aware MEL: What is the evidence? - Apgar, Marina; Higdon, Grace Lyn (2018)

Contribution analysis for adaptive management - Apgar, Marina; Hernandez, Kevin; Ton, Giel (Briefing Paper, 2020)
This briefing note shares practical learning on the use of contribution analysis for adaptive management (CA4AM). It examines how the approach enables programmes to work with theories of change in a practical, reflexive way, and how, combined with assessing evidence of a programme’s contribution to change, its findings can inform programme adaptation. It also examines both how and to what extent CA enables AM through the experiences of four large complex programmes all working towards systems-level change and employing a structured process of reflection on theories of change. Key messages CA4AM can enable programmes to work with theories of change in a practical, reflexive way. It is particularly useful for programmes operating in conditions of complexity, when it is difficult to discern attribution and when systems-level change is the goal. A range of enabling factors help CA4AM to be used most effectively, including contractual flexibility; embedded monitoring, evaluation and learning (MEL); and supportive leadership.

Revealing the Relational Mechanisms of Research for Development Through Social Network Analysis - Apgar, Marina; Fournie, Guillaume; Haesler, Barbara; Higdon, Grace Lyn; Kenny, Leah; Oppel, Annalena; Pauls, Evelyn; Smith, Matthew; Snijder, Mieke; Vink, Daan; Hossain, Mazeda (The European Journal of Development Research, 2023)
Achieving impact through research for development programmes (R4D) requires engagement with diverse stakeholders across the research, development and policy divides. Understanding how such programmes support the emergence of outcomes, therefore, requires a focus on the relational aspects of engagement and collaboration. Increasingly, evaluation of large research collaborations is employing social network analysis (SNA), making use of its relational view of causation. In this paper, we use three applications of SNA within similar large R4D programmes, through our work within evaluation of three Interidsiplinary Hubs of the Global Challenges Research Fund, to explore its potential as an evaluation method. Our comparative analysis shows that SNA can uncover the structural dimensions of interactions within R4D programmes and enable learning about how networks evolve through time. We reflect on common challenges across the cases including navigating different forms of bias that result from incomplete network data, multiple interpretations across scales, and the challenges of making causal inference and related ethical dilemmas. We conclude with lessons on the methodological and operational dimensions of using SNA within monitoring, evaluation and learning (MEL) systems that aim to support both learning and accountability.

Reframing, refining, and reconceptualising the worst forms of child labour through participatory adaptive programming - Apgar, Marina; Burns, Danny (Journal of modern slavery, 2021)
This article explores the potential of using participatory action research as an adaptive programming modality to drive learning and innovation to tackle the drivers of (and seek to eliminate) the Worst Forms of Child Labour. We draw on our experience from early phases of implementation of a large-scale action research programme, which despite the constraints covid-19 posed in moving to full implementation and participatory engagement with children and other stakeholders on the ground, is already generating rich learning about the opportunities and challenges of designing programmes that respond to the complex reality of WFCL. We share early learning about what it takes to be fully open to using the lived experience of programme development, and early findings from scoping and mapping of the dynamics of social norms, business practices and urban neighbourhoods and supply chains influencing WFCL in Bangladesh and Nepal, to frame and reframe the questions and response strategies and operationalise a participatory adaptive intent to work with hidden and complex dynamics that characterise the WFCL.

Innovating for inclusive rigour in peacebuilding evaluation - Apgar, Marina; Báez-Silva, Ángela Maria; Deng, Ayak Chol; Fairey, Tiffany; Rohrbach, Livia; Alamoussa, Dioma; Bradburn, Helene; Cubillos, Edwin; Gray, Stephen; Wingender, Leslie (Institute of Development Studies, 2022)
Inclusive and rigorous peacebuilding evaluation is both vital and complex. In this blog we share examples of how we are innovating our methodologies to move towards participatory and adaptive practice.

Getting beneath the surface in program planning, monitoring and evaluation: Learning from use of participatory action research and theory of change in the CGIAR Research Program on Aquatic Agricultural Systems - Apgar, J Marina; Allen, Will; Albert, Joelle; Douthwaite, Boru; Paz Ybarnegaray, Rodrigo; Lunda, Jeston (Action Research, 2017)
Many rural poor and marginalized people strive to make a living in social-ecological systems that are characterized by multiple and often inequitable interactions across agents, scale and space. Uncertainty and inequality in such systems require research and development interventions to be adaptive, support learning and to engage with underlying drivers of poverty. Such complexity-aware approaches to planning, monitoring and evaluating development interventions are gaining strength, yet, there is still little empirical evidence of what it takes to implement them in practice. In this paper, we share learning from an agricultural research program that used participatory action research and theory of change to foster learning and support transformative change in aquatic agricultural systems. We reflect on our use of critical reflection within participatory agricultural research interventions, and our use of theory of change to collectively surface and revisit assumptions about how change happens. We share learning on the importance of being strengths-based in engaging stakeholders across scales and building a common goal as a starting point, and then staging a more critical practice as capacity is built and opportunities for digging deeper emerge.

3 fallacies of embracing complexity - Ang, Yuen Yuen (UNDP Blog, 2018)
Global development is seeing an exciting paradigm shift. Increasingly, leaders and practitioners recognize that development is not a “complicated” challenge that can be neatly parsed out into separate problems and siloed departments, like assembling a car. Rather, the various tasks of development—poverty eradication, improving governance, climate action, gender equality, and so on—are all connected, making development a “complex” challenge.

Combining Big Data and Thick Data to Improve Services Delivery - Ang, Yuen Yuen (IBM Center for The Business of Government, 2018)

Complexity & Development 2.0: From Agreeing We Should Adapt to Creating the Conditions that Enable Adaptation - Ang, Yuen Yuen (2018)
The field of global development has reached a critical turning point. Almost gone is the mechanical, one-size-fits-all “good governance” paradigm of the past. In its place is a growing embrace of complexity and systems thinking. While this is an encouraging shift in the right direction, the discussion mostly ends by concluding that we should adapt. Yuen Yuen Ang urges that it’s time to take our conversation on “complexity & development” to the next level: how to enable adaptation. Effective adaptation doesn’t automatically arise just because we desire it—rather, it requires certain enabling conditions. In How China Escaped the Poverty Trap (2016), she introduces a generic complexity 2.0 framework that she terms “directed improvisation.” Effective adaptation, Yuen Yuen argues, requires a paradoxical blend of top-down direction and bottom-up improvisation. Such a system can be created—and produce dramatic results and adaptive solutions—even within a closed political regime like China.

Going Local 2.0: How to Reform Development Agencies to Make Localized Aid More Than Talk - Ang, Yuen Yuen (Stanford Social Innovation Review, 2018)
More and more global aid agencies believe they should replace one-size-fits-all best practices with locally tailored solutions, but they must shift from just agreeing to “go local” to preparing development experts for the task by taking on three major problems with their internal practices.

How Beijing Commands: Grey, Black, and Red Directives from Deng to Xi - Ang, Yuen Yuen (The China Quarterly, 2022)

How China Escaped the Poverty Trap - Ang, Yuen Yuen (Cornell University Press, 2016)
Before markets opened in 1978, China was an impoverished planned economy governed by a Maoist bureaucracy. In just three decades it evolved into the world's second-largest economy and is today guided by highly entrepreneurial bureaucrats. In How China Escaped the Poverty Trap, Yuen Yuen Ang explains this astonishing metamorphosis. Rather than insist that either strong institutions of good governance foster markets or that growth enables good governance, Ang lays out a new, dynamic framework for understanding development broadly. Successful development, she contends, is a coevolutionary process in which markets and governments mutually adapt.By mapping this coevolution, Ang reveals a startling conclusion: poor and weak countries can escape the poverty trap by first harnessing weak institutions―features that defy norms of good governance―to build markets. Further, she stresses that adaptive processes, though essential for development, do not automatically occur. Highlighting three universal roadblocks to adaptation, Ang identifies how Chinese reformers crafted enabling conditions for effective improvisation.How China Escaped the Poverty Trap offers the most complete synthesis to date of the numerous interacting forces that have shaped China’s dramatic makeover and the problems it faces today. Looking beyond China, Ang also traces the coevolutionary sequence of development in late medieval Europe, antebellum United States, and contemporary Nigeria, and finds surprising parallels among these otherwise disparate cases. Indispensable to all who care about development, this groundbreaking book challenges the convention of linear thinking and points to an alternative path out of poverty traps.
Keywords: IMPORTANT

Integrating Big Data and Thick Data to Transform Public Services Delivery - Ang, Yuen Yuen (IBM Center for The Business of Government, 2019)
This report explains how public organizations can combine big data and thick data to transform public services delivery—a strategy called mixed analytics. Governments can greatly enhance the value of big data by combining it with “thick” data—rich qualitative information about users, such as their values, goals, and consumption behavior, obtained by observing or interacting with them in their daily lives. Big data holds great promise for improving public services delivery and innovation in government, but they are not a panacea . Having lots of data can be overwhelming or have little utility if the data are “thin”—that is, they lack meaning for users or fail to capture issues that matter most . By yielding insights into what citizens really care about and how they consume services, thick data can inform both the collection and analysis of big data. This report introduces the concept of “mixed analytics,” integrating big data and thick data to transform government decision making, public services delivery, and communication. The report presents three case studies of organizations that employ mixed analytics at the international, federal, and city level, respectively. Together, this research offers a set of transferable lessons for agencies at all levels of government: Lesson 1: Big data is a means to an end, rather than an end. Lesson 2: Thick data can identify unexpected problems or previously unexpressed needs. Lesson 3: Thick data can inform the analysis of big data. Lesson 4: Mixed analytics can offer both scale and depth. Lesson 5: Applying technology is a social activity, not an isolated technical task. Lesson 6: The best solutions are not always high-tech. The report concludes with five actionable recommendations for public managers. Make data and technology relevant to the people who use it. Leverage thick data at appropriate stages of the problem-solving process. Build an interdisciplinary team of quantitative and qualitative experts who work closely with stakeholders. Combine big and thick data to improve communication Improve government agencies’ knowledge of mixed research methods. This report builds on multiple past IBM Center reports about how government can leverage data and analytics to improve decisions, including Data-Driven Government: The Role of Chief Data Officers, by Jane Wiseman; Ten Actions to Implement Big Data Initiatives: A Study of 65 Cities, by Alfred Ho and Bo McCall, and Realizing The Promise of Big Data, by Kevin DeSouza. The report also complements several chapters that assess the past and potential use of data across the public sector in our recent book, Government For The Future: Reflection and Vision For Tomorrow’s Leaders. At a time when the US and other governments continue to enhance their use of data as a strategic asset for transformation, we hope that Professor Ang’s report provides useful insights for government managers and stakeholders.

Making Details Matter: How to Reform Aid Agencies to Generate Contextual Knowledge - Ang, Yuen Yuen (SSRN, 2014)
My essay considers a central problem of reinventing foreign aid in the twenty-first century: how to reform aid agencies to enable a “best-fit” approach to development assistance. For the past decades, the aid community has tried to transplant best practices from the developed world to the developing world. Increasingly, however, it is recognized that copying best practices does not work and may even backfire; rather, aid programs work best when they are tailored to local contexts. Yet while the idea of a best-fit approach is widely embraced in principle, actualizing it is easier said than done. For meaningful changes to take root in practice, we must first identify the obstacles to localizing development assistance and suggest ways to address these problems. To this end, I propose a three-pronged strategy to promote the generation of contextual knowledge among aid professionals, a necessary condition for crafting solutions that can fit various local contexts, namely: (1) build a bank of knowledge about unorthodox practices that work, (2) diversify expertise within aid agencies; and (3) carve experimental pockets. My proposal does not fit neatly into any one of the six themes specified in the GDN competition; rather, it concerns all of the themes. Whether it is to use aid to improve governance, apply information technology, or design financial instruments, the overarching challenge is to empower and incentivize aid professionals to learn and apply contextual knowledge to creatively solve problems in developing societies.
Keywords: Contextual knowledge; Localization; Public policy; foreign aid; international development

PDIA in action - Andrews, Matt; Samji, Salimah (Center for International Development, Harvard University, 2021)
Learning from our experience in 2020, we asked the alumni of our HKS Implementing Public Policy (IPP) Executive Education program, if they wanted to work with our students on their nominated problems. Eight IPP alumni, William Keith Young, Adaeze Oreh, Milzy Carrasco, Kevin Schilling, Artem Shaipov, George Imbenzi, David Wuyep, and Raphael Kenigsberg, who had been trained on PDIA and implementation, signed up to work with our students. Thirty-seven students signed up to take the course beginning January 26th, 2021. The students worked across eight teams and adopted a problem driven approach to foster learning that could help their authorizers develop an action learning strategy to their nominated challenge. This book highlights the students’ work drawing from their blogs as well as the event series. There are 8 sections, one for each of the teams and the problems they worked on during the course. We hope you enjoy reading their stories! Scan the QR Code at the end of each section to learn more.

Building State Capability: Evidence, Analysis, Action - Andrews, Matt; Pritchett, Lant; Woolcock, Michael (Oxford University Press, 2017)
This is an open access title available under the terms of a CC BY-NC-ND 4.0 licence. It is free to read at Oxford Scholarship Online and offered as a free PDF download from OUP and selected open access locations. Governments play a major role in the development process, and constantly introduce reforms and policies to achieve developmental objectives. Many of these interventions have limited impact, however; schools get built but children don't learn, IT systems areintroduced but not used, plans are written but not implemented. These achievement deficiencies reveal gaps in capabilities, and weaknesses in the process of building state capability.This book addresses these weaknesses and gaps. It starts by providing evidence of the capability shortfalls that currently exist in many countries, showing that many governments lack basic capacities even after decades of reforms and capacity building efforts. The book then analyses this evidence, identifying capability traps that hold many governments back - particularly related to isomorphic mimicry (where governments copy best practice solutions from other countries that make them look morecapable even if they are not more capable) and premature load bearing (where governments adopt new mechanisms that they cannot actually make work, given weak extant capacities). The book then describes a process that governments can use to escape these capability traps. Called PDIA (problem driveniterative adaptation), this process empowers people working in governments to find and fit solutions to the problems they face. The discussion about this process is structured in a practical manner so that readers can actually apply tools and ideas to the capability challenges they face in their own contexts. These applications will help readers devise policies and reforms that have more impact than those of the past.
Keywords: IMPORTANT

Doing Iterative and Adaptive Work - Andrews, Matt; Pritchett, Lant; Woolcock, Michael (CID Working Paper, 2016)
Many of the challenges in international development are complex in nature. They involve many actors in uncertain contexts and with unclear solutions. Our work has proposed an approach to addressing such challenges, called Problem Driven Iterative Adaptation (PDIA). This paper is the most recent in a series intended to show how one can do PDIA, building on the first paper, "Doing Problem Driven Work.” The current paper addresses a key part of the approach one moves to once a problem has been identified, performing real-time experimental iterations. This is intended as a practical paper that builds on experience and embeds exercises for readers who are actually involved in this kind of work.
Keywords: IMPORTANT

Doing Problem Driven Work - Andrews, Matt; Pritchett, Lant; Woolcock, Michael (HKS Working Paper, 2015)
Keywords: IMPORTANT

Escaping Capability Traps through Problem-Driven Iterative Adaptation (PDIA) - Andrews, Matt; Pritchett, Lant; Woolcock, Michael (HKS Working Paper, 2012)
Keywords: IMPORTANT

Building capability by delivering results: Putting Problem-Driven Iterative Adaptation (PDIA) principles into practice - Andrews, Matt; Pritchett, Lant; Samji, Salimah; Woolcock, Michael (A Governance Practitioner’s Notebook: Alternative Ideas and Approaches, 2015)
The Governance Practitioner’s Notebook takes an unusual approach for the OECD-DAC Network on Governance (GovNet). It brings together a collection of specially written notes aimed at those who work as governance practitioners within development agencies. It does so, however, without attempting to offer definitive guidance – instead aiming to stimulate thinking and debate. To aid this process the book is centred on a fictional Governance Adviser. The Notebook’s format provides space for experts to speak on today’s governance issues: politics, public sector reform and stakeholder engagement. It encourages debate, charts the evolution of donor thinking, and highlights future challenges in the age of the Sustainable Development Goals. Each section introduces both technical issues and major areas of debate, providing ideas for future development support to institutional reform.
Keywords: IMPORTANT

A Guide to Peer-to-Peer Learning - Andrews, Matt; Manning, Nick (Effective Institutions Platform, 2016)

Learning to Target for Economic Diversification: PDIA in Sri Lanka - Andrews, Matt; Ariyasinghe, Duminda; Batuwanthudawa, Thamari; Darmasiri, Shivanthika; Silva, Nilupul; Harrington, Peter; Jayasinghe, Prasanna; Jayasinghe, Upul; Jayathilake, Gamini; Karunaratne, Jayani; Katugampala, Lalit; Liyanapathirane, Jeewani; et al. (CID Faculty Working Paper 332, 2017)

Explaining positive deviance in public sector reforms in development - Andrews, Matt (World Institute for Development Economic Research (UNU-WIDER), 2013)
Public sector reforms are commonplace in developing countries. Much of the literature about these reforms reflects on their failures. This paper asks about the successes and investigates which of two competing theories best explain why some reforms exhibi
Keywords: Development; Innovation; Reform; case survey; governance; growth; leadership

Getting Real about Unknowns in Complex Policy Work - Andrews, Matt (Research on Improving Systems of Education (RISE), 2021)
As with all public policy work, education policies are demanding. Policy workers need to ‘know’ a lot—about the problems they are addressing, the people who need to be engaged, the promises they can make in response, the context they are working in, and the processes they will follow to implement. Most policy workers answer questions about such issues within the structures of plan and control processes used to devise budgets and projects. These structures limit their knowledge gathering, organization and sense-making activities to up-front planning activities, and even though sophisticated tools like Theories of Change suggest planners ‘know’ all that is needed for policy success, they often do not. Policies are often fraught with ‘unknowns’ that cannot be captured in passive planning processes and thus repeatedly undermine even the best laid plans. Through a novel strategy that asks how much one knows about the answers to 25 essential policy questions, and an application to recent education policy interventions in Mozambique, this paper shows that it is possible to get real about unknowns in policy work. Just recognizing these unknowns exist—and understanding why they do and what kind of challenge they pose to policy workers—can help promote a more modest and realistic approach to doing complex policy work.

The Limits of Institutional Reform in Development: Changing Rules for Realistic Solutions - Andrews, Matt (Cambridge University Press, 2013)
Institutional reforms are common across the globe. Think of efforts to build new governments in Afghanistan and Iraq; or decades worth of interventions intended to improve fiscal management, reduce corruption or introduce efficient public sector service delivery in African countries.These reforms often have limited results, however. They lead to new laws that are not properly implemented, and new organizations that have poor capacities and fail to function as needed. In this book, Matt Andrews explains why reform results are frequently limited and suggests ways to overcome these limits. In the first half of the book, Andrews argues that reforms fail to make governments better when they are introduced as signals to gain short-term support--from donors and others. Reforms as signals introduce unrealistic best practices that do not fit developing country contexts and are not considered relevant by implementing agents. The result is a set of new forms that do not function properly. Andrews uses examples to prove this point, ranging from efforts to introduce fiscal rules in Argentina to reforms aimed at international accounting standard adoption in many African countries, and anti corruption interventions in Malawi and Uganda. In the second half of the book, Andrews notes that there are instances where reforms are not being introduced as signals, and are having more of an impact on government effectiveness. Examples include local government reforms in Rwanda, anti corruption initiatives in Indonesia, and a variety of initiatives ranging from results based management to civil service modernization and internal control regime adoption in governments like Kenya, Kosovo and Afghanistan. Andrews uses these examples to discuss ways in which reforms can actually provide realistic solutions to governance challenges in developing countries. Lessons from these experiences suggest that reform limits can be overcome by focusing interventions on problem solving, and promoting incremental and localized processes to find solutions, involving multiple agents who can authorize and implement reforms.

LAC MEL Specialists Use Peer Network to Improve Quality and Use of Evidence - Anderson, Todd M; Prevatt, Amy (CLA Case Competition, 2019)
Based on consultations and stocktakings with LAC Missions, the LAC Bureau identified Mission Monitoring, Evaluation and Learning (MEL) Specialists as influential decision makers regarding data and evidence use. The LAC Bureau aims to improve the quality and use of evidence for decision making, and ultimately to improve development outcomes.

Time to Listen: Hearing People on the Receiving End of International Aid - Anderson, Mary B.; Brown, Dayna; Jean, Isabella (CDA Collaborative Learning Projects, 2012)
Does the way international assistance is organized make sense? Is it working as we mean it to? This book approaches these questions through the experiences of people living on the receiving side of international assistance. It reports on the ideas, insights, and analyses of almost 6,000 people across 20 countries where international aid has been provided. From such a range of locations and people, one might expect vastly different ideas and opinions. However, remarkably consistent patterns and common judgments emerged. In the midst of difference, there was striking unanimity and consistency about the processes and the effects of the international aid system. Using their words, their experiences, and their ideas, this book describes why the cumulative impacts of international aid efforts have not met expectations. It describes a way forward to make changes that, according to those on the receiving end, will lead to more effective and lasting results.

How Do Donor-led Empowerment and Accountability Activities Take Scale into Account? Evidence from DFID Programmes in Contexts of Fragility, Conflict and Violence - Anderson, Colin; Fox, Jonathan; Gaventa, John (IDS, 2020)
Development donors invest significantly in governance reform, including in contexts characterised by conflict and fragility. However, there is relatively little comparative study of their change strategies, and little understanding of what works and why. This paper explores the strategies of six recent DFID-funded programmes in Mozambique, Myanmar, and Pakistan with empowerment and accountability aims. Document review and field interviews are used to analyse the application of multi-scalar or multi-level change strategies, since such approaches are hypothesised to potentially generate more leverage for public accountability reforms. Analysis suggests that these strategies can strengthen citizen ability to navigate governance systems to resolve problems and claim accountability, and can bolster pro-accountability coalitions’ internal solidarity and external legitimacy. Multi-level strategies also appear associated with establishing more significant pressure for reform than exclusively local or national approaches. Yet conventional project reporting focuses on counting activities and outputs rather than analysing the dynamic, interactive processes at work in these strategies, and few evaluations are publicly accessible. To fully understand what kinds of action strengthen citizen demands for accountability requires a more transparent and rigorous approach to learning from donor-led governance interventions.

The Collaborative, Participatory & Empowerment Evaluation Topical Interest Group - American Evaluation Association
designed to foster collaboration, share knowledge, and promote scholarship, this TIG is a division of the American Evaluation Association (AEA).

Doing Accountability Differently: A 'Vertically Integrated' Approach - Amakom, Uzochukwu; Fashola, Temitope; Gay, Charles; Shutt, Cathy (Christian Aid, 2018)

Making humanitarian response more flexible: bibliography - ALNAP (2018)

An introduction to systems thinking and tools for systems thinking - Allen, Will; Kilvington, Margaret (Learning fro Sustainability, 2018)
Systems thinking is an approach to integration that is based on the belief that the component parts of a system will act differently when isolated from the system’s environment or other parts of the system. THIS PAGE PROVIDES PLENTY OF RESOURCES ON SYSTEM THINKING

Navigating complexity: adaptive management and organizational learning in a development project in Northern Uganda - Allana, Amir; Sparkman, Timothy (Knowledge Management for Development Journal, 2014)
Adaptive management is a management paradigm for intervening in complex, unpredictable systems where continual learning and adaptation is vital for success. This management approach requires a fundamentally different set of tools, processes, and most importantly, staff behaviors and organizational culture than ‘traditional’ management. A facilitative approach to development, where the goal is creating systemic change that spreads in networks of local businesses, government, and civil society organizations, necessitates an adaptive approach. Mercy Corps Uganda’s Northern Karamoja Growth, Health, and Governance Program (GHG) has been an ongoing experiment in applying the principles of adaptive management and facilitation. This article details two examples from GHG where adaptation has occurred, and provides an analysis of the tools, processes, and organizational culture that exists. Three salient takeaways for effective facilitation are elaborated on: the importance of staff behaviors and underlying beliefs, particularly with regards to ‘failure’, flexibility to experiment, dissent & debate, and curiosity with the subject matter of their work; importance of consistent messaging from senior management with regards to the same; and tools and processes playing a support function to these behaviours, rather than being their source. Lessons and implications are drawn out from the GHG experience for funders and implementing organizations wanting to apply adaptive management in the context of development programming. These include building flexibility into budgeting and contracts, rethinking the structure and content of reports, and utilizing alternative hiring criteria to attract senior managers who are more likely to succeed at adaptive management.

Navigating complexity: Adaptive management in the Northern Karamoja Growth, Health & Governance program - Allana, Amir (MercyCorps, 2014)
Development actors increasing agree that managing programs adaptively – especially complex interventions – can improve their effectiveness. But what does adaptive management look like in practice?
Keywords: IMPORTANT; Practice

Where have we got to on adaptive learning, thinking and working politically, doing development differently etc? Getting beyond the People’s Front of Judea - Algoso, Dave; Hudson, Alan (From Poverty to Power, 2016)
Dave Algoso and Alan Hudson at Global Integrity compare and contrast 9 different initiatives that are all heading in roughly the right direction in aid reform
Keywords: IMPORTANT

How to Build Movements with Cyclical Patterns in Mind - Non Profit News | Nonprofit Quarterly - Algoso, Dave; Guerzovich, Florencia; Gattoni, Soledad (Nonprofit Quarterly, 2023)
The world changes too much for anyone who is invested in social change work to imagine that this work is linear and predictable. Opportunities come and go, whether caused by a pandemic or political shifts. This much most social movement leaders and activists intuitively understand. But what can be done with this realization? How might movement groups better prepare for moments of opportunity? We want to explore how we can create the changes we want to see by responding to the changes that are outside our control.

Adapting Aid: Lessons from six case studies - Algoso, Dave; Beloe, Jon; Hemberger, Alison; Hill, Philippa; Proud, Emma (MercyCorps & IRC, 2016)
Keywords: IMPORTANT

Eight Practices for Strategic Agility - Algoso, Dave (Stanford Social Innovation Review, 2019)
Rather than a glossy brochure that no one reads, your strategy should be an ongoing practice that informs your decisions and adapts as circumstances change. A Viewpoint from the Summer 2019 issue.

How INGOs are Doing Development Differently - Algoso, Dave (Care, IRC, Mercy Corps, Oxfam & World Vision, 2017)

More books on complexity than you can shake a stick at—plus a mea culpa - Algoso, Dave (Praxis, 2016)
Last month, Duncan Green was kind enough to post my overly ambitious multi-book review on complexity thinking in development on his From Poverty to Power blog. It covered three books: Ben Ramalingam’s Aid on the Edge of Chaos; Jean Boulton, Peter Allen, and Cliff Bowman’s Embracing Complexity; and Danny Burns and Stuart Worsley’s Navigating Complexity in International Development. It...

Data-powered positive deviance: Combining traditional and non-traditional data to identify and characterise development-related outperformers - Albanna, Basma; Heeks, Richard; Pawelke, Andreas; Boy, Jeremy; Handl, Julia; Gluecker, Andreas (Development Engineering, 2022)
The positive deviance approach in international development scales practices and strategies of positively-deviant individuals and groups: those who are able to achieve significantly better development outcomes than their peers despite having similar resources and challenges. This approach relies mainly on traditional data sources (e.g. surveys and interviews) for identifying those positive deviants and for discovering their successful solutions. The growing availability of non-traditional digital data (e.g. from remote sensing and mobile phones) relating to individuals, communities and spaces enables data innovation opportunities for positive deviance. Such datasets can identify deviance at geographic and temporal scales that were not possible before. But guidance is needed on how this new data can be employed in the positive deviance approach, and how it can be combined with more traditional data to gain deeper, more meaningful, and context-aware insights. This paper presents such guidance through a data-powered method that combines both traditional and non-traditional data to identify and understand positive deviance in new ways and domains. This method has been developed iteratively through six development projects covering five different domains – sustainable cattle ranching, agricultural productivity, rangeland management, research performance, crime control – with global and local development partners in six countries. The projects combine different types of non-traditional data with official statistics, administrative data and interviews. Here, we describe a structured method for data-powered positive deviance developed from the experience of these projects, and we reflect on lessons learned. We hope to encourage and guide greater use of this new method; enabling development practitioners to make more effective use of the non-traditional digital datasets that are increasingly available.

Positive deviance, big data, and development: A systematic literature review - Albanna, Basma; Heeks, Richard (The Electronic Journal of Information Systems in Developing Countries, 2019)
Positive deviance is a growing approach in international development that identifies those within a population who are outperforming their peers in some way, eg, children in low-income families who are well nourished when those around them are not. Analysing and then disseminating the behaviours and other factors underpinning positive deviance are demonstrably effective in delivering development results. However, positive deviance faces a number of challenges that are restricting its diffusion. In this paper, using a systematic literature review, we analyse the current state of positive deviance and the potential for big data to address the challenges facing positive deviance. From this, we evaluate the promise of “big data-based positive deviance”: This would analyse typical sources of big data in developing countries—mobile phone records, social media, remote sensing data, etc—to identify both positive deviants and the factors underpinning their superior performance. While big data cannot solve all the challenges facing positive deviance as a development tool, they could reduce time, cost, and effort; identify positive deviants in new or better ways; and enable positive deviance to break out of its current preoccupation with public health into domains such as agriculture, education, and urban planning. In turn, positive deviance could provide a new and systematic basis for extracting real-world development impacts from big data.
Keywords: big data; developing countries; machine learning; mobile data; positive deviance; systematic literature review

Politics matters, so what? Time for bigger bets (and more learning) on adaptive programming - Alan (www.globalintegrity.org, 2016)
By Alan Hudson, Executive Director, Global Integrity, July 26, 2016 Politics matters. Context too. And blueprints have limited value. Our strategy is based on these insights, so we’re totally on board. A World Development Report (WDR) that puts power and politics...

Try, learn, adapt, repeat: T/A Learn reflections - Alan (www.globalintegrity.org, 2015)
By Alan Hudson — November 16, 2015. I spent last week in Rio de Janeiro (tough assignment, I know), participating in the Transparency and Accountability Initiative’s third T/A Learn Annual Workshop. As the report of the second Annual Workshop, held...

Afterword: Alabert Hirschman observed - Alacevich, Michele (Development Projects Observed, 2015)
Albert Hirschman considered Development Projects Observed a natural sequel to his earlier work. As Hirschman put it to an old acquaintance at the World Bank, his previous books identified the inner and often hidden mechanisms of development sequences: “Having worked out a few basic hypotheses …, I could perhaps test them (and hit on some new ones) by looking at Bank-financed projects that have had enough time to give rise to such sequences.”¹ But as much as it appeared as the logical evolution of Hirschman’s intellectual trajectory, this new research was also the offshoot of an important crisis in development

Non-Violent Communication in everyday life - Aktar, Afrin; Goon, Swmaranika; Pal, Ujjal; Zaman, Thahina (CLARISSA, 2021)
In March, members of the team in Bangladesh (Community Mobilisers and Lead Community Mobilisers) participated in a training course in non-violent communication (NVC). NVC is a communication technique that prioritises listening over speaking. It aims to improve communication by achieving a deeper understanding of our emotions and values and what we observe in the behaviour of others. The training was facilitated by Paul Kahawatte, a UK-based mediator, facilitator and trainer, Roufun Naher, a lecturer at the University of Dhaka and NVC practitioner in Bangladesh, Neil Howard, co-lead of Social Protection for CLARISSA and Jiniya Afroze, CLARISSA Country Coordinator in Bangladesh. The aim of the training was to support the community mobilisers to explore and practice different ways of communicating and facilitating. This will help them support communities to come together and build their collective power for the social protection ‘cash plus’ pilot. Lemon Roy, a community mobiliser was grateful for the experience; “I had little knowledge about Non-Violent Communication before participating in the training. I didn’t know that passive listening and the communication gap between two people could lead to conflict. I have learned about tools and techniques in conflict mediation and the importance of understanding the feelings and needs of other people.”.

Unconditional cash support sparks transformation: A story from CLARISSA Social Protection Intervention in Dhaka - Aktar, Afrin; Debnath, Raj; Kashfi, Sharmin (CLARISSA, 2023)
The CLARISSA Social Protection (SP) intervention provided six months of unconditional cash transfers to every household in the Gojmohol neighbourhood,...

Is (Web) Science Ready for Empowerment? - Akkermans, Hans; Gyan, Nana Baah; Bon, Anna; Tuyp, Wendelien; Grewal, Aman; Boyera, Stéphane; Allen, Mary (ACM WebSci'11, 2011)

Creative Activities for Work Teams and Communities of Practice. A guide for designing participatory, productive, and engaging sessions - AKF (2023)
This free guide aims to build the skills and confidence of people designing and facilitating participatory, productive, and engaging workshop sessions for work teams and communities of practice by providing them with a series of creative activities that can be run in-person or virtually. By the time they have gone through this guide, session leads will be able to: Describe ways in which creative activities can be of value to teams and communities of practice, Design both in-person and virtual workshop sessions that incorporate creative activities, Facilitate participatory, productive and engaging workshop sessions that incorporate one or several creative activities, and Lead an effective debrief following a creative activity. This toolkit was developed by the Aga Khan Foundation.

Virtual Facilitation Techniques for Work Teams and Communities of Practice. A guide for designing and running dynamic and engaging remote sessions and meetings - AKF (2023)
This free guide aims to build the skills and confidence of people designing and facilitating virtual meetings, including online community of practice sessions. By the time they have gone through this guide, community of practice facilitators will be able to: Design an engaging and participatory virtual session, Develop an effective, varied and realistic agenda for a virtual session, Apply a wide range of virtual facilitation techniques, and Using technology to facilitate virtual sessions. This toolkit was developed by the Aga Khan Foundation.

Supply Chains, the Informal Economy, and the Worst Forms of Child Labour - Aked, Jody (CLARISSA Working Paper, 2021)
As a cohort of people, ‘children in work’ have become critical to the everyday functioning of diverse supply chain systems. This Working Paper considers diverse commodity chains (leather, waste, recycling and sex) to explore the business realities that generate child labour in its worst forms. A review of the literature finds that occurrence of the worst forms of child labour (WFCL) in supply chain systems is contingent on the organising logics and strategies adopted by actors in both the formal and informal economies. Piecing together the available evidence, the paper hypothesises that a supply chain system is sensitive to the use of WFCL when downward pressure to take on business risk cannot be matched by the economic resilience to absorb that risk. Emergencies and persistent stressors may increase risk and reduce resilience, shifting norms and behaviour. There is a need for further work to learn from business owners and workers in the informal economy.

Beyond Tweets and Screams: Action for Empowerment and Accountability in Nigeria – The Case of the #BBOG Movement - Aina, Tade Akin; Atela, Martin; Ojebode, Ayo; Dayil, Plangsat; Aremu, Fatai (IDS, 2019)
This paper explores the nature, role and dynamics of new forms of social and political action as pathways to empowerment and accountability in fragile conflict- and violence-affected settings in Africa. Through an in-depth analysis of the case of the Bring Back Our Girls (#BBOG) movement in Nigeria and a multi-methods approach, the paper provides new knowledge that addresses evidence gaps in the following areas: (1) the multiple ways through which social and political action play out in fragile, conflict- and violence-affected settings; (2) whether the conditions in which new forms of social and political action applicable to fragile, conflict- and violence-affected settings – the settings most dominant in African countries – emerge as currently projected in the literature; and (3) whether these social and political actions necessarily produce accountability and empowerment in fragile, conflict- and violence-affected settings. Although often expressed as contentious and/or unruly politics, experiences from the BBOG movement suggest that the new forms of social and political action possess a wide range of implications for citizen action and governance, including leading to multiple forms of empowerment in fragile settings.

’Adaptive Authoritarianism’ in Contemporary China: Identifying Zones of Legitimacy Building - Ahlers, Anna Lisa; Schubert, Gunter (Reviving Legitimacy: Lessons for and from China, 2011)
Anna L. Ahlers and Gunter Schubert, ’Adaptive Authoritarianism’ in Contemporary China: Identifying Zones of Legitimacy Building, in: Deng Zhenglai and Guo Sujian (eds), Reviving Legitimacy: Lessons for and from China, Lanham: Lexington Books (2011),

Agile Glossary and Terminology - Agile Alliance (Agile Alliance, 2015)
Learn the unique terminology used in Agile development from the experts at Agile Alliance.

Manifesto for Agile Software Development - Agile Alliance (Agile Alliance, 2001)

Subway Map to Agile Practices - Agile Alliance (Agile Alliance, 2015)
AGILE 101

Can policy-makers think like designers? - Adler, Natalia (A workshop on Doing Development Differently, 2014)
This is the poster used by Natalia Adler, from Unicef, at the Doing Development Differently workshop in 2014. Can policy-makers think like designers? Since 2012, UNICEF has been supporting two autonomous governments in Nicaragua to develop empathy-driven policies for children. While policies are the final product, the goal of the Designing for Children Initiative is to make government officials think like designers. A good policy or service is nothing without active and engaged frontline workers. From the onset, our objective was to transform public employees into ‘public entrepreneurs.’ In doing so, we have disrupted the traditional assumptions about participatory governance, where it normally suffices to hold a meeting and have people talk about their needs and aspirations. For participation to work, government officials have to relearn how to listen. They have to let go of pre-determined assumptions and their expert status, and put themselves in the shoes of the people they are trying to reach. This is the process of empathy-driven policy-making. For the past two years, we have used a Human Centered Design (HCD) approach to make this happen. While HCD is often seen as an innovation in itself, for us, it’s just a means to an end: common-sense development.

Adapt LLC - Adapt LLC (Adapt Consult)
Doing Development Differently… Adapt LLC provides expertise to international development practitioners seeking to improve the impacts of their investments through a more in-depth understanding of the political and economic risks and opportunities associated with programming in complex environments. The Adapt team enhances the design and implementation of international development assistance strategies, programs, and projects around the world.

+Acumen MOOC: Adaptive Leadership: Mobilizing for Change and Disrupting the Status Quo - +Acumen (NovoEd, 2017)
This four-module course by +Acumen and Adaptive Change Advisors on adaptive leadership, which is a practical leadership framework developed at Harvard Kennedy School for driving systemic change, particularly during times of uncertainty or when there are no easy answers.

Lean Data Approaches to Measure Social Impact - ACUMEN (NovoEd, 2018)

Accountability Learning and Planning System - ActionAid (ActionAid International, 2000)

Alps - Accountability, Learning and Planning System - ActionAid (ActionAid International, 2006)
Alps is ActionAid’s over arching accountability framework, containing within it our programme planning system. Alps is distinctive in that it is strongly driven by principles, and sets out necessary personal attitudes and behaviours alongside organisational processes for planning, strategy formulation, learning, monitoring reviews/evaluations and audit. Alps defines our standards, not only about what we do but also how we do it. Alps is part of ActionAid’s human rights-based work.

Alps - Accountability, Learning and Planning System - 2011 Update - ActionAid (ActionAid International, 2011)
Alps is ActionAid’s over arching accountability framework, containing within it our programme planning system. Alps is distinctive in that it is strongly driven by principles, and sets out necessary personal attitudes and behaviours alongside organisational processes for planning, strategy formulation, learning, monitoring reviews/evaluations and audit. Alps defines our standards, not only about what we do but also how we do it. Alps is part of ActionAid’s human rights-based work.

Notes to Accompany ALPS - ActionAid (ActionAid International, 2001)

Reflect Action Methods - ActionAid (2009)
Reflect is an innovative approach to adult learning and social change, which fuses the theories of Brazilian educator Paulo Freire with participatory methodologies. It was developed in the 1990s through pilot projects in Bangladesh, Uganda and El Salvador and is now used by over 500 organisations in over 70 countries worldwide. Organisations working with Reflect won UNESCO literacy prizes in 2003, 2005, 2007, 2008 and 2010. This site is aimed at Reflect practitioners and others who want to know more about the approach. We want it to be truly interactive and the more stories, graphics and photos are submitted, the better the site will be. Please add your comments to tell us about the work you are doing with Reflect and join the discussion forum to share your ideas and experiences of working with the approach.

Appreciative Inquiry: An approach for learning and change based on our own best practices​ - Acosta, Anne; Douthwaite, Boru (ILAC Brief, 2005)
Since it was conceptualized in the late 1980s as a research methodology and change paradigm, the technique of ‘appreciative inquiry’ (AI) has proved to be highly effective for capturing the positive features of an organization or social system and energizing the members to strive for higher levels of performance. This Brief outlines the basic principles and methods of AI, describes various domains in which it has been undertaken and provides a recent example of its use in a centre affiliated with the Consultative Group on International Agricultural Research (CGIAR).

From Data to Wisdom - Ackoff, Russell (Journal of Applied Systems Analysis, 1989)

A new pathway: how can funders support meaningful monitoring, evaluation, and learning practice in the field? - Blog post on Better Evaluation - Acevedo, Andrea; Colnar, Megan (2023)
How can donors and grantees work together to create effective monitoring, evaluation, and learning (MEL) practices that drive field-wide transformation?

Getting strategic: vertically integrated approaches - Aceron, Joy; Isaac, Francis (Institute of Development Studies, 2016)
Holding power to account requires understanding where power lies and how it is exercised. It entails understanding how decisions are made, who makes them and what decision criteria are used to make them. Vertically integrated civil society action takes into account how power is exercised and how decisions are made in a given policy, programme or process.

Going vertical: citizen-led reform campaigns in the Philippines - Aceron, Joy; Isaac, Francis (Making All Voices Count Research Report, 2016)
The Philippines has a long history of state–society engagement to introduce reforms in government and politics. Forces from civil society and social movements have interfaced with reform-oriented leaders in government on a range of social accountability initiatives – to make governance more responsive, to introduce policy reforms, and to make government more accountable. Several theoretical propositions on which strategic approaches work best for social accountability initiatives have been put forward – including the idea of vertically integrated civil society monitoring and advocacy. This multi-authored research report uses vertical integration as a framework for examining seven successful civil society social accountability initiatives in the Philippines, looking at what made them successful, and how the gains they realised can be deepened and sustained.

Thinking big: How to use theory of change for systems change - Abercrombie, Rob; Boswell, Katie; Thomasoo, Rosanna (NPC, 2018)
We think that, applied well, theory of change can support charities and funders to take a systemic approach to their work. This report identifies five common pitfalls that organisations fall into when using theory of change, and walks through five rules of thumb that will help organisations to use the approach to tackle complex problems. We think that, applied well, theory of change can support charities and funders to take a systemic approach to their work. This report identifies five common pitfalls that organisations fall into when using theory of change, and walks through five rules of thumb that will help organisations to use the approach to tackle complex problems.

Historical roots of Agile Methods: Where did “Agile Thinking” come from? - Abbas, Noura; Gravell, Andrew M.; Wills, Gary B. (Springer Berlin Heidelberg, 2008)
The appearance of Agile methods has been the most noticeable change to software process thinking in the last fifteen years [16], but in fact many of the “Agile ideas” have been around since 70’s or even before. Many studies and reviews have been conducted about Agile methods which ascribe their emergence as a reaction against traditional methods. In this paper, we argue that although Agile methods are new as a whole, they have strong roots in the history of software engineering. In addition to the iterative and incremental approaches that have been in use since 1957 [21], people who criticised the traditional methods suggested alternative approaches which were actually Agile ideas such as the response to change, customer involvement, and working software over documentation. The authors of this paper believe that education about the history of Agile thinking will help to develop better understanding as well as promoting the use of Agile methods. We therefore present and discuss the reasons behind the development and introduction of Agile methods, as a reaction to traditional methods, as a result of people’s experience, and in particular focusing on reusing ideas from history.
Keywords: Agile methods; Software Development

18F Method Cards - A collection of tools to bring human-centered design into your project - 18F-GSA (18F-GSA, US Governement, 2017)
18F’s method cards describe how our organization puts human-centered design into practice.

About the BEAM Exchange (BEAM Exchange, 2018)

Adaptive management in aid programmes (BEAM Exchange, 2014)
Aid works better – especially in complex and conflict-affected environments – when there is scope to trial and adapt programme strategies. So argues Amir Allana in Navigating Complexity, a case study of Northern Karamoja's Growth, Health and Governance (GHG) Programme that is adopting an 'adaptive management' approach. This BEAM Exchange webinar invited Amir, Tim Sparkman and Peter Roggekamp to discuss lessons from their work in Uganda and Cambodia. What does adaptive management look like in practice? And what does it require of managers and funders to make it happen? Tim and Peter share their experiences of applying adaptive management techniques to the Growth, Health and Governance Programme and the Cambodia Agricultural Value Chain Program (CAVAC). Amir highlight insights from the Navigating Complexity report, including the importance of office culture, consistent management signals, and the role of supportive tools and processes.

Book Review: Branko Milanovic's brilliant take on Global Inequality (From Poverty to Power, 2016)
Some of my favourite development economists are nomads, people with feet in different regions, which seems to make them better able to identify interesting patterns and similarities/differences bet...

Busan Outcome Document (2011)

Center for Theory of Change (Theory of Change Community)

Collaboration for Impact (2018)
The site seeks to accelerate the adoption of system collaboration and Collective Impact in Australia. It does so by creating knowledge, network and tools; shining a light on successful Collective Impact initiatives; and building an Australian community of practice.

Conflict and Human Rights Shareweb (2019)

Donor Committee for Enterprise Development (DCED, 2018)
The DCED is a forum for learning about the most effective ways to fulfil the SDGs – based on practical experience in Private Sector Development (PSD).

Feedback Commons (Keystone Accountability, 2018)

Fostering Learning and Effectiveness through Collaboration: A New Collaborative on Transparency, Accountability, and Participation (TAI, 2018)
What are the features, values, and practices of effective learning organizations? How do learning practices contribute to more effective programming? And, how can collaborations between academics, researchers and practitioners better support learning organizations in the global South?  These are just a few of the questions that a new global learning collaborative seeks to explore. In …

Making All Voices Count's Research & Evidence Strategy (2014)
Building an evidence-base on what works in technology for voice, transparency and accountability, how, and why.

mHero and the Principles for Digital Development: Development Done Right | iHRIS (2016)

MIRADI: Adaptive Management Software for Conservation Projects (2018)

Monitoring and Measuring Results in Private Sector Development (Swiss Agency for Development and Cooperation SDC, 2016)

Nairobi Outcome Document (2016)

NetworkWeaver (NetworkWeaver, 2018)
Weaving Smart Networks

Organizational Development Network - Advancing the science, practice and impact of Dialogic Organizational Development
Dialogic OD is a label used to distinguish a mindset about organizations, leadership and change that is different from foundational Diagnostic OD. Gervase Bushe and Bob Marshak introduced the concept in 2009 to show that new forms of organization development had emerged since the mid 1980s (like appreciative inquiry, future search, open space, and world café, among many others) that did not conform with, and in some ways violated, central principles of OD found in textbooks and taught in graduate programs. Since then, theory and research on Dialogic OD has expanded rapidly. Bushe and Marshak argue that each OD practitioner is likely to use a unique combination of diagnostic and dialogic mindsets in their practice. In the 2015 textbook on Dialogic Organization Development, Edgar Schein describes Dialogic OD as a return to the roots of OD’s original spirit of inquiry, and opined that Dialogic OD may have emerged because organizational problems are now more complex, ambiguous and uncertain. Since then, the idea that Diagnostic OD is better suited to complicated, technical problems and Dialogic OD better suited to complex, adaptive challenges, has been echoed by others.

Positive Deviance Initiative (2016)

Swiss Programme for Research on Global Issues for Development (r4d programme) - NFP [Nr.] (2018)
Research and innovation are decisive factors for sustainable global development. With their knowledge, skills, experience and worldwide network of partners, researchers can make an important contribution towards the solution of global problems.