Guidelines for good market development program design. A managers’ perspective - Bekkers, Harald; Roggekamp, Peter (BEAM Exchange, 2014)
Good programme design is a key factor contributing to the success of market development programmes. Unfortunately, too many current designs have major flaws which prevent programmes becoming successful even before they get started. This document considers what makes market development programmes consistently successful and how to prevent programme design from being a hurdle to sustainable and efficient impact as scale.

Guidelines for designing and monitoring social accountability interventions - Franco, Erika Lopez; Shankland, Alex (IDS, 2018)

Guidelines and good practices for evidence-informed policy-making in a government department - Wills, Alf; Tshangela, Mapula; Shaxson, Louise; Datta, Ajoy; Matomela, Bongani (2016)

Guide to Evaluating Participatory Processes - Practical guide - Parés, Marc (2013)
Guide to Evaluating Participatory Processes - Practical guide
Keywords: Academia; Biology; Computer Science; Earth Sciences; Geography; History; Law; Math; Medicine; Philosophy; Physics; Political science; Psychology; Religion; Research; economics; universities

Guidance: Rigor and Reproducibility in Grant Applications | grants.nih.gov - NIH (National Institutes of Health, 2018)

Guidance on using the revised Logical Framework - How to note - DFID (DFID, 2011)
The principal changes to the logframe from the earlier (2008) 4 x 4 matrix are: - Objectively Verifiable Indicators (OVIs) have been separated into their component elements (Indicator, Baseline and Target), and Milestones added. - Means of Verification has been renamed ‘Source’. - Inputs are now quantified in terms of funds (expressed in Sterling for DFID and all partners) and use of DFID staff time (expressed as annual Full-Time Equivalents (FTEs). - A DFID Share box now indicates the financial value of DFID’s Inputs as a percentage of the whole. - Assumptions are shown at Purpose and Output level only. - Risks are shown at Activities level only, but also rated at Output level; - At the Output level, the Impact Weighting is now shown in the logframe together with a Risk Rating for individual Outputs. - Activities are now shown separately (so do not normally appear in the logframe sent for approval), although they can be added to the logframe if this is more suitable for your purposes. - A renewed emphasis on the use of disaggregated beneficiary data within indicators, baselines and targets. The Logical Framework (logframe) was significantly re-designed in February 2009. In January 2011 a slightly amended logframe template was introduced at the same time as the launch of the new DFID Business Case. Given the extent of changes that took place in February 2009 it was only necessary to make minor amendments in January 2011. These amendments are as follows: - Results Chain terminology aligned across DFID (the terms Goal and Purpose in the old logframe template have been replaced by Impact and Outcome) - Rows added to allow achieved results to be captured alongside the planned results determined at project design stage - Word version of the logframe template removed – excel is the preferred format for logical frameworks from January 2011 - Indicator numbering introduced within logframe template

Guidance on tasks and deliverables for different evaluation phases - Methods Lab (2015)
This tool describes the five key phases of evaluation, from planning and design, to implementation and communication of results. It provides a list of the main tasks and deliverables for each phase, intended for use by anyone managing an impact evaluation. This tool was developed by Irene Guijt, Simon Hearn, Tiina Pasanen and Patricia Rogers for use in Methods Lab projects. It follows to some extent the BetterEvaluation Rainbow Framework.

Grow - Scale - Impact: How to help inclusive business achieve scale - Pérez Castro, Andrea A.; Tora, Krisztina (GIZ, 2016)

Grounded Theory Method in HCI and CSCW - Muller, Michael J; Kogan, Sandra (IBM Center for Social Software, 2010)

Governance for Growth in Vanuatu: Review of a decade of thinking and working politically - Hadley, Sierd; Tilley, Helen (ODI, 2017)

Governance and Service Delivery: Practical Applications of Social Accountability Across Sectors - Wetterberg, Anna; Brinkerhoff, Derick W.; Hertz, Jana C. (RTI Press, 2016)
The six case studies in Governance and Service Delivery: Practical Applications of Social Accountability Across Sectors illustrate the multiple ways citizen participation in accountability – called social accountability – can lead to positive effects on governance, citizen empowerment, and service delivery. Drawing on their extensive experience implementing international donor-funded programs and projects, the authors examine six recent RTI International projects in Africa and Asia. The analysis focuses on both intended and actual effects, and it unpacks the influence of context on implementation and outcomes achieved, as well as on prospects for sustainability. Donors, academics, policy makers, practitioners and others interested in sustainable development and governance will find this systematic review invaluable.

Going vertical: citizen-led reform campaigns in the Philippines - Aceron, Joy; Isaac, Francis (Making All Voices Count Research Report, 2016)
The Philippines has a long history of state–society engagement to introduce reforms in government and politics. Forces from civil society and social movements have interfaced with reform-oriented leaders in government on a range of social accountability initiatives – to make governance more responsive, to introduce policy reforms, and to make government more accountable. Several theoretical propositions on which strategic approaches work best for social accountability initiatives have been put forward – including the idea of vertically integrated civil society monitoring and advocacy. This multi-authored research report uses vertical integration as a framework for examining seven successful civil society social accountability initiatives in the Philippines, looking at what made them successful, and how the gains they realised can be deepened and sustained.

Going Local 2.0: How to Reform Development Agencies to Make Localized Aid More Than Talk - Ang, Yuen Yuen (Stanford Social Innovation Review, 2018)
More and more global aid agencies believe they should replace one-size-fits-all best practices with locally tailored solutions, but they must shift from just agreeing to “go local” to preparing development experts for the task by taking on three major problems with their internal practices.

Glossary of Knowledge Management - Serrat, Olivier (Knowledge Solutions: Tools, Methods, and Approaches to Drive Organizational Performance, 2017)
The knowledge management discipline can be cryptic. These Knowledge Solutions define its most common concepts in simple terms.
Keywords: Definitions; Knowledge management; Taxonomy; Terminology

Global mapping of technology for transparency and accountability - Avila, Renata; Feigenblatt, Hazel; Heacock, Rebekah; Heller, Nathaniel (Transparency & Accountability Initiative, 2010)

Global Learning for Adaptive Management (GLAM) - DFID (DFID Development Tracker, 2016)
To improve the value for money and impact of development interventions by promoting the use of better monitoring, evaluation and learning strategies that support development programmes to gather and respond to emergent evidence on the effects of interventions, as well as changes in the development context. The Global Learning for Adpative Management programme will make available technical assistance to DFID, USAID and partners to develop this ‘adaptive management’ approach, as well as build an evidence base to understand links between the use of adaptive management and more effective, efficient, relevant and sustainable development interventions.

Global governance and local peace: accountability and performance in international peacebuilding - Campbell, Susanna (Cambridge University Press, 2018)
Local peacebuilding and global accountability -- The country context--Burundi from 1999 to 2014 -- Ingos in peacebuilding--globally unaccountable, locally adaptive -- International organizations in peacebuilding--globally accountable, locally constrained -- Bilateral development donors--accountable for global targets, not local change

Global Delivery at the World Bank Group - GDI (Global Delivery Initiative, 2014)

Global Affairs Canada: an approach to development innovation - GAC (Global Affairs Canada, 2018)
For Global Affairs Canada, development innovation includes new or improved business models, policy practices, approaches, technologies, behavioral insights or ways of delivering products and services that benefit and empower the poorest and most vulnerable people in developing countries.

GLAM Monitoring and Learning Strategy (internal document) - GLAM (ODI, 2018)

GIZ’s evaluation system - GIZ Evaluation Unit (GIZ GmbH, 2018)
This provides a general description of GIZ’s new evaluation system, which is designed to implement our new policy and to achieve the objectives of evaluation reform. This general description is accompanied by two other documents that provide supplementary details of GIZ’s key evaluation instruments: project evaluations for BMZ business and corporate strategic evaluations. They are geared in particular to GIZ staff members who commission, implement or support evaluations and to evaluators who conduct evaluations on behalf of GIZ as well as repre-sentatives of commissioning parties, clients and cooperation partners. The current versions are very much a ‘work in progress’ in many regards. We are well aware that we will only be able to meet our own high expectations of our evaluations if we continue to develop their quality in dialogue with project staff, our commissioning parties, clients and partners and with evaluation practitioners and experts from the academic, scientific and research community. We look forward to taking on this task and to sharing our experience with all interested parties.

GIZ's Evaluation Policy: Principles, guidelines and requirements of our evaluation practice - GIZ (GIZ, 2018)

Getting strategic: vertically integrated approaches - Aceron, Joy; Isaac, Francis (Institute of Development Studies, 2016)
Holding power to account requires understanding where power lies and how it is exercised. It entails understanding how decisions are made, who makes them and what decision criteria are used to make them. Vertically integrated civil society action takes into account how power is exercised and how decisions are made in a given policy, programme or process.

Getting Strategic about Technology and Accountability: MAVC Learning and Inspiration Event in Manila - Halloran, Brendan (Politics, Governance and Development, 2016)
Two years ago, Jonathan Fox released a widely read and well-received paper looking at the evidence for citizen-led accountability.  In it, he noted the proliferation of short-term, isolated interve…

Getting Past the Rhetoric #2: Managing for ‘Thinking and Working Politically’ in Large Facilities - Tyrrel, Lavinia (Abt Associates - Governance Soapbox, 2018)

Getting Past the Rhetoric #1: Tendering for ‘Thinking and Working Politically’ in Large Facilities - Tyrrel, Lavinia (Abt Associates - Governance Soapbox, 2018)

Getting good at disruption in an uncertain world: Insights from Southern NGO leaders - Buckley, Lila; Ward, Halina (International Institute for Environment and Development, 2016)
Keywords: IMPORTANT

Getting beneath the surface in program planning, monitoring and evaluation: Learning from use of participatory action research and theory of change in the CGIAR Research Program on Aquatic Agricultural Systems - Apgar, J Marina; Allen, Will; Albert, Joelle; Douthwaite, Boru; Paz Ybarnegaray, Rodrigo; Lunda, Jeston (Action Research, 2017)
Many rural poor and marginalized people strive to make a living in social-ecological systems that are characterized by multiple and often inequitable interactions across agents, scale and space. Uncertainty and inequality in such systems require research and development interventions to be adaptive, support learning and to engage with underlying drivers of poverty. Such complexity-aware approaches to planning, monitoring and evaluating development interventions are gaining strength, yet, there is still little empirical evidence of what it takes to implement them in practice. In this paper, we share learning from an agricultural research program that used participatory action research and theory of change to foster learning and support transformative change in aquatic agricultural systems. We reflect on our use of critical reflection within participatory agricultural research interventions, and our use of theory of change to collectively surface and revisit assumptions about how change happens. We share learning on the importance of being strengths-based in engaging stakeholders across scales and building a common goal as a starting point, and then staging a more critical practice as capacity is built and opportunities for digging deeper emerge.

Gendered voices for climate action - Fisher, Susannah; Shakya, Clare (iied Working Paper, 2018)
Including local women and men who have experienced the impacts of climate change in decision-making forums is key to developing a just process hearing local voices and experiences. The Paris Agreement signs up the international community to delivering countrydriven, gender-responsive climate action. Including local women and men will be critical to this transition and to achieving the scale and pace of change needed through both procedural and distributional justice. However, there is scarce evidence on how these voices and experiences can impact on and strengthen climate responses, or how best to enable and facilitate this participation.

Future oriented conservation: knowledge governance, uncertainty and learning - Wyborn, Carina; van Kerkhoff, Lorrae; Dunlop, Michael; Dudley, Nigel; Guevara, Oscar (Biodiversity and Conservation, 2016)
Despite signiﬁcant progress in understanding climate risks, adaptation efforts in biodiversity conservation remain limited. Adaptation requires addressing immediate conservation threats while also attending to long term, highly uncertain and potentially transformative future changes. To date, conservation research has focused more on projecting climate impacts and identifying possible strategies, rather than understanding how governance enables or constrains adaptation actions. We outline an approach to futureoriented conservation that combines the capacities to anticipate future ecological change; to understand the implications of that change for social, political and ecological values; and the ability to engage with the governance (and politics) of adaptation. Our approach builds on the adaptive management and governance literature, however we explicitly address the (often contested) rules, knowledge and values that enable or constrain adaptation. We call for a broader focus that extends beyond technical approaches to acknowledge the sociopolitical challenges inherent to adaptation. More importantly, we suggest that conservation policy makers and practitioners can use this approach to facilitate learning and adaptation in the context of complexity, transformational change and uncertainty.

Future Democracies - Laboratory of Collective Intelligence for Participatory Democracy - Yago Bermejo (MediaLab Prado, 2019)
The Laboratory of Collective Intelligence for Participatory Democracy (2016-2019) is a project that arose out of Medialab Prado in coordination with the Government Area of Citizen Participation, Transparency and Open Government of the city of Madrid. Its work has been very connected with the analysis, reflection and innovation found on the digital participation platform Decide Madrid. The project has also organised many workshops and conferences that have brought together hundreds of people from the world of participatory democracy. This publication summarises the project’s core working principles as well as the open activities it has hosted over the course of these three years. These pilot experiences offer a possible vision of the future of democratic governance.
Keywords: Democracy

From Thinking Politically To Working Politically: Are We Really Doing Development Any Differently? (2019)
In June it will be seven years since the Center for Global Development published the Problem Driven Iterative Adaptation (PDIA) paper. Few academic papers have had such an impact on development thinking. The question is whether the paper – and subsequent debate and experimentation - have had a demonstrable and beneficial impact on development practice. Experience to date suggests thinking politically is easier than working politically. Practitioners in development agencies, governments and contractors find that convincing theoretical literature can be tricky to operationalize in a fractious environment. Still, a growing number of programs have set out to be politically informed in design and politically savvy in implementation. Engaging the reality of politics creates the potential to promote change. To consider these issues, Abt Associates is hosting a workshop for international development practitioners in Washington, DC, on June 18th, 2019, to discuss the successes and failures in operationalizing this agenda. The session will allow practitioners to discuss experiences in TWP and propose solutions or approaches. Please join us in this Innovations in Governance series finale where four internationally renowned thinkers and doers will join us to offer their perspectives.

From Silos to Synergy: Learning from politically informed, gender aware programs - Derbyshire, Helen; Siow, Orlanda; Gibson, Sam; Hudson, David; Roche, Chris (DLP Program, 2018)

From purposeful to meaningful adaptive programming: how about adaptive operations? - Nixon, Nicola (Devpolicy Blog from the Development Policy Centre, 2019)
Embracing adaptive programming approaches but sticking with rigid, traditional systems-based operations? It's time to take a long hard look at all the boring stuff, says Nicola Nixon.

From principle to practice: Implementing the Principles for Digital Development - Waugaman, Adele (2016)
The Principles for Digital Development (download PDF here) find their roots in the efforts of individuals, development organizations, and donors alike who have called for a more concerted effort by donors and implementing partners to institutionalize lessons learned in the use of information and communication technologies (ICTs) in development projects.
Keywords: IMPORTANT

From poverty to power: how active citizens and effective states can change the world - Green, Duncan (2012)
From Poverty to Power From Poverty to Power argues that a radical redistribution of power, opportunities, and assets rather than traditional models of charitable or government aid is required to break the cycle of poverty and inequality. The forces driving this transformation are active citizens and effective states. Published in association with Oxfam GB. Full description

From political economy analysis to doing development differently: a learning experience - Booth, David; Harris, Daniel; Wild, Leni (ODI, 2016)
A study of how development efforts can be strengthened by an awareness of political economy, reflecting on the experiences of PoGo policy researchers.
Keywords: IMPORTANT

From ‘Feedback Loops’ to ‘Responsive Governance’ - McGee, Rosie (Making All Voices Count, 2014)
Part of a sequenced learning process for MAVC's Research and Evidence Component, the summary of a thematic discussion led by researcher Rosemary McGee.

From Design Thinking to Systems Change - Conway, Rowan; Masters, Jeff; Thorold, Jake (RSA, 2017)
This report takes the Small Business Research Initiative (SBRI), a twophase pre-procurement innovation programme that aims to match social challenges with new ideas, as its primary case study. It suggests augmenting the excellent design thinking deployed through SBRI with a think like a system, act like an entrepreneur lens in order to drive better social outcomes from SBRI-originating innovations. Programmes like SBRI have great potential to drive change and address pressing challenges, but must be guided by a more developed understanding of how change happens. The stakes are too high to not raise our game when it comes to social innovation. Wicked problems can be overcome but will require sophisticated theories of change able to account for the complexity and unpredictability of modern life. We offer think like a system, act like an entrepreneur as a contribution to this effort. This report is based on the following research: •• Desk based review of relevant literature; •• A series of structured interviews with participants in SBRI competitions; and •• An expert round table that brought together policymakers, commissioners, procurement experts and entrepreneurs to consider and develop initial findings.
Keywords: ⛔ No DOI found

From best practice to best fit: understanding and navigating wicked problems in international development - Ramalingam, Ben; Laric, Miguel; Primrose, John (Working Paper, 2014)

From analysis to action: operationalising learning and adaptation in Savings at the Frontier - Salehi, Yusef (2016)
Keywords: IMPORTANT

From a last mile century to a first mile revolution: the future is OSLO - Indy Johar (Medium, 2013)
The Future is OSLO

Framework for the Biodiversity Cross-Mission Learning Program - Environmental Incentives; Foundations of Success; ICF International (Measuring Impact, 2015)
USAID’s Bureau for Economic Growth, Education, and the Environment/Office of Forestry and Biodiversity (E3/FAB) launched the five-year Measuring Impact program (MI) to help strengthen USAID’s biodiversity programs by building the capacity of the Agency to design, monitor, and evaluate effective programs and by enhancing the evidence base that informs programming decisions. A core programmatic strategy of MI is to “Improve Biodiversity Conservation Approaches” with the life-of-project (LOP) objective that E3/FAB and the Missions engaged with MI have a greater understanding of conditions under which commonly deployed conservation strategic approaches are effective. To accomplish this, MI is working with E3/FAB to design and implement a Biodiversity Cross-Mission Learning Program (Learning Program) to systematically capture and share lessons on theories of change1 (TOCs) for common conservation strategic approaches in the USAID biodiversity portfolio.

Fostering Systems Change - Gopal, Srik; Kania, John (2015)
Five simple rules for foundations seeking to create lasting social change.

Fostering Learning in Large Programmes and Portfolios: Emerging Lessons from Climate Change and Sustainable Development - Harvey, Blane; Pasanen, Tiina; Pollard, Alison; Raybould, Julia (Sustainability, 2017)
In ﬁelds like climate and development, where the challenges being addressed can be described as “wicked”, learning is key to successful programming. Useful practical and theoretical work is being undertaken to better understand the role of reﬂexive learning in bringing together different knowledge to address complex problems like climate change. Through a review of practical cases and learning theories commonly used in the areas of resilience, climate change adaptation and environmental management, this article: (i) reviews the theories that have shaped approaches to reﬂexive learning in large, highly-distributed climate change and resilience-building programmes for development; and (ii) conducts a comparative learning review of key challenges and lessons emerging from early efforts to promote and integrate reﬂexive learning processes in programmes of this nature. Using a case study approach, the authors focus on early efforts made in four large, inter-related (or nested) programmes to establish, integrate and sustain learning processes and systems. Eight themes emerged from the review and are considered from the perspective of learning programmes as emergent communities of practice. By investigating how these themes play out in the nested programming, the paper contributes to the limited existing body of evidence on learning in large climate change programmes and identiﬁes areas where future efforts might focus.

Fostering Learning and Effectiveness through Collaboration: A New Collaborative on Transparency, Accountability, and Participation (TAI, 2018)
What are the features, values, and practices of effective learning organizations? How do learning practices contribute to more effective programming? And, how can collaborations between academics, researchers and practitioners better support learning organizations in the global South?  These are just a few of the questions that a new global learning collaborative seeks to explore. In …

Fostering Innovation and Entrepreneurialism in Public Sector Reform - Cummings, Clare (Public Administration and Development, 2015)
Summary There is growing recognition within the international development sector that there is a need for a new, more effective approach to engaging in public sector reform. This article builds on an emerging body of work that advocates more entrepreneurial and adaptive public sector reform programming. Drawing on knowledge and theory from public sector management, psychology and entrepreneurialism, this article aims to understand what motivates public sector workers to work entrepreneurially and suggest how these ideas can be applied to the way in which international development agencies engage in public sector reform work. This requires a shift in thinking from predesigned, large, externally led programmes promoting international best practice to interventions in which agencies adopt the role of a facilitator, helping to establish the enabling conditions for local partners to work entrepreneurially, developing their own solutions to the problems that they identify in their work. Copyright ? 2015 John Wiley & Sons, Ltd.
Keywords: Development; Entrepreneurship; Innovation; Public sector reform; motivation

Forward Look - A vision for the World Bank Group in 2030 - World Bank (World Bank, 2016)

Forget the money: De-linking technical assistance - Manuel, Clare (Synthesis Paper, 4, 2016)

Foresight and International Development* - Bingley, Kate (IDS Bulletin, 2016)
This article provides an overview of the use of foresight-type approaches and techniques in policy-related work in international development. It draws primarily on published and grey literatures, as well as select interviews with foresight practitioners. It begins with a brief introduction to the approaches and tools used in the field of strategic foresight, and then a broad mapping of the foresight landscape as relevant to international development. It provides reflections on the evidence of use and impact of foresight initiatives, and makes suggestions around future directions for foresight in international development.

Foreign policy and complex adaptive systems: Exploring new paradigms for analysis and action - Frej, William; Ramalingam, Ben (Santa Fe Institute, 2011)

Food security indicators - FAO (ESS Website, 2018)

Follow the rabbit: Field Guide to Systemic Design - CoLab (Government of Alberta, 2018)
This Field Guide is designed to support budding systemic designers to facilitate and lead systemic design projects. It’s designed for systemic designers who have basic familiarity with SD concepts and are looking for some practical tips and tools to put theory into action. While this Guide was designed with Government of Alberta staff in mind, the content applies across a range of issue areas, sectors, and intersections. The Guide goes through a systemic design project from concept to implementation. It takes you through the workshop planning process, and discusses workshop roles and client relations. In the FAQs, you’ll find explanations to some commonly asked questions about systemic design concepts to help you introduce others to SD and bring them along with you. You’ll get the most out of this field guide if you approach it with a systemic design mindset – open, willing to try things, a desire to learn, and a ‘yes, and’ orientation. Think of this field guide as an early stop in your systemic design journey: it’s not the end of your learning – it’s a jumping off point. In the Additional Resources section, you can connect to more in-depth explorations of concepts and practical applications.

FOCEVAL – Promoting Evaluation Capacities in Costa Rica:: Smart(er) Implementation with Capacity WORKS? - Storm, Sabrina (Transformation, Politics and Implementation, 2017)
The National Monitoring and Evaluation System of Costa Rica and its corresponding laws were established during the 1990s. Since then, the country has endeavored to implement monitoring and evaluation (M&E) activities as part of its public policy framework. Nevertheless, hardly any systematic evaluations had been conducted, and monitoring activities had been reduced mainly to the institutional self-reporting of implementation compliance. Persisting regional disparities and growing levels of inequality among the population raised the level of pressure on the government to present reliable information on the effectiveness of public interventions. Hence, results-oriented evaluations were promoted by some Costa Rican departments as

Five Ways to Make a Difference: Perceptions of Practitioners Working in Urban Neighborhoods - Durose, Catherine; van Hulst, Merlijn; Jeffares, Stephen; Escobar, Oliver; Agger, Annika; de Graaf, Laurens (Public Administration Review, 2015)

Five tips for evaluating your impact in international development - Pasanen, Tiina (The Guardian, 2015)
How can impact evaluations actually be helpful? These top tips will save development professionals time, energy and money. International development can be messy – with uncertain, complex settings, and multiple partners with different interests, goals and capacities. At the same time, we are under increasing pressure to demonstrate impact. We have to show that our projects have made a real change in people’s lives and that donor or taxpayer money hasn’t been wasted. But impact evaluations, which are often seen as a solution to this, aren’t always used. And in some cases they are unhelpful. So how can we improve the quality of impact evaluations, so that they produce results that are useful? This is where evaluability assessments come in. By asking whether we should evaluate a project – and if so, when and how – these assessments can improve the quality of impact evaluations. Here are five other reasons to assess evaluability before starting an impact evaluation.

FeverBee’s Online Community Strategy Template - FeverBee (FeverBee, 2018)

Feedback Commons (Keystone Accountability, 2018)

Feasibility Study: Crowdsourcing High-frequency food price data in rural Indonesia - UN Global Pulse (2015)

Fall in love with the solution, not the problem - Quaggiotto, Giulio; Leurs, Bas; Hazeldine, Shaun (Nesta, 2016)
Three strategies for international development organisations to solve problems without defining them.

Failing Forward: How CARE is focusing on what goes wrong to improve impact - Janoch, Emily (2019 CLA Case Competition, 2019)
CARE's Failing Forward initiative is sparking opportunities to showcase the ideas that don't work so we can spend more time implementing the ones that do. It's changing the conversation inside the organization, and leading to changes in the way we design and implement programs. It's also allowing us to make connections across a global portfolio of more than 900 projects, and with new and different partners to learn from their experience.

Failing Forward: How CARE is Focusing on What Goes Wrong to Improve Impact - Janoch, Emily (CLA Case Competition, 2019)
CARE's Failing Forward initiative is sparking opportunities to showcase the ideas that don't work so we can spend more time implementing the ones that do. It's changing the conversation inside the organization, and leading to changes in the way we design and implement programs.

Facilities deserve a place in development - Lacy, Jacqui de (Devpolicy Blog from the Development Policy Centre, 2017)
There are different ways to deliver aid programs, DFAT often uses facilities in its larger relationships, what are the rationales and risks behind this?

Facilitator's Guide to Participatory Decision-Making - Kaner, Sam (John Wiley & Sons, 2014)
Unleash the transformative power of face to face groups The third edition of this ground–breaking book continues to advance its mission to support groups to do their best thinking. It demonstrates that meetings can be much more than merely an occasion for solving a problem or creating a plan. Every well–facilitated meeting is also an opportunity to stretch and develop the perspectives of the individual members, thereby building the strength and capacity of the group as a whole. This fully updated edition of The Facilitator′s Guide to Participatory Decision–Making guides readers through the struggle and the satisfaction of putting participatory values into practice, helping them to fulfill the promise of effective group decision–making. With previous editions already embraced by business and community leaders and consulting professionals around the world, this new book is even more insightful and easy to use. New for this edition: 60 pages of brand new skills and tools Many new case examples Major expansion and reorganization of the advanced sections of the book. New chapter: Teaching A Group About Group Dynamics Doubled in size: Classic Facilitator Challenges. Substantially improved: Designing Realistic Agendas now three chapters, with wise, insightful answers to the most vexing questions about meeting design.

Facilitating collaborative problem solving with human-centred design: the Making All Voices Count governance programme in 12 countries of Africa and Asia - Jackson, Carl (Knowledge Management for Development Journal, 2015)
Keywords: IMPORTANT

Extreme Teaming: Lessons in Complex, Cross-Sector Leadership - Edmondson, Amy C.; Harvey, Jean-Francois (Emerald Group Publishing Limited, 2017)
Extreme Teaming Lessons in Complex, Cross-Sector Leadership Today’s global enterprises increasingly involve collaborative work by teams of experts operating across different professions, organizations, and industries. Extreme Teaming provides new insights into the world of complex, cross industry projects and the ways they must be managed. Leading experts Amy Edmondson and Jean-François Harvey analyze contemporary cases that expose the complex demands of cross-boundary collaboration on management, and inform our understanding of teams. Containing powerful insights and practical guidelines that allow managers to bridge professional divides and organizational boundaries in order to work together effectively, this is a new exploration of the challenges involved in today’s global enterprises. The authors demonstrate that the work done in the modern organization is less and less about looking inward and creating strong teams inside the company, and more about teaming across boundaries – that often are in flux. Extreme Teaming is a must-read book for all courses related to leading open innovation; teamwork and collaboration; project management; and cross-boundary work.

External validity and policy adaptation. From impact evaluation to policy design - Williams, Martin J. (BSG Working Paper Series, 2017)
With the growing number of rigorous impact evaluations worldwide, the question of how best to apply this evidence to policymaking processes has arguably become the main challenge for evidence-based policymaking. How can policymakers predict whether a policy will have the same impact in their context as it did elsewhere, and how should this influence the design and implementation of policy? This paper introduces a simple and flexible framework to address these questions of external validity and policy adaptation. I show that failures of external validity arise from an interaction between a policy’s theory of change and a dimension of the context in which it is being implemented, and develop a method of “mechanism mapping” that maps a policy’s theory of change against salient contextual assumptions to identify external validity problems and suggest appropriate policy adaptations. In deciding whether and how to adapt a policy in a new context, I show there is a fundamental informational trade-o↵ between the strength and relevance of evidence on the policy from other contexts and the policymaker’s knowledge of the local context. This trade-o↵ can guide policymakers’ judgments about whether policies should be copied exactly from elsewhere, adapted, or invented anew.

External Review of the Building Effective and Accessible Markets (BEAM) programme - Snedden, Ewan; Freer, Gordon; Moffatt, Michelle (WYG International, 2016)

Exploring ways to reconcile accountability and learning in the evaluation of niche experiments - Regeer, Barbara J.; de Wildt-Liesveld, Renée; van Mierlo, Barbara; Bunders, Joske F. G. (Evaluation, 2016)
While evaluation is seen as a mechanism for both accountability and learning, it is not self-evident that the evaluation of niche experiments focuses on both accountability and learning at the same time. Tensions exist between the accountability-oriented needs of funders and the learning needs of managers of niche experiments. This article explores the differences in needs and expectations of funders and managers in terms of upwards, downwards and internal accountability. The article shows that as the multi-stakeholder contexts in which niche experiments take place give rise to various requirements, tensions in evaluation are essentially a specific manifestation of tensions between niche experiments and their multiple contexts. Based on our findings, an adjusted accountability framework is proposed, including several strategies that can reconcile a learning approach with accountability needs in niche experiments aiming to change current practices in a more sustainable direction.

Exploring the prospects for adaptive governance in marine transboundary conservation in East Africa - Tuda, Arthur Omondi; Kark, Salit; Newton, Alice (Marine Policy, 2019)
This article explores the prospects for adaptive governance in a proposed marine transboundary conservation initiative in East Africa. Adaptive governance that involves interdependent state and non-state actors learning and taking action on joint environmental problems is suggested for effective transboundary resource governance. Using the concept of adaptive co-management, the current multi-stakeholder marine governance systems in southern Kenya and northern Tanzania are compared to illuminate opportunities and constraints for adaptive marine transboundary conservation governance between Kenya and Tanzania. The concept of networks and the formal method of social network analysis (SNA) are applied as the main methodological device. Using questionnaire and semi-structured interviews, social network data of 70 organizations (local resources users, government agencies and NGOs) was generated from Kenya (n = 33) and Tanzania (n = 37). Results show the existence of strong collaboration networks for marine resource governance in both Kenya and Tanzania. Social proximity is the common driver of network formation. Collaboration networks in Kenya and Tanzania have contributed to enhanced learning among marine resource managers. Conclusions point to the need to focus on common challenges relating to low levels of rule-compliance, limited access to information on the state of resources and poor integration of science into marine management decisions. Finally, differences in views regarding the state of marine ecosystems need to be addressed to improve prospects for joint problem-solving in marine transboundary conservation.
Keywords: Adaptive co-management; Adaptive governance; Collaboration; Marine transboundary conservation; Social network analysis

Exploring the potential for interactive radio to improve accountability and responsiveness to small-scale farmers in Tanzania - Gilberds, Heather; Handforth, Calum; Leclair, Mark (IDS, 2016)
What are the possibilities of using new digital technologies alongside radio to help ensure that agricultural development projects are farmer-centred, and meet the needs of the rural citizens they intend to serve? This research assesses Farm Radio International’s Listening Post – a model that combines radio and digital technologies with the aim of collecting and aggregating farmer feedback to aid decision-making and adaptive project implementation. The research shows that linking a mobile-based crowd-sourcing tool with radio is effective at ensuring engagement from a large number of farmers, who felt it was a useful way of raising their questions and concerns to NGOs, policy-makers and experts. The model has clear potential to strengthen the chain of relationships between citizens, extension services suppliers, projects and policymakers. It has also demonstrated its potential to collect real-time feedback from farmers that could be used to aid decision-making and improve accountability in agricultural development initiatives, helping to ensure they are more responsive to farmers. However, it also concludes that ‘closing the feedback loop’ – ensuring that farmer’s comments, questions and concerns are responded to – is a challenge for the Listening Post. Sometimes, the interactive radio programmes close the loop by disseminating answers to questions or concerns raised by farmers, or by connecting farmers to extension services – but only when a solid and systematic process had been developed for this to happen. Therefore is critical to identify and incentivise actors who are capable of responding during the design phase of a Listening Post, and to ensure that they are involved in every stage of the process. It also important that project partners who are interested in beneficiary feedback identify the flexible or actionable points in their project frameworks from the outset, rather than collecting data that they are not able to use to adapt their programmes

Exploring the new world: practical insights for funding, commissioning and managing in complexity - Lowe, Toby; Plimmer, Dawn (Collaborate & Newcastle University, 2019)
The world is complex. If we want to contribute to creating positive social outcomes, we must learn to embrace this complexity. This is the New World that funders and commissioners are discovering: • People are complex: everyone’s life is different, everyone’s strengths and needs are different. • The issues we care about are complex: issues – like homelessness – are tangled and interdependent. • The systems that respond to these issues are complex: the range of people and organisations involved in creating ‘outcomes’ in the world are beyond the management control of any person or organisation. Building on the findings from our previous report, A Whole New World, we have spent the last 12 months working with a growing movement of funders, public sector commissioners, and organisations working on the ground to begin to explore this New World: to find examples of practice that will help people to navigate it effectively. This report explores the key features of their response: working in a way that is human, prioritises learning and takes a systems approach.

Exploring new territories for evaluation - Provocative propositions - Sharp, Cathy (2018)

Exploring new territories for evaluation - Sharp, Cathy (Collective Leadership for Scotland, 2018)
Drawing on action research, this paper recasts evaluation as ‘action inquiry’, an embedded evaluative learning practice that can help navigate complexity when enacting collective leadership. It is offered as an invitation to inquiry amongst a reasonably well-informed audience of policy makers and practitioners who work in and for public services. It will particularly interest those who provide research, evaluation and facilitation support, and those seeking to develop a more relational approach to research and evaluation. Action inquiry is a model of practising change together in environments where ‘nothing is clear, and everything keeps changing’ that significantly challenges the prevailing discourse on evaluation. Action inquiry can be wrapped around and enmeshed within initiatives and programmes that work with complexity - anywhere where success will depend on the quality of relationships that can be developed. The paper reviews some important interrelated concepts that underpin the ideals of collective leadership and public service reform and which confront deeply embedded traditional notions of leadership, expertise and participation. These offer important challenges to ideas about how change happens and recognise that relationships are at the heart of practising change. This warrants a re-examination of the high expectations of evidence-based or informed practice. Collective leadership makes new demands of evidence as it rests on help to determine ‘wise actions’ in real-life situations. This confronts the practical reality of how to work together in conditions often expressed as ‘dynamic’ or ‘turbulent’ and the added human complexities of power, emotions and relationships; too often these elements are denied or avoided aspects of a change process. Facilitated action inquiry makes these elements part of the conversations, in the midst of ‘work-as-we-are-doing-it’, to increase areas of choice for individuals and a group as a whole. Within public policy, there is an overwhelming amount of evidence about ‘what works’ yet change seems to be stubborn and slow. The paper explores some of the deep-rooted vestiges of a ‘hierarchy of evidence’ and assumptions about standardisation and generalisability that act as a ‘barrier to transformation’. These include the narrow framing of what counts as evidence and consequent relegation of community perspectives, lived experience and practice-based evidence. The paper acknowledges the clear appetite for different approaches to evaluation, especially those that better reflect deeply held values and avoid creating a culture of ‘gaming’, rooted in fear of failure and loss of funding, at the expense of learning. The need for new forms of developmental evaluative thinking, collaborative inquiry and action research to create embedded learning is well overdue. Action inquiry is a desirable and necessary response to the complex situations and challenges of human services and recognises the essentialness of knowledge co-production. It is a model of co-creation at every stage and endorses the idea that people learn from participation in evaluation and by testing theories of change through action. Action inquiry builds on the idea of inquiry, or a moment-to-moment awareness and quality of attention and draws from several elements of action research practice. It sees inquiry as an intervention in itself, one that furthermore, explicitly seeks to enhance the probability of the success of a programme, focus on learning, the collaborative development of practice-based knowledge and positive relationships. The paper highlights the importance of building inquiry into living systems, the role of facilitation, systemic inquiry, and evaluative thinking. It proposes an expansion of ideas of appreciation as a relational and collaborative practice that is a driver of emergence. Social recognition that acknowledges someone’s social value to the community and implies mutual moral obligations to cooperation and participation is particularly crucial in a work context that requires successful coordination and multiple contributions to achieve results across hierarchies of position, professional rank and sectors. Hence, appreciation goes beyond the idea of positivity to include social recognition, valuing more explicit forms of inquiry, building participants’ aspirations to design new social systems and acting in new ways to embed change. In developing this discussion, the paper contributes to emerging dialogues about the need for a model of ‘5th generation evaluation’. Such a model would be based on the idea that appreciative and challenging inquiry that is contextual, relational and open-minded will create better opportunities for change and development. The paper sets out some ‘provocative propositions’ that can help us to navigate this terrain, perhaps of a fledgling ‘5th generation approach’ to inquiry. Facilitated action inquiry can hold the key to developing both new knowledge and an adaptive, collaborative and improvisational skill-set, able to respond in new ways to systemic and complex issues on the ground. It’s common to hear the expression ‘it’s all about relationships’ and it is clearly time to shift our focus to relationships; not relationships as ‘things’, but as co-created and dynamic relational processes in which we are embedded. In this way we can bring new qualities to our talking to each other about our various and shared visions of a better future.

Explaining positive deviance in public sector reforms in development - Andrews, Matt (World Institute for Development Economic Research (UNU-WIDER), 2013)
Public sector reforms are commonplace in developing countries. Much of the literature about these reforms reflects on their failures. This paper asks about the successes and investigates which of two competing theories best explain why some reforms exhibi
Keywords: Development; Innovation; Reform; case survey; governance; growth; leadership

Experiential Learning: Fostering lasting behavioural change - Norton, Brittney M. (Prospects practice paper, 4, 2016)
The term ‘experiential learning’ is often mistakenly used to describe any learning experience that involves participatory activities. This paper presents an overview of what true experiential learning is and how this form of learning allows for real-world application - thus promoting lasting behavioural change. Furthermore, it explains how experiential learning was adopted as the methodological core of the Prospects psychosocial programme in Liberia, which builds ‘pre-employment skills’ in 3,500 vulnerable youth aged 15-24 years. The goal of the paper is to provide other youth and education development practitioners with a resource that can be used to strengthen youth and behaviour change programmes through the incorporation of the experiential learning theory. In addition, this paper explains its application in the Prospects programme, before concluding with key considerations of such a design. 1

Evidence to Action: Research to Address Illegal Wildlife Trade - Milner-Gulland, E. J.; Cugniere, Laure; Hinsley, Amy; Phelps, Jacob; Rolfe, Michael 't Sas; Veríssimo, Diogo (2018)
Tools and expertise to improve the evidence base for national and international Illegal Wildlife Trade policy already exist but are underutilised. Tapping into these resources would produce substantive benefits for wildlife conservation and associated sectors, enabling governments to better meet their obligations under the Sustainable Development Goals and international biodiversity conventions. This can be achieved through enhanced funding support for inter-sectoral research collaborations, engaging researchers in priority setting and programme design, increasing developing country research capacity and engaging researchers and community voices in policy processes. This briefing, addressed to policy makers and practitioners, is part of the 2018 Evidence to Action: Research to Address Illegal Wildlife Trade event programme, organised by five of the UK’s most active IWT research institutions, to support the London 2018 IWT Conference.

Evidence Base for Collaborating, Learning, and Adapting: summary of the literature review (2017 update) - Learning Lab (USAID, 2017)
LEARN and USAID/PPL are managing an area of work known as Evidence Base for Collaborating, Learning, and Adapting (EB4CLA). The purpose of this work is to answer key learning questions:

Evidence Base for Collaborating, Learning, and Adapting: summary of the literature review (2016.08) - Learning Lab (USAID, 2016)
LEARN and USAID/PPL are managing an area of work known as Evidence Base for Collaborating, Learning, and Adapting (EB4CLA). The purpose of this work is to answer key learning questions:

Everyday Political Analysis - Hudson, David; Marquette, Heather; Waldock, Sam (Developmental Leadership Program, 2016)
This short note introduces a stripped-back political analysis framework designed to help frontline development practitioners make quick but politically-informed decisions. It aims to complement more in-depth political analysis by helping programming staff to develop the 'craft' of political thinking in a way that fits their everyday working practices. Everyday Political Analysis involves two steps: Understanding interests: What makes people tick? Understanding change: What space and capacity do people have to effect change? For each step five questions, accompanied by prompts, aim to help staff to conduct quick political analysis. The EPA framework can be used at any stage of the aid management cycle, and can help users to respond rapidly to unexpected change. We are keen to hear back from people on their experience of using EPA to help us adapt the framework. Was it useful (or not)? Do people tend to use just one or both steps? Are there missing statements or prompts that would improve the analysis? Please email us at info [at] dlprog.org.

Evaluation of the market systems development approach: Lessons for expanded use and adaptive management at Sida Volume II: Case studies - Ruffer, Tim; Bailey, Helen; Dahlgren, Stefan; Spaven, Patrick; Winters, Mark (Evaluation of the market systems development approach, 2018)
This report presents the findings of an evaluation of Sida’s management of the market systems development (MSD) approach. It aims to inform thinking on how Sida can best manage its growing portfolio of MSD programs. Beyond this, it provides insights relevant to Sida’s wider support to complex and adaptive programs. The evaluation identified several factors that affect Sida’s ability to ensure that conducive conditions are in place for effective MSD programs and good development programming more generally. Sida’s relatively flexible framework of rules, guidelines and systems for project management provide the space needed for staff to innovate and manage adaptively. But for this to happen consistently and effectively, Sida needs to invest more deliberately in building the capacity of its staff in relevant areas. In addition, leadership and incentives are key to shaping a culture of active experimentation and learning. This needs to be supported with clearer guidance for those involved in the design and appraisal of MSD projects; and strengthened oversight of project performance, including through adjustments to Sida’s contracts and funding agreements.

Evaluation of the market systems development approach: Lessons for expanded use and adaptive management at Sida Volume I: Evaluation Report - Ruffer, Tim; Bailey, Helen; Dahlgren, Stefan; Spaven, Patrick; Winters, Mark (Evaluation of the market systems development approach, 2018)
This report presents the findings of an evaluation of Sida’s management of the market systems development (MSD) approach. It aims to inform thinking on how Sida can best manage its growing portfolio of MSD programs. Beyond this, it provides insights relevant to Sida’s wider support to complex and adaptive programs. The evaluation identified several factors that affect Sida’s ability to ensure that conducive conditions are in place for effective MSD programs and good development programming more generally. Sida’s relatively flexible framework of rules, guidelines and systems for project management provide the space needed for staff to innovate and manage adaptively. But for this to happen consistently and effectively, Sida needs to invest more deliberately in building the capacity of its staff in relevant areas. In addition, leadership and incentives are key to shaping a culture of active experimentation and learning. This needs to be supported with clearer guidance for those involved in the design and appraisal of MSD projects; and strengthened oversight of project performance, including through adjustments to Sida’s contracts and funding agreements.

Evaluation of development programmes - OECD - OECD (2019)
Data & research on evaluation of development programmes inc. paris declaration, budget support, multilateral effectiveness, impact evaluation, joint evaluations, governance, aid for trade

Evaluation for equitable development results - Segone, M (UNICEF, 2011)
This document is made up of a range of Evaluation Working Papers (EWP) focused on evaluation for equitable development. Put together by evaluation specialists they present strategic evaluation findings, lessons learned and innovative approaches and methodologies. Part 1:Evaluation and equity Evaluation to accelerate progress towards equity, social justice and human rights 2 Human rights and gender equality in evaluation 13 When human rights is the starting point for evaluation 25 Strengthening Equity- focused evaluations through insights from feminist theory and approaches 39 Decolonizing evaluation in a developing world. Implications and cautions for Equity-focused evaluations 59 Part 2: Methodological implications for Equity-focused evaluations Methodological issues to design and implement equity-focused evaluations 86 Developmental evaluation for Equity-focused evaluations 102 Systems thinking and Equity-focused evaluations 115 Methodological challenges in using programme theory to evaluate pro-poor and equity-focused programmes by Patricia Rogers, Royal Melbourne Institute of Technology University and Richard Hummelbrunner, Independent consultant 142 Case Study and equity in Evaluationby Saville Kushner, University of the West of Englan 172 Values-Engaged Evaluationsby Jennifer Greene, University of Illinois at Urbana-Champaign 192 Part 3: Examples of Equity-focused evaluations Evaluating the contribution of UNDP to equity-focused public policies in Brazil and China 210 Using a human rights approach to evaluate ILO’s discrimination strategy 222 CONEVAL experience in evaluating interventions for Indigenous populations in Mexico 244 UNICEF supported evaluations with elements of equity-focused evaluations 258

Evaluation Building Blocks: A Guide - McKegg, K.; Oakden, J.; Wehipeihana, N.; King, Julian (Kinnect Group, 2018)
This guide presents the Kinnect Group’s approach to evaluation. Developed through ten years of collaborative work, it has been well tested. While there are many other guides to evaluation and different ways to do it, our approach emphasises the place of evaluative reasoning in evaluation. The guide draws on the work of many evaluation theorists, as well as our practice-based body of knowledge. We hope that using the building blocks as explained in the guide will help you in your endeavour to do credible and useful evaluation. We find this approach works for us. Our clients tell us it gives them meaningful and insightful findings that they can use to take action. Our view to intellectual property is simple. What you read is yours to use. We just ask that you cite the guide when you draw from it. For those of you who engage with this guide and use it in your practice, we look forward to hearing from you. We are particularly interested in feedback where you find aspects of the guide work well for you. In that way we can all continue to build and evolve our practice.

Evaluation at IDRC - IDRC (IDRC, 2017)

Evaluation and learning in complex, rapidly changing health systems: China’s management of health sector reform - Xiao, Yue; Husain, Lewis; Bloom, Gerald (Globalization and Health, 2018)
Healthcare systems are increasingly recognised as complex, in which a range of non-linear and emergent behaviours occur. China’s healthcare system is no exception. The hugeness of China, and the variation in conditions in different jurisdictions present very substantial challenges to reformers, and militate against adopting one-size-fits-all policy solutions. As a consequence, approaches to change management in China have frequently emphasised the importance of sub-national experimentation, innovation, and learning. Multiple mechanisms exist within the government structure to allow and encourage flexible implementation of policies, and tailoring of reforms to context. These limit the risk of large-scale policy failures and play a role in exploring new reform directions and potentially systemically-useful practices. They have helped in managing the huge transition that China has undergone from the 1970s onwards. China has historically made use of a number of mechanisms to encourage learning from innovative and emergent policy practices. Policy evaluation is increasingly becoming a tool used to probe emergent practices and inform iterative policy making/refining. This paper examines the case of a central policy research institute whose mandate includes evaluating reforms and providing feedback to the health ministry. Evaluation approaches being used are evolving as Chinese research agencies become increasingly professionalised, and in response to the increasing complexity of reforms. The paper argues that learning from widespread innovation and experimentation is challenging, but necessary for stewardship of large, and rapidly-changing systems.

Evaluating with the Success Case Method - Gram, Tom (Performance X Design, 2011)
The method was developed by Robert Brinkerhoff as an alternative (or supplement) to the Kirkpatrick approach and its derivatives. It is very simple and fast (which is part of it’s appeal) and goes something like this: Step 1. Identify targeted business goals and impact expectations Step 2. Survey a large representative sample of all participants in a program to identify high impact and low impact cases Step 3. Analyze the survey data to identify: a small group of successful participants a small group unsuccessful participants Step 4. Conduct in-depth interviews with the two selected groups to: document the nature and business value of their application of learning identify the performance factors that supported learning application and obstacles that prevented it. Step 5. Document and disseminate the story report impact applaud successes use data to educate managers and organization The process produces two key outputs In-depth stories of documented business effect that can be disseminated to a variety of audiences Knowledge of factors that enhance or impede the effect of training on business results. Factors that are associated with successful application of new skills are compared and contrasted with those that impede training. It answers practical and common questions we have about training and other initiatives: What is really happening? Who’s using what, and how well? Who’s not using things as planned? What’s getting used, and what isn’t? Which people and how many are having success? Which people and how many are not? What results are being achieved? What value, if any, is being realized? What goals are being met? What goals are not? Is the intervention delivering the promised and hoped for results? What unintended results are happening? What is the value of the results? What sort of dollar or other value can be placed on the results? Does the program appear to be worthwhile? Is it producing results worth more than its costs? What is its return on investment? How much more value could it produce if it were working better? How can it be improved? What’s helping? What’s getting in the way? What could be done to get more people to use it? How can everyone be more like those few who are most successful?

Evaluating the impact of flexible development interventions using a ‘loose’ theory of change: Reflections on the Australia-Mekong NGO Engagement Platform - Davies, Rick (Methods Lab, 2016)
• For some interventions, tight and testable theories of change are not appropriate – for example, in fast moving humanitarian emergencies or participatory development programmes, a more flexible approach is needed. • However, it is still possible to have a flexible project design and to draw conclusions about causal attribution. This middle path involves ‘loose’ theories of change, where activities and outcomes may be known, but the likely causal links between them are not yet clear. • In this approach, data is collected ‘after the event’ and analysed across and within cases, developing testable models for ‘what works’. More data will likely be needed than for projects with a ‘tight’ theory of change, as there is a wider range of relationships between interventions and outcomes to analyse. The theory of change plays an important role in guiding the selection of data types. • While loose theories of change are useful to identify long term impacts, this approach can also support short cycle learning about the effectiveness of specific activities being implemented within a project’s lifespan.

Evaluating the Design of Behavior Change Interventions: A Case Study of Rhino Horn in Vietnam - Olmedo, Alegría; Sharif, Vian; Milner‐Gulland, E. J. (Conservation Letters, 2018)
Behavioral change interventions are increasingly widely used in conservation. Several projects addressing rhino horn consumption were recently launched in Vietnam. We used key informant interviews, document analysis, and marketing theory to explore their strategies for intervention design, implementation, monitoring, and evaluation. We developed a framework to evaluate whether they followed best practice and identify implementation challenges. Interventions could make greater use of key project design steps, including basing interventions on robust research to understand the behavior in question, identifying the target audience whose behavior interventions aim to change, and developing measures that can provide reliable evidence of success or not. Challenges include the need for law enforcement to complement campaigns; improving cooperation between NGOs; and clearly defining aims of demand-reduction initiatives. Using best practice from other fields and considering demand reduction within the wider context of wildlife, trade policy will help address these challenges.
Keywords: Awareness; Ilegal wildlife trade; Project design; demand reduction; social marketing

Evaluating Systems Change Results: an inquiry framework - Cabaj, Mark (Tamarack Institute, 2018)

Evaluating megaprojects: From the ‘iron triangle’ to network mapping - Lehtonen, Markku (Evaluation, 2014)
Evaluation literature has paid relatively little attention to the specific needs of evaluating large, complex industrial and infrastructure projects, often called ‘megaprojects’. The abundant megaproject governance literature, in turn, has largely focused on the so-called ‘megaproject pathologies’, i.e. the chronic budget overruns, and failure of such projects to keep to timetables and deliver the expected social and economic benefits. This article draws on these two strands of literature, identifies shortcomings, and suggests potential pathways towards an improved evaluation of megaprojects. To counterbalance the current overemphasis on relatively narrowly defined accountability as the main function of megaproject evaluation, and the narrow definition of project success in megaproject evaluation, the article argues that conceptualizing megaprojects as dynamic and evolving networks would provide a useful basis for the design of an evaluation approach better able to promote learning and to address the socio-economic aspects of megaprojects. A modified version of ‘network mapping’ is suggested as a possible framework for megaproject evaluation, with the exploration of the multiple accountability relationships as a central evaluation task, designed to reconcile learning and accountability as the central evaluation functions. The article highlights the role of evaluation as an ‘emergent’ property of spontaneous megaproject ‘governing’, and explores the challenges that this poses to the role of the evaluator.

Evaluating interventions that prevent or counter violent extremism: A practical guide - Hofman, Joanna; Sutherland, Alex (RAND Corporation, 2018)

Evaluating Humanitarian Innovation - Obrecht, Alice; Warner, Alexandra; Dillon, Neil (HIF-ALNAP working paper, 2017)

Evaluating development projects: exploring a synthesis model of the logical framework approach and outcome mapping - Yang, Ting (2018)
Under the current results-driven development agenda, sound evaluation, and a corresponding evaluation toolkit, need to be in place to examine whether and to what extent development interventions have achieved their targeted objectives and results, and to generate lessons for further development learning and improvement. My review of the literature shows that innovative and appropriate evaluation approaches are needed to address key challenges in evaluation such as the tension between learning and accountability objectives, the need to unpack the mechanisms linking outputs and outcomes or goal, and to add an actor perspective. Irrespective of project type, the Logical Framework Approach (LFA) is often a standard requirement of major official donor agencies on projects they fund, so as to fulfil bureaucratic imperatives. However, it is often considered inadequate in addressing key challenges in development evaluation. Given the dominant status of the LFA with such strong support from donors, it is helpful to seek a ‘middle way’: a combination of the LFA with other approaches in order to address some of its inadequacies, while satisfying donor agencies’ requirements. A synthesis of the LFA and Outcome Mapping (OM) is one such option. This thesis explores the practical value and usefulness of a synthesis model empirically. Applying the model in two case study aid projects, I found that it serves well as a theory-based evaluation tool with a double-stranded (actor strand and results chain) theory of change. The model helps reconcile learning and accountability and add explanatory power and an explicit actor perspective. It also helps establish causation and enable attribution claims at various results levels with its different elements. The model has some limitations but my results suggest it can be usefully adopted. The choice of its application depends on project evaluation context and purpose in specific cases.

Evaluating Countering Violent Extremism: Practice and Progress - Fink, Naureen Chowdhury; Romaniuk, Peter; Barakat, Rafia (Center on Global Counterterrorism Cooperation, 2013)

Evaluating Complexity. Propositions for improving practice - Preskill, Hallie; Gopal, Srik (FSG, 2014)
9 propositions can help evaluators measure progress on complex social problems.
Keywords: IMPORTANT

Evaluating complex interventions: A theory-driven realist-informed approach - Douthwaite, Boru; Mayne, John; McDougall, Cynthia; Paz-Ybarnegaray, Rodrigo (Evaluation, 2017)
There is a growing recognition that programs that seek to change people’s lives are intervening in complex systems, which puts a particular set of requirements on program monitoring and evaluation. Developing complexity-aware program monitoring and evaluation systems within existing organizations is difficult because they challenge traditional orthodoxy. Little has been written about the practical experience of doing so. This article describes the development of a complexity-aware evaluation approach in the CGIAR Research Program on Aquatic Agricultural Systems. We outline the design and methods used including trend lines, panel data, after action reviews, building and testing theories of change, outcome evidencing and realist synthesis. We identify and describe a set of design principles for developing complexity-aware program monitoring and evaluation. Finally, we discuss important lessons and recommendations for other programs facing similar challenges. These include developing evaluation designs that meet both learning and accountability requirements; making evaluation a part of a program’s overall approach to achieving impact; and, ensuring evaluation cumulatively builds useful theory as to how different types of program trigger change in different contexts.

Evaluating Community Impact: Capturing and Making Sense of Community Outcomes - Tamarack Institute (2017)

Evaluability assessment for Impact Evaluation: Guidance, checklists and decision support - Peersman, Greet; Guijt, Irene; Pasanen, Tiina (ODI, 2015)
This guidance note focuses on the utility of, and guidance for, evaluability assessment before undertaking an impact evaluation. The primary audience for this guidance note is evaluators conducting an evaluability assessment for impact evaluation. The secondary audience is people commissioning or managing an evaluability assessment for impact evaluation, as well as funders of an impact evaluation. Sections one and two provide an overview of evaluability assessment and how it can be used for impact evaluation. Section three provides guidance for planning to undertake an evaluability assessment for impact evaluation. This is informative for all intended users of the guidance note. Section four includes checklists and decision support for evaluability assessments. The checklist is geared to those conducting the evaluability assessment and can be adapted to suit a particular context or purpose. The decision support provides those conducting an evaluability assessment with evidence-based recommendations for impact evaluation funders and commissioners, about whether, when and how to proceed with the evaluation. Sections five and six provide guidance on what to do after the assessment is concluded, and offer lessons learned from evaluability assessments in practice.

EvalC3 - Davies, Rick (2019)
Tools for developing, exploring and evaluating predictive models of expected outcomes

Essentials of Utilization-Focused Evaluation - Patton, Michael Quinn (SAGE Publications, 2011)
Based on Michael Quinn Patton's best-selling Utilization-Focused Evaluation, this briefer book provides an overall framework and essential checklist steps for designing and conducting evaluations that actually get used. The new material and innovative graphics present the utilization-focused evaluation process as a complex adaptive system, incorporating current understandings about systems thinking and complexity concepts. The book integrates theory and practice, is based on both research and professional experience, and offers new case examples and cartoons with Patton's signature humor.
Keywords: Reference / Research; Research

Escaping Capability Traps through Problem-Driven Iterative Adaptation (PDIA) - Andrews, Matt; Pritchett, Lant; Woolcock, Michael (HKS Working Paper, 2012)
Keywords: IMPORTANT

EPE Week: Lee-Anne Molony on Collaborative Outcomes Reporting - Molony, L A (AEA365 a Tip-a-day by and for Evaluators, 2013)
This AEA365 blog, written by Lee-Anne Molony a Principal Consultant at Clear Horizon, provides a brief overview on Collaborative Outcomes Reporting (COR).

Engines of Change: What Civic Tech can learn from Social Movements - Purpose; Omidyar Network (Omidyar Network and Purpose, 2016)

Engineering Complex Scaled Up Innovations in the Humanitarian and Development Sector - McClure, Dan; Gray, Ian (ThoughtWorks, 2015)

Engaging Stakeholders in Planning for Sea Level Rise and Resilience - Yusuf, Juita-Elena (Wie); John, Burton St; Covi, Michelle; Nicula, J. Gail (Journal of Contemporary Water Research & Education, 2018)
This case study describes a region-wide, multi-sectoral, and whole-of-community stakeholder engagement approach for addressing sea level rise (SLR) and flooding. This approach was implemented through a university-led community engagement event, the Hampton Roads Resilient Region Reality Check (H4RC), which allowed an examination of its effectiveness as a mechanism for capturing community-wide perceptions regarding SLR, flooding, and associated risks; engaging stakeholders in discussion within and across different groups; and assessing community willingness to address flooding and SLR. The results show that the event helped participants broaden their perspectives and understanding of flooding and SLR. In an approach that called for participants to engage in social learning across social networks, the event had some effect on individual efficacy. However, there was little impact on participants' already-established perception that the region does not possess significant willingness to take action.
Keywords: Social learning; flooding; resilience; social capital; stakeholder engagement

Engaging Farmers in Climate Change Adaptation Planning: Assessing Intercropping as a Means to Support Farm Adaptive Capacity - Himanen, Sari J.; Mäkinen, Hanna; Rimhanen, Karoliina; Savikko, Riitta (Agriculture, 2016)
Agriculture is one of the most vulnerable and adaptation-prone sources of livelihood facing climate change. Joint adaptation planning by farmers and researchers can help develop practically feasible and environmentally and economically sound adaptation actions as well as encourage the proactive building of farm adaptive capacity. Here, the perceptions of Finnish farmers and rural stakeholders regarding intercropping, the cultivation of two or more crop genotypes together in time and space, as a means to prepare for climate change, were collected in an open workshop. Our aim was to identify the potentials and challenges associated with intercropping, its role as an adaptation strategy, and in farm adaptive capacity. Qualitative analysis revealed better yield security, increased nutrient and protein self-sufficiency, soil conservation and maintenance, reduced pathogen pressure and regulation of water dynamics as the main perceived potentials of intercropping. Potentials relating to the farm economy and environment were also recognized. The main challenges associated with intercropping were related to the lack of information on crop variety performance and optimal yielding in mixtures, industry and policy requirements for seed purity, more complicated crop management and harvesting, and the economic risks associated with experimenting with novel mixtures. Nitrogen-fixing legumes; deep-rooted species, such as lucerne (Medicago sativa L.); special crops, such as herbs in forage mixtures; and autumn-sown winter oilseeds and cereals were highlighted as the most promising intercrops. Because the recognized potentials relate to the safeguarding of field cropping from anticipated climate change and the associated weather variability, we conclude that intercropping can serve as one adaptation strategy to strengthen the adaptive capacity of Finnish farms. However, assuring markets and policies that allow the development of intercropping, performing experiments to assess the benefits and implement options in practice, and providing farmers and farm advisors with more knowledge on the method represent the critical prerequisites for the broader adoption of intercropping.
Keywords: Adaptation planning; Adaptive capacity; Climate change; ecological intensification; intercropping; yield security

Enabling collaborative synthesis in multi-partner programmes - Cochrane, Logan; Cundill, Georgina (Development in Practice, 2018)
Multi-partner consortia have emerged as an important modality for knowledge generation to address complex sustainability challenges. Establishing eﬀective multi-partner consortia involves signiﬁcant investment. This article shares lessons from the Collaborative Adaptation Research Initiative in Africa and Asia (CARIAA), which aims to support policy and practice for climate change adaptation through a consortium model. Key lessons include the need to facilitate collaborative spaces to build trust and identify common interests, while accepting that this is not a guarantee of success; the importance of programmatic leadership to achieve synthesis; and the value of strategic planning in supporting motivation and alignment between partners.

Enabling collaboration and synthesis: in brief - Cochrane, Logan; Cundill Kemp, Georgina; Landry, Marie-Eve; Lee, Rebecca (INBRIEF, 2017)
Collaborative Adaptation Research Initiative in Africa and Asia (CARIAA) aims to build the resilience of vulnerable populations in climate change hotspots by building new knowledge and capacities to support better informed policy and practice. The program connects more than 450 researchers and practitioners from over 40 organizations. Initially, CARIAA was not directive in terms of the specific topics or forms of synthesis. This approach allowed new ideas to emerge from interactions between members of the network. However, as the brief outlines, lessons learned at mid-term regarding cross- consortia collaboration and synthesis indicate that greater programmatic leadership is required.

Enabling citizen-driven improvement of public services. Leveraging Technology to Strengthen Accountability in Nigerian Healthcare - ICT4SA (World Bank and Reboot, 2015)
Effective public service delivery begins with knowing whether the services offered are working as intended. We launched a system to enable citizen input on the delivery of public services.

Empowerment evaluation - Fettermen, David (2014)

Empowerment and Accountability in difficult settings: What are we learning? - Gaventa, John; Oswald, Katy (Key messages emerging from theAction for Empowerment and Accountability Programme, 2019)
Empowerment and Accountability in Difficult Settings: What Are We Learning? Key Messages Emerging from the Action for Empowerment and Accountability Programme Empowerment and accountability have long been part of the international development vocabulary and a core part of governance, social development and civil society programmes. Yet, much of what has been learnt about these approaches has been drawn from studies in somewhat stable, open and middle-income places around the world. Less is known about how empowerment and accountability are achieved through social and political action in more difficult settings – those faced by institutional fragility, conflict, violence, and closing civic space. This document highlights key messages emerging from the work of the Action for Empowerment and Accountability Research Programme (A4EA), and the implications for how donors, policy makers and practitioners support strategies for empowerment and accountability in fragile, conflict and violence affected settings (FCVAS). Our eight key messages have strong implications for the theories of change used for effective programming in the field.

Emerging Technologies: Smarter ways to fight wildlife crime - Cressa, Douglas; Zommers, Zinta (Environmental Development, 2014)
The illegal trade of animals—for luxury goods, traditional medicine or cultural ceremonies, pets, entertainment, and even research—is a major threat to wildlife conservation and welfare (Baker et al., 2013). Poachers and illegal traders use highly sophisticated and rapidly changing techniques to avoid detection. To keep pace with the "war on wildlife", conservation and law enforcement communities have started to adopt cutting-edge military tools and techniques. High-tech equipment can magnify counter-poaching efforts without requiring armies of rangers or risking lives. Tools include acoustic traps, mobile technology, mikrokopters, radio frequency identification tags, encrypted data digital networks, camera traps, DNA testing, radio collars, metal scanners, and satellite imagery.

Emerging Opportunities: Monitoring and Evaluation in a Tech-Enabled World - Raftree, Linda; Bamberger, Michael (Discussion Paper, 2014)
Monitoring and evaluation practice is using information and communication technologies for more timely data, and more inclusive voice and feedback.

Embracing the Chaotic: Cynefin and Humanitarian Response - Morealtitude (WanderLust, 2010)
Note: Many thanks to Dave Snowden for his willingness to comment on this article prior to release Because I’m always a little behind the eight-ball, a friend of mine* recently introduced me to the …

Embracing Emergence: How Collective Impact Addresses Complexity - Kania, John; Kramer, Mark (Stanford Social Innovation Review, 2013)
Collective impact is upending conventional wisdom on how we achieve social progress.

Embracing Complexity: Strategic Perspectives for an Age of Turbulence - Boulton, Jean G.; Allen, Peter M.; Bowman, Cliff (Oxford University Press, 2015)
The book describes what it means to say the world is complex and explores what that means for managers, policy makers and individuals. The first part of the book is about the theory and ideas of complexity. This is explained in a way that is thorough but not mathematical. It compares differing approaches, and also provides a historical perspective, showing how such thinking has been around since the beginning of civilisation. It emphasises the difference between a complexity worldview and the dominant mechanical worldview that underpins much of current management practice. It defines the complexity worldview as recognising the world is interconnected, shaped by history and the particularities of context. The comparison of the differing approaches to modelling complexity is unique in its depth and accessibility. The second part of the book uses this lens of complexity to explore issues in the fields of management, strategy, economics, and international development. It also explores how to facilitate others to recognise the implications of adopting a complex rather than a mechanical worldview and suggests methods of research to explore systemic, path-dependent emergent aspects of situations. The authors of this book span both science and management, academia and practice, thus the explanations of science are authoritative and yet the examples of changing how you live and work in the world are real and accessible. The aim of the book is to bring alive what complexity is all about and to illustrate the importance of loosening the grip of a modernist worldview with its hope for prediction, certainty and control.

Eight Practices for Strategic Agility - Algoso, Dave (Stanford Social Innovation Review, 2019)
Rather than a glossy brochure that no one reads, your strategy should be an ongoing practice that informs your decisions and adapts as circumstances change. A Viewpoint from the Summer 2019 issue.

Effective Governance. Strategy for Australia’s aid investments - DFAT (Department of Foreign Affairs and Trade, Australian Government, 2015)

edX MOOC: Adaptive Leadership in Development - edX (edX, 2016)
Gain the skills to be an adaptive leader in development and learn how to uncover local solutions to complex problems in developing countries.

Dynamic gridlock: Adaptive Humanitarian Action in the Democratic Republic of Congo - Obrecht, Alice (ALNAP Country Study, 2018)

DRG Center practical tips to implement more context-adaptive programming - Jacobstein, David (#AdaptDev, 2018)
Hi #adaptdevers, The cross-sectoral programs team in the DRG Center have been working in support of ways that programming can be made more aware of, and responsive to, context considerations that have to do with incentives and relationships between different actors - often called power dynamics or political economy. Following several discussions with field folks, we were asked to come up with simple and practical tips that could help staff to implement more context-adaptive programming, or to better Work Politically to use TWP jargon. With assistance from many of our field staff and partners, we've put together three documents (to start) that have now been cleared to share publicly: • Tips on Making Your Solicitations Invite Context-Driven Adaptive Programming - suggestions spanning most sections of a typical solicitation to better weave in Thinking and Working Politically. I'm particularly excited that this document contains embedded links to excerpts from a set of solicitations that model good practice. You can also find a summary of all the excerpts from real solicitations, and links onward to the public posting of those solicitations on Fed Biz Ops or Grants.gov, here. • Tips on Power Dynamics and Theories of Change - suggestions relevant to those working on project designs and logic models. • Tips for Better Use of Advisory Councils - suggestions to set up better advisory bodies that help you to incorporate political economy more automatically into your programmatic adjustments. I hope that you'll find these practical and helpful! Feedback is always welcome. Best, David

(Draft) Managing to Adapt - Analysing Adaptive Management for Planning, Monitoring, Evaluation, and Learning (draft) - Schlingheider, Annika; Pellfolk, Erica; Maneo, Gabriele; Desai, Harsh (Oxfam Research Reports, 2017)
Adaptive management is at the heart of ‘doing development differently’ (Wild et al., 2016). Whether it is here to stay depends on how much it is mainstreamed into existing development programming by donors and implementers alike, especially in planning, monitoring, evaluation, and learning (PMEL) cycles. In this report, we find that mainstreaming adaptive management in PMEL involves three strategies: 1. planning for flexibility; 2. developing locally owned monitoring and evaluation (M&E); and 3. creating an enabling environment for learning. Adopting these strategies contributes to virtuous cycles of PMEL

Donor perceptions and processes: Risks to success of adaptive programming? - Browse - Knowledge Platform Security & Rule of Law - Donovan, Elbereth (Knowledge Platform - Security & Rule of Law, 2017)
Keywords: IMPORTANT

Donor Committee for Enterprise Development (DCED, 2018)
The DCED is a forum for learning about the most effective ways to fulfil the SDGs – based on practical experience in Private Sector Development (PSD).

Doing Urban Development Differently - Jones, Harry; Adhikari, Bishnu (From Poverty to Power, 2017)
Guest post from Harry Jones and Bishnu Adhikari, both of Palladium on what urban aid and development can learn from the Doing Development Differently movement

Doing things differently: Rethinking monitoring and evaluation to understand change - Saferworld (Saferworld, 2016)
Doing things differently: Rethinking monitoring and evaluation to understand change Learning paper Over the past four years, Saferworld has put in place a way of monitoring, evaluating and learning from our work focused on behaviour and relationship change. This paper outlines the process we have gone through to adapt, embed, and embrace an approach inspired by Outcome Mapping and Outcome Harvesting (OH). Key benefits of our monitoring, evaluation and learning approach are: It is simple, but promotes complex discussion and analysis. It allows conflict- and gender-sensitivity to be built into programmes; it promotes analysis of and adaptation to context. Bringing front-line staff and partners into wider conversations with others substantially increases cross-organisational learning.

Doing the Transparent State: open government data as performance indicators - Ruppert, Evelyn; others (A World of Indicators: The production of knowledge and justice in an interconnected world, 2013)

Doing Problem Driven Work - Andrews, Matt; Pritchett, Lant; Woolcock, Michael (HKS Working Paper, 2015)
Keywords: IMPORTANT

Doing legal empowerment differently: Learning from pro-poor litigation in Bangladesh - O'Neil, Tam; Valters, Craig; Farid, Cynthia (ODI, 2015)
Marginalised groups and their allies can, and do, use the law and justice systems, including public interest litigation, to improve their access to rights, goods and services.  Yet there is no automatic link between legal action and improved outcomes for poor people.  Where some minimum conditions are met – a progressive legal framework, a sympathetic judiciary and legal advocacy organisation – pro-poor litigation is a potential tool in disputes over rights and resources.  But concrete benefits for poor people also requires state action to enforce progressive rulings.  Much depends on whether claimants, legal activists and state reformers cooperate around shared interests, and whether activists can negotiate power and interest structures to motivate government action to implement rulings.  Effective strategies are those that link litigation with grassroots legal action and other forms of political and social activism and advocacy.  Donors need to fund legal advocacy organisations in ways that enable them to select social issues that are locally relevant and political feasible – and allows activists and reformers to work in politically smart and adaptive ways.

Doing Iterative and Adaptive Work - Andrews, Matt; Pritchett, Lant; Woolcock, Michael (CID Working Paper, 2016)
Many of the challenges in international development are complex in nature. They involve many actors in uncertain contexts and with unclear solutions. Our work has proposed an approach to addressing such challenges, called Problem Driven Iterative Adaptation (PDIA). This paper is the most recent in a series intended to show how one can do PDIA, building on the first paper, "Doing Problem Driven Work.” The current paper addresses a key part of the approach one moves to once a problem has been identified, performing real-time experimental iterations. This is intended as a practical paper that builds on experience and embeds exercises for readers who are actually involved in this kind of work.
Keywords: IMPORTANT

Doing Digital Development Differently: Lessons in adaptive management from technology for governance initiatives in Kenya - Prieto Martin, Pedro; Hernandez, Kevin; Faith, Becky; Ramalingam, Ben (MAVC Research Report, 2017)
Development projects don’t always work as planned. This has long been acknowledged by those in the sector, and has led to several approaches that seek to solve complex development problems through enabling and encouraging greater adaptiveness and learning within projects (e.g. Doing Development Differently and Problem-Driven Iterative Adaptation). Digital development projects experience many of these issues. Using technology for transparency and accountability (Tech4T&A) projects in Kenya as case studies, our research analysed the many different theoretical approaches to learning and adaptation, and then tested how these play out in reality. Firstly, we conducted an extensive review of the literature on the spectrum of approaches to adaptive learning. The findings were used to develop a framework through which to analyse adaptiveness at the different layers of complexity in projects (e.g. software design and development, programme design and management). The second part of the research consisted of interviews and focus group discussions with participants in Tech4T&A projects in Kenya. Respondents helped us identify the main characteristics of adaptiveness in their projects (e.g. who needs to adapt, and how and when) and the challenges and issues that inhibit projects’ abilities to be adaptive. This process also revealed how accountability interplays with adaptiveness, and considered how better collaboration flows can enable adaptiveness. From our literature review and empirical study, we draw several conclusions for increasing adaptiveness in digital development projects. These include simplifying the many adaptive theories that abound, increasing responsiveness to project beneficiaries and stakeholders, and for stakeholders to keep on experimenting, networking and advocating.
Keywords: IMPORTANT

Doing Development Differently: Who we are, What we’re doing, What we’re learning - ODI (ODI, 2016)
In November 2014, the doing development differently community got together in Harvard to discuss what successful development interventions look like. Two years on, our community is broader than aid. It's broader than donors. It's about all organisations delivering change, producing real solutions to real problems that have real impact. It's about building trust, empowering people and promoting sustainability. Over the past two years, the community has been putting these ideas into practice across the world – being honest about what we are learning, including where we are not getting things right. This document aims to be an entry point for anyone interested in doing development differently. It explores the things that could be stopping you including: You are constrained by a disabling environment. You have strict reporting requirements. People aren't sold on the idea. It's easier to do things the way you've always done it. It's difficult to do in practice. If you are facing one, several, or all of these constraints, this document gives you a place to start. Using 43 case studies from practitoners within donors, governments, implementing organisations and NGOs across the globe, we aim to draw out some key lessons learnt, and give some advice for people considering this approach. These case studies are split across five categories: Swimming against the tide. Working in and with government. Feedback loops and data. Organisational change. Diffusion. Finally, if you want to know more, the document concludes with a suggested list of blogs and reports which explain these concepts in more detail.
Keywords: IMPORTANT

Doing Development Differently: Understanding the Landscape and Implications of New Approaches to Governance and Public-sector Reforms - Fritz, Verena (Transformation, Politics and Implementation, 2017)
Seeking to accelerate development, the agencies and individuals involved have regularly advanced new ideas of how external support can function better, deliver more, and achieve greater impact. There has been a particular flourishing of new ideas within the broad field of governance and public-sector reforms in the 2000s. This chapter starts off with a review of the “landscape of new ideas,” focusing on five proposed approaches in particular: political economy analysis (PEA), Problem-Driven Iterative Adaptation (PDIA), Doing Development Differently (DDD), Thinking and Working Politically (TWP), and the “science of delivery.” It sets out the “problem-diagnostic” that underpins each of these

Doing Development Differently in the Global South - Workshop Report - CARE (CARE and British Council, 2018)

Doing Development Differently at the World Bank: updating the plumbing to fit the architecture - Bain, Katherine A.; Booth, David; Wild, Leni (ODI, 2016)
Keywords: IMPORTANT

Doing Development Differently at Scale - Pellini, Arnaldo; Karetji, Petrarca C.; Soekadis, Ade (Knowledge, Politics and Policymaking in Indonesia, 2018)
In recent decades there has been an increasing recognition that politics and political institutions matter for development. There is also a much greater interest in contextually grounded approaches. This has stemmed from an acknowledgement that purely technocratic approaches to development often result in failure because they do not take into account the nature of political institutions. Nor do they take account of the context in a particular developing country and the interests and incentives of powerful national actors. Policy processes are embedded in specific social, political and organisational contexts. Approaches that focus on implementing universal best practices in evidence-informed policymaking are unlikely to be successful. Instead, what is needed is an approach that takes the local context as the starting point for understanding what issues are relevant to policymakers and developing contextually appropriate solutions. The authors of this chapter reflect on the management approaches and systems that may be required to enable and support large-scale development programmes to be flexible and adaptive to the local policy context and circumstances. The chapter argues that such programmes struggle to adopt adaptive management principle and that to do that at scale requires some changes in the way such programmes are managed.
Keywords: Adaptive Management; Doing development differently; Evidence-informed policymaking; Indonesia; Knowledge sector; Thinking and working politically

Doing Anti-corruption Differently - Jorge (www.globalintegrity.org, 2016)
By Alan Hudson, Executive Director Corruption and how to tackle it is center-stage in London this week, with the spotlight brighter than ever as a result of the Panama Papers. This is welcome news. The Anti-Corruption Summit, hosted by the...

Doing adaptive management at Sida - Tim Ruffer; Helen Bailey; Stefan Dahlgren; Patrick Spaven; Mark Winters (Evaluation Brief, 2019)
Lessons from the market systems development approach The evaluation focuses on Sida’s management of MSD projects. The projects are applying the MSD approach with an aim to: - contribute to improved MSD programming by Sida through better management practices across the project cycle - generate recommendations on how Sida can create conducive conditions for systems approaches and adaptive programming more generally. The brief assesses Sida’s organisational capacity for adaptive management in three dimensions: - leadership and culture - staff capacities - skills, and systems and procedures

Doing Accountability Differently: A 'Vertically Integrated' Approach - Amakom, Uzochukwu; Fashola, Temitope; Gay, Charles; Shutt, Cathy (Christian Aid, 2018)

Does Foreign Aid Really Work? An Updated Assessment - Riddell, Roger C. (Social Science Research Network, 2014)
This paper draws widely from the book Does foreign aid really work?, building on that discussion to provide an updated answer to the question based on recent evidence and contemporary debates on aid effectiveness. It starts with a brief discussion (Section 2) of the question: ‘does emergency aid work?’ This is important to the wider debate for two linked reasons. Firstly, the harshest critics of development aid are all supportive of emergency aid, with a number calling for its expansion in spite of evidence of major weaknesses and failures. Secondly, the sharp historical distinction made between emergency and development aid is becoming increasingly strained, as more emergency aid is being used a year or more after emergencies strike to rebuild lives and restore livelihoods, while more development aid is used to directly save lives.The rest of the paper focuses exclusively on development aid. Section 3 provides a rapid overview of the evidence of the impact of individual aid projects, including those of Non-Governmental Organisations (NGOs). The picture is overwhelmingly positive: well over 75% of projects meet their immediate objectives and impact has improved, even though sustaining benefits remains a challenge and there continue to be aid failures. Section 4 reviews the evidence of the wider and long-term impact of aid at the sectoral and country level, including a brief discussion of academic studies on aid and growth. Though there are still major gaps in our knowledge, the quality of the data is improving. However, there is little firm, quantitative evidence to show the specific contribution that aid has made to help achieve the Millennium Development Goals (MDGs), which are a central purpose of many current donor programmes (Section 4.2). More widely, there is plenty of evidence to suggest that aid has contributed positively to both growth and wider development at the sector and country level, though some studies suggest aid has had little or no impact. Overall, the contribution that aid makes to aggregate development is lower than the public has been led to believe (Sections 4.1, 4.3. and 4.4). Assessments of the overall impact of the aid provided by NGOs are rare, but a recent study suggests it has been positive (Section 4.5). More attention is now given to the issues of corruption in aid. Although there is evidence of aid funds being used for corrupt purposes and of aid worsening corruption, on balance it remains a minor issue. Aid has had significant successes in helping the poor affected by corruption (Section 4.6). The second half of the paper shifts focus to the large gap between what aid has done and what it might do. Section 5 discusses a range of inefficiencies within and across the aid system and their costs in terms of reduced impact, including the way aid is allocated, its growing complexity, and the volatility and unpredictability of aid flows (Section 5.1 and 5.2). These inefficiencies place in a different light the evidence of aid’s overall positive impact. The paper looks at the different initiatives that have been mounted to begin to address these weaknesses and failures, including the 2005 Paris Declaration, and discusses why donors have failed to honour the promises they have made to change the ways that they give aid.In Section 5.3, the paper argues that the most critical debate about whether aid works concerns the assessment of whether the short-term, immediate and extensive benefits that aid undoubtedly brings are outweighed by the direct and indirect systemic problems that it risks creating or accentuating. As increasingly over the last decade donors have channelled more of their aid into short-term, quick-impact projects, assessing the wider negative systemic effects of aid has become even more important. Some recent studies suggest that aid’s systemic problems are large and growing, narrowing the gap between aid’s harshest critics and broader analyses of aid impact. Against the backdrop of already too many proposals of how to make aid work better, Section 6 lays out nine concrete proposals for doing this: deepening knowledge of local contexts; ensuring short-term uses of aid are consistent with and supportive of long-term development, and that all aid is more closely related to overall recipient development goals and processes; helping build local capacities for recipients to be able to coordinate aid better; moving from rhetoric to reality in learning lessons from aid; using aid to help the poor in middle income countries; reducing volatility in aid at the country level; encouraging budget support by addressing donor-country political concerns; and rethinking ways of communicating about aid. Section 7 concludes. It suggests that, paradoxically, aid’s impact may well have been harmed by focussing too narrowly on trying to make short-term aid work better, and that the main focus of attention needs to widen to assess how aid can contribute more to a recipient’s own development goals. Additionally, donors need to help build the capacity of developing countries and developing country scholars to enable them to play a bigger role in helping to answer the question of whether aid works; unsettling though their assessments may be.

DIY - Development Impact and You: Practical Tools to Trigger and Support Social Innovation - Nesta (NESTA, 2014)
Keywords: IMPORTANT

Dispatch on Switzerland's International Cooperation 2017–2020: Key points in brief - SDC (Federal Department of Foreign Affairs FDFA, 2016)
The “Dispatch on Switzerland’s International Cooperation, 2017–2020”, is a report to the Swiss Parliament outlining the strategic priorities proposed by the Federal Council in this area. It includes framework credits for each of the five policy instruments used by Switzerland to implement its international cooperation strategy. These are implemented by the Swiss Agency for Development and Cooperation (SDC) of the Federal Department of Foreign Affairs (FDFA), the Human Security Division (HSD) of the FDFA’s Directorate of Political Affairs, and the State Secretariat for Economic Affairs (SECO) of the Federal Department of Economic Affairs, Education and Research (EAER).

Discussion Note: Complexity-Aware Monitoring - USAID (Program Cycle, 2018)
This Discussion Note complements ADS 201 and outlines general principles and promising approaches for monitoring complex aspects of USAID development assistance. Complexity-aware monitoring is a type of complementary monitoring that is useful when results are difficult to predict due to dynamic contexts or unclear cause and-effect relationships. Key principles of the Program Cycle include applying analytic rigor, managing adaptively, and utilizing a range of approaches to achieve results. ADS 201.3.5.5 identifies three types of program monitoring – performance, context, and complementary. All USAID programming incorporates performance monitoring and should include context monitoring. Performance monitoring “is the ongoing and systematic collection of performance indicator data and other quantitative or qualitative information to reveal whether implementation is on track and whether expected results are being achieved.” Context monitoring is “[t]he systematic collection of information about conditions and external factors relevant to the implementation and performance of an operating units (OU’s) strategy, projects, and activities.” As stated in ADS 201.3.5.5, complementary monitoring may be used by missions and Washington OUs to complement performance and context monitoring in situations where results are difficult to predict due to dynamic contexts or unclear cause-andeffect relationships. This Discussion Note provides an explanation of when to use complementary monitoring approaches that are complexity-aware and summarizes the three principles of complexity-aware monitoring.

Digital Technologies for Democratic Governance in Latin America: Opportunities and Risks - Breuer, Anita; Welp, Yanina (Routledge, 2016)
This book is the first to comprehensively analyse the political and societal impacts of new Information and Communication Technologies (ICT) in a region of the Global South. It evaluates under what conditions some Latin American governments and people have succeeded in taking up the opportunities related to the spread of ICTs, while others are confronted with the pessimist scenario of increased, digitally induced social and democratic cleavages. Specifically, the book examines if and how far the spread and use of new ICT affected central aims of democratic governance such as reducing socio-economic and gender inequality; strengthening citizen participation in political decision making; increasing the transparency of legislative processes; improving administrative processes; providing free access to government data and information; and expanding independent spaces of citizen communication. The country case and cross-country explore a range of bottom-up driven initiatives to reinforce democracy in the region. The book offers researchers and students an interdisciplinary approach to these issues by linking it to established theories of media and politics, political communication, political participation, and governance. Giving voice to researchers native to the region and with direct experience of the region, it uniquely brings together contributions from political scientists, researchers in communication studies and area studies specialists who have a solid record in political activism and international development co-operation.

Digital Principles into practice - HDIF (HDIF, 2018)

Digital lives in Ghana, Kenya and Uganda - Bailur, Savita; Donner, Jonathan; Locke, Chris; Schoemaker, Emrys; Smart, Charlotte (Caribou Digital, 2015)
Keywords: IMPORTANT

Digital Development: Elements of a Critical ICT4D Theory and Praxis - Bon, Anna; Akkermans, Hans (Springer International Publishing, 2019)
In recent years, critical research literature in ICT4D has grown. It is widely accepted that theory is to inform practice. However, the inverse directionality, practice informs theory, is much less present in ICT4D, including in critical research. In this paper, we discuss ways how ICT4D research and theory may be better informed by practice—in terms of (i) recognizing praxis-oriented research paradigms and integrating their results, (ii) development of foundational theories, (iii) critical analysis of ICT4D emerging policies, and (iv) positioning ICT4D in the wider development debate. This suggests several elements or directions in which critical research has the potential to push current boundaries of ICT4D in terms of content as well as relevance.
Keywords: Action research paradigm; Critical research; Network complexity theory; Principles for Digital Development

Digital Democracy: The Tools Transforming Political Engagement - Simon, Julie; Bass, Theo; Boelman, Victoria; Mulgan, Geoff (NESTA, 2017)
This paper shares lessons from Nesta’s research into some of the pioneering innovations in digital democracy which are taking place across Europe and beyond

Diffusion Of Innovations Theory, Principles, And Practice - Dearing, James W.; Cox, Jeffrey G. (Health Affairs, 2018)
Aspects of the research and practice paradigm known as the diffusion of innovations are applicable to the complex context of health care, for both explanatory and interventionist purposes. This article answers the question, “What is diffusion?” by identifying the parameters of diffusion processes: what they are, how they operate, and why worthy innovations in health care do not spread more rapidly. We clarify how the diffusion of innovations is related to processes of dissemination and implementation, sustainability, improvement activity, and scale-up, and we suggest the diffusion principles that can be readily used in the design of interventions.

Dialogic Organization Development: The Theory and Practice of Transformational Change - Bushe, Gervase R. (EDS Publications Ltd., 2015)
Dialogic Organization Development is a compelling alternative to the classical action research approach to planned change. Organizations are seen as fluid,socially constructed realities that are continuously created through conversations and images change happens when those conversations and images change. Leaders and consultants can help foster, support, or accelerate the emergence of transformational possibilities by encouraging disruptions to taken-for-granted ways of thinking and acting and the use of generative images to stimulate new organizational conversations and narratives. Dialogic OD is a different mindset, but it s also the previously unrecognized underpinning of such diverse change methods as Appreciative Inquiry, the Art of Convening, Future Search, and many more.

DFID Smart Rules: Better Programme Delivery - version X (2019.01) - DFID (DFID, 2019)

DFID Smart Rules: Better Programme Delivery - version VIII (2018.04) - DFID (DFID, 2017)
Smart Rules provide the operating framework for the Department for International Development’s (DFID’s) programmes.
Keywords: IMPORTANT; Practice

DFID Smart Rules: Better Programme Delivery - version VII (2017.04) - DFID (DFID, 2017)
Smart Rules provide the operating framework for the Department for International Development’s (DFID’s) programmes.
Keywords: IMPORTANT; Practice

DFID Smart Rules: Better Programme Delivery - version VI (2016.10) - DFID (DFID, 2016)
Smart Rules provide the operating framework for the Department for International Development’s (DFID’s) programmes.
Keywords: IMPORTANT; Practice

DFID’s partnerships with civil society organisations - A performance review - ICAI (Independent Commission for Aid Impact (ICAI), 2019)
DFID values civil society organisations (CSOs), but its funding and partnership practices do not fully support the long-term health of the civil society sector.

DFID’s approach to value for money in programme and portfolio management - ICAI (Independent Commission for Aid Impact (ICAI), 2018)
ICAI published this review on DFID’s approach to value for money in February 2018, and as value for money is both a process and an outcome and cuts across all aspects of DFID’s operations, did not score this review. We made five recommendations and published a follow-up to this review in July 2019. All UK government departments are required to achieve value for money in their use of public funds. In recent years, DFID has been working to build value for money considerations further into its management processes and its relationships with implementers and multilateral partners, establishing itself as a global champion on value for money. Review This review was published in February. Though it was not rated, ICAI made five recommendations, and found that the Department for International Development’s approach to value for money was helping to make UK aid spending go further, but improvements are still needed. Findings This review found that DFID has strengthened its processes and systems for ensuring it gains maximum value for each pound spent, has taken swift remedial action to tackle under-performing programmes, and has become a strong global champion on value for money. The review also found that DFID has been diligent in its efforts to cut waste, detect fraud, and improve efficiency, and that this work is improving the return on the UK investment in aid. However, the review found that DFID’s approach was not adequately reporting and capturing results and value for money at the country portfolio level, or how programmes work together to deliver lasting impact, including reducing future dependency on aid. It also found that weakness in the annual review process could undermine DFID’s approach to value for money. It found that targets were frequently revised, and that there could be pressures for optimistic scoring of programmes. Recommendations Based on this review, we made the following five recommendations to help DFID improve its approach to value for money still further: DFID country offices should articulate cross-cutting value for money objectives at the country portfolio level, and should report periodically on progress at that level. Drawing on its experience with introducing adaptive programming, DFID should encourage programmes to experiment with different ways of delivering results more cost-effectively, particularly for more complex programming. DFID should ensure that principles of development effectiveness – such as ensuring partner country leadership, building national capacity and empowering beneficiaries – are more explicit in its value for money approach. Programmes should reflect these principles in their value for money frameworks, and where appropriate incorporate qualitative indicators of progress at that level. DFID should be more explicit about the assumptions underlying the economic case in its business cases, and ensure that these are taken into account in programme monitoring. Delivery plans should specify points in the programme cycle when the economic case should be fully reassessed. Senior responsible owners should also determine whether a reassessment is needed following material changes in the programme, results targets or context. Annual review scores should include an assessment of whether programmes are likely to achieve their intended outcomes in a cost-effective way. DFID should consider introducing further quality assurance into the setting and adjustment of logframe targets.

DFID’s approach to delivering impact - ICAI (Independent Commission for Aid Impact (ICAI), 2015)
UK aid, at its best, makes a real and positive difference to the lives and livelihoods of poor people around the world. Ensuring the best possible performance across a large and multifaceted aid programme is, however, a complex management challenge. This report reviews ICAI’s previous 44 reports and looks at how well DFID ensures positive, long-term, transformative impact across its work.

DFID Data Disaggregation Action Plan: Better Data for Better Lives - DFID (2017)

Developmental Evaluation in Practice: Tips, Tools, and Templates - USAID (USAID, 2017)

Developmental Evaluation: How Barriers & Enablers Emerge Over Time - Baylor, Rebecca; Fatehi, Y. K.; Esper, H. (USAID, 2019)

Developmental evaluation: Bridging the gaps between proposal, program, and practice , Developmental evaluation: Bridging the gaps between proposal, program, and practice - Lawrence, Rachael B.; Rallis, Sharon F.; Davis, Laura C.; Harrington, Karen (Evaluation, 2018)
Developmental evaluation supports grant-funded initiatives seeking innovation and change. Programs born from aspirational grant language and plans often need guidance as they work toward creating workable models for social innovation. This article describes the challenge of designing and implementing complex programs and presents a case that illustrates how a program moves from proposal to practice. The Massachusetts Charter Public School Association Capacity Building Network is a complex intervention, funded by the US federal government, aimed at raising school capacity to serve students with disabilities and English language learners. Developmental evaluation served to bridge the gaps between an aspirational proposal, an ambitious and ambiguous program plan, and emerging practices to serve this population of students. Jointly reviewing our experience in this developmental evaluation, the evaluation team and a program director share important thematic lessons learned about the developmental evaluation approach., Developmental evaluation supports grant-funded initiatives seeking innovation and change. Programs born from aspirational grant language and plans often need guidance as they work toward creating workable models for social innovation. This article describes the challenge of designing and implementing complex programs and presents a case that illustrates how a program moves from proposal to practice. The Massachusetts Charter Public School Association Capacity Building Network is a complex intervention, funded by the US federal government, aimed at raising school capacity to serve students with disabilities and English language learners. Developmental evaluation served to bridge the gaps between an aspirational proposal, an ambitious and ambiguous program plan, and emerging practices to serve this population of students. Jointly reviewing our experience in this developmental evaluation, the evaluation team and a program director share important thematic lessons learned about the developmental evaluation approach.

Developmental Evaluation: Applying Complexity Concepts to Enhance Innovation and Use - Patton, Michael Quinn (Guilford Press, 2010)
Developmental evaluation (DE) offers a powerful approach to monitoring and supporting social innovations by working in partnership with program decision makers. In this book, eminent authority Michael Quinn Patton shows how to conduct evaluations within a DE framework. Patton draws on insights about complex dynamic systems, uncertainty, nonlinearity, and emergence. He illustrates how DE can be used for a range of purposes: ongoing program development, adapting effective principles of practice to local contexts, generating innovations and taking them to scale, and facilitating rapid response in crisis situations. Students and practicing evaluators will appreciate the book's extensive case examples and stories, cartoons, clear writing style, "closer look" sidebars, and summary tables. Provided is essential guidance for making evaluations useful, practical, and credible in support of social change.See also Developmental Evaluation Exemplars, edited by Michael Quinn Patton, Kate McKegg, and Nan Wehipeihana, which presents 12 in-depth case studies.
Keywords: Evaluation; Measurement; Organizational effectiveness; Project management

Developmental Evaluation - Tamarack Community (2012)
This webpage from Tamarack Community provides links to audio files of an interview between Mark Cabaj and Michael Patton in which they discuss the topic of development evaluation. This is an "approach that has proven particularly useful and effective at addressing the unique challenges of evaluating the real nature of community change work. Learning Objectives: To deepen appreciation for the role of evaluation and evaluation thinking To explore the distinctions between various types of evaluation To understand more about developmental evaluation and its use To investigate concepts, tools and resources available to support working with developmental evaluation methods" (Tamarack Community)

Development is Going Digital: What is the role of INGOs? ICT for Development programmes in the Horn, East and Central Africa - Haikin, Matt; Flatters, George (Policy & Practice, 2017)
Development is going digital and INGOs like Oxfam have a vital convening role to play. This paper draws on ICT for Development in Oxfam’s programmes in the Horn, East and Central Africa to consider what this role is.  In order to realise the opportunities

Development innovation: Taking the high road - Ramalingam, Ben (Nesta, 2015)
In part two of his blog on the challenges facing development innovation, Ben Ramalingam suggests how we might take the ‘high road’ to bring about long-lasting and meaningful innovation in the sector.

Development innovation: Fad, silo or catalyst? - Ramalingam, Ben (Nesta, 2015)
In the first of a two-part blog piece, Ben Ramalingam, affiliate of the Overseas Development Institute / Institute of Development Studies, discusses the challenges of implementing innovation in to the development sector - from keeping expectations realistic to managing the innovation process and assessing impact and value.

Development entrepreneurship: how donors and leaders can foster institutional change - Faustino, Jaime; Booth, David (Working Politically in Practice, 2014)
Various communities of practice have been established recently to advance the general idea of thinking and working politically in development agencies. This paper makes a contribution by describing the practice of what has been called development entrepreneurship and explaining some of the ideas from outside the field of development that have inspired it.

Development Aid Confronts Politics: The Almost Revolution - Carothers, Thomas; de Gramont, Diane (EDS Publications Ltd., 2013)
A new lens on development is changing the world of international aid. The overdue recognition that development in all sectors is an inherently political process is driving aid providers to try to learn how to think and act politically. Major donors are pursuing explicitly political goals alongside their traditional socioeconomic aims and introducing more politically informed methods throughout their work. Yet these changes face an array of external and internal obstacles, from heightened sensitivity on the part of many aid-receiving governments about foreign political interventionism to inflexible aid delivery mechanisms and entrenched technocratic preferences within many aid organizations. This pathbreaking book assesses the progress and pitfalls of the attempted politics revolution in development aid and charts a constructive way forward.

Developing ICT Services in a Low-Resource Development Context - Bon, Anna; Akkermans, Hans; Gordijn, Jaap (Complex Systems Informatics and Modeling Quarterly, 2016)
Keywords: IMPORTANT

Developing and Using Useful ToCs - Mayne, John (Evergreen Briefing Note, 2018)
Developing useful theories of change (ToCs) for anintervention, especially complex interventions, is not straightforward nor something done in a couple of hourswith limited effort. Agreement is needed on a number of basic concepts and terms, key challenges need to be addressed, and there are numerous issues that need to be considered.Butthe effort is worth the investment of resources and time.Theory-based evaluation approaches need good ToCs.Some of the ideas here are discussed in the context of agricultural research for nutrition and health interventions by Mayne and Johnson (2015) (10) Developing and Using Useful ToCs. Available from: https://www.researchgate.net/publication/323868372_Developing_and_Using_Useful_ToCs [accessed Jun 13 2018].

Developing an effective adaptive monitoring network to support integrated coastal management in a multiuser nature reserve - Vugteveen, Pim; van Katwijk, Marieke M.; Rouwette, Etiënne; Lenders, H. J. Rob; Hanssen, Lucien (Ecology and Society, 2015)
ABSTRACT. We elaborate the necessary conceptual and strategic elements for developing an effective adaptive monitoring network to support Integrated Coastal Management (ICM) in a multiuser nature reserve in the Dutch Wadden Sea Region. We discuss quality criteria and enabling actions essential to accomplish and sustain monitoring excellence to support ICM. The Wadden Sea Long-Term Ecosystem Research project (WaLTER) was initiated to develop an adaptive monitoring network and online data portal to better understand and support ICM in the Dutch Wadden Sea Region. Our comprehensive approach integrates ecological and socioeconomic data and links research-driven and policy-driven monitoring for system analysis using indicators of pressures, state, benefits, and responses. The approach and concepts we elaborated are transferable to other coastal regions to accomplish ICM in complex social-ecological systems in which scientists, multisectoral stakeholders, resource managers, and governmental representatives seek to balance long-term ecological, economic, and social objectives within natural limits.
Keywords: IMPORTANT

Desired and unwanted: Policy work by participation practitioners - Escobar, Óliver (XXII World Congress of Political Science: ‘Reordering Power, Shifting Boundaries’, 2012)
"This paper presents findings from doctoral research in a Scottish Local Authority Area, where I am developing an interpretive ethnography of ‘the work of participation’ (cf. Colebatch, 2005c, 2006). My focus is on engagement practitioners:
Keywords: Academia; Biology; Computer Science; Earth Sciences; Geography; History; Law; Math; Medicine; Philosophy; Physics; Political science; Psychology; Religion; Research; economics; universities

Designing Regenerative Cultures - Wahl, Daniel Christian (Triarchy Press Ltd, 2016)
This is a 'Whole Earth Catalog' for the 21st century: an impressive and wide-ranging analysis of what's wrong with our societies, organizations, ideologies, worldviews and cultures - and how to put them right. The book covers the finance system, agriculture, design, ecology, economy, sustainability, organizations and society at large. In this remarkable book, Daniel Wahl explores ways in which we can reframe and understand the crises that we currently face, and he explores how we can live our way into the future. Moving from patterns of thinking and believing to our practice of education, design and community living, he systematically shows how we can stop chasing the mirage of certainty and control in a complex and unpredictable world. The book asks how can we collaborate in the creation of diverse regenerative cultures adapted to the unique biocultural conditions of place? How can we create conditions conducive to life? *** "This book is a valuable contribution to the important discussion of the worldview and value system we need to redesign our businesses, economies, and technologies - in fact, our entire culture - so as to make them regenerative rather than destructive." --Fritjof Capra, author of The Web of Life, co-author of The Systems View of Life: A Unifying Vision *** "This is an excellent addition to the literature on ecological design and it will certainly form a keystone in the foundations of the new MA in Ecological Design Thinking at Schumacher College, Devon. It not only contains a wealth of ideas on what Dr Wahl has termed 'Designing Regenerative Cultures' but what is probably more important, it provides some stimulating new ways of looking at persistent problems in our contemporary culture and hence opens up new ways of thinking and acting in the future." -- Seaton Baxter OBE, Prof. in Ecological Design Thinking, Schumacher College, UK [Subject: Systems Thinking, Education, Social Anthropology, Environmentalism, Ecology, Regenerative Culture, Sociology]

Designing Participatory Meetings and Brownbags: A TOPS Quick Guide to Linking Development Practitioners - TOPS (Technical and Operational Performance Support Program, USAID, 2013)
Visit this resource for ideas on designing meetings and brownbags that are engaging and build inopportunities for participant dialogue.

Designing Monitoring and Evaluation Approaches for Learning An FOS How-To Guide - FOS (FOS, 2019)

Designing for Public Services - Nesta (Nesta/IDEO, 2016)

Design thinking: understand - improve - apply - Plattner, Hasso; Meinel, Christoph; Leifer, Larry (Understanding innovation, 2011)
"In this book, the researchers take a system's view that begins with a demand for deep, evidence-based understanding of design thinking phenomena. They continue with an exploration of tools which can help improve the adaptive expertise needed for design thinking. The final part of the book concerns design thinking in information technology and its relevance for business process modeling and agile software development, i.e. real world creation and deployment of products, services, and enterprise systems"--Cover
Keywords: Creative ability in business; Creative thinking; Designer; Industrial management; Innovation; Organizational change; Product development; System design; Technological innovations

Design Thinking is not enough – Innovate DFID - DFID, Innovate (Medium, 2016)
By Tamara Giltsoff

Design Thinking for Social Innovation - Brown, Tim; Wyatt, Jocelyn (Stanford Social Innovation Review, 2010)
Designers have traditionally focused on enhancing the look and functionality of products. Recently, they have begun using design techniques to tackle more complex problems, such as finding ways to provide low-cost healthcare throughout the world. Businesses were the first to embrace this new approach—called design thinking—and nonprofits are beginning to adopt it too.

Design reality gap issues within an ICT4D project: an assessment of Jigawa State Community Computer Center - Kanya, Rislana Abdulazeez; Good, Alice (Sixth International Conference on Information and Communication Technologies and Development (ICTD2013), 2013)
This paper evaluates the Jigawa State Government Community Computer centre project using the design reality gap framework. The purpose of this was to analyse the shortfall between design expectations and implementation realities, in order to find out the current situation of the project. Furthermore to analyse whether it would meet the key stakeholder’s expectation. The Majority of Government ICT Projects is classified as either failure or partial failure. Our research will underpin a case study of Jigawa State Community Computer centre project. To support our argument we conducted a qualitative case study in which we conducted two interviews, and a survey to find out the current status of the project. Our findings indicated that the project can be classified as a partial failure because few of the centres, operating in some areas of the community, offered socio-economic advancement. Our results also indicated that the project design reality gap is quite significant. This gap occurred due to poor planning and implementation of the project. Our research reveals several challenges that could impact upon the successfulness of the project provide a foundation for further research and add to existing literature. The challenges associated with the project we identify include poor state of power supply, lack of community engagement during the project planning and implementation, lack of commitment from the Local Government Administration. Our research has potential in adding ICT4D literature, reveals how ICT4D project evaluation can be carried out using design reality gap framework and provides guideline for policy makers.

Design for Sustainability: Countering the Drivers of Unsustainability in Development Projects - Meyer, Isabel; Marais, Mario Alphonso (The Journal of Community Informatics, 2015)
Development projects aim to make a difference in communities. Funders often require that the intervention needs to last, or that it needs to effect change that will last, over an extended period of time. In short, a sustainable difference needs to be made and this is an important measure of success.  However, there are inherent drivers in the combined system of donors and beneficiaries that counter sustainability. This article explores these drivers, and identifies project design elements that need to be in place in order to promote sustainability by countering the drivers. This approach is applied to an education project, and implications for ICT4D projects are developed.
Keywords: Development; Project design; Systems Approaches; sustainability

Demanding Good Governance: Lessons from Social Accountability Initiatives in Africa - McNeil, Mary; Malena, Carmen (World Bank Publications, 2010)
Accountability is the cornerstone of good governance. Unless public officials can be held to account, then critical benefits associated with good governance, such as social justice, poverty reduction and development remain elusive. The impacts of non-responsive and unaccountable governance are perhaps most harshly felt by the citizens of Africa, where corruption and governance failures are broadly acknowledged as a principal obstacle to the achievement Over the past decade, a range of social accountability practices―such as participatory budgeting, independent budget analysis, participatory monitoring of public expenditure and citizen evaluation of public services have been experimented with in many Africa countries. Their outcomes and lessons have, thus far, received little attend and documentation. This volume aims to make a contribution towards filling this gap by describing and analyzing a selection of social accountability initiatives from seven Sub-Saharan countries.

Delivering large-scale IT projects on time, on budget, and on value - Bloch, Michael; Blumberg, Sven; Laartz, Jürgen (McKinsey on Business Technology, 2012)
Large IT efforts often cost much more than planned; some can put the whole organization in jeopardy. The companies that defy these odds are the ones that master key dimensions that align IT and business value.

Delivering institutional reform at scale: Problem-driven approaches supported by adaptive programming - Manuel, Clare (Synthesis Paper, 2, 2016)
LASER synthesis papers aim to help donors and other stakeholders better understand why and how to approach investment climate reform programming differently. The papers reflect emerging best practice and lessons learnt on what works and what does not work in doing development differently. The papers have been peer-reviewed by experts in the field including senior advisers at DFID, World Bank, IFC and the Donor Committee for Enterprise Development (amongst others). Second synthesis paper - Delivering institutional reform at scale: problem-driven approaches supported by adaptive programming February 2016 Drawing on our experience on the ground, this peer-reviewed paper suggests how problem driven approaches, supported by adaptive programme management can be implemented at scale in relation to donor programming aimed at institutional reform and improving state capability.
Keywords: IMPORTANT

Deliberation and Development: Rethinking the Role of Voice and Collective Action in Unequal Societies - Heller, Patrick; Rao, Vijayendra (World Bank Publications, 2015)
This book marries two fields that rarely converse with one another:deliberative democracy and development studies. The study ofdeliberation―which explores normative and practical questions aroundgroup-based decision making through discussion or debate―has emergedas a critical area of study over the past two decades. Concurrently, the fieldof development has seen a spurt of interest in community-led developmentand participation premised on the ability of groups to arrive at decisionsand manage resources via a process of discussion and debate. Despite thegrowing interest in both fields, they have rarely engaged with one another.This book, which brings together new essays by some of the leading scholarsin the field, deepens our understanding of participatory decision makingin developing countries while initiating a new field of study for scholarsof deliberation. In the process, it sheds light on how to best design andimplement policies to strengthen the role of participation in development.

Definitions of Agile Software Development and Agility - Laanti, Maarit; Similä, Jouni; Abrahamsson, Pekka (Springer, 2013)
The Agile Manifesto and Agile Principles are typically referred to as the definitions of "agile" and "agility". However, many other definitions exist in the literature. Thus the different definitions provide interesting source for research. For each definition we examine where their emphasis is and compare that to the emphases found in the Agile Principles.

Defining the burden of proof in conservation - Salafsky, Nick; Redford, Kent H. (Biological Conservation, 2013)
Conservationists often must take action in the face of uncertainty about the costs and benefits of different options. Although this uncertainty can be paralyzing when the stakes are high, there is obviously a cost to inaction as well as action, and decision makers need to be encouraged to act when appropriate. Many other fields of human endeavor such as law, medicine, and public safety have formally developed the “burden and standards of proof” that decision makers have to meet in choosing to take action. In this paper, we review the standards developed in these other fields to help define a similar framework for conservation. Specifically we propose that a conservation decision maker must assume the burden of proof when there is a decision to act that substantially affects others, in which the decision maker has professional standing, where there is not immediate urgency, and where there is some, but not complete certainty about the outcomes of acting versus not acting. Once these initial tests have been met, in situations in which the decision maker is more worried about the consequences of not acting, then a relatively low standard of proof is required for taking action. If the decision maker is concerned with the consequences of acting in error, but the action is relatively reversible, then a medium standard of proof is required. And finally, if there are concerns about the consequences of acting in error, but the action is relatively irreversible, then a high standard of proof is required.
Keywords: Biodiversity; Burden of proof; Decision theory; Standard of proof; Uncertainty

Defining and using evidence in conservation practice - Salafsky, Nick; Boshoven, Judith; Burivalova, Zuzana; Dubois, Natalie S.; Gomez, Andres; Johnson, Arlyne; Lee, Aileen; Margoluis, Richard; Morrison, John; Muir, Matthew; Pratt, Stephen C.; Pullin, Andrew S.; et al. (Conservation Science and Practice, 2019)
There is growing interest in evidence-based conservation, yet there are no widely accepted standard definitions of evidence, let alone guidance on how to use it in the context of conservation and natural resource management practice. In this paper, we first draw on insights of evidence-based practice from different disciplines to define evidence as being the “relevant information used to assess one or more hypotheses related to a question of interest.” We then construct a typology of different kinds of information, hypotheses, and evidence and show how these different types can be used in different steps of conservation practice. In particular, we distinguish between specific evidence used to assess project hypotheses and generic evidence used to assess generic hypotheses. We next build on this typology to develop a decision tree to support practitioners in how to appropriately use available specific and generic evidence in a given conservation situation. Finally, we conclude with a discussion of how to better promote and enable evidence-based conservation in both projects and across the discipline of conservation. Our hope is that by understanding and using evidence better, conservation can both become more effective and attract increased support from society.
Keywords: Adaptive Management; Evidence-based conservation; Evidence-based practice; Natural resource management; Project management; biodiversity; environmental evidence

Deconstructing Development Discourse: Buzzwords and Fuzzwords - Cornwall, Andrea; Eade, Deborah (Oxfam & Practical Action Publishing, 2010)
Writing from diverse locations, contributors critically examine some of the key terms in current development discourse. Why should language matter to those who are doing development? Surely, there are more urgent things to do than sit around mulling over semantics?

Decision-Making and Data Use Landscaping [at DFID] - Powell, Josh; Orton-Vipond, Sarah; Bhatia, Vinisha; Kilroy, Annie (Development Gateway, 2018)
As DFID aims to harness the Data Revolution, ensuring that data1 drive decision-making, public accountability, and the achievement of the Sustainable Development Goals (SDGs), ensuring that systems, processes, and skills for data are aligned with these objectives is paramount. Across sector policy teams, country offices, and various analytical and technical cadres, different strengths and weaknesses, as well as needs and ambitions exist. To inform a strategic approach to data, as framed in its forthcoming Data Roadmap, DFID collaborated with Development Gateway to perform a Decision and Data Use Landscaping study. This report details lessons learned from approximately 60 interviews across 4 DFID country offices, all sector policy teams, senior managers, and various analytical and technical cadres and offices.

Decide Madrid: a case study on e-Participation - Pina, Vicente; Torres, Lourdes; Royo, Sonia; García-Rayado, Jaime (XX International AECA Congress, 2019)
This paper analyses the award-winning e-participation initiative of the city council of Madrid, Decide Madrid, to discover the critical success factors at contextual, organizational and individual level. This analysis is carried out with desk research and semi-structured interviews. Results show that the most relevant success factors are the socio-economic context, the commitment of the city council, the method used to recruit the workers and the knowledge of senior managers about citizen participation and ICTs. However, the lack of transparency and poor functioning of some of its participation activities are negatively affecting its performance.

Debiasing: a systematic discipline and delight for development professionals - Chambers, Robert (Institute of Development Studies, 2018)
Do we really need debiasing, yet another word?  Yes, unless anyone can improve on it, because we need a word to describe a rigorous discipline we development professionals need for grounded realism. This has been coming on me slowly. But now explorations and ‘aha!’ moments in India have accumulated and combined into an epiphany.  For […]

Dealing with uncertainty: Reflections on donor preferences for pre-planned project models - Tyrrel, Lavinia; Cole, William (Working Politically in Practice, 2016)
Keywords: IMPORTANT

DE 201: A practitioner's guide to developmental evaluation - Dozois, Elizabeth; Langlois, Marc; Blanchet-Cohen, Natasha (J.W. McConnell Family Foundation, 2010)
Developmental evaluation has emerged fairly recently as a way to support adaptive learning in complex and emergent initiatives. Combining the rigour of evaluation with the flexibility and imagination required for development, this new form of evaluation brings critical thinking to bear on the creative process in initiatives involving high levels of uncertainty, innovation, emergence, and social complexity. This guidebook from J.W. McConnell Family Foundation and the International Institute for Child Rights and Development explores key aspects of Developmental Evaluation (DE) and offers suggestions for implementing the practice and includes a range of DE resources to help with its facilitation. Contents What is developmental evaluation? When is developmental evaluation appropriate? What competencies are needed to be an effective DE? How is developmental evaluation practiced? Challenges Appendices Assessing Readiness Stakeholder analysis Sample Learning Framework Systems Analysis Framework - Key Components Data Analysis Resources

Data and monitoring needs for a more ecological agriculture - Zaks, David P. M.; Kucharik, Christopher J. (Environmental Research Letters, 2011)
Information on the life-cycle environmental impacts of agricultural production is often limited. As demands grow for increasing agricultural output while reducing its negative environmental impacts, both existing and novel data sources can be leveraged to provide more information to producers, consumers, scientists and policy makers. We review the components and organization of an agroecological sensor web that integrates remote sensing technologies and in situ sensors with models in order to provide decision makers with effective management options at useful spatial and temporal scales for making more informed decisions about agricultural productivity while reducing environmental burdens. Several components of the system are already in place, but by increasing the extent and accessibility of information, decision makers will have the opportunity to enhance food security and environmental quality. Potential roadblocks to implementation include farmer acceptance, data transparency and technology deployment.

DAC Peer Review of Switzerland - 2019 - OECD (OECD, 2019)

DAC Peer Review of Switzerland - 2013 - OECD (OECD, 2013)

DAC - Mid-term Peer Review of Switzaerland - 2016 - OECD (OECD, 2016)