Evaluating Systems Change Results: an inquiry framework - Cabaj, Mark (Tamarack Institute, 2018)

Evaluating the Design of Behavior Change Interventions: A Case Study of Rhino Horn in Vietnam - Olmedo, Alegría; Sharif, Vian; Milner‐Gulland, E. J. (Conservation Letters, 2018)
Behavioral change interventions are increasingly widely used in conservation. Several projects addressing rhino horn consumption were recently launched in Vietnam. We used key informant interviews, document analysis, and marketing theory to explore their strategies for intervention design, implementation, monitoring, and evaluation. We developed a framework to evaluate whether they followed best practice and identify implementation challenges. Interventions could make greater use of key project design steps, including basing interventions on robust research to understand the behavior in question, identifying the target audience whose behavior interventions aim to change, and developing measures that can provide reliable evidence of success or not. Challenges include the need for law enforcement to complement campaigns; improving cooperation between NGOs; and clearly defining aims of demand-reduction initiatives. Using best practice from other fields and considering demand reduction within the wider context of wildlife, trade policy will help address these challenges.
Keywords: Awareness; Ilegal wildlife trade; Project design; demand reduction; social marketing

Evaluating the impact of flexible development interventions using a ‘loose’ theory of change: Reflections on the Australia-Mekong NGO Engagement Platform - Davies, Rick (Methods Lab, 2016)
• For some interventions, tight and testable theories of change are not appropriate – for example, in fast moving humanitarian emergencies or participatory development programmes, a more flexible approach is needed. • However, it is still possible to have a flexible project design and to draw conclusions about causal attribution. This middle path involves ‘loose’ theories of change, where activities and outcomes may be known, but the likely causal links between them are not yet clear. • In this approach, data is collected ‘after the event’ and analysed across and within cases, developing testable models for ‘what works’. More data will likely be needed than for projects with a ‘tight’ theory of change, as there is a wider range of relationships between interventions and outcomes to analyse. The theory of change plays an important role in guiding the selection of data types. • While loose theories of change are useful to identify long term impacts, this approach can also support short cycle learning about the effectiveness of specific activities being implemented within a project’s lifespan.

Evaluating with the Success Case Method - Gram, Tom (Performance X Design, 2011)
The method was developed by Robert Brinkerhoff as an alternative (or supplement) to the Kirkpatrick approach and its derivatives. It is very simple and fast (which is part of it’s appeal) and goes something like this: Step 1. Identify targeted business goals and impact expectations Step 2. Survey a large representative sample of all participants in a program to identify high impact and low impact cases Step 3. Analyze the survey data to identify: a small group of successful participants a small group unsuccessful participants Step 4. Conduct in-depth interviews with the two selected groups to: document the nature and business value of their application of learning identify the performance factors that supported learning application and obstacles that prevented it. Step 5. Document and disseminate the story report impact applaud successes use data to educate managers and organization The process produces two key outputs In-depth stories of documented business effect that can be disseminated to a variety of audiences Knowledge of factors that enhance or impede the effect of training on business results. Factors that are associated with successful application of new skills are compared and contrasted with those that impede training. It answers practical and common questions we have about training and other initiatives: What is really happening? Who’s using what, and how well? Who’s not using things as planned? What’s getting used, and what isn’t? Which people and how many are having success? Which people and how many are not? What results are being achieved? What value, if any, is being realized? What goals are being met? What goals are not? Is the intervention delivering the promised and hoped for results? What unintended results are happening? What is the value of the results? What sort of dollar or other value can be placed on the results? Does the program appear to be worthwhile? Is it producing results worth more than its costs? What is its return on investment? How much more value could it produce if it were working better? How can it be improved? What’s helping? What’s getting in the way? What could be done to get more people to use it? How can everyone be more like those few who are most successful?

Evaluation and learning in complex, rapidly changing health systems: China’s management of health sector reform - Xiao, Yue; Husain, Lewis; Bloom, Gerald (Globalization and Health, 2018)
Healthcare systems are increasingly recognised as complex, in which a range of non-linear and emergent behaviours occur. China’s healthcare system is no exception. The hugeness of China, and the variation in conditions in different jurisdictions present very substantial challenges to reformers, and militate against adopting one-size-fits-all policy solutions. As a consequence, approaches to change management in China have frequently emphasised the importance of sub-national experimentation, innovation, and learning. Multiple mechanisms exist within the government structure to allow and encourage flexible implementation of policies, and tailoring of reforms to context. These limit the risk of large-scale policy failures and play a role in exploring new reform directions and potentially systemically-useful practices. They have helped in managing the huge transition that China has undergone from the 1970s onwards. China has historically made use of a number of mechanisms to encourage learning from innovative and emergent policy practices. Policy evaluation is increasingly becoming a tool used to probe emergent practices and inform iterative policy making/refining. This paper examines the case of a central policy research institute whose mandate includes evaluating reforms and providing feedback to the health ministry. Evaluation approaches being used are evolving as Chinese research agencies become increasingly professionalised, and in response to the increasing complexity of reforms. The paper argues that learning from widespread innovation and experimentation is challenging, but necessary for stewardship of large, and rapidly-changing systems.

Evaluation at IDRC - IDRC (IDRC, 2017)

Evaluation Building Blocks: A Guide - McKegg, K.; Oakden, J.; Wehipeihana, N.; King, Julian (Kinnect Group, 2018)
This guide presents the Kinnect Group’s approach to evaluation. Developed through ten years of collaborative work, it has been well tested. While there are many other guides to evaluation and different ways to do it, our approach emphasises the place of evaluative reasoning in evaluation. The guide draws on the work of many evaluation theorists, as well as our practice-based body of knowledge. We hope that using the building blocks as explained in the guide will help you in your endeavour to do credible and useful evaluation. We find this approach works for us. Our clients tell us it gives them meaningful and insightful findings that they can use to take action. Our view to intellectual property is simple. What you read is yours to use. We just ask that you cite the guide when you draw from it. For those of you who engage with this guide and use it in your practice, we look forward to hearing from you. We are particularly interested in feedback where you find aspects of the guide work well for you. In that way we can all continue to build and evolve our practice.

Evaluation for equitable development results - Segone, M (UNICEF, 2011)
This document is made up of a range of Evaluation Working Papers (EWP) focused on evaluation for equitable development. Put together by evaluation specialists they present strategic evaluation findings, lessons learned and innovative approaches and methodologies. Part 1:Evaluation and equity Evaluation to accelerate progress towards equity, social justice and human rights 2 Human rights and gender equality in evaluation 13 When human rights is the starting point for evaluation 25 Strengthening Equity- focused evaluations through insights from feminist theory and approaches 39 Decolonizing evaluation in a developing world. Implications and cautions for Equity-focused evaluations 59 Part 2: Methodological implications for Equity-focused evaluations Methodological issues to design and implement equity-focused evaluations 86 Developmental evaluation for Equity-focused evaluations 102 Systems thinking and Equity-focused evaluations 115 Methodological challenges in using programme theory to evaluate pro-poor and equity-focused programmes by Patricia Rogers, Royal Melbourne Institute of Technology University and Richard Hummelbrunner, Independent consultant 142 Case Study and equity in Evaluationby Saville Kushner, University of the West of Englan 172 Values-Engaged Evaluationsby Jennifer Greene, University of Illinois at Urbana-Champaign 192 Part 3: Examples of Equity-focused evaluations Evaluating the contribution of UNDP to equity-focused public policies in Brazil and China 210 Using a human rights approach to evaluate ILO’s discrimination strategy 222 CONEVAL experience in evaluating interventions for Indigenous populations in Mexico 244 UNICEF supported evaluations with elements of equity-focused evaluations 258

Evaluation of development programmes - OECD - OECD (2019)
Data & research on evaluation of development programmes inc. paris declaration, budget support, multilateral effectiveness, impact evaluation, joint evaluations, governance, aid for trade

Evaluation of the market systems development approach: Lessons for expanded use and adaptive management at Sida Volume I: Evaluation Report - Ruffer, Tim; Bailey, Helen; Dahlgren, Stefan; Spaven, Patrick; Winters, Mark (Evaluation of the market systems development approach, 2018)
This report presents the findings of an evaluation of Sida’s management of the market systems development (MSD) approach. It aims to inform thinking on how Sida can best manage its growing portfolio of MSD programs. Beyond this, it provides insights relevant to Sida’s wider support to complex and adaptive programs. The evaluation identified several factors that affect Sida’s ability to ensure that conducive conditions are in place for effective MSD programs and good development programming more generally. Sida’s relatively flexible framework of rules, guidelines and systems for project management provide the space needed for staff to innovate and manage adaptively. But for this to happen consistently and effectively, Sida needs to invest more deliberately in building the capacity of its staff in relevant areas. In addition, leadership and incentives are key to shaping a culture of active experimentation and learning. This needs to be supported with clearer guidance for those involved in the design and appraisal of MSD projects; and strengthened oversight of project performance, including through adjustments to Sida’s contracts and funding agreements.

Evaluation of the market systems development approach: Lessons for expanded use and adaptive management at Sida Volume II: Case studies - Ruffer, Tim; Bailey, Helen; Dahlgren, Stefan; Spaven, Patrick; Winters, Mark (Evaluation of the market systems development approach, 2018)
This report presents the findings of an evaluation of Sida’s management of the market systems development (MSD) approach. It aims to inform thinking on how Sida can best manage its growing portfolio of MSD programs. Beyond this, it provides insights relevant to Sida’s wider support to complex and adaptive programs. The evaluation identified several factors that affect Sida’s ability to ensure that conducive conditions are in place for effective MSD programs and good development programming more generally. Sida’s relatively flexible framework of rules, guidelines and systems for project management provide the space needed for staff to innovate and manage adaptively. But for this to happen consistently and effectively, Sida needs to invest more deliberately in building the capacity of its staff in relevant areas. In addition, leadership and incentives are key to shaping a culture of active experimentation and learning. This needs to be supported with clearer guidance for those involved in the design and appraisal of MSD projects; and strengthened oversight of project performance, including through adjustments to Sida’s contracts and funding agreements.

Everyday Political Analysis - Hudson, David; Marquette, Heather; Waldock, Sam (Developmental Leadership Program, 2016)
This short note introduces a stripped-back political analysis framework designed to help frontline development practitioners make quick but politically-informed decisions. It aims to complement more in-depth political analysis by helping programming staff to develop the 'craft' of political thinking in a way that fits their everyday working practices. Everyday Political Analysis involves two steps: Understanding interests: What makes people tick? Understanding change: What space and capacity do people have to effect change? For each step five questions, accompanied by prompts, aim to help staff to conduct quick political analysis. The EPA framework can be used at any stage of the aid management cycle, and can help users to respond rapidly to unexpected change. We are keen to hear back from people on their experience of using EPA to help us adapt the framework. Was it useful (or not)? Do people tend to use just one or both steps? Are there missing statements or prompts that would improve the analysis? Please email us at info [at] dlprog.org.

Evidence Base for Collaborating, Learning, and Adapting: summary of the literature review (2016.08) - Learning Lab (USAID, 2016)
LEARN and USAID/PPL are managing an area of work known as Evidence Base for Collaborating, Learning, and Adapting (EB4CLA). The purpose of this work is to answer key learning questions:

Evidence Base for Collaborating, Learning, and Adapting: summary of the literature review (2017 update) - Learning Lab (USAID, 2017)
LEARN and USAID/PPL are managing an area of work known as Evidence Base for Collaborating, Learning, and Adapting (EB4CLA). The purpose of this work is to answer key learning questions:

Evidence to Action: Research to Address Illegal Wildlife Trade - Milner-Gulland, E. J.; Cugniere, Laure; Hinsley, Amy; Phelps, Jacob; Rolfe, Michael 't Sas; Veríssimo, Diogo (2018)
Tools and expertise to improve the evidence base for national and international Illegal Wildlife Trade policy already exist but are underutilised. Tapping into these resources would produce substantive benefits for wildlife conservation and associated sectors, enabling governments to better meet their obligations under the Sustainable Development Goals and international biodiversity conventions. This can be achieved through enhanced funding support for inter-sectoral research collaborations, engaging researchers in priority setting and programme design, increasing developing country research capacity and engaging researchers and community voices in policy processes. This briefing, addressed to policy makers and practitioners, is part of the 2018 Evidence to Action: Research to Address Illegal Wildlife Trade event programme, organised by five of the UK’s most active IWT research institutions, to support the London 2018 IWT Conference.

Experiential Learning: Fostering lasting behavioural change - Norton, Brittney M. (Prospects practice paper, 4, 2016)
The term ‘experiential learning’ is often mistakenly used to describe any learning experience that involves participatory activities. This paper presents an overview of what true experiential learning is and how this form of learning allows for real-world application - thus promoting lasting behavioural change. Furthermore, it explains how experiential learning was adopted as the methodological core of the Prospects psychosocial programme in Liberia, which builds ‘pre-employment skills’ in 3,500 vulnerable youth aged 15-24 years. The goal of the paper is to provide other youth and education development practitioners with a resource that can be used to strengthen youth and behaviour change programmes through the incorporation of the experiential learning theory. In addition, this paper explains its application in the Prospects programme, before concluding with key considerations of such a design. 1

Explaining positive deviance in public sector reforms in development - Andrews, Matt (World Institute for Development Economic Research (UNU-WIDER), 2013)
Public sector reforms are commonplace in developing countries. Much of the literature about these reforms reflects on their failures. This paper asks about the successes and investigates which of two competing theories best explain why some reforms exhibi
Keywords: Development; Innovation; Reform; case survey; governance; growth; leadership

Exploring new territories for evaluation - Sharp, Cathy (Collective Leadership for Scotland, 2018)
Drawing on action research, this paper recasts evaluation as ‘action inquiry’, an embedded evaluative learning practice that can help navigate complexity when enacting collective leadership. It is offered as an invitation to inquiry amongst a reasonably well-informed audience of policy makers and practitioners who work in and for public services. It will particularly interest those who provide research, evaluation and facilitation support, and those seeking to develop a more relational approach to research and evaluation. Action inquiry is a model of practising change together in environments where ‘nothing is clear, and everything keeps changing’ that significantly challenges the prevailing discourse on evaluation. Action inquiry can be wrapped around and enmeshed within initiatives and programmes that work with complexity - anywhere where success will depend on the quality of relationships that can be developed. The paper reviews some important interrelated concepts that underpin the ideals of collective leadership and public service reform and which confront deeply embedded traditional notions of leadership, expertise and participation. These offer important challenges to ideas about how change happens and recognise that relationships are at the heart of practising change. This warrants a re-examination of the high expectations of evidence-based or informed practice. Collective leadership makes new demands of evidence as it rests on help to determine ‘wise actions’ in real-life situations. This confronts the practical reality of how to work together in conditions often expressed as ‘dynamic’ or ‘turbulent’ and the added human complexities of power, emotions and relationships; too often these elements are denied or avoided aspects of a change process. Facilitated action inquiry makes these elements part of the conversations, in the midst of ‘work-as-we-are-doing-it’, to increase areas of choice for individuals and a group as a whole. Within public policy, there is an overwhelming amount of evidence about ‘what works’ yet change seems to be stubborn and slow. The paper explores some of the deep-rooted vestiges of a ‘hierarchy of evidence’ and assumptions about standardisation and generalisability that act as a ‘barrier to transformation’. These include the narrow framing of what counts as evidence and consequent relegation of community perspectives, lived experience and practice-based evidence. The paper acknowledges the clear appetite for different approaches to evaluation, especially those that better reflect deeply held values and avoid creating a culture of ‘gaming’, rooted in fear of failure and loss of funding, at the expense of learning. The need for new forms of developmental evaluative thinking, collaborative inquiry and action research to create embedded learning is well overdue. Action inquiry is a desirable and necessary response to the complex situations and challenges of human services and recognises the essentialness of knowledge co-production. It is a model of co-creation at every stage and endorses the idea that people learn from participation in evaluation and by testing theories of change through action. Action inquiry builds on the idea of inquiry, or a moment-to-moment awareness and quality of attention and draws from several elements of action research practice. It sees inquiry as an intervention in itself, one that furthermore, explicitly seeks to enhance the probability of the success of a programme, focus on learning, the collaborative development of practice-based knowledge and positive relationships. The paper highlights the importance of building inquiry into living systems, the role of facilitation, systemic inquiry, and evaluative thinking. It proposes an expansion of ideas of appreciation as a relational and collaborative practice that is a driver of emergence. Social recognition that acknowledges someone’s social value to the community and implies mutual moral obligations to cooperation and participation is particularly crucial in a work context that requires successful coordination and multiple contributions to achieve results across hierarchies of position, professional rank and sectors. Hence, appreciation goes beyond the idea of positivity to include social recognition, valuing more explicit forms of inquiry, building participants’ aspirations to design new social systems and acting in new ways to embed change. In developing this discussion, the paper contributes to emerging dialogues about the need for a model of ‘5th generation evaluation’. Such a model would be based on the idea that appreciative and challenging inquiry that is contextual, relational and open-minded will create better opportunities for change and development. The paper sets out some ‘provocative propositions’ that can help us to navigate this terrain, perhaps of a fledgling ‘5th generation approach’ to inquiry. Facilitated action inquiry can hold the key to developing both new knowledge and an adaptive, collaborative and improvisational skill-set, able to respond in new ways to systemic and complex issues on the ground. It’s common to hear the expression ‘it’s all about relationships’ and it is clearly time to shift our focus to relationships; not relationships as ‘things’, but as co-created and dynamic relational processes in which we are embedded. In this way we can bring new qualities to our talking to each other about our various and shared visions of a better future.

Exploring new territories for evaluation - Provocative propositions - Sharp, Cathy (2018)

Exploring the new world: practical insights for funding, commissioning and managing in complexity - Lowe, Toby; Plimmer, Dawn (Collaborate & Newcastle University, 2019)
The world is complex. If we want to contribute to creating positive social outcomes, we must learn to embrace this complexity. This is the New World that funders and commissioners are discovering: • People are complex: everyone’s life is different, everyone’s strengths and needs are different. • The issues we care about are complex: issues – like homelessness – are tangled and interdependent. • The systems that respond to these issues are complex: the range of people and organisations involved in creating ‘outcomes’ in the world are beyond the management control of any person or organisation. Building on the findings from our previous report, A Whole New World, we have spent the last 12 months working with a growing movement of funders, public sector commissioners, and organisations working on the ground to begin to explore this New World: to find examples of practice that will help people to navigate it effectively. This report explores the key features of their response: working in a way that is human, prioritises learning and takes a systems approach.

Exploring the potential for interactive radio to improve accountability and responsiveness to small-scale farmers in Tanzania - Gilberds, Heather; Handforth, Calum; Leclair, Mark (IDS, 2016)
What are the possibilities of using new digital technologies alongside radio to help ensure that agricultural development projects are farmer-centred, and meet the needs of the rural citizens they intend to serve? This research assesses Farm Radio International’s Listening Post – a model that combines radio and digital technologies with the aim of collecting and aggregating farmer feedback to aid decision-making and adaptive project implementation. The research shows that linking a mobile-based crowd-sourcing tool with radio is effective at ensuring engagement from a large number of farmers, who felt it was a useful way of raising their questions and concerns to NGOs, policy-makers and experts. The model has clear potential to strengthen the chain of relationships between citizens, extension services suppliers, projects and policymakers. It has also demonstrated its potential to collect real-time feedback from farmers that could be used to aid decision-making and improve accountability in agricultural development initiatives, helping to ensure they are more responsive to farmers. However, it also concludes that ‘closing the feedback loop’ – ensuring that farmer’s comments, questions and concerns are responded to – is a challenge for the Listening Post. Sometimes, the interactive radio programmes close the loop by disseminating answers to questions or concerns raised by farmers, or by connecting farmers to extension services – but only when a solid and systematic process had been developed for this to happen. Therefore is critical to identify and incentivise actors who are capable of responding during the design phase of a Listening Post, and to ensure that they are involved in every stage of the process. It also important that project partners who are interested in beneficiary feedback identify the flexible or actionable points in their project frameworks from the outset, rather than collecting data that they are not able to use to adapt their programmes

Exploring the prospects for adaptive governance in marine transboundary conservation in East Africa - Tuda, Arthur Omondi; Kark, Salit; Newton, Alice (Marine Policy, 2019)
This article explores the prospects for adaptive governance in a proposed marine transboundary conservation initiative in East Africa. Adaptive governance that involves interdependent state and non-state actors learning and taking action on joint environmental problems is suggested for effective transboundary resource governance. Using the concept of adaptive co-management, the current multi-stakeholder marine governance systems in southern Kenya and northern Tanzania are compared to illuminate opportunities and constraints for adaptive marine transboundary conservation governance between Kenya and Tanzania. The concept of networks and the formal method of social network analysis (SNA) are applied as the main methodological device. Using questionnaire and semi-structured interviews, social network data of 70 organizations (local resources users, government agencies and NGOs) was generated from Kenya (n = 33) and Tanzania (n = 37). Results show the existence of strong collaboration networks for marine resource governance in both Kenya and Tanzania. Social proximity is the common driver of network formation. Collaboration networks in Kenya and Tanzania have contributed to enhanced learning among marine resource managers. Conclusions point to the need to focus on common challenges relating to low levels of rule-compliance, limited access to information on the state of resources and poor integration of science into marine management decisions. Finally, differences in views regarding the state of marine ecosystems need to be addressed to improve prospects for joint problem-solving in marine transboundary conservation.
Keywords: Adaptive co-management; Adaptive governance; Collaboration; Marine transboundary conservation; Social network analysis

Exploring ways to reconcile accountability and learning in the evaluation of niche experiments - Regeer, Barbara J.; de Wildt-Liesveld, Renée; van Mierlo, Barbara; Bunders, Joske F. G. (Evaluation, 2016)
While evaluation is seen as a mechanism for both accountability and learning, it is not self-evident that the evaluation of niche experiments focuses on both accountability and learning at the same time. Tensions exist between the accountability-oriented needs of funders and the learning needs of managers of niche experiments. This article explores the differences in needs and expectations of funders and managers in terms of upwards, downwards and internal accountability. The article shows that as the multi-stakeholder contexts in which niche experiments take place give rise to various requirements, tensions in evaluation are essentially a specific manifestation of tensions between niche experiments and their multiple contexts. Based on our findings, an adjusted accountability framework is proposed, including several strategies that can reconcile a learning approach with accountability needs in niche experiments aiming to change current practices in a more sustainable direction.

External Review of the Building Effective and Accessible Markets (BEAM) programme - Snedden, Ewan; Freer, Gordon; Moffatt, Michelle (WYG International, 2016)

External validity and policy adaptation. From impact evaluation to policy design - Williams, Martin J. (BSG Working Paper Series, 2017)
With the growing number of rigorous impact evaluations worldwide, the question of how best to apply this evidence to policymaking processes has arguably become the main challenge for evidence-based policymaking. How can policymakers predict whether a policy will have the same impact in their context as it did elsewhere, and how should this influence the design and implementation of policy? This paper introduces a simple and flexible framework to address these questions of external validity and policy adaptation. I show that failures of external validity arise from an interaction between a policy’s theory of change and a dimension of the context in which it is being implemented, and develop a method of “mechanism mapping” that maps a policy’s theory of change against salient contextual assumptions to identify external validity problems and suggest appropriate policy adaptations. In deciding whether and how to adapt a policy in a new context, I show there is a fundamental informational trade-o↵ between the strength and relevance of evidence on the policy from other contexts and the policymaker’s knowledge of the local context. This trade-o↵ can guide policymakers’ judgments about whether policies should be copied exactly from elsewhere, adapted, or invented anew.

Extreme Teaming: Lessons in Complex, Cross-Sector Leadership - Edmondson, Amy C.; Harvey, Jean-Francois (Emerald Group Publishing Limited, 2017)
Extreme Teaming Lessons in Complex, Cross-Sector Leadership Today’s global enterprises increasingly involve collaborative work by teams of experts operating across different professions, organizations, and industries. Extreme Teaming provides new insights into the world of complex, cross industry projects and the ways they must be managed. Leading experts Amy Edmondson and Jean-François Harvey analyze contemporary cases that expose the complex demands of cross-boundary collaboration on management, and inform our understanding of teams. Containing powerful insights and practical guidelines that allow managers to bridge professional divides and organizational boundaries in order to work together effectively, this is a new exploration of the challenges involved in today’s global enterprises. The authors demonstrate that the work done in the modern organization is less and less about looking inward and creating strong teams inside the company, and more about teaming across boundaries – that often are in flux. Extreme Teaming is a must-read book for all courses related to leading open innovation; teamwork and collaboration; project management; and cross-boundary work.

Facilitating collaborative problem solving with human-centred design: the Making All Voices Count governance programme in 12 countries of Africa and Asia - Jackson, Carl (Knowledge Management for Development Journal, 2015)
Keywords: IMPORTANT

Facilitator's Guide to Participatory Decision-Making - Kaner, Sam (John Wiley & Sons, 2014)
Unleash the transformative power of face to face groups The third edition of this ground–breaking book continues to advance its mission to support groups to do their best thinking. It demonstrates that meetings can be much more than merely an occasion for solving a problem or creating a plan. Every well–facilitated meeting is also an opportunity to stretch and develop the perspectives of the individual members, thereby building the strength and capacity of the group as a whole. This fully updated edition of The Facilitator′s Guide to Participatory Decision–Making guides readers through the struggle and the satisfaction of putting participatory values into practice, helping them to fulfill the promise of effective group decision–making. With previous editions already embraced by business and community leaders and consulting professionals around the world, this new book is even more insightful and easy to use. New for this edition: 60 pages of brand new skills and tools Many new case examples Major expansion and reorganization of the advanced sections of the book. New chapter: Teaching A Group About Group Dynamics Doubled in size: Classic Facilitator Challenges. Substantially improved: Designing Realistic Agendas now three chapters, with wise, insightful answers to the most vexing questions about meeting design.

Facilities deserve a place in development - Lacy, Jacqui de (Devpolicy Blog from the Development Policy Centre, 2017)
There are different ways to deliver aid programs, DFAT often uses facilities in its larger relationships, what are the rationales and risks behind this?

Failing Forward: How CARE is focusing on what goes wrong to improve impact - Janoch, Emily (2019 CLA Case Competition, 2019)
CARE's Failing Forward initiative is sparking opportunities to showcase the ideas that don't work so we can spend more time implementing the ones that do. It's changing the conversation inside the organization, and leading to changes in the way we design and implement programs. It's also allowing us to make connections across a global portfolio of more than 900 projects, and with new and different partners to learn from their experience.

Failing Forward: How CARE is Focusing on What Goes Wrong to Improve Impact - Janoch, Emily (CLA Case Competition, 2019)
CARE's Failing Forward initiative is sparking opportunities to showcase the ideas that don't work so we can spend more time implementing the ones that do. It's changing the conversation inside the organization, and leading to changes in the way we design and implement programs.

Fall in love with the solution, not the problem - Quaggiotto, Giulio; Leurs, Bas; Hazeldine, Shaun (Nesta, 2016)
Three strategies for international development organisations to solve problems without defining them.

Feasibility Study: Crowdsourcing High-frequency food price data in rural Indonesia - UN Global Pulse (2015)

Feedback Commons (Keystone Accountability, 2018)

FeverBee’s Online Community Strategy Template - FeverBee (FeverBee, 2018)

Five tips for evaluating your impact in international development - Pasanen, Tiina (The Guardian, 2015)
How can impact evaluations actually be helpful? These top tips will save development professionals time, energy and money. International development can be messy – with uncertain, complex settings, and multiple partners with different interests, goals and capacities. At the same time, we are under increasing pressure to demonstrate impact. We have to show that our projects have made a real change in people’s lives and that donor or taxpayer money hasn’t been wasted. But impact evaluations, which are often seen as a solution to this, aren’t always used. And in some cases they are unhelpful. So how can we improve the quality of impact evaluations, so that they produce results that are useful? This is where evaluability assessments come in. By asking whether we should evaluate a project – and if so, when and how – these assessments can improve the quality of impact evaluations. Here are five other reasons to assess evaluability before starting an impact evaluation.

Five Ways to Make a Difference: Perceptions of Practitioners Working in Urban Neighborhoods - Durose, Catherine; van Hulst, Merlijn; Jeffares, Stephen; Escobar, Oliver; Agger, Annika; de Graaf, Laurens (Public Administration Review, 2015)

FOCEVAL – Promoting Evaluation Capacities in Costa Rica:: Smart(er) Implementation with Capacity WORKS? - Storm, Sabrina (Transformation, Politics and Implementation, 2017)
The National Monitoring and Evaluation System of Costa Rica and its corresponding laws were established during the 1990s. Since then, the country has endeavored to implement monitoring and evaluation (M&E) activities as part of its public policy framework. Nevertheless, hardly any systematic evaluations had been conducted, and monitoring activities had been reduced mainly to the institutional self-reporting of implementation compliance. Persisting regional disparities and growing levels of inequality among the population raised the level of pressure on the government to present reliable information on the effectiveness of public interventions. Hence, results-oriented evaluations were promoted by some Costa Rican departments as

Follow the rabbit: Field Guide to Systemic Design - CoLab (Government of Alberta, 2018)
This Field Guide is designed to support budding systemic designers to facilitate and lead systemic design projects. It’s designed for systemic designers who have basic familiarity with SD concepts and are looking for some practical tips and tools to put theory into action. While this Guide was designed with Government of Alberta staff in mind, the content applies across a range of issue areas, sectors, and intersections. The Guide goes through a systemic design project from concept to implementation. It takes you through the workshop planning process, and discusses workshop roles and client relations. In the FAQs, you’ll find explanations to some commonly asked questions about systemic design concepts to help you introduce others to SD and bring them along with you. You’ll get the most out of this field guide if you approach it with a systemic design mindset – open, willing to try things, a desire to learn, and a ‘yes, and’ orientation. Think of this field guide as an early stop in your systemic design journey: it’s not the end of your learning – it’s a jumping off point. In the Additional Resources section, you can connect to more in-depth explorations of concepts and practical applications.

Food security indicators - FAO (ESS Website, 2018)

Foreign policy and complex adaptive systems: Exploring new paradigms for analysis and action - Frej, William; Ramalingam, Ben (Santa Fe Institute, 2011)

Foresight and International Development* - Bingley, Kate (IDS Bulletin, 2016)
This article provides an overview of the use of foresight-type approaches and techniques in policy-related work in international development. It draws primarily on published and grey literatures, as well as select interviews with foresight practitioners. It begins with a brief introduction to the approaches and tools used in the field of strategic foresight, and then a broad mapping of the foresight landscape as relevant to international development. It provides reflections on the evidence of use and impact of foresight initiatives, and makes suggestions around future directions for foresight in international development.

Forget the money: De-linking technical assistance - Manuel, Clare (Synthesis Paper, 4, 2016)

Forward Look - A vision for the World Bank Group in 2030 - World Bank (World Bank, 2016)

Fostering Innovation and Entrepreneurialism in Public Sector Reform - Cummings, Clare (Public Administration and Development, 2015)
Summary There is growing recognition within the international development sector that there is a need for a new, more effective approach to engaging in public sector reform. This article builds on an emerging body of work that advocates more entrepreneurial and adaptive public sector reform programming. Drawing on knowledge and theory from public sector management, psychology and entrepreneurialism, this article aims to understand what motivates public sector workers to work entrepreneurially and suggest how these ideas can be applied to the way in which international development agencies engage in public sector reform work. This requires a shift in thinking from predesigned, large, externally led programmes promoting international best practice to interventions in which agencies adopt the role of a facilitator, helping to establish the enabling conditions for local partners to work entrepreneurially, developing their own solutions to the problems that they identify in their work. Copyright ? 2015 John Wiley & Sons, Ltd.
Keywords: Development; Entrepreneurship; Innovation; Public sector reform; motivation

Fostering Learning and Effectiveness through Collaboration: A New Collaborative on Transparency, Accountability, and Participation (TAI, 2018)
What are the features, values, and practices of effective learning organizations? How do learning practices contribute to more effective programming? And, how can collaborations between academics, researchers and practitioners better support learning organizations in the global South?  These are just a few of the questions that a new global learning collaborative seeks to explore. In …

Fostering Learning in Large Programmes and Portfolios: Emerging Lessons from Climate Change and Sustainable Development - Harvey, Blane; Pasanen, Tiina; Pollard, Alison; Raybould, Julia (Sustainability, 2017)
In ﬁelds like climate and development, where the challenges being addressed can be described as “wicked”, learning is key to successful programming. Useful practical and theoretical work is being undertaken to better understand the role of reﬂexive learning in bringing together different knowledge to address complex problems like climate change. Through a review of practical cases and learning theories commonly used in the areas of resilience, climate change adaptation and environmental management, this article: (i) reviews the theories that have shaped approaches to reﬂexive learning in large, highly-distributed climate change and resilience-building programmes for development; and (ii) conducts a comparative learning review of key challenges and lessons emerging from early efforts to promote and integrate reﬂexive learning processes in programmes of this nature. Using a case study approach, the authors focus on early efforts made in four large, inter-related (or nested) programmes to establish, integrate and sustain learning processes and systems. Eight themes emerged from the review and are considered from the perspective of learning programmes as emergent communities of practice. By investigating how these themes play out in the nested programming, the paper contributes to the limited existing body of evidence on learning in large climate change programmes and identiﬁes areas where future efforts might focus.

Fostering Systems Change - Gopal, Srik; Kania, John (2015)
Five simple rules for foundations seeking to create lasting social change.

Framework for the Biodiversity Cross-Mission Learning Program - Environmental Incentives; Foundations of Success; ICF International (Measuring Impact, 2015)
USAID’s Bureau for Economic Growth, Education, and the Environment/Office of Forestry and Biodiversity (E3/FAB) launched the five-year Measuring Impact program (MI) to help strengthen USAID’s biodiversity programs by building the capacity of the Agency to design, monitor, and evaluate effective programs and by enhancing the evidence base that informs programming decisions. A core programmatic strategy of MI is to “Improve Biodiversity Conservation Approaches” with the life-of-project (LOP) objective that E3/FAB and the Missions engaged with MI have a greater understanding of conditions under which commonly deployed conservation strategic approaches are effective. To accomplish this, MI is working with E3/FAB to design and implement a Biodiversity Cross-Mission Learning Program (Learning Program) to systematically capture and share lessons on theories of change1 (TOCs) for common conservation strategic approaches in the USAID biodiversity portfolio.

From a last mile century to a first mile revolution: the future is OSLO - Indy Johar (Medium, 2013)
The Future is OSLO

From analysis to action: operationalising learning and adaptation in Savings at the Frontier - Salehi, Yusef (2016)
Keywords: IMPORTANT

From best practice to best fit: understanding and navigating wicked problems in international development - Ramalingam, Ben; Laric, Miguel; Primrose, John (Working Paper, 2014)

From Design Thinking to Systems Change - Conway, Rowan; Masters, Jeff; Thorold, Jake (RSA, 2017)
This report takes the Small Business Research Initiative (SBRI), a twophase pre-procurement innovation programme that aims to match social challenges with new ideas, as its primary case study. It suggests augmenting the excellent design thinking deployed through SBRI with a think like a system, act like an entrepreneur lens in order to drive better social outcomes from SBRI-originating innovations. Programmes like SBRI have great potential to drive change and address pressing challenges, but must be guided by a more developed understanding of how change happens. The stakes are too high to not raise our game when it comes to social innovation. Wicked problems can be overcome but will require sophisticated theories of change able to account for the complexity and unpredictability of modern life. We offer think like a system, act like an entrepreneur as a contribution to this effort. This report is based on the following research: •• Desk based review of relevant literature; •• A series of structured interviews with participants in SBRI competitions; and •• An expert round table that brought together policymakers, commissioners, procurement experts and entrepreneurs to consider and develop initial findings.
Keywords: ⛔ No DOI found

From ‘Feedback Loops’ to ‘Responsive Governance’ - McGee, Rosie (Making All Voices Count, 2014)
Part of a sequenced learning process for MAVC's Research and Evidence Component, the summary of a thematic discussion led by researcher Rosemary McGee.

From political economy analysis to doing development differently: a learning experience - Booth, David; Harris, Daniel; Wild, Leni (ODI, 2016)
A study of how development efforts can be strengthened by an awareness of political economy, reflecting on the experiences of PoGo policy researchers.
Keywords: IMPORTANT

From poverty to power: how active citizens and effective states can change the world - Green, Duncan (2012)
From Poverty to Power From Poverty to Power argues that a radical redistribution of power, opportunities, and assets rather than traditional models of charitable or government aid is required to break the cycle of poverty and inequality. The forces driving this transformation are active citizens and effective states. Published in association with Oxfam GB. Full description

From principle to practice: Implementing the Principles for Digital Development - Waugaman, Adele (2016)
The Principles for Digital Development (download PDF here) find their roots in the efforts of individuals, development organizations, and donors alike who have called for a more concerted effort by donors and implementing partners to institutionalize lessons learned in the use of information and communication technologies (ICTs) in development projects.
Keywords: IMPORTANT

From purposeful to meaningful adaptive programming: how about adaptive operations? - Nixon, Nicola (Devpolicy Blog from the Development Policy Centre, 2019)
Embracing adaptive programming approaches but sticking with rigid, traditional systems-based operations? It's time to take a long hard look at all the boring stuff, says Nicola Nixon.

From Silos to Synergy: Learning from politically informed, gender aware programs - Derbyshire, Helen; Siow, Orlanda; Gibson, Sam; Hudson, David; Roche, Chris (DLP Program, 2018)

From Thinking Politically To Working Politically: Are We Really Doing Development Any Differently? (2019)
In June it will be seven years since the Center for Global Development published the Problem Driven Iterative Adaptation (PDIA) paper. Few academic papers have had such an impact on development thinking. The question is whether the paper – and subsequent debate and experimentation - have had a demonstrable and beneficial impact on development practice. Experience to date suggests thinking politically is easier than working politically. Practitioners in development agencies, governments and contractors find that convincing theoretical literature can be tricky to operationalize in a fractious environment. Still, a growing number of programs have set out to be politically informed in design and politically savvy in implementation. Engaging the reality of politics creates the potential to promote change. To consider these issues, Abt Associates is hosting a workshop for international development practitioners in Washington, DC, on June 18th, 2019, to discuss the successes and failures in operationalizing this agenda. The session will allow practitioners to discuss experiences in TWP and propose solutions or approaches. Please join us in this Innovations in Governance series finale where four internationally renowned thinkers and doers will join us to offer their perspectives.

Future Democracies - Laboratory of Collective Intelligence for Participatory Democracy - Yago Bermejo (MediaLab Prado, 2019)
The Laboratory of Collective Intelligence for Participatory Democracy (2016-2019) is a project that arose out of Medialab Prado in coordination with the Government Area of Citizen Participation, Transparency and Open Government of the city of Madrid. Its work has been very connected with the analysis, reflection and innovation found on the digital participation platform Decide Madrid. The project has also organised many workshops and conferences that have brought together hundreds of people from the world of participatory democracy. This publication summarises the project’s core working principles as well as the open activities it has hosted over the course of these three years. These pilot experiences offer a possible vision of the future of democratic governance.
Keywords: Democracy

Future oriented conservation: knowledge governance, uncertainty and learning - Wyborn, Carina; van Kerkhoff, Lorrae; Dunlop, Michael; Dudley, Nigel; Guevara, Oscar (Biodiversity and Conservation, 2016)
Despite signiﬁcant progress in understanding climate risks, adaptation efforts in biodiversity conservation remain limited. Adaptation requires addressing immediate conservation threats while also attending to long term, highly uncertain and potentially transformative future changes. To date, conservation research has focused more on projecting climate impacts and identifying possible strategies, rather than understanding how governance enables or constrains adaptation actions. We outline an approach to futureoriented conservation that combines the capacities to anticipate future ecological change; to understand the implications of that change for social, political and ecological values; and the ability to engage with the governance (and politics) of adaptation. Our approach builds on the adaptive management and governance literature, however we explicitly address the (often contested) rules, knowledge and values that enable or constrain adaptation. We call for a broader focus that extends beyond technical approaches to acknowledge the sociopolitical challenges inherent to adaptation. More importantly, we suggest that conservation policy makers and practitioners can use this approach to facilitate learning and adaptation in the context of complexity, transformational change and uncertainty.

Gendered voices for climate action - Fisher, Susannah; Shakya, Clare (iied Working Paper, 2018)
Including local women and men who have experienced the impacts of climate change in decision-making forums is key to developing a just process hearing local voices and experiences. The Paris Agreement signs up the international community to delivering countrydriven, gender-responsive climate action. Including local women and men will be critical to this transition and to achieving the scale and pace of change needed through both procedural and distributional justice. However, there is scarce evidence on how these voices and experiences can impact on and strengthen climate responses, or how best to enable and facilitate this participation.

Getting beneath the surface in program planning, monitoring and evaluation: Learning from use of participatory action research and theory of change in the CGIAR Research Program on Aquatic Agricultural Systems - Apgar, J Marina; Allen, Will; Albert, Joelle; Douthwaite, Boru; Paz Ybarnegaray, Rodrigo; Lunda, Jeston (Action Research, 2017)
Many rural poor and marginalized people strive to make a living in social-ecological systems that are characterized by multiple and often inequitable interactions across agents, scale and space. Uncertainty and inequality in such systems require research and development interventions to be adaptive, support learning and to engage with underlying drivers of poverty. Such complexity-aware approaches to planning, monitoring and evaluating development interventions are gaining strength, yet, there is still little empirical evidence of what it takes to implement them in practice. In this paper, we share learning from an agricultural research program that used participatory action research and theory of change to foster learning and support transformative change in aquatic agricultural systems. We reflect on our use of critical reflection within participatory agricultural research interventions, and our use of theory of change to collectively surface and revisit assumptions about how change happens. We share learning on the importance of being strengths-based in engaging stakeholders across scales and building a common goal as a starting point, and then staging a more critical practice as capacity is built and opportunities for digging deeper emerge.

Getting good at disruption in an uncertain world: Insights from Southern NGO leaders - Buckley, Lila; Ward, Halina (International Institute for Environment and Development, 2016)
Keywords: IMPORTANT

Getting Past the Rhetoric #1: Tendering for ‘Thinking and Working Politically’ in Large Facilities - Tyrrel, Lavinia (Abt Associates - Governance Soapbox, 2018)

Getting Past the Rhetoric #2: Managing for ‘Thinking and Working Politically’ in Large Facilities - Tyrrel, Lavinia (Abt Associates - Governance Soapbox, 2018)

Getting Strategic about Technology and Accountability: MAVC Learning and Inspiration Event in Manila - Halloran, Brendan (Politics, Governance and Development, 2016)
Two years ago, Jonathan Fox released a widely read and well-received paper looking at the evidence for citizen-led accountability.  In it, he noted the proliferation of short-term, isolated interve…

Getting strategic: vertically integrated approaches - Aceron, Joy; Isaac, Francis (Institute of Development Studies, 2016)
Holding power to account requires understanding where power lies and how it is exercised. It entails understanding how decisions are made, who makes them and what decision criteria are used to make them. Vertically integrated civil society action takes into account how power is exercised and how decisions are made in a given policy, programme or process.

GIZ's Evaluation Policy: Principles, guidelines and requirements of our evaluation practice - GIZ (GIZ, 2018)

GIZ’s evaluation system - GIZ Evaluation Unit (GIZ GmbH, 2018)
This provides a general description of GIZ’s new evaluation system, which is designed to implement our new policy and to achieve the objectives of evaluation reform. This general description is accompanied by two other documents that provide supplementary details of GIZ’s key evaluation instruments: project evaluations for BMZ business and corporate strategic evaluations. They are geared in particular to GIZ staff members who commission, implement or support evaluations and to evaluators who conduct evaluations on behalf of GIZ as well as repre-sentatives of commissioning parties, clients and cooperation partners. The current versions are very much a ‘work in progress’ in many regards. We are well aware that we will only be able to meet our own high expectations of our evaluations if we continue to develop their quality in dialogue with project staff, our commissioning parties, clients and partners and with evaluation practitioners and experts from the academic, scientific and research community. We look forward to taking on this task and to sharing our experience with all interested parties.

GLAM Monitoring and Learning Strategy (internal document) - GLAM (ODI, 2018)

Global Affairs Canada: an approach to development innovation - GAC (Global Affairs Canada, 2018)
For Global Affairs Canada, development innovation includes new or improved business models, policy practices, approaches, technologies, behavioral insights or ways of delivering products and services that benefit and empower the poorest and most vulnerable people in developing countries.

Global Delivery at the World Bank Group - GDI (Global Delivery Initiative, 2014)

Global governance and local peace: accountability and performance in international peacebuilding - Campbell, Susanna (Cambridge University Press, 2018)
Local peacebuilding and global accountability -- The country context--Burundi from 1999 to 2014 -- Ingos in peacebuilding--globally unaccountable, locally adaptive -- International organizations in peacebuilding--globally accountable, locally constrained -- Bilateral development donors--accountable for global targets, not local change

Global Learning for Adaptive Management (GLAM) - DFID (DFID Development Tracker, 2016)
To improve the value for money and impact of development interventions by promoting the use of better monitoring, evaluation and learning strategies that support development programmes to gather and respond to emergent evidence on the effects of interventions, as well as changes in the development context. The Global Learning for Adpative Management programme will make available technical assistance to DFID, USAID and partners to develop this ‘adaptive management’ approach, as well as build an evidence base to understand links between the use of adaptive management and more effective, efficient, relevant and sustainable development interventions.

Global mapping of technology for transparency and accountability - Avila, Renata; Feigenblatt, Hazel; Heacock, Rebekah; Heller, Nathaniel (Transparency & Accountability Initiative, 2010)

Glossary of Knowledge Management - Serrat, Olivier (Knowledge Solutions: Tools, Methods, and Approaches to Drive Organizational Performance, 2017)
The knowledge management discipline can be cryptic. These Knowledge Solutions define its most common concepts in simple terms.
Keywords: Definitions; Knowledge management; Taxonomy; Terminology

Going Local 2.0: How to Reform Development Agencies to Make Localized Aid More Than Talk - Ang, Yuen Yuen (Stanford Social Innovation Review, 2018)
More and more global aid agencies believe they should replace one-size-fits-all best practices with locally tailored solutions, but they must shift from just agreeing to “go local” to preparing development experts for the task by taking on three major problems with their internal practices.

Going vertical: citizen-led reform campaigns in the Philippines - Aceron, Joy; Isaac, Francis (Making All Voices Count Research Report, 2016)
The Philippines has a long history of state–society engagement to introduce reforms in government and politics. Forces from civil society and social movements have interfaced with reform-oriented leaders in government on a range of social accountability initiatives – to make governance more responsive, to introduce policy reforms, and to make government more accountable. Several theoretical propositions on which strategic approaches work best for social accountability initiatives have been put forward – including the idea of vertically integrated civil society monitoring and advocacy. This multi-authored research report uses vertical integration as a framework for examining seven successful civil society social accountability initiatives in the Philippines, looking at what made them successful, and how the gains they realised can be deepened and sustained.

Governance and Service Delivery: Practical Applications of Social Accountability Across Sectors - Wetterberg, Anna; Brinkerhoff, Derick W.; Hertz, Jana C. (RTI Press, 2016)
The six case studies in Governance and Service Delivery: Practical Applications of Social Accountability Across Sectors illustrate the multiple ways citizen participation in accountability – called social accountability – can lead to positive effects on governance, citizen empowerment, and service delivery. Drawing on their extensive experience implementing international donor-funded programs and projects, the authors examine six recent RTI International projects in Africa and Asia. The analysis focuses on both intended and actual effects, and it unpacks the influence of context on implementation and outcomes achieved, as well as on prospects for sustainability. Donors, academics, policy makers, practitioners and others interested in sustainable development and governance will find this systematic review invaluable.

Governance for Growth in Vanuatu: Review of a decade of thinking and working politically - Hadley, Sierd; Tilley, Helen (ODI, 2017)

Grounded Theory Method in HCI and CSCW - Muller, Michael J; Kogan, Sandra (IBM Center for Social Software, 2010)

Grow - Scale - Impact: How to help inclusive business achieve scale - Pérez Castro, Andrea A.; Tora, Krisztina (GIZ, 2016)

Guidance on tasks and deliverables for different evaluation phases - Methods Lab (2015)
This tool describes the five key phases of evaluation, from planning and design, to implementation and communication of results. It provides a list of the main tasks and deliverables for each phase, intended for use by anyone managing an impact evaluation. This tool was developed by Irene Guijt, Simon Hearn, Tiina Pasanen and Patricia Rogers for use in Methods Lab projects. It follows to some extent the BetterEvaluation Rainbow Framework.

Guidance on using the revised Logical Framework - How to note - DFID (DFID, 2011)
The principal changes to the logframe from the earlier (2008) 4 x 4 matrix are: - Objectively Verifiable Indicators (OVIs) have been separated into their component elements (Indicator, Baseline and Target), and Milestones added. - Means of Verification has been renamed ‘Source’. - Inputs are now quantified in terms of funds (expressed in Sterling for DFID and all partners) and use of DFID staff time (expressed as annual Full-Time Equivalents (FTEs). - A DFID Share box now indicates the financial value of DFID’s Inputs as a percentage of the whole. - Assumptions are shown at Purpose and Output level only. - Risks are shown at Activities level only, but also rated at Output level; - At the Output level, the Impact Weighting is now shown in the logframe together with a Risk Rating for individual Outputs. - Activities are now shown separately (so do not normally appear in the logframe sent for approval), although they can be added to the logframe if this is more suitable for your purposes. - A renewed emphasis on the use of disaggregated beneficiary data within indicators, baselines and targets. The Logical Framework (logframe) was significantly re-designed in February 2009. In January 2011 a slightly amended logframe template was introduced at the same time as the launch of the new DFID Business Case. Given the extent of changes that took place in February 2009 it was only necessary to make minor amendments in January 2011. These amendments are as follows: - Results Chain terminology aligned across DFID (the terms Goal and Purpose in the old logframe template have been replaced by Impact and Outcome) - Rows added to allow achieved results to be captured alongside the planned results determined at project design stage - Word version of the logframe template removed – excel is the preferred format for logical frameworks from January 2011 - Indicator numbering introduced within logframe template

Guidance: Rigor and Reproducibility in Grant Applications | grants.nih.gov - NIH (National Institutes of Health, 2018)

Guide to Evaluating Participatory Processes - Practical guide - Parés, Marc (2013)
Guide to Evaluating Participatory Processes - Practical guide
Keywords: Academia; Biology; Computer Science; Earth Sciences; Geography; History; Law; Math; Medicine; Philosophy; Physics; Political science; Psychology; Religion; Research; economics; universities

Guidelines and good practices for evidence-informed policy-making in a government department - Wills, Alf; Tshangela, Mapula; Shaxson, Louise; Datta, Ajoy; Matomela, Bongani (2016)

Guidelines for designing and monitoring social accountability interventions - Franco, Erika Lopez; Shankland, Alex (IDS, 2018)

Guidelines for good market development program design. A managers’ perspective - Bekkers, Harald; Roggekamp, Peter (BEAM Exchange, 2014)
Good programme design is a key factor contributing to the success of market development programmes. Unfortunately, too many current designs have major flaws which prevent programmes becoming successful even before they get started. This document considers what makes market development programmes consistently successful and how to prevent programme design from being a hurdle to sustainable and efficient impact as scale.

Guidelines for the Monitoring System for Development-Related Changes (MERV) - SDC (SDC, 2013)

Guidelines for writing a case study on implementation (Book Project: Smart Implementation in Governance) - Kirsch, Renate (GIZ GmbH, 2015)

Guiding questions to help narrow the scope of an evaluation - Methods Lab (2015)
Time and budget constraints can mean that programmes are not able to assess all possible evaluation questions; this is especially true for multi-component or multi-site programmes operating in challenging environments. This tool identifies areas of enquiry to help programmes prioritise the number of questions and measurement indicators used. This tool was developed by Anne Buffardi for use in in Methods Lab projects.

Handbook of Democratic Innovation and Governance - Elstub, Stephen; Escobar, Oliver (Edward Elgar Publishing, 2019)
Democratic innovations are proliferating in politics, governance, policy, and public administration. These new processes of public participation are reimagining the relationship between citizens and institutions. This Handbook advances understanding of democratic innovations, in theory and practice, by critically reviewing their importance throughout the world. The overarching themes are a focus on citizens and their relationship to these innovations, and the resulting effects on political equality. The Handbook therefore offers a definitive overview of existing research on democratic innovations, while also setting the agenda for future research and practice.
Keywords: Democracy; Public Affairs & Administration

Has Kenya’s ICT revolution triggered more citizen participation? - Salome, Nyambura (MAVC Research Briefing, 2016)
Lessons from policy in Ghana, Kenya, South Africa and Tanzania
Keywords: IMPORTANT

Help! I’m hiring new staff and I want them to work adaptively. A guide to hiring adaptive employees. - Salib, Monalisa (USAID LEARN, 2019)
If you are involved in hiring, this tool can help you increase your chances of selecting staff members skilled in adaptive management. It will help answer the questions: Which competencies should I recruit for in order to hire more adaptive employees? Which desired qualifications should I incorporate into position descriptions to attract adaptive employees? Which interview questions should I ask to screen for adaptive employee competencies? You should use this tool when you decide to hire a new individual - whether that be a direct hire, contractor, or consultant - or when you are changing an individual’s current position description or scope of work. Specifically, this tool will help you determine which qualifications are most important for the position and offers interview questions you can use with candidates. --- You’re in luck! We just released a new guide for hiring adaptive employees. This visual and user-friendly tool will help increase your chances of selecting staff skilled in adaptive management. Here are the details: What do you mean by “work adaptively” or an “adaptive employee”? An adaptive employee is an individual who, in collaboration with relevant stakeholders, systematically acquires and uses knowledge to make decisions and adjustments in their work in order to achieve greater impact. Why focus on adaptive employees? Why does it matter? With industries, technologies, and organizations in a near constant state of flux, leaders are recognizing the importance of adaptability as a critical capacity. Because of this, it’s not surprising that a recent LinkedIn report found that adaptability was the most important soft skill hiring managers were screening for. In addition, evidence shows that teams that apply more data-driven and adaptive leadership practices perform better than those that focus less on these practices. It makes sense, then, that managers want to recruit more team members who are skilled in adaptive management - it helps achieve results. Who should use this tool? Anyone involved in hiring, responsible for developing or approving position descriptions, participating in interviews, and/or approving new hires. When should I use this tool? Use it as soon as you decide to hire a new staff person or when you’re adjusting existing position descriptions. What does the tool help me do? It will help you think through: Which competencies should I recruit for in order to hire more adaptive employees? Which desired qualifications should I incorporate into position descriptions to attract adaptive employees? (You can even copy and paste qualification language from the tool into scopes of work!) Which interview questions should I ask to screen for adaptive employee competencies? (You can copy and paste interview questions into your screening, interview, and reference check protocols!) Has the tool been tested? Yes, staff within USAID’s Global Development Lab and on the USAID LEARN contract tested the tool, providing feedback on the minimum viable product and subsequent versions. Testers confirmed finding the tool useful, and gave it a 9/10 score when asked if they would recommend it to colleagues. Some specific feedback from testers included: "The tool prompts deeper thought on what you might need and want in a job candidate" and "All [of the interview questions I used from the tool] worked well." What about the existing staff? How can they gain greater skills in adaptive management? Existing USAID staff and implementing partners can access online training in collaborating, learning, and adapting (CLA) and find resources in the CLA Toolkit. USAID staff also have access to an internal CLA community of practice and five-day, in-person CLA training.

HERMES (Highly Extensible Resource for Modeling Event-Driven Supply Chains) - HERMES (hermes.psc.edu, 2015)
Systems modeling is a tool for policymakers and program managers to capture all the direct and indirect effects of changes to a supply chain, identify sustainable solutions to the root causes of issues and save time, effort and resources in costly trial and error. Without systems modeling, evaluating the supply chain, identifying gaps, and implementing solutions can be insufficient, unsustainable and costly. Stock2 Vaccine supply chains are complex systems, comprising all the equipment, personnel, policies and processes needed to deliver a vaccine from its point of origin to the population. Understanding how the various components of a vaccine supply chain interact with each other is critical to evaluating supply chain function, identifying the root causes of issues and formulating sustainable solutions. HERMES (Highly Extensible Resource for Modeling Event-Driven Supply Chains) is a software program that allows users to generate a detailed computer simulation model of a supply chain

Hospicing The Old - Robinson, Cassie (Stewarding Loss, 2019)
In 2010 I was introduced to the Berkana Institutes’s Two Loop model, and I come back to it again and again. As I’ve moved across different…

How Adaptive Management is challenging the monitoring and evaluation of complex programmes - Simister, Nigel (INTRAC, 2018)
By Nigel Simister Adaptive management is a broad approach designed to support development or humanitarian programmes in complex or uncertain …

How can we be more adaptive? AdaptScan identifies critical areas for improvement - Mercy Corps (Mercy Corps & IRC, 2017)
Keywords: IMPORTANT

How Change Happens - Green, Duncan (OUP Oxford, 2016)
Human society is full of would-be 'change agents', a restless mix of campaigners, lobbyists, and officials, both individuals and organizations, set on transforming the world. They want to improve public services, reform laws and regulations, guarantee human rights, get a fairer deal for those on the sharp end, achieve greater recognition for any number of issues, or simply be treated with respect. Striking then, that not many universities have a Department of Change Studies, to which social activists can turn for advice and inspiration. Instead, scholarly discussions of change are fragmented with few conversations crossing disciplinary boundaries, rarely making it onto the radars of those actively seeking change. This book bridges the gap between academia and practice, bringing together the best research from a range of academic disciplines and the evolving practical understanding of activists to explore the topic of social and political change. Drawing on many first-hand examples from the global experience of Oxfam, one of the world's largest social justice NGOs, as well as the author's insights from studying and working on international development, it tests ideas on How Change Happens and offers the latest thinking on what works to achieve progressive change. This is an open access title available under the terms of a CC BY-NC-ND 4.0 International licence. It is available as a free PDF download from OUP and selected open access locations.

How China Escaped the Poverty Trap - Ang, Yuen Yuen (Cornell University Press, 2016)
Before markets opened in 1978, China was an impoverished planned economy governed by a Maoist bureaucracy. In just three decades it evolved into the world's second-largest economy and is today guided by highly entrepreneurial bureaucrats. In How China Escaped the Poverty Trap, Yuen Yuen Ang explains this astonishing metamorphosis. Rather than insist that either strong institutions of good governance foster markets or that growth enables good governance, Ang lays out a new, dynamic framework for understanding development broadly. Successful development, she contends, is a coevolutionary process in which markets and governments mutually adapt.By mapping this coevolution, Ang reveals a startling conclusion: poor and weak countries can escape the poverty trap by first harnessing weak institutions―features that defy norms of good governance―to build markets. Further, she stresses that adaptive processes, though essential for development, do not automatically occur. Highlighting three universal roadblocks to adaptation, Ang identifies how Chinese reformers crafted enabling conditions for effective improvisation.How China Escaped the Poverty Trap offers the most complete synthesis to date of the numerous interacting forces that have shaped China’s dramatic makeover and the problems it faces today. Looking beyond China, Ang also traces the coevolutionary sequence of development in late medieval Europe, antebellum United States, and contemporary Nigeria, and finds surprising parallels among these otherwise disparate cases. Indispensable to all who care about development, this groundbreaking book challenges the convention of linear thinking and points to an alternative path out of poverty traps.
Keywords: IMPORTANT

How Decent is Decent Work? Using SenseMaker to understand workers’ experiences - Mager, Franziska; Smith, Becca; Gujit, Irene (Oxfam, 2018)

How DFID can better manage complexity in development programming - Sharp, Samuel; Valters, Craig; Whitty, Brendan (ODI, 2019)
The United Kingdom’s (UK) Department for International Development (DFID) is an ambitious government department that is committed to reducing poverty and conflict overseas. Many of the issues on which DFID works are complex; whether focused on climate change, gender equality, health or other priorities, simple solutions rarely exist. And to tackle these complex challenges, DFID staff must interact with unpredictable systems of political, organisational and individual behaviours and incentives. There is a risk that complex problems spur more complicated programmes; that the complexity of development challenges is addressed through designing programmes with too many projects and implementers. While there may be valid reasons for this, too many of these complicated programmes will overburden staff. This briefing note is the outcome of an ongoing process within DFID to confront these issues and answer the question: how can DFID design and manage programmes to address complex development challenges without creating too much staff workload? Key messages DFID deals with complex problems, which require flexible systems to support testing, learning and adaptation. • Complex problems do not necessarily require workload-heavy delivery structures, but simpler approaches depend on delivery partners’ experience and competence. • However, programmes that tackle complex problems do require more hands-on engagement and face more workload from inflexible compliance requirements. We suggest that DFID and similar agencies: • Pay closer attention to delivery options in programme design, making use of existing options where possible and, where not, fostering local organisations through long-term investments. • Encourage programme designers to articulate how ambition matches resources and consider ‘good enough’ design. • Reduce the burden of compliance by cultivating partner autonomy, reassessing results and valuefor-money requirements and promoting more flexible contracting and procurement. • Plan and prioritise management resources across a portfolio of programmes to make sure they can be focused in the right areas, where the complexity of the problem requires greater engagement.

How DFID learns - ICAI (Independent Commission for Aid Impact (ICAI), 2014)
Excellent learning is essential for UK aid to achieve maximum impact and value for money. We take learning to mean the extent to which DFID uses information and experience to influence its decisions. Each ICAI review assesses how well learning takes place. Our reports to date indicate a mixed performance. This review seeks to identify the way DFID learns and what inhibits it from doing so consistently. We drew on our reviews, assessed data from DFID’s own surveys and carried out interviews inside and outside the department. Review DFID generates considerable volumes of information, much of which, such as funded research, is publicly available. DFID does not clearly or consistently link this investment to how it can deliver better impact. We made five recommendations and gave an amber-red score. Findings DFID does not clearly identify how its investment in learning links to its performance and delivering better impact. DFID has the potential to be excellent at organisational learning if its best practices become common. DFID staff learn well as individuals. They are highly motivated and DFID provides opportunities and resources for them to learn. DFID is not yet, however, managing all the elements that contribute to how it learns as a single, integrated system. DFID does not review the costs, benefits and impact of learning. Insufficient priority is placed on learning during implementation. The emphasis on results can lead to a bias to the positive. Learning from both success and failure should be systematically encouraged. Recommendations DFID needs to focus on consistent and continuous organisational learning based on the experience of DFID, its partners and contractors and the measurement of its impact, in particular during the implementation phase of its activities. All DFID managers should be held accountable for conducting continuous reviews from which lessons are drawn about what works and where impact is actually being achieved for intended beneficiaries. All information commissioned and collected (such as annual reviews and evaluations) should be synthesised so that the relevant lessons are accessible and readily usable across the organisation. The focus must be on practical and easy-to-use information. Knowhow should be valued as much as knowledge. Staff need to be given more time to acquire experience in the field and share lessons about what works and does not work on the ground. DFID needs to continue to encourage a culture of free and full communication about what does and does not work. Staff should be encouraged always to base their decisions on evidence, without any bias to the positive.

How did China create “Directed Improvisation”? - Pritchett, Lant (Building State Capability, 2017)
written by Lant Pritchett Yuen-Yuen Ang, a Professor of Political Science at University of Michigan came to speak at Harvard the other day and I was lucky enough to hear her presentation.  Her most…
Keywords: IMPORTANT

How do we know what works? Interview with Prof. Michael Woolcock, Lead Social Scientist at the World Bank - Evans, Alice (2018)
Dr Alice Evans and leading experts discuss growth, governance, & gender inequalities. Alice is a Lecturer at King's College London, and Faculty Associate at the Harvard Kennedy School.

How Does Developing Enforcement Capacity Reduce Wildlife Crime? Increasing program effectiveness by understanding common assumptions - Price, Claire; Buff, Jesse; Peabody, Shawn; Present, Tess; Lauck, Liz (USAID, 2018)
"How Does Developing Enforcement Capacity Reduce Wildlife Crime?" summarizes findings from the literature around four key assumptions about capacity building for enforcement and prosecution. Using the experiences and evidence summarized in this brief, program designers and implementing partners should be able to increase the effectiveness and efficiency of their capacity building efforts.

How Does the World Bank Build Citizen Engagement Commitments into Project Design? Results from Pilot Assessments in Mozambique, Myanmar, Nigeria, and Pakistan - Nadelman, Rachel; Le, Ha; Sah, Anjali (Working Paper 525, 2019)
How and to what degree is the World Bank putting its new institutional citizen engagement (CE) commitments into practice? This question guides an independent assessment that the Accountability Research Center (ARC) at American University has undertaken as part of the Institute of Development Studies-led Action for Empowerment and Accountability (A4EA) research programme’s investigation into how external actors can best support local processes of and conditions for empowerment and accountability. This report investigates the World Bank’s incorporation of CE into project design, the critical early stage of donor engagement. To accomplish this, ARC reviewed the World Bank’s fiscal year 2015–17 investment project portfolios for four A4EA priority countries, Mozambique, Myanmar, Nigeria, and Pakistan, which covers 57 projects that range from US$19 million to U$600 million. The analysis determines whether projects commit to seeking a strategic approach to CE, which involves combining multiple tactics so that the whole could be greater than the sum of the parts. This assessment of CE commitments is intended to help to inform possible national, civil society organisation strategies to monitor whether and how these commitments are actually implemented in practice.

How ICT4D Research Fails the Poor - Harris, Roger W. (Information Technology for Development, 2016)
Research can improve development policies and practices and funders increasingly require evidence of such socioeconomic impact from their investments. This article questions whether information and communication technologies for development (ICT4D) research conforms to the requirements for achieving socioeconomic impact. We report on a literature review of the impact of research in international development and a survey of ICT4D researchers who assessed the extent to which they follow practices for achieving socioeconomic impact. The findings suggest that while ICT4D researchers are interested in influencing both practice and policy, they are less inclined toward the activities that would make this happen, especially engaging with users of their research and communicating their findings to a wider audience. Their institutions do not provide incentives for researchers to adopt these practices. ICT4D researchers and their institutions should engage more closely with the users of their research through more and better communications with the public, especially through the use of information and communication technologies.
Keywords: ICT4D; Policy making; Practice; Research; impact

How INGOs are Doing Development Differently - Algoso, Dave (Care, IRC, Mercy Corps, Oxfam & World Vision, 2017)

How should donors manage adaptively? Market Systems Development as a case study. - Itad (Itad, 2018)
Itad has recently completed a strategic evaluation for Sida to help them work through how they can best manage programmes that are adaptive and apply systems approaches - the conclusions from the evaluation about what funders need to do to manage adaptively are of broader relevance.

How to Apply Agile Principles to International Development M&E - ICTworks (ICT Works, 2016)
We all want to be good at our jobs. We want to accomplish the things we set out to do. If we aren’t accomplishing them, we want to figure out why or try new solutions. The trend toward Adaptive M&E is exactly that: a desire to be better at our jobs. Similar trends exist in the software world (agile) and in manufacturing and start-ups (lean). But by any name, this process of seeking to improve is about speeding up decision-making and solution delivery by focusing on incremental, iterative planning and execution. I presented on agile with Monalisa Salib of USAID Learning Lab at MERL Tech 2016. I talked about creating software; she talked about creating evaluation tools for USAID missions, but we were describing the same basic process: Develop, Release, Reflect, and Adapt. You can see our slide deck for more details, but here are the key discussion takeaways that won’t show up there: 1: Eyes on the Prize Any effort should start with a clear definition of what you’re hoping to achieve.
Keywords: IMPORTANT

How to create an agile organization - survey - Salo, Olli; Ahlbäck, Karin; Fahrbach, Clemens; Murarka, Monica (McKinsey & Co., 2017)
Eighteen practices for organizational agility The survey asked respondents about a series of specific actions that underlie each of the 18 practices (9 of them stable, and 9 dynamic) of organizational agility; all of the practices are summarized in the table below. To rate respondents’ organizations, we asked how frequently their performance units engaged in each action that supports a given practice.

How to design a monitoring and evaluation framework for a policy research project - Pasanen, Tiina; Shaxson, Louise (Methods Lab, 2016)
Policy research projects face a number of challenges: policy processes are complex, involve multiple actors and often feature a significant time-lag between research and what may or may not happen as a result of it. To complicate matters further, the scope and scale of policy research projects are increasingly moving away from single research studies towards multi-component, multi-site and multi-sector endeavours. These factors mean that developing an overarching monitoring and evaluation framework can be challenging. But it is not impossible. The guidance note aims to support the first steps in designing and structuring the M&E framework (that is, what aspects or areas of policy research projects to monitor and evaluate, why, when and how). It builds on an M&E framework for policy research projects developed and tested by the Research and Policy in Development (RAPID) programme of the Overseas Development Institute, and draws on case studies of how it has been used by RAPID and other research teams.

How to Make ‘Cash Plus’ Work: Linking Cash Transfers to Services and Sectors - Roelen, Keetie; Devereux, Stephen; Abdulai, Abdul-Gafaru; Martorano, Bruno; Palermo, Tia; Ragno, Luigi Peter (UNICEF Office of Research, 2017)
The broad-ranging benefits of cash transfers are now widely recognized. However, the evidence base highlights that they often fall short in achieving longer-term and second-order impacts related to nutrition, learning outcomes and morbidity. In recognition of these limitations, several ‘cash plus’ initiatives have been introduced, whereby cash transfers are combined with one or more types of complementary support. This paper aims to identify key factors for successful implementation of these increasingly popular ‘cash plus’ programmes, based on (i) a review of the emerging evidence base of ‘cash plus’ interventions and (ii) an examination of three case studies, namely, Chile Solidario in Chile, IN-SCT in Ethiopia and LEAP in Ghana. The analysis was guided by a conceptual framework proposing a menu of ‘cash plus’ components. The assessment of three case studies indicated that effective implementation of ‘cash plus’ components has indeed contributed to greater impacts of the respective programmes. Such initiatives have thereby addressed some of the non-financial and structural barriers that poor people face and have reinforced the positive effects of cash transfer programmes. In design of such programmes, further attention should be paid to the constraints faced by the most vulnerable and how such constraints can be overcome. We conclude with recommendations regarding the provision of complementary support and cross-sectoral linkages based on lessons learned from the case studies. More research is still needed on the impact of the many variations of ‘cash plus’ programming, including evidence on the comparative roles of individual ‘plus’ components, as well as the knowledge, attitudes and behaviour pathways which influence these impacts.

How to manage complexity: four lessons for DFID’s new Secretary of State - LearnAdapt (Medium, 2019)
Lessons for DFID in addressing complex problems without creating complicated, workload heavy programmes.

How-To Note: Conduct a Data Quality Assessment - USAID (2017)

How-To Note: Developing a Project Logic Model (and its Associated Theory of Change) - USAID (Program Cycle, 2017)
This How-To Note describes considerations for developing a project logic model, as well as steps for thinking through a more complete theory of change (TOC). A logic model is a graphic or visual depiction that summarizes key elements of a TOC, and it is often used as a facilitation tool during the design process. There are many types of logic models, including but not limited to logical frameworks (logframes), results chains, results frameworks, and local actor-oriented models, among others. The project logic model and its associated TOC are included in the Project Appraisal Document (PAD) that approves a project design (see ADS 201.3.3.13). While this How-To Note focuses on logic models at the project level, logic models are also used at the strategy level (specifically, results frameworks – see Box 1), and often at the activity level. The concepts and steps presented here are generally applicable to the process of developing logic models and TOCs throughout the Program Cycle.

How to set up and manage an adaptive programme - Lessons from the Action on Climate Today (ACT) Programme - Cooke, Katherine (OPM, 2017)

How useful are RCTs in evaluating transparency and accountability projects? - Leavy, Jennifer (MAVC Working Paper, 2014)

How We Model Matters: A Manifesto for the Next Generation of Program Theorizing - Whynot, Jane; Lemire, Sebastian; Montague, Steve (Canadian Journal of Program Evaluation, 2019)
In this concluding article, grounded on the exemplary contributions contained in the preceding pages, the guest editors scale the proverbial soapbox and present a manifesto to guide the pursuit and advancement of the next generation of program theorizing. Formulating ten declarations for program theory development and examination, the modest hope of the authors is to motivate and inspire reflective evaluation practitioners to broaden their views, approaches, and techniques for future program theorizing.

Human-Centered, Systems-Minded Design - Both, Thomas (Stanford Social Innovation Review, 2018)
Both human-centered and systems-thinking methods fit within an effective design approach, and can work in conjunction to address social challenges.

Humanitarian Innovation Guide - ELHRA (2018)
The Humanitarian Innovation Guide is a growing online resource to help individuals and organisations define humanitarian problems and successfully develop innovative solutions. ABOUT THE GUIDE The humanitarian sector is investing in exciting innovations, but it is not yet producing a steady pipeline of well-designed solutions that effectively address problems, evidence their impact, and have the potential to be scaled. As stated in a recent independent evaluation of the Humanitarian Innovation Fund (Triple Line, 2017), a review of the innovation ecosystem in 2015 found significant gaps in resources for innovation, including a shortage of guidance on the skills needed to manage successful innovation projects. These findings were echoed in the research carried out to inform this resource. As part of a grant agreement with the European Commission to provide financial and technical support to emerging humanitarian innovations, and in line with our strategic aim to develop the sector’s skills and capabilities in humanitarian innovation, this resource aims to translate our own learning, along with learning from across the sector, into a practical, grounded guide for innovators working in humanitarian contexts. Building on our unique position in the sector, the Humanitarian Innovation Guide is designed to provide targeted support to individuals and organisations attempting to develop innovative solutions to the challenges facing humanitarian assistance, resulting in a more effective humanitarian response. The Guide is written with two audiences in mind: humanitarian practitioners who are seeking to develop a new approach to their work and want to apply an innovation lens to solving problems; and social entrepreneurs from outside the sector who have identified an opportunity to engage with the sector and need a humanitarian framework to contextualise their innovation plans. We also hope that it will be a useful resource for innovation managers who are tasked with supporting innovation in their agencies, labs or networks. As the first step-by-step guide to managing innovation in the humanitarian sector, we hope that its continued development will better enable individuals, organisations and the wider sector to: Plan the activities involved in an innovation process and systematise the management of innovation. Manage a successful innovation project and generate evidence for innovation. Ensure that innovations are developed in an ethical way, with full consideration of risks and responsibilities.

I know what I know (but how do I know what I don’t?) - Smit, Debbie; de Lanerolle, Indra; Braam, Tamara; Byrne, Deborah; Legong, Gontse (MAVC, 2017)
An important support function of Making All Voices Count South Africa is to design, plan and facilitate community of practice gatherings for sustained learning and sharing across Making All Voices Count grantees. This report aims to capture the content of a one-day Making All Voices Count South African Community of Practice (CoP) Meeting held in November 2016. The South African MAVC CoP has been running for three years and has met between two and four times a year. It is a space for MAVC grantees and others working to foster innovation in the fields of transparency and accountability, to share experiences and knowledge, and collaborate in learning and improving work.

ICT4D and Sustainability - Marais, Mario (The International Encyclopedia of Digital Communication and Society, 2015)

ICT4D Blog » A citizen participation ecosystem - Peña-López, Ismael (ICTlogy, 2019)
Research on the Information Society, the Digital Divide and Information and Communication Technologies for development

Identifying general trends and patterns in complex systems research: An overview of theoretical and practical implications - Preiser, Rika (Systems Research and Behavioral Science, 2019)
Research on complex systems is becoming more prolific, and there is a need to provide some point of orientation to researchers and practitioners that are drawing on the body of literature that informs the field of complexity research. In this paper, I aim to give an overview of the development of the field and offer some overarching trends and patterns that are recognizable in research on complex systems. The paper then draws on the work to provide six organizing principles of complex systems to inform practical implications and methods for studying and understanding complex systems.
Keywords: complex systems; heuristics; trends and patterns

If politics is the problem, how can external actors be part of the solution? - Devarajan, Shantayanan; Khemani, Stuti (The World Bank, 2016)
Despite a large body of research and evidence on the policies and institutions needed to generate growth and reduce poverty, many governments fail to adopt these policies or establish the institutions. Research advances since the 1990s have explained this syndrome, which this paper generically calls
Keywords: IMPORTANT

Illegal Wildlife Trade - Frameworks - Oxford Martin Programme on the Illegal Wildlife Trade (Oxford Martin Programme on the Illegal Wildlife Trade, 2018)
Our frameworks will enable a better understanding of wildlife product consumer preferences and motivations to effectively influence the system.

Illogical Framework: The Importance of Monitoring and Evaluation in International Development Studies - Pomerantz, Jessica R. (Cornell Policy Review, 2011)
In my brief experience with monitoring and evaluation, I have become convinced that it is critically important both as an international development project component and as a field of academic study. Throughout my academic career at Cornell University, I have, at times, argued that monitoring and evaluation has actually impeded development efforts, but here I wish to amend my opinion. Bad monitoring and evaluation can sabotage development projects and our meaningful interpretation of development impacts; failures can appear to be successes and vice-versa. As a student and practitioner of monitoring and evaluation, I have drawn the conclusions listed below and I submit them for your consideration. • Monitoring and evaluation is a key element of the international development industry applicable to many areas of public administration, domestic and international. • International development failures could be discovered and averted or corrected given proper monitoring and evaluation activities. • Anecdotal evidence from development activities in Afghanistan provides one example of the international community’s lack of attention to monitoring and evaluation concerning an ongoing development catastrophe. • Higher education ought to be filling the monitoring and evaluation knowledge gap but to date is failing to do so.

Impact assessment and the quest for the Holy Grail - Bymolt, Roger (KIT Working papers, 2015)

Impact evaluation for portfolio programmes on policy influence: Reflections on the Indonesian Poverty Reduction Support Facility - Mackenzie, Jessica; Hearn, Simon (Methods Lab, 2016)
• Donors are increasingly using portfolio-based programmes that embrace ‘good failure’ and adaptive, political programming. • However, measuring the impact of these programmes is challenging, especially for those working on policy influence and building country systems; not only do you need to measure the positive and negative impact of the overall portfolio, but also the different pathways tested. • Programmes, therefore, need a light-touch monitoring and evaluation system that allows it to remain flexible. • Good practice examples of portfolio-based programmes present six strategies to evaluate impact: 1. Develop appropriate logic models 2. Collect observational data throughout implementation 3. Develop stories of change or case studies 4. Understand causal relationships without a counterfactual 5. Purposefully select which activities to study 6. Be explicit about how impact will be valued across the portfolio. • These strategies are only useful if monitoring and evaluation is placed at the centre of programme decision-making.

Impact Evaluation of Development Interventions: A Practical Guide - White, Howard; Raitzer, David (Asian Development Bank, 2017)
This book offers guidance on the principles, methods, and practice of impact evaluation. It contains material for a range of audiences, from those who may use or manage impact evaluations to applied researchers.

Impact Evaluation: Taking stock and looking ahead - Conference report - Visser, Irene; Guijt, Irene; Kusters, Cecile (Centre for Development Innovation, 2013)
This report summarises the presentations and discussions of the Conference ‘Impact evaluation. Taking stock and looking ahead’, which took place in Wageningen on March 25 and 26, 2013. The Conference was organised and funded by the Centre for

Implementation of the first adaptive management plan for a European migratory waterbird population: The case of the Svalbard pink-footed goose Anser brachyrhynchus - Madsen, Jesper; Williams, James Henty; Johnson, Fred A.; Tombre, Ingunn M.; Dereliev, Sergey; Kuijken, Eckhart (Ambio, 2017)
An International Species Management Plan for the Svalbard population of the pink-footed goose was adopted under the Agreement on the Conservation of African-Eurasian Migratory Waterbirds in 2012, the first case of adaptive management of a migratory waterbird population in Europe. An international working group (including statutory agencies, NGO representatives and experts) agreed on objectives and actions to maintain the population in favourable conservation status, while accounting for biodiversity, economic and recreational interests. Agreements include setting a population target to reduce agricultural conflicts and avoid tundra degradation, and using hunting in some range states to maintain stable population size. As part of the adaptive management procedures, adjustment to harvest is made annually subject to population status. This has required streamlining of monitoring and assessment activities. Three years after implementation, indicators suggest the attainment of management results. Dialogue, consensus-building and engagement among stakeholders represent the major process achievements.
Keywords: Adaptive harvest management; Human–wildlife conflict; Population target; Stakeholder involvement; Structured decision-making; Tundra degradation

Implementing Adaptive Approaches in Real World Scenarios: A Nigeria Case Study, with Lessons for Theory and Practice - Bridges, Kate; Woolcock, Michael (The World Bank, 2019)
How does adaptive implementation work in practice? Drawing on extensive interviews and observations, this paper contrasts the ways in which an adaptive component of a major health care project was implemented in three program and three matched comparison states in Nigeria. The paper examines the bases on which claims and counterclaims about the effectiveness of these approaches were made by different actors, concluding that resolution requires any such claims to be grounded in a fit-for-purpose theory of change and evaluation strategy. The principles of adaptive development may be gaining broad acceptance, but a complex array of skills, expectations, political support, empirical measures, and administrative structures needs to be deftly integrated if demonstrably positive operational results are to be obtained, especially when undertaken within institutional systems, administrative logics, and political imperatives that are predisposed to serve rather different purposes.

Implementing Innovation: A User's Manual for Open Government Programs - Reboot Design (2015)
This guide draws from our experience around the world helping government reformers achieve real change. It is a practical resource for anyone working to implement an open government initiative.

Implementing the new development agenda - Wrap up - Teskey, Graham (DDD 4 Conference, 2017)

Importing Democracy: The Role of NGO's in South Africa, Tajikistan, & Argentina - Fisher, Julie (Kettering Foundation Press, 2013)
While street protesters demanding democratic reforms make headlines in the international news, Importing Democracy: The Role of NGOs in South Africa, Tajikistan, and Argentina, written by Julie Fisher and published by the Kettering Foundation Press, focuses on a quieter movement led by democratization NGOs. In South Africa, the Good Governance Learning Network shares participatory tools to make local governments more responsive. In Tajikistan, Jahan teaches local police about human rights. In Argentina, seven democratization NGOs sponsor public deliberations in local communities and have organized a nationwide citizens network to combat municipal government corruption. The book is organized around three chapters for each country, South Africa, Tajikistan, and Argentina. The first chapter of each country s section begins with the historical, political, and economic context and continues with a discussion of the general contours of civil society. The second chapter in each section deals with the role of democratization NGOs in promoting both loyal opposition and law-based civil liberties. The third chapter focuses on their role in promoting political culture and political participation. Loyal opposition and law-based civil liberties help define democratization at the national level, whereas changes in political culture and increased political participation often occur throughout society. Following the nine country chapters, the book concludes with a comparative overview and implications for international policy. Fisher, a former Kettering Foundation program officer, writes that the idea that democracy can be exported has lost credibility in recent years. In many countries, however, democratization NGOs are importing democratic ideas and recovering local democratic traditions.

Improving development aid design and evaluation: Plan for sailboats, not trains - Kleinfeld, Rachel (Carnegie Endowment for International Peace, 2015)
How do reforms that require political engagement differ from traditional technical reforms? Why is political engagement different, and what are the implications for design and evaluation? How should development programmes that engage politics be designed? And how can those who fund or implement such programmes evaluate whether their efforts are contributing to reform? This report …
Keywords: IMPORTANT

Improving our legacy: Incorporation of adaptive management into state wildlife action plans - Fontaine, Joseph J. (Journal of Environmental Management, 2011)
The loss of biodiversity is a mounting concern, but despite numerous attempts there are few large scale conservation efforts that have proven successful in reversing current declines. Given the challenge of biodiversity conservation, there is a need to develop strategic conservation plans that address species declines even with the inherent uncertainty in managing multiple species in complex environments. In 2002, the State Wildlife Grant program was initiated to fulﬁll this need, and while not explicitly outlined by Congress follows the fundamental premise of adaptive management, ‘Learning by doing’. When action is necessary, but basic biological information and an understanding of appropriate management strategies are lacking, adaptive management enables managers to be proactive in spite of uncertainty. However, regardless of the strengths of adaptive management, the development of an effective adaptive management framework is challenging. In a review of 53 State Wildlife Action Plans, I found a keen awareness by planners that adaptive management was an effective method for addressing biodiversity conservation, but the development and incorporation of explicit adaptive management approaches within each plan remained elusive. Only w25% of the plans included a framework for how adaptive management would be implemented at the project level within their state. There was, however, considerable support across plans for further development and implementation of adaptive management. By furthering the incorporation of adaptive management principles in conservation plans and explicitly outlining the decision making process, states will be poised to meet the pending challenges to biodiversity conservation.

Improving the Evaluability of INGO Empowerment and Accountability Programmes - Shutt, Cathy; McGee, Rosemary (CDI Practice Paper 1, 2013)

Improving the Measurement of Civic Space - Malena, Carmen (Transparency & Accountability Initiative, 2015)
‘Civic space’ – i.e. the freedom and means to speak, access information, associate, organise, and participate in public decision-making – is essential to the healthy functioning and development of any society and a precondition for accountable governance and social justice. When civic space is restricted, human and civil rights are denied, government accountability is jeopardised, …

In pursuit of the deep and sustained shifts necessary for adaptive management - Simister, Nigel (INTRAC, 2018)
The final blog in Nigel Simister's series on adaptive management and the M&E of complex projects and programmes.

In the goldfish bowl: science and technology policy dialogues in a digital world - Latta, Susie; Mulcare, Charlotte; Zacharzewski, Anthony (Sciencewise, 2013)

Inclusion in Making All Voices Count - Kelbert, Alexandra (Making All Voices Count, 2014)
Part of a sequenced learning process for MAVC's Research and Evidence Component, the summary of a thematic discussion led by researcher Evangelia Berdou.

Inclusive rigour for complexity - Chambers, Robert (Journal of Development Effectiveness, 2015)
Rigour can be reductionist or inclusive. To learn about and understand conditions of complexity, emergence, nonlinearity and unpredictability, the inclusive rigour of mixed methods has been a step in the right direction. From analysis of mixed methods and participatory approaches and methods, this article postulates canons for inclusive rigour for research and evaluation for complexity: eclectic methodological pluralism; improvisation and innovation; adaptive iteration; triangulation; plural perspectives; optimal ignorance and appropriate imprecision; and being open, alert and inquisitive. Inclusive rigour is inherent in participatory methods and approaches, visualisations, group-visual synergy, the democracy of the ground and participatory statistics. Transparent reflexivity, personal behaviour and attitudes, and good facilitation are fundamental. Fully inclusive rigour for complexity demands many personal, institutional and professional revolutions.
Keywords: Evaluation; Mixed methods; Participatory methodologies; Research; paradigm; rigour

Inclusive Social Accountability - Counterpart International (Counterpart International, 2019)
Counterpart International’s Inclusive Social Accountability (ISA) developmental framework integrates elements of social inclusion and community accountability into one comprehensive approach.

Incorporating Geospatial Analysis into USAID Biodiversity Program Design - Higgins, Mark; Salafsky, Nick; Adeney, Marion; Petrova, Silvia (Measuring Impact, 2017)
Incorporating Geospatial Analysis into USAID Biodiversity Program Design is a biodiversity case example that describes the use of geospatial data and analysis for program design, including examples and lessons learned.

Increasing Participation in Evaluation - Baker, A; Bruner, B (Effectiveness Initiatives, 2014)
The Increasing Participation in Evaluation bulletin was developed by Anita Baker with Beth Bruner to help organizations integrate evaluative thinking into their organizational practice. This three page bulletin discusses how Organization Staff, Evaluators, and Funders are typically involved in participatory evaluation. The guide also defines the term "Ripple", and how to accomplish Ripple as well as examining what it looks like when Executive Leaders and Management Staff use Evaluative Thinking. Contents How are Organization Staff, Evaluators, and Funders typically involved in Participatory Evaluation? Organization Managers/Staff Roles Evaluators/Roles Funder Roles Shared Roles What is Ripple Anyway? How do you accomplish Ripple? What does it look like when Executive Leaders and Management Staff Use Evaluative Thinking? How to use these Bulletins

Incubating policy for economic transformation: Lessons from Nepal - Booth, David (ODI, 2018)

Independent Evaluation of SDC’s Results-Based Management System with a Focus on Poverty Reduction - Schmidt, Martin; Palenberg, Markus; Vähämäki, Janet (SDC, 2017)
The SDC's results-based management system ensures that the processes and instruments used for the design and implementation of programs and projects contribute to achieving the desired results (outputs, outcomes and impact). The independent evaluation examines the extent to which these processes and tools have fostered the results culture within the organization, and improved competencies for results-based management decisions, learning and communication. In addition to the findings, conclusions and recommendations, the evaluation report includes the management response of SDC’s Directorate.

Independent Review of Facilities: review and management response - DFAT (DFAT, 2018)

Information lives of the poor: fighting poverty with technology - Elder, Laurent; Samarajiva, Rohan; Gillwald, Alison; Galperin, Hernan (2014)

Information Technology and Political Engagement: Mixed Evidence from Uganda - Grossman, Guy (2016)

ING’s agile transformation - Schlatmann, Bart (McKinsey Quarterly, 2017)
Two senior executives from the global bank describe their recent journey.

Innovating for pro-poor services: why politics matter - Mason, Nathaniel; Doczi, Julian; Cummings, Clare (2016)
This report addresses how politics matter for innovations. How politically smart approaches can help deliver access to services.

Innovating in Development Learning Event - Challenge brief - Vogel, Isabel (Ideas to Impact / iMC, 2017)

Innovation at UNDP: from weekend sport to daily practice - Quaggiotto, Giulio; Begovic, Mellie (Nesta, 2016)
The experience of working to embed innovation approaches in everyday practice at UNDP's country offices
Keywords: IMPORTANT

Innovation for International Development: Navigating the Paths and Pitfalls - Ramalingam, Ben; Bound, Kirsten (NESTA, 2016)
Experiences, insights and practical advice from over 20 leading practitioners in innovation for international development, brought together in one collection.

Innovation in development: Sharing learning, improving impact - Vogel, Isabel; Minkley, Gabrielle; Chowdhury, Sajid (Ideas to Impact / IMC Worldwide, 2017)

Innovation lifecycle and the missing middle - Gray, Ian; McClure, Dan (Managing Humanitarian Innovation - The cutting edge of aid, 2018)

Innovation, Sustainability, Development: A New Manifesto - STEPS Centre (STEPS Centre, 2010)
Our Manifesto project publication is available in print, on CD or to view online. Innovation, Sustainability, Development: A New Manifesto recommends new ways of linking science and innovation to development for a more sustainable, equitable and resilient future. The multimedia version, with added audio, video and background documents, is available on CD and online.

Innovation teams and labs. A Practice Guide - Puttick, Ruth (Nesta, 2014)

Institutional Change, Political Economy, and State Capabilities : Learning from Edo State, Nigeria - Bain, Katherine; Porter, Doug; Watts, Michael (Justice, security, and development series, 2015)
This paper is one of a series aimed at deepening the World Bank’s capacity to follow through on commitments made in response to the World Development Report (WDR) 2011, which gave renewed prominence to the nexus between conflict, security, and development. Nigeria is a remarkable illustration of how deeply intractable the cycle of poverty, conflict, and fragility can become when tied to the ferocious battles associated with the political economy of oil. This paper places the corpus of analytic and programmatic work concerning institutional reform in conversation with a now substantial body of work on resource politics and most especially, the debate over the politico-institutional character (sometimes called political settlements or pacting arrangements associated with the order of power) and reform landscape of the petro-state. Recent institution reform policy writing appears to have little to say about the political and economic conditions in which crises and institutional disjunctures may authorize, and thereby enable, agents to embark on institutional reforms. The authors focus on Edo state for two reasons. First, it does not on its face appear to be an obvious location in which to explore a reform experience, given its entanglement in the Niger Delta conflict and the maladies typically associated with state fragility. Second, Edo is of interest also because of the changes that its experience is contributing to the World Bank country team’s effort to engage operationally across all its instruments with the political economy of institutional reform in Nigeria, its largest client country in Africa.

Integrating Big Data and Thick Data to Transform Public Services Delivery - Ang, Yuen Yuen (IBM Center for The Business of Government, 2019)
This report explains how public organizations can combine big data and thick data to transform public services delivery—a strategy called mixed analytics. Governments can greatly enhance the value of big data by combining it with “thick” data—rich qualitative information about users, such as their values, goals, and consumption behavior, obtained by observing or interacting with them in their daily lives. Big data holds great promise for improving public services delivery and innovation in government, but they are not a panacea . Having lots of data can be overwhelming or have little utility if the data are “thin”—that is, they lack meaning for users or fail to capture issues that matter most . By yielding insights into what citizens really care about and how they consume services, thick data can inform both the collection and analysis of big data. This report introduces the concept of “mixed analytics,” integrating big data and thick data to transform government decision making, public services delivery, and communication. The report presents three case studies of organizations that employ mixed analytics at the international, federal, and city level, respectively. Together, this research offers a set of transferable lessons for agencies at all levels of government: Lesson 1: Big data is a means to an end, rather than an end. Lesson 2: Thick data can identify unexpected problems or previously unexpressed needs. Lesson 3: Thick data can inform the analysis of big data. Lesson 4: Mixed analytics can offer both scale and depth. Lesson 5: Applying technology is a social activity, not an isolated technical task. Lesson 6: The best solutions are not always high-tech. The report concludes with five actionable recommendations for public managers. Make data and technology relevant to the people who use it. Leverage thick data at appropriate stages of the problem-solving process. Build an interdisciplinary team of quantitative and qualitative experts who work closely with stakeholders. Combine big and thick data to improve communication Improve government agencies’ knowledge of mixed research methods. This report builds on multiple past IBM Center reports about how government can leverage data and analytics to improve decisions, including Data-Driven Government: The Role of Chief Data Officers, by Jane Wiseman; Ten Actions to Implement Big Data Initiatives: A Study of 65 Cities, by Alfred Ho and Bo McCall, and Realizing The Promise of Big Data, by Kevin DeSouza. The report also complements several chapters that assess the past and potential use of data across the public sector in our recent book, Government For The Future: Reflection and Vision For Tomorrow’s Leaders. At a time when the US and other governments continue to enhance their use of data as a strategic asset for transformation, we hope that Professor Ang’s report provides useful insights for government managers and stakeholders.

Integrating farmers’ adaptive knowledge into flood management and adaptation policies in the Vietnamese Mekong Delta: A social learning perspective - Tran, Thong Anh; Rodela, Romina (Global Environmental Change, 2019)
Flood management and adaptation are important elements in sustaining farming production in the Vietnamese Mekong Delta (VMD). While over the past decades hydraulic development introduced by the central government has substantially benefited the rural economy, it has simultaneously caused multiple barriers to rural adaptation. We investigate the relational practices (i.e., learning interactions) taking place within and across the flood management and adaptation boundaries from the perspective of social learning. We explore whether and how adaptive knowledge (i.e., experimental and experiential knowledge) derived from farmers’ everyday adaptation practices contributes to local flood management and adaptation policies in the selected areas. We collected data through nine focus groups with farmers and thirty-three interviews with government officials, environmental scientists, and farmers. Qualitative analysis suggests that such processes are largely shaped by the institutional context where the boundary is embedded. This study found that while the highly bureaucratic operation of flood management creates constraints for feedback, the more informal arrangements set in place at the local level provide flexible platforms conducive to open communication, collaborative learning, and exchange of knowledge among the different actors. This study highlights the pivotal role of shadow systems that provide space for establishing and maintaining informal interactions and relationships between social actors (e.g., interactions between farmers and extension officials) in stimulating and influencing, from the bottom-up, the emergence of adaptive knowledge about flood management and adaptation in a local context.
Keywords: Adaptation; Flood management; Knowledge brokers; Mekong Delta; Shadow systems; Social learning; Vietnamese

Integrating Food Security and Wild Caught Fisheries Management in USAID Programming. Illustrative situation Model and Results Chain - Swaminathan, Vinaya; Best, Barbara; Ajroud, Brittany (Measuring Impact, 2018)
Integrating Food Security and Wild Caught Fisheries Management in USAID Programming is an illustrative case example that demonstrates how two adaptive management programming tools—situation models and results chains—can help USAID staff and its partners to better design and implement activities that integrate food security, nutrition, and sustainable management of wild fisheries.

Integrating Livelihood and Conservation Goals: A retrospective Analysis of World Bank Projects - Gómez, Andrés; Price, Claire (Measuring Impact, 2017)
Using a database of World Bank project evaluations, E3/FAB’s new study called “Integrating Livelihood and Conservation Goals: A Retrospective Analysis of World Bank Projects” aims to contribute to building the evidence base around the integration of biodiversity conservation and livelihood goals.

Integrating modelling and smart sensors for environmental and human health - Reis, Stefan; Seto, Edmund; Northcross, Amanda; Quinn, Nigel W. T.; Convertino, Matteo; Jones, Rod L.; Maier, Holger R.; Schlink, Uwe; Steinle, Susanne; Vieno, Massimo; Wimberly, Michael C. (Environmental Modelling & Software, 2015)
Sensors are becoming ubiquitous in everyday life, generating data at an unprecedented rate and scale. However, models that assess impacts of human activities on environmental and human health, have typically been developed in contexts where data scarcity is the norm. Models are essential tools to understand processes, identify relationships, associations and causality, formalize stakeholder mental models, and to quantify the effects of prevention and interventions. They can help to explain data, as well as inform the deployment and location of sensors by identifying hotspots and areas of interest where data collection may achieve the best results. We identify a paradigm shift in how the integration of models and sensors can contribute to harnessing ‘Big Data’ and, more importantly, make the vital step from ‘Big Data’ to ‘Big Information’. In this paper, we illustrate current developments and identify key research needs using human and environmental health challenges as an example.
Keywords: Big data; Environmental health; Environmental sensors; Integrated modelling; Population health

Integrating Top-Down and Bottom-Up Adaptation Planning to Build Adaptive Capacity: A Structured Learning Approach - Butler, J. R. A.; Wise, R. M.; Skewes, T. D.; Bohensky, E. L.; Peterson, N.; Suadnya, W.; Yanuartati, Y.; Handayani, T.; Habibi, P.; Puspadi, K.; Bou, N.; Vaghelo, D.; et al. (Coastal Management, 2015)
Climate adaptation planning provides an opportunity to enhance the adaptive capacity of stakeholders across multiple levels. However, reviews of standard top-down and bottom-up approaches indicate that the value of multistakeholder involvement is not fully recognized or incorporated into guidelines. Focusing on provinces in Indonesia and Papua New Guinea within the Coral Triangle region, we present a novel integrated top-down and bottom-up planning approach. Based on Participatory Systemic Inquiry the process involves three stages of workshops intentionally designed to promote social learning, knowledge exchange, empowerment and social networks among multilevel stakeholders. Stage 1 workshops engage government, nongovernment and science stakeholders at the provincial level to analyze sub-districts' vulnerability and design appropriate adaptation strategies. Stage 2 engages local government, non-government and community stakeholders within vulnerable sub-districts identified in Stage 1. Stage 3 combines Stage 1 and 2 stakeholders to refine adaptation strategies and design action plans for sub-districts. Evaluation demonstrated that different stakeholder groups' perceptions of community adaptation needs varied significantly, justifying the approach. In terms of adaptive capacity, the primary outcome for all stakeholder groups was innovative ideas, suggesting that social learning and knowledge exchange had occurred. Empowerment was a secondary outcome. We discuss how the approach could be further refined.
Keywords: Climate change; Coral Triangle; Evaluation; Social learning; knowledge cultures

International Development - Illusions and Realities - Black, Maggie (New Internationalist, 2015)
Part of the popular, reissued NoNonsense series from New Internationalist'Development' is often misunderstood and can embrace everything from building a large dam to planting trees. The idea can often mask confusion, contradiction, deceit and corruption. This book is essential reading for anyone wanting to know what development actually is. It covers all the key themes and critically suggests ways to bring the poor and marginalised into the process.

Interview protocol: Most significant turning points - Prieto Martin, Pedro; Faith, Becky (Institute of Development Studies, 2017)
This interview protocol was used for a research project on adaptiveness in technology for governance initiatives in Kenya. For more information, please read the research report at: Prieto Martin, P.; Hernandez, K.; Faith, B. and Ramalingam, B. (2017) Doing Digital Development Differently: Lessons in adaptive management from technology for governance initiatives in Kenya, MAVC Research Report, Brighton: Institute of Development Studies, ids.ac.uk/project/making-all-voices-count
Keywords: IMPORTANT

Into the unknown: explorations in development practice - Chambers, Robert (Practical Action Publishing, 2014)
Into the Unknown reflects on the journey of learning, and encourages readers to learn from observation, curiosity, critical feedback, play and fun. This book will be of interest to development professionals, including academics, students, NGO workers and the staff of international agencies

Introducción a Capacity WORKS + Proyectos de Cambio - Informe - Cuéllar, Daniel (GIZ, 2014)

Introduction - Localising Governance: An Outlook on Research and Policy - Joshi, Anuradha; Schultze-Kraft, Markus (IDS Bulletin, 2014)

Introduction: Making change happen–citizen action and national policy reform - Gaventa, John; McGee, Rosemary (Citizen Action and National Policy Reform. London: Zed Books, 2010)

Introduction: Opening Governance – Change, Continuity and Conceptual Ambiguity - McGee, Rosie; Edwards, Duncan (IDS Bulletin, 2016)

Introduction to International Futures - International Futures at the Pardee Center (International Futures Help System, 2016)
INTERNATIONAL FUTURES HELP SYSTEM Introduction to IFs Purposes Visual Representation of IFs Issues and Modules Quick Survey of IFs Issues and Modules IFs Background IFs Geographic Representation of the World IFs Time Horizon Instructional Use Acknowledgements Feedback Support for IFs Use Development Mode Features

Introduction to Project Management - The Case of GIZ Projects - Lacroix, Eric J. (Khulna University, 2015)
In this course, we will try to understand the complexity and the structure of the management of a development project implemented by the GIZ for the German Development cooperation. We will first define the frame of management and project to enter in the map of two logics. Then, we will overfly the 5 success factors of Capacity WORKS, with concrete examples in annexe. It will bring us to the monitoring, the standard processes in GIZ cooperation projects, the basic tools for project implementation and the evaluation. The team building will allow us to enter in a specific aspect of project management.

Introduction to SAVI's way of working - State Accountability and Voice Initiative (SAVI) - SAVI (DFID, 2015)
We are piloting a new approach to identifying and supporting our partners.  This breaks with convention in many ways in order to improve impact.  SAVI Approach Papers summarise key aspects of SAVI’s way of working. They explain what we do, and why – and link to relevant tools and frameworks. Our approach is summarized in...
Keywords: Practice

Introduction - Towards Systemic Approaches to Evaluation and Impact - Befani, Barbara; Ramalingam, Ben; Stern, Elliot (IDS Bulletin, 2015)

Investment Climate Reform: Doing it Differently - Manuel, Clare (Synthesis Paper, 1, 2015)
LASER synthesis papers aim to help donors and other stakeholders better understand why and how to approach investment climate reform programming differently. The papers reflect emerging best practice and lessons learnt on what works and what does not work in doing development differently. The papers have been peer-reviewed by experts in the field including senior advisers at DFID, World Bank, IFC and the Donor Committee for Enterprise Development (amongst others). First synthesis paper - Investment climate reform: doing it differently June 2015 Drawing on our experience in taking a problem-focused approach to investment climate reform, we discuss in this peer-reviewed paper emerging lessons and reflections on what works, both in recent literature and in practical experience.

Is DFID Getting Real About Politics?A stocktake of how DFID has adopted a politically-informed approach (2010-2015) - Piron, Laure-Hélène; Baker, Aislin; Savage, Laura; Wiseman, Katie (DFID, 2016)
1. Background This internal stocktake assesses whether DFID is “getting real about politics” - how it is taking power and politics into account in all its operations. Country Poverty Reduction Diagnostics undertaken by DFID teams identify politics as the most frequent barrier to poverty reduction and growth. The UK 2015 Aid Strategy has committed DFID to spending 50% of Official Development Assistance (ODA) in fragile states. This requires a “patient, long-term approach” to addressing barriers to peace and stability which are fundamentally political, rather than purely financial or technical. The stocktake is based on three DFID offices case studies (Democratic Republic of the Congo, Pakistan and Malawi) as well as extensive internal and external consultations between June and December 2015. It provides illustrations of how DFID is evolving but does not systematically offer evidence of development impacts or non-country work, as this would have required a different methodology. 2. What does it mean to take politics into account? Politically-informed approaches are based on a large body of evidence that confirms the importance of institutions and politics for sustainable development. External assistance needs to support locally-led change. Success depends on timing, context, political processes and local actors. Desirable outcomes are hard to achieve and difficult to predict. Politically-informed approaches improve development effectiveness through:  The ‘what’: political goals, using development assistance to shift how power is distributed in the economy and society. The two main elements are: aiming for long term transformation of institutions; and supporting locally-led change processes more likely to be sustainable and successful: locallyowned (i.e. with local salience) and locally-negotiated.  The ‘how’: politically-smart methods, with greater realism and feasibility. The three main elements are: understanding power and politics in a specific context in order to identify opportunities and barriers for change; influencing and stakeholder management skills; and proactive risk management. To influence DFID operations, a politically-informed approach needs to be iterative, not one-off. The explicit understanding of context, whether formal setpiece studies or more routine analysis, should inform policy and programme decisions, from high level strategic choices, to day-to-day implementation, for both international policy and country support. This is a dynamic process: as the context evolves and lessons are learned about what works, analyses and decisions are updated. These are the principles behind the ‘flexible and adaptive’ agenda.

Is It Time for a New Paradigm for "Citizen Engagement"? The Role of Context and What the Evidence Tells Us - O'Meally, Simon (People, Spaces, Deliberation, 2013)
The meteoric rise of "citizen engagement" Almost all development agencies promote some form of citizen engagement and accountability, often framed as 'voice', 'demand-side governance', 'demand for good governance' or 'social accountability'. The current World Bank president, Jim Yong Kim, recently put it that, "citizen voice can be pivotal in providing the demand-side pressure on government, service providers, and organizations such as the World Bank that is needed to encourage full and swift response to citizen needs". There has, in turn, been a mushrooming of useful operational guidance on different "tools" for social accountability - i.e. steps, inputs and methodologies - that guide discrete interventions, ranging from citizen score cards to participatory expenditure tracking. One might, however, be forgiven for thinking that some of the debates on citizen engagement need an injection of realism; especially as contextual factors can make or break a "tool's" implementation. A review of experience to date would be one good place to start.

Is there a new Washington Consensus? An analysis of five World Development Reports - Evans, Alice (From Poverty to Power, 2017)
Is there a new Washington Consensus? Alice Evans analyses the last five World Development Reports and finds significant changes in orthodoxy, but also big gaps

Is (Web) Science Ready for Empowerment? - Akkermans, Hans; Gyan, Nana Baah; Bon, Anna; Tuyp, Wendelien; Grewal, Aman; Boyera, Stéphane; Allen, Mary (ACM WebSci'11, 2011)

It’s About Results at Scale, Not Collective Impact - Edmondson, Jeff; Santhosh-Kumar, Parvathi (Stanford Social Innovation Review, 2017)
Improving outcomes at scale requires a paradigm shift in how we work.

It's All About MeE: Using Structured Experiential Learning (“e”) to Crawl the Design Space - Pritchett, Lant; Samji, Salimah; Hammer, Jeffrey (Working Paper 322, 2013)
This paper argues that within-project variations in design can serve as their own counterfactual, reducing the incremental cost of evaluation and increasing the direct usefulness of evaluation to implementing agencies. It suggests combining monitoring (‘M’), structured experiential learning (‘e’), and evaluation (‘E’) so as to facilitate innovation and organisational capability building while also providing accountability …

It's the politics! Can donors rise to the challenge? - Unsworth, Sue (A Governance Practitioner’s Notebook: Alternative Ideas and Approaches, 2015)
The Governance Practitioner’s Notebook takes an unusual approach for the OECD-DAC Network on Governance (GovNet). It brings together a collection of specially written notes aimed at those who work as governance practitioners within development agencies. It does so, however, without attempting to offer definitive guidance – instead aiming to stimulate thinking and debate. To aid this process the book is centred on a fictional Governance Adviser. The Notebook’s format provides space for experts to speak on today’s governance issues: politics, public sector reform and stakeholder engagement. It encourages debate, charts the evolution of donor thinking, and highlights future challenges in the age of the Sustainable Development Goals. Each section introduces both technical issues and major areas of debate, providing ideas for future development support to institutional reform.
Keywords: IMPORTANT

Iterative Design and Monitoring for Adaptive Management: How Causal Link Monitoring can help - Britt, Heather; Hummelbrunner, Richard; Green, Jackie (Better Evaluation, 2018)
Development actors are embracing the concept and practice of adaptive management, using evidence to inform ongoing revisions throughout implementation. In this guest blog, Heather Britt, Richard Hummelbrunner and Jackie Greene discuss a practical approach that donors and partners can use to agree on what’s most important to monitor as a project continues to evolve.

Joint Programming in Conflict-Affected and Fragile States - Koenig, Sibylle; Brusset, Emery (Joint Programming - EU, 2019)
On the Joint Programming Practices from the EU in fragile settings

Journalism from the ‘Silicon Savannah’: The Vexed Relationship Between Nairobi’s Newsmakers and its ICT4D Community - Benequista, Nicholas (Stability: International Journal of Security and Development, 2015)
During the course of a year-long knowledge exchange initiative called the Networked News Lab, a small group of Kenyan journalists and a PhD researcher from the London School of Economics and Political Science sought to identify opportunities for collaboration between newsmakers and practitioners from the field of information and communication technologies for development (ICT4D). In almost every instance, the project failed to promote cooperation between the two groups, though it succeeded in highlighting the fundamental issues that separate them. Drawing from interviews, project documents and participant observation, this chapter describes the incompatibilities between the two communities and what they suggest about current efforts to strengthen journalism in Africa through the application of ICTs.
Keywords: ICT4D; ICTs; IMPORTANT; Journalism; Kenya; state-building

Kenya’s tech community will not save journalism - Benequista, Nicholas (Briefing Note, 1, 2014)

Knowing When to Adapt - A Decision Tree - USAID (USAID LEARN, 2019)
This job aid will help you determine how best to resolve a challenge/issue in programming. It is intended for use by USAID, partners, and others working in international development.

Knowledge Management strategies and processes in Agile Software Development: a systematic literature review - Ouriques, Raquel Andrade Barros; Wnuk, Krzysztof; Gorschek, Tony; Svensson, Richard Berntsson (CoRR, 2018)
Knowledge-intensive companies that adopt Agile Software Development (ASD) relay on efficient implementation of Knowledge Management (KM) strategies to promotes different Knowledge Processes (KPs) to gain competitive advantage. This study aims to

Knowledge Retention and Transfer (KRT) - Model and Matsurity Matrix - USAID (CLA Toolkit, 2019)
The Maturity Matrix is not a standalone tool. It is one of the three major components of the KRT Model. The Maturity Matrix is meant to be used in conjunction with the KRT Toolkit and Implementation Plan.

LAC MEL Specialists Use Peer Network to Improve Quality and Use of Evidence - Anderson, Todd M; Prevatt, Amy (CLA Case Competition, 2019)
Based on consultations and stocktakings with LAC Missions, the LAC Bureau identified Mission Monitoring, Evaluation and Learning (MEL) Specialists as influential decision makers regarding data and evidence use. The LAC Bureau aims to improve the quality and use of evidence for decision making, and ultimately to improve development outcomes.