Case studies of results-based management by providers: Canada - OECD (OECD, 2017)

Case studies of results-based management by providers: New Zealand - OECD (OECD, 2017)

Case studies of results-based management by providers: Sweden - OECD (OECD, 2017)

Case studies of results-based management by providers: Switzerland - OECD (OECD, 2017)

Case studies of results-based management by providers: The Netherlands - OECD (OECD, 2017)

Case studies of results-based management by providers: United Kingdom - OECD (OECD, 2017)

Case studies of results-based management by providers: World Bank Group - OECD (OECD, 2017)

DAC - Mid-term Peer Review of Switzaerland - 2016 - OECD (OECD, 2016)

DAC Peer Review of Switzerland - 2009 - OECD (OECD, 2009)

DAC Peer Review of Switzerland - 2013 - OECD (OECD, 2013)

DAC Peer Review of Switzerland - 2019 - OECD (OECD, 2019)

Evaluation Feedback for Effective Learning and Accountability - OECD (Evaluation and Aid Effectiveness, 2001)
Evaluation is a key tool in improving the quality and effectiveness of development co-operation. The Development Assistance Committee (DAC) Working Party in Aid Evaluation is the only international forum where bilateral and multilateral evaluation experts meet regularly. Its Members meet to improve evaluation practice and to learn lessons from development co-operation activities and experiences. A series "Evaluation and Aid Effectiveness" was launched to share selected pieces of work with a broader public. This publication is composed of two parts: The Workshop Report, based on the fore-mentioned meeting, highlights the various issues raised, topics of discussion and different feedback systems, and outlines the areas identified by participants as most relevant for improving evaluation feedback. The Background Synthesis Report, intended as a starting point for discussion at the workshop, outlines the main concerns and challenges facing evaluation feedback and the means to address these. The report is based on an analysis of questionnaire results, and a review of previous initiatives in this area. The print version of the above report is available free of charge from the Development Co-operation Directorate; please send an email to dac.contact@oecd.org to get a copy.

Evaluation of development programmes - OECD - OECD (2019)
Data & research on evaluation of development programmes inc. paris declaration, budget support, multilateral effectiveness, impact evaluation, joint evaluations, governance, aid for trade

Glossary of Key Terms in Evaluation and Results Based Management - OECD (OECD, 2009)

Innovation for Development Impact: Lessons from the OECD Development Assistance Committee - OECD (The Development Dimension, 2020)
The development co-operation community needs to innovate to meet the global challenges ahead. Although it has an established track record for innovating partnerships, funding instruments and technologies, they are not enough to deliver the Sustainable Development Goals. This report synthesises the lessons emerging from an OECD Development Assistance Committee peer learning exercise on how innovation efforts can be strengthened, individually and collectively, to achieve the 2030 Agenda. The report is organised around three blocks – strategy, management and culture; organisation and collaboration; and, the innovation process – and provides recommendations on how innovation can best benefit poor and vulnerable people around the world.

OECD Development Assistance Committee Peer Review of Canada - OECD (OECD, 2018)

Systems Approaches to Public Sector Challenges: Working with Change - OECD (OECD, 2017)

Quality standards for development evaluation - OECD; Development Assistance Committee (OECD, 2010)
Based on a broad international consultative process, the DAC Quality Standards for Development Evaluation are a reference guide to good practice in development evaluation. With a view to improving the quality of evaluation processes and products, and facilitating collaboration, this reference guide lays out standards for each phase of a typical evaluation process: from defining purpose, to planning, designing, implementing, reporting, and learning from and using evaluation results.

Minister of International Development and La Francophonie Mandate Letter (November 12, 2015) - Office of the Prime Minister (2015)
Dear Minister:I am honoured that you have agreed to serve Canadians as Minister of International Development and La Francophonie. You will be part of a strong team of ministers led by the Minister of Foreign Affairs.

A Holistic Approach to Evaluating Research - Ofir, Zenda; Schwandt, Thomas; Duggan, Colleen; McLean, Robert (IDRC, 2016)

Open Governance Research Exchange - OGRX (Your hub for quantitative and qualitative research on innovations in governance, 2017)
Your hub for quantitative and qualitative research on innovations in governance.

Lessons Learned From the Use of the Most Significant Change Technique for Adaptive Management of Complex Health Interventions - Ohkubo, Saori; Mwaikambo, Lisa; Salem, Ruwaida M.; Ajijola, Lekan; Nyachae, Paul; Sharma, Mukesh Kumar (Global Health: Science and Practice, 2022)
Introduction:The Most Significant Change (MSC) technique is a complex-aware monitoring and evaluation tool, widely recognized for various adaptive management purposes. The documentation of practical examples using the MSC technique for an ongoing monitoring purpose is limited. We aim to fill the current gap by documenting and sharing the experience and lessons learned of The Challenge Initiative (TCI), which is scaling up evidence-based family planning (FP) and adolescent and youth sexual and reproductive health (AYSRH) interventions in 11 countries in Asia and sub-Saharan Africa. Methods:The qualitative assessment took place in early 2021 to document TCI’s use and adaptation of MSC and determine its added value in adaptive management, routine monitoring, and cross-learning efforts. Focus group discussions and key informant interviews were conducted virtually with staff members involved in collecting and selecting MSC stories. Results:TCI has had a positive experience with using MSC to facilitate adaptive management in multiple countries. The use of MSC has created learning opportunities that have helped diffuse evidence-based FP and AYSRH interventions both within and across countries. The responsive feedback step in the MSC process was viewed as indispensable to learning and collaboration. There are several necessary inputs to successful use of the method, including buy-in about the benefits, training on good interviewing techniques and qualitative research, and dedicated staff to manage the process. Conclusion:Our assessment results suggest that the MSC technique is an effective qualitative data collection tool to strengthen routine monitoring and adaptive management efforts that allows for flexibility in how project stakeholders implement the process. The MSC technique could be an important tool for global health practitioners, policy makers, and researchers working on complex interventions because they continually need to understand stakeholders’ needs and priorities, learn from lessons and evidence-based practices, and be agile about addressing potential challenges.

Instituciones que aprenden - Informe para la XXVII Cumbre Iberoamericana de Jefes y Jefas de Estado y de Gobierno - Oliván Cortés, Raúl (XXVII Cumbre Iberoamericana, 2020)
La pandemia de la Covid-19 ha brotado súbitamente en un momento de transición para las instituciones y organizaciones sociales de medio mundo. Cuando aún no se había superado la crisis de confianza de la última década entre los ciudadanos y los gobiernos, en un momento de gran impulso de las iniciativas de gobierno abierto, planes de innovación y transformación digital, para hacer más democráticas y eficientes las administraciones a través de programas políticos encaminados a configurar un nuevo contrato social, verde y digital en el marco de la Agenda 2030; el coronavirus ha provocado un shock histórico alterando el decurso del siglo XXI y exigiendo una aceleración de todos los procesos en marcha. La complejidad y dimensión de la pandemia ha puesto de manifiesto la necesidad de arquitecturas institucionales más flexibles, ágiles y resilientes, que sean capaces de incorporar toda la energía civil para aprender de su talento y creatividad, dándole mayor protagonismo a la ciudadanía (activistas, emprendedores, tejido asociativo, academia, makers…) no solo en la toma de decisiones sino también en el diseño e implementación de las estrategias. Conforme se centrifugaba a millones de empleados a teletrabajar desde sus casas, se hacía más poderosa la idea de pensar las organizaciones como flujos más allá de los organigramas estáticos de lugares y personas. Transformar las jerarquías en redes, concebirlas como cuerpos sociales dinámicos, no solo amplía su radio de acción y su conectividad exterior, también reactiva sus fortalezas internas, aflora los liderazgos ocultos, multiplica el valor social producido y maximiza el uso eficiente de los recursos en una época de limitaciones. Los laboratorios de innovación pública, social o ciudadana, o laboratorios de gobierno, junto a otras formas de innovación abierta y diseño social, se reivindican como proyectos inspiradores de un cambio de paradigma: de las instituciones que ordenan a las instituciones que aprenden. Pensar las organizaciones bajo el prisma de la ciencia de redes y la ética de los rizomas –nodos, enlaces, hubs, comunidades…– nos debería permitir una aproximación a la compleja y escurridiza tarea de configurar los ecosistemas de innovación y creatividad en el ámbito de lo público y lo social. Este informe propone un modelo denominado Hexágono de la Innovación Pública (HIP) que promueve un cambio sistémico a través de seis vectores (OPEN_ abierto, TRANS_ transversal, FAST_ ágil, PROTO_ modelado, CO_ colaborativo y TEC_ tecnológico) basados en las propiedades de las redes y en el análisis de 105 metodologías que usan las agencias más innovadoras del mundo. Se incluyen una herramienta de autodiagnóstico y el HIP-SIM, una primera aproximación a un software abierto para visualizar, modelar y simular la creación de ecosistemas innovadores con el que queremos propiciar una comunidad y un debate internacional.

Instituições que aprendem - Relatório para a XXVII Cimeira Ibero-Americana de Chefes de Estado e de Governo - Oliván Cortés, Raúl (XXVII Cimeira Ibero-Americana, 2020)
A pandemia da Covid-19 surgiu de súbito num momento de transição para as instituições e organizações sociais de meio mundo. Antes ainda de se ter ultrapassado a crise de confiança da última década entre os cidadãos e os governos, num momento de grande impulso das iniciativas de governo aberto e de planos de inovação e transformação digital para tornar mais democráticas e eficientes as administrações através de programas políticos encaminhados para configurar um novo contrato social, verde e digital no quadro da Agenda 2030, o coronavírus provocou um choque histórico alterando o curso do século XXI e exigindo uma aceleração de todos os processos em andamento. A complexidade e dimensão da pandemia revelou a necessidade de arquiteturas institucionais mais flexíveis, ágeis e resilientes, capazes de incorporar toda a energia civil para aprenderem com o seu talento e criatividade, dando maior protagonismo à cidadania (ativistas, empreendedores, tecido associativo, academia, makers…) não só na tomada de decisões mas também na configuração e implementação das estratégias. À medida que se afastava do centro a milhões de empregados para teletrabalhar a partir de casa, tornou-se mais poderosa a ideia de pensar nas organizações como fluxos que excedem os organigramas estáticos de lugares e pessoas. Transformar as hierarquias em redes, concebê-las como corpos sociais dinâmicos, não só alarga o seu raio de ação e conectividade externa, mas também reativa as suas forças internas, aflora as lideranças ocultas, multiplica o valor social produzido e maximiza o uso eficiente dos recursos numa época de limitações. Os laboratórios de inovação pública, social ou cidadã, ou laboratórios de governo, a par de outras formas de inovação aberta e configuração social, reivindicam-se como projetos inspiradores de uma mudança de paradigma: de instituições que ordenam para instituições que aprendem. Pensar nas organizações sob a ótica da ciência das redes e da ética dos rizomas – nós, ligações, hubs, comunidades… – deverá permitir-nos uma aproximação à complexa e subtil tarefa de configurar os ecossistemas de inovação e criatividade no âmbito público e social. Este relatório propõe um modelo denominado Hexágono da Inovação Pública (HIP) que promove uma transformação sistémica através de seis vetores (OPEN_ aberto, TRANS_ transversal, FAST_ rápido, PROTO_ modelado, CO_ colaborativo e TEC_ tecnológico) baseados nas propriedades das redes e na análise de 105 metodologias usadas pelas agências mais inovadoras do mundo. Incluem-se uma ferramenta de autodiagnóstico e o HIP-SIM, uma primeira aproximação a um software aberto para visualizar, modelar e simular a criação de ecossistemas inovadores com o qual queremos favorecer a comunidade e o debate internacional.

Modelo HIP - Hexágono de la innovación pública - Oliván Cortés, Raúl (2021)

The limits of agile - can we apply it to policy making? - Ollerhead, Lisa (2015)
Bringing new policy tools and techniques to the UK Government

Evaluating the Design of Behavior Change Interventions: A Case Study of Rhino Horn in Vietnam - Olmedo, Alegría; Sharif, Vian; Milner‐Gulland, E. J. (Conservation Letters, 2018)
Behavioral change interventions are increasingly widely used in conservation. Several projects addressing rhino horn consumption were recently launched in Vietnam. We used key informant interviews, document analysis, and marketing theory to explore their strategies for intervention design, implementation, monitoring, and evaluation. We developed a framework to evaluate whether they followed best practice and identify implementation challenges. Interventions could make greater use of key project design steps, including basing interventions on robust research to understand the behavior in question, identifying the target audience whose behavior interventions aim to change, and developing measures that can provide reliable evidence of success or not. Challenges include the need for law enforcement to complement campaigns; improving cooperation between NGOs; and clearly defining aims of demand-reduction initiatives. Using best practice from other fields and considering demand reduction within the wider context of wildlife, trade policy will help address these challenges.
Keywords: Awareness; Ilegal wildlife trade; Project design; demand reduction; social marketing

Locally Led Development: Engaging Local Stakeholders in Building An Evidence Base - OLS (USAID Learning Lab, 2020)
For two months in the Spring of 2020, the Office of Local Sustainability’s Evidence and Learning Team invited staff from across USAID to join us in exploring how the Agency approaches its generation and use of evidence from the perspective of locally led development. Our seven-part Standards of Evidence for Locally Led Development series brought together eight expert presenters and more than 670 participants to engage in conversations ranging from scientific research to complexity-aware monitoring to ethical considerations when conducting research. This blog post outlines some of our key learnings from the series and introduces the topics covered by each presenter. We encourage you to explore the event resources and share your own takeaways in the comment section below. WHAT DID WE LEARN? Three key themes emerged from the series: - Practitioners should draw on a wide range of evidence to inform locally led development programming. - Local actors can provide important contextual knowledge throughout the life of your research programs. - Prioritize local actors as the end users for your evidence to help them build their own self-reliance and increase sustainability of your program’s outcomes.

20 years of Outcome Mapping - OM Learning Community (2021)
Celebrate 20 years of Outcome Mapping with the OMLC Stewards plus special guests in this 90 minute webinar. - Exclusive celebration video with Outcome Blues soundtrack - Fireside chat with the authors of the OM manual, Sarah Earl, Fred Carden & Terry Smutylo, to hear their views on OM’s evolution since the 2001 publication - Launch of our new paper presenting core concepts of OM and guiding practices for transformative change, with special guests Sonal Zaveri & Julius Nyangaga as discussants

20 years of Outcome Mapping - Evolving practices for transformative change - OM Learning Community (Outcome Mapping Learning Community, 2021)
To mark 20 years of Outcome Mapping, this paper presents the core principles and concepts that are foundational to using the approach. It also presents a set of guiding practices to support transformative change. The OMLC Stewards presented the paper at a special webinar on 29th April 2021 - see the link below for the recording.

Systems Practice - Omidyar Group (The Omidyar Group, 2017)

Using Participatory Statistics to Examine the Impact of Interventions to Eradicate Slavery: Lessons from the Field - Oosterhoff, P.; Bharadwaj, S.; Burns, D.; Raj, A. M.; Nanda, R. B.; Narayanan, P. (2016)
This CDI Practice Paper by Pauline Oosterhoff, Sowmyaa Bharadwaj, Danny Burns, Aruna Mohan Raj, Rituu B. Nanda and Pradeep Narayanan reflects on the use of participatory statistics to assess the impact of interventions to eradicate slavery and bonded labour. It deals with: (1) the challenges of estimating changes in the magnitude of various forms of slavery; (2) the potential of combining participatory approaches with statistical principles to generate robust data for assessing impact of slavery eradication; and (3) the practical and ethical questions in relation to working with people living within a context of modern slavery. The paper draws lessons from the realities of using participatory statistics to support the evaluation of a slavery eradication programme in North India.

Modern Slavery Prevention and Responses in South Asia: An Evidence Map - Oosterhoff, Pauline; Yunus, Raudah; Jensen, Charity; Somerwell, Francis; Pocock, Nicola (DFID, 2018)
The Asia Pacific region has the highest numbers of both slavery and child labour victims in the world. The International Labour Organization (ILO) estimates that 16.5 million and 8.4 million persons were in situations of forced labour and forced marriage respectively in Asia Pacific, among 40.3 million in modern slavery globally (ILO & Walk Free Foundation 2017). Although there is a growing body of research and evaluations on specific sub-sectors and interest in the worst forms of labour exploitation, there has not yet been a systematic scoping or synthesis of studies that would help policymakers understand ‘what works’ to reduce the prevalence of “Modern Slavery”2(MS) in the countries of interest to DFID (India, Nepal, Bangladesh, Pakistan). India alone is estimated to have the largest number of people in modern slavery in the world with nearly 8 million persons living in modern slavery, followed by 3.2 million, 592,000 and 171,000 in Pakistan, Bangladesh and Nepal respectively (Walk Free Foundation 2018). While systematic reviews on interventions to reduce prevalence of trafficking, forced or bonded labour or slavery exist, these have not focused on the target countries of interest to DFID. Previous systematic reviews have focussed on particular types of MS, such as cross border sexual exploitation globally (van der Laan et al. 2011), labour exploitation in Europe (Cockbain et al. 2018) or community based interventions for safer migration programming in low and middle income countries (LMICs) (Zimmerman et al. 2016). Several systematic reviews focus specifically on health needs and post-trafficking care or interventions (Ottisova et al. 2016; Muraya & Fry 2016; Hemmings et al. 2016; Dell et al. 2017), and research methods and tools used in trafficking in persons (TIP) and health research (Cannon et al. 2018; Doherty et al. 2016). A recent global review and evidence map of MS interventions found some interventions in the target countries, but the outcomes were not specified (Bryant & Joudo 2018). In this Evidence Map, we address this gap in evidence for India, Nepal, Bangladesh and Pakistan specifically, scoping the range of Modern Slavery interventions and outcomes for specific target populations (survivors, employers, landlords, services providers, criminal justice officials) and at different levels (individual, community, state). This map is foremost targeted to DFID and its partners in India, Nepal, Bangladesh and Pakistan to inform evidence-based policy making. We hope that it is also useful to academics and practitioners working to address modern slavery, or in intervention areas or locations described.

Adaptive management in wildlife conservation - Organ, John; Decker, D.J.; McDonald, J.E.; Riley, Shawn; Mahoney, S.P. (The Wildlife Society Techniques Manual, 2012)
Adaptive management in wildlife conservation emerged from the wildlife profession's search for better solutions to increasingly complex conservation challenges. Adaptive management is an effective process for wildlife managers to employ to (1) deal with uncertainty in the management system, (2) learn from their management actions, and (3) achieve desired results. Being adaptable or flexible in your management approach is not the same as managing adaptively or conducting adaptive management. Adaptive management requires adhering to a stepwise process and fully executing each step. A critical step is rigorous monitoring and assessment of management interventions. Without this, wildlife managers cannot achieve the essence of adaptive management, which is the explicit goal of learning more about the management system after each management action.

“Shared learning” for building urban climate resilience – experiences from Asian cities - Orleans Reed, Sarah; Friend, Richard; Toan, Vu Canh; Thinphanga, Pakamas; Sutarto, Ratri; Singh, Dilip (Environment and Urbanization, 2013)
This paper considers how resilience thinking and, in particular, its emphasis on learning has been applied in 10 cities in Vietnam, India, Thailand and Indonesia. Applying a “shared learning” approach in the Asian Cities Climate Change Resilience Network (ACCCRN) has helped to create or strengthen networks, build appreciation for complexity and uncertainty among stakeholders, provide a space for deliberating concepts such as vulnerability and resilience, and build knowledge and capacities for stakeholders to engage and represent their own interests. Shared learning approaches face considerable challenges navigating politicized urban environments, in which the nature and value of existing systems − and therefore the value of building resilience − are contested. This article suggests that deliberate, strategic intervention by facilitators may contribute to more transformative change on behalf of equitable, socially just outcomes – and thus cautions against seeing urban climate vulnerability as a technical challenge, or shared learning as a “toolkit” for building resilience.

‘Learning journeys’ for adaptive management – Where does it take us? - Ørnemark, Charlotte (GPSA Note 12, 2016)
Keywords: IMPORTANT

What about the results? Lessons from long-term process support to strengthen results-based management (RBM) for Swedish framework NGOs operating in Western Balkans - Ørnemark, Charlotte (NCG, 2012)
The observations and lessons outlined in this report should be seen as a contribution to the on-going learning and reflections in a wider debate on how to assess and monitor results from support to CSOs involved in complex social change processes (as opposed to those involved in more ‘classical’ service delivery and more linear development assistance). These lessons, though directly emerging from the consultancy to provide long-term RBM support to the FOs and their local CSO partners in Western Balkans, also draws on previous experiences and similar lessons from on-going initiatives. In brief, these lessons call for a certain ‘paradigm shift’ in the way we (as development professionals) regard and support RBM for actors involved in less tangible social change processes in highly contested political spaces. Some of these shifts, outlined in more detail in different sections of this report, are summarised below:  From a linear, aggregated cause-and-effect thinking around RBM to one that is linked to human factors and is embedded in systems,  From pushing the burden of reporting down in the system to a clearer division of labour between donors, recipient governments, intermediary agents like framework NGOs (FOs) and implementing CSOs with each actor using its comparative advantage and appropriate role in the RBM process and for its own learning,  From operating in programming and results frameworks where outcomes and impact are largely assumed to be predictable to setting up systems that deal with uncertainty and that capture emerging result patterns through tracking of gradual changes,  From a project/programme perspective to a focus on institutions and systems as actors and arenas for change, where organisations are enabled to act as change agents towards clearly identified processes of social transformation,  From SMART to REAL results frameworks – although SMART (specific, measurable, achievable, realistic and time-bound) principles are always good to keep in mind for the formulation of goals and indicators, the danger of coming up with fictive measuring frameworks calls for a more systematic incorporation of ‘real’ concerns, including making RBM processes rights-based, embedded in local realities (and empowering for those involved), aligned to national reform efforts and national and international human rights commitments, and learning-oriented for all different levels of operations.

Planning & Navigating Social Change - Tools for Pacific voyagers - Orr, Douglas Epeli; Cavatore, Maria; Aston, Tom (Oxfam, 2019)
Those working to advance social change in the Pacific must understand, work with and respond to the complex and changing relationships and dynamics of power that exist within such networks, and situate their work in the context of decolonisation and self-determination. But the approaches typically used by NGOs to plan our projects and programs have failed to give due attention to these dynamics. The result is that our projects fall short of achieving their promise, despite being technically sound and logical. In this guide, we aim to provide a process and tools that prioritise and draw out the rich, often implicit, knowledge that Pacific Islanders have about our contexts to plan and manage social change initiatives in a manner that values and responds to this ocean of relationships. Drawing from these relationship-based and voyaging traditions, we emphasise the need to: • frame change in terms of change in people; e.g. change in the situation of groups of people and change in terms of the actions of people; • identify a course towards a particular ‘change destination’; • identify the signs to read that will help you determine where you are on that course; • be constantly attentive to the context as you make the journey, and • be ready to adjust your course in response to changes in your context. To support this way of working, we have selected a number of tools from approaches that we believe fit. In particular, we have drawn from the following approaches: • Thinking and Working Politically and other ‘politically aware’ approaches that recognise that the success of a development initiative will be determined by the interplay of relationships of power within a particular context, not just by having the right technical solution; • Outcome mapping because of its focus on people and relationships, and on identifying and supporting needed changes in the behaviour of specific actors in relation to others. This is key, because it’s individuals who deliver services, individuals who decide whether to allocate budget and resources to address a problem, and individuals who make the choice to believe whether a service is relevant to solving their issues; • Adaptive management because of its recognition that development initiatives take place within complex and changing systems and need to adjust to these and that this requires ongoing learning and adaptation.

Monitoring and Adaptive Management - Ossi, Damien (District of Columbia Wildlife Action Plan - 2015, 2015)

A diagnostic approach for going beyond panaceas - Ostrom, E. (Proceedings of the National Academy of Sciences, 2007)

VOICES - On speaking, mediation, representation and listening - Oswald, Katy (Think Piece, 2014)

Which Voices Are Heard and By Whom? - Oswald, Katy (Making All Voices Count, 2014)

Knowledge Management strategies and processes in Agile Software Development: a systematic literature review - Ouriques, Raquel Andrade Barros; Wnuk, Krzysztof; Gorschek, Tony; Svensson, Richard Berntsson (CoRR, 2018)
Knowledge-intensive companies that adopt Agile Software Development (ASD) relay on efficient implementation of Knowledge Management (KM) strategies to promotes different Knowledge Processes (KPs) to gain competitive advantage. This study aims to

Outcome Mapping FAQs - Outcome Mapping Learning Community (Outcome Mapping, 2018)
FAQ resource produced by IDRC's Evaluation Unit, with contributions from Kaia Ambrose, Sarah, Earl, Jan Van Ongevalle and Julius Nyangaga. We've added it here to answer some of your OM related questions.

Outcome Harvesting. Understanding social change results in complex circumstances - outcomeharvesting.net (2018)
This website is a source of applications, events and resources to support the development of a community of practitioners of Outcome Harvesting. BetterEvaluation is one of three sister sites of OutcomeHarvesting.net

‘We have the solutions to our own problems’: supporting civil society in Nigeria - Overseas Development Institute (ODI, 2014)
Development assistance programmes that start from careful analysis of what works and what does not in country contexts are still remarkably rare. One reason for this is the lack of well documented examples of ‘politically’ smart aid programming. To this end, this film outlines the experience of the State Accountability and Voice Initiative (SAVI), a DFID-funded programme in Nigeria providing support to civil society, media and governmental organisations.

Illegal Wildlife Trade - Frameworks - Oxford Martin Programme on the Illegal Wildlife Trade (Oxford Martin Programme on the Illegal Wildlife Trade, 2018)
Our frameworks will enable a better understanding of wildlife product consumer preferences and motivations to effectively influence the system.

Applied political economy analysis for human rights programs and campaigns: A guide for practitioners (second edition) - Pact (Pact, 2023)
This updated guide provides practical guidance to practitioners in the human rights sector and beyond on how to integrate Applied Political Economy Analysis

Organizational Performance Index (OPI) Handbook: A practical guide to the OPI tool for practitioners and development professionals - Pact (Pact, 2015)
This handbook is intended to help development practitioners understand how to use and what can be achieved by Pact’s Organizational Performance Index (OPI) tool. This document will provide practitioners and managers with information on how the Pact OPI works and how best to incorporate it in their country strategy or program.

Context-sensitive engagement: Lessons learned from Swiss experiences in South Asia for aid effectiveness in fragile scenarios - Paffenholz, Thania; Jütersonke, Oliver (CCDP, 2008)

A conceptual framework for analysing adaptive capacity and multi-level learning processes in resource governance regimes - Pahl-Wostl, Claudia (Global Environmental Change, 2009)
Governance failures are at the origin of many resource management problems. In particular climate change and the concomitant increase of extreme weather events has exposed the inability of current governance regimes to deal with present and future challenges. Still our knowledge about resource governance regimes and how they change is quite limited. This paper develops a conceptual framework addressing the dynamics and adaptive capacity of resource governance regimes as multi-level learning processes. The influence of formal and informal institutions, the role of state and non-state actors, the nature of multi-level interactions and the relative importance of bureaucratic hierarchies, markets and networks are identified as major structural characteristics of governance regimes. Change is conceptualized as social and societal learning that proceeds in a stepwise fashion moving from single to double to triple loop learning. Informal networks are considered to play a crucial role in such learning processes. The framework supports flexible and context sensitive analysis without being case study specific. First empirical evidence from water governance supports the assumptions made on the dynamics of governance regimes and the usefulness of the chosen approach. More complex and diverse governance regimes have a higher adaptive capacity. However, it is still an open question how to overcome the state of single-loop learning that seem to characterize many attempts to adapt to climate change. Only further development and application of shared conceptual frameworks taking into account the real complexity of governance regimes can generate the knowledge base needed to advance current understanding to a state that allows giving meaningful policy advice.
Keywords: Adaptive capacity; Adaptive governance; Climate change adaptation; Complexity; Institutions; Resource management; Social learning

MOTION HANDBOOK - Developing a transformative Theory of Change - Palavicino, Carla Alvial; Matti, Cristian; Witte, Jenny (TIPC, Utrecht University, 2022)
A step-by-step guide on how to develop a Transformative Theory of Change, for innovation projects, programmes and organisations working on systems transformation. The MOTION project was initiated with one key question in mind: how can we help projects and organisations be more transformative, using the framework and concept provided by the multi-level perspective? And what kind of tools, methods and frameworks can we co-design that translate scientific concepts into practises relevant for policy practitioners? This led us into a co-creation journey during which researchers from the Transformative Innovation Policy Consortium (TIPC) and EIT Climate-KIC project partners experimented, reflected and learned from each other in building the approach that we share in this handbook. Through this journey, we gained a deeper understanding of what the portfolio approach means in a transformative system change context and which skills and competences are needed to facilitate processes of co-creation in the science-policy-practice interface. We had the opportunity to configure the key building blocks of our theoretical approach, the Transformative Outcomes, into practical insights and actions that can easily be applied by innovation organisations at many levels. This handbook is the culmination of the journey as it translates key learnings from the MOTION project into practical insights that are relevant to practitioners working on systems transformation.

Participación Digital y mejora democrática - un camino sin regreso - Palumbo, Gabriel (Luminate, 2020)
Tal vez como nunca en la historia, la democracia está siendo desafiada. A los viejos problemas, muchos no resueltos, se les agregan otros, surgidos de la incorporación de tecnología, de la emergencia de nuevos actores sociales y de una conformación de la subjetividad política cada vez más compleja e impredecible. Esta investigación pretende reactualizar algunos de los debates clásicos sobre la democracia y, al mismo tiempo, sentar una base empírica que permita pensar de qué modo la tecnología puede ayudar a mejorar la calidad democrática. Según nuestro estudio, la participación digital está promoviendo modificaciones importantes en la relación entre la ciudadanía y los decisores políticos, ampliando los espacios de proximidad. Esto sucede, fundamentalmente, porque baja las barreras de acceso y de incentivos haciendo más fácil, más efectiva y más mensurable la participación ciudadana. La participación digital aumenta el períodos electorales y fomenta el asociacionismo cívico como posibilidad cierta y concreta de presionar sobre los decisores políticos. Lo digital habilita, además, participación offline y genera espacios de pedagogía cívica. La capacidad de implantación de temas en el debate público que se puede generar a través de ella mejora la conversación pública, impactando a su vez la calidad de la democracia. Desde Luminate entendemos que favorecer los espacios de rendición de cuentas desde la ciudadanía hacia el poder político es una apuesta al robustecimiento de la experiencia democrática y que las formas digitales están abriendo posibilidades que otras opciones no pudieron lograr. Esperamos que esta investigación sea un aporte para poder seguir reflexionando y avanzando en esta dirección.

Guide to Evaluating Participatory Processes - Practical guide - Parés, Marc (2013)
Guide to Evaluating Participatory Processes - Practical guide
Keywords: Academia; Biology; Computer Science; Earth Sciences; Geography; History; Law; Math; Medicine; Philosophy; Physics; Political science; Psychology; Religion; Research; economics; universities

The Politics of Evidence: From Evidence-based Policy to the Good Governance of Evidence - Parkhurst, Justin (Taylor & Francis, 2017)
The Open Access version of this book, available at http://www.tandfebooks.com/, has been made available under a Creative Commons Attribution-Non Commercial-No Derivatives 3.0 license. There has been an enormous increase in interest in the use of evidence for public policymaking, but the vast majority of work on the subject has failed to engage with the political nature of decision making and how this influences the ways in which evidence will be used (or misused) within political areas. This book provides new insights into the nature of political bias with regards to evidence and critically considers what an ‘improved’ use of evidence would look like from a policymaking perspective. Part I describes the great potential for evidence to help achieve social goals, as well as the challenges raised by the political nature of policymaking. It explores the concern of evidence advocates that political interests drive the misuse or manipulation of evidence, as well as counter-concerns of critical policy scholars about how appeals to ‘evidence-based policy’ can depoliticise political debates. Both concerns reflect forms of bias – the first representing technical bias, whereby evidence use violates principles of scientific best practice, and the second representing issue bias in how appeals to evidence can shift political debates to particular questions or marginalise policy-relevant social concerns. Part II then draws on the fields of policy studies and cognitive psychology to understand the origins and mechanisms of both forms of bias in relation to political interests and values. It illustrates how such biases are not only common, but can be much more predictable once we recognise their origins and manifestations in policy arenas. Finally, Part III discusses ways to move forward for those seeking to improve the use of evidence in public policymaking. It explores what constitutes ‘good evidence for policy’, as well as the ‘good use of evidence’ within policy processes, and considers how to build evidence-advisory institutions that embed key principles of both scientific good practice and democratic representation. Taken as a whole, the approach promoted is termed the ‘good governance of evidence’ – a concept that represents the use of rigorous, systematic and technically valid pieces of evidence within decision-making processes that are representative of, and accountable to, populations served.
Keywords: Public policy; Social Policy; Social Services & Welfare

The politics of evidence: from evidence-based policy to the good governance of evidence - Parkhurst, Justin O. (Routledge studies in governance and public policy, 2017)
Introduction -- Evidence-based policymaking : an important first step, and the need to take the next -- Bias and the politics of evidence -- The overt politics of evidence : bias and the pursuit of political interests -- The subtle politics of evidence - the cognitive-political origins of bias -- What is "good evidence for policy"? From hierarchies to appropriate evidence -- What is the "good use of evidence" for policy? -- From evidence-based policy to the good governance of evidence

Are we suffering from obsessive measurement disorder? - Pasanen, Tiina (From Poverty to Power, 2019)
ODI’s Tiina Pasanen argues that more data doesn’t necessarily mean we make better decisions, and sets out some ideas for how we count what counts.

Five tips for evaluating your impact in international development - Pasanen, Tiina (The Guardian, 2015)
How can impact evaluations actually be helpful? These top tips will save development professionals time, energy and money. International development can be messy – with uncertain, complex settings, and multiple partners with different interests, goals and capacities. At the same time, we are under increasing pressure to demonstrate impact. We have to show that our projects have made a real change in people’s lives and that donor or taxpayer money hasn’t been wasted. But impact evaluations, which are often seen as a solution to this, aren’t always used. And in some cases they are unhelpful. So how can we improve the quality of impact evaluations, so that they produce results that are useful? This is where evaluability assessments come in. By asking whether we should evaluate a project – and if so, when and how – these assessments can improve the quality of impact evaluations. Here are five other reasons to assess evaluability before starting an impact evaluation.

Monitoring and evaluation: five reality checks for adaptive management - Pasanen, Tiina (ODI Insight, 2017)

Monitoring And Evaluation Tools and Approaches to Support Adaptive Management - Pasanen, Tiina (GLAM, 2019)

Supporting adaptive management. Monitoring and evaluation tools and approaches - Pasanen, Tiina; Barnett, Inka (ODI, 2019)
Key messages • This working paper introduces a set of monitoring and evaluation (M&E) tools and approaches, discussing their potential usefulness in supporting adaptive management in development and humanitarian programmes. It emphasises adaptive programmes characterised by complex aspects, such as: (1) they are innovative; (2) they have uncertain pathways for change; or (3) they operate in uncertain or unstable environments. • The majority of these tools have been used in international development for many years. However, adaptive management brings additional challenges for monitoring and evaluating programmes, as they require intentional M&E design from the start that is oriented towards both learning and accountability. • All of the tools and approaches introduced in this paper have potential to support learning and adaptation, although in various ways and at different stages of a programme. Some tools can support strategic planning and diagnosing throughout a programme – especially during design and inception – while others can help analyse causal relationships at specific points in a programme. It is important to tailor the approach used for its intended purpose. However, whether learning and adaptation happens depends also on factors other than the choice of M&E methods. • For some of these approaches a considerable body of evidence already exists but, for many, more practical examples and systematic analysis is needed. In addition to building the evidence base concerned with which approaches are suitable for different types of adaptive programmes and why, it is also important to improve understanding of the enabling environmental conditions necessary for the tools and approaches outlined here to facilitate and strengthen evaluative thinking, evidence-informed decision-making and ongoing programme iteration.

How to design a monitoring and evaluation framework for a policy research project - Pasanen, Tiina; Shaxson, Louise (Methods Lab, 2016)
Policy research projects face a number of challenges: policy processes are complex, involve multiple actors and often feature a significant time-lag between research and what may or may not happen as a result of it. To complicate matters further, the scope and scale of policy research projects are increasingly moving away from single research studies towards multi-component, multi-site and multi-sector endeavours. These factors mean that developing an overarching monitoring and evaluation framework can be challenging. But it is not impossible. The guidance note aims to support the first steps in designing and structuring the M&E framework (that is, what aspects or areas of policy research projects to monitor and evaluate, why, when and how). It builds on an M&E framework for policy research projects developed and tested by the Research and Policy in Development (RAPID) programme of the Overseas Development Institute, and draws on case studies of how it has been used by RAPID and other research teams.

We created this MEL system for you, now please own it! | From Poverty to Power - Pasanen, Tiina; Yanguas, Pablo (2021)
External consultants, learning partners or critical friends -whatever we call them- can seldom change the system or organisational (learning) culture from outside. So, how can Monitoring Evaluation and Learning (MEL) consultants support real change instead of creating tools or processes that are quickly forgotten without any real institutional ownership? Consultants and learning partners can seldom change a learning system or organisational culture – it just does not work like that. What they can do is interrogate existing practices, introduce new ideas, and support change processes already underway to ensure lasting, transformative impact. They are a multiplier for homegrown change initiatives, not a substitute. Don’t hire MEL consultants to sell you tools or tell you what to do. Hire them because you need a critical friend along the path of organisational change. Own, then learn – not the other way round.

Your Company’s Secret Change Agents - Pascale, Richard T.; Sternin, Jerry (Harvard Business Review, 2005)
Somewhere in your organization, groups of people are already doing things differently and better. To create lasting change, find these areas of positive deviance and fan their flames.

The Power of Positive Deviance - Pascale, Richard; Sternin, Jerry; Sternin, Monique (Harvard Business Press, 2010)

The Rapid Evaluation, Action, and Learning (REAL) Approach: A toolkit to measure and refine changes and interventions in health campaigns - PATH (PATH, 2019)
Evaluative Tools for Improving Campaign Effectiveness Public health campaigns aim to control disease or deliver essential health services and products through time-limited and periodic channels. Many countries use campaigns to augment or replace routine service delivery, to target certain populations, or to accelerate progress towards coverage targets. The Root Cause Analysis and Rapid Evaluation, Action, and Learning toolkits present a systematic but flexible approach to identifying the root causes of campaign bottlenecks and then designing, testing, and refining solutions to optimize potential impact. These toolkits respond to a growing need for “fit-for-purpose” rapid-testing, adaptive learning approaches to evaluation and the need for a culture shift toward iterative adaptation and improvement that integrates measurement and evidence-informed decision-making into daily practice. The comprehensive package of toolkits, in addition to the individual Root Cause Analysis toolkit and Rapid Evaluation, Action, and Learning toolkit are available below.

Complexity-Aware Monitoring Discussion Note (Screencast) - Patsalides, Melissa; Britt, Heather (USAID, 2014)
This screencast covers information included in the Complexity-Aware Monitoring Discussion Note. The topic is intended for those seeking cutting-edge solutions to monitoring complex aspects of strategies and projects. Most of the principles and promising approaches discussed here have a significant body of theory and practice behind them, but many have not been used to monitor USAID strategies and projects. We have much to learn about whether and how they can be applied successfully in the Agency. USAID's Office of Learning, Evaluation, and Research in the Bureau of Policy, Planning, and Learning has partnered with different groups through monitoring trials to build a body of evidence regarding which complexity-aware M&E approaches are most effective in the contexts of USAID programming.

Developmental Evaluation - Patton, Michael Quinn (The Nonprofit Quarterly, 2006)

Developmental Evaluation: Applying Complexity Concepts to Enhance Innovation and Use - Patton, Michael Quinn (Guilford Press, 2010)
Developmental evaluation (DE) offers a powerful approach to monitoring and supporting social innovations by working in partnership with program decision makers. In this book, eminent authority Michael Quinn Patton shows how to conduct evaluations within a DE framework. Patton draws on insights about complex dynamic systems, uncertainty, nonlinearity, and emergence. He illustrates how DE can be used for a range of purposes: ongoing program development, adapting effective principles of practice to local contexts, generating innovations and taking them to scale, and facilitating rapid response in crisis situations. Students and practicing evaluators will appreciate the book's extensive case examples and stories, cartoons, clear writing style, "closer look" sidebars, and summary tables. Provided is essential guidance for making evaluations useful, practical, and credible in support of social change.See also Developmental Evaluation Exemplars, edited by Michael Quinn Patton, Kate McKegg, and Nan Wehipeihana, which presents 12 in-depth case studies.
Keywords: Evaluation; Measurement; Organizational effectiveness; Project management

Essentials of Utilization-Focused Evaluation - Patton, Michael Quinn (SAGE Publications, 2011)
Based on Michael Quinn Patton's best-selling Utilization-Focused Evaluation, this briefer book provides an overall framework and essential checklist steps for designing and conducting evaluations that actually get used. The new material and innovative graphics present the utilization-focused evaluation process as a complex adaptive system, incorporating current understandings about systems thinking and complexity concepts. The book integrates theory and practice, is based on both research and professional experience, and offers new case examples and cartoons with Patton's signature humor.
Keywords: Reference / Research; Research

Principles-Focused Evaluation: The GUIDE - Patton, Michael Quinn (Guilford Press, 2017)
How can programs and organizations ensure they are adhering to core principles--and assess whether doing so is yielding desired results? From evaluation pioneer Michael Quinn Patton, this book introduces the principles-focused evaluation (P-FE) approach and demonstrates its relevance and application in a range of settings. Patton explains why principles matter for program development and evaluation and how they can serve as a rudder to navigate the uncertainties, turbulence, and emergent challenges of complex dynamic environments. In-depth exemplars illustrate how the unique GUIDE framework is used to determine whether principles provide meaningful guidance (G) and are useful (U), inspiring (I), developmentally adaptable (D), and evaluable (E). User-friendly features include rubrics, a P-FE checklist, firsthand reflections and examples from experienced P-FE practitioners, sidebars and summary tables, and end-of-chapter application exercises.

Utilization-Focused Evaluation (U-FE) Checklist - Patton, Michael Quinn (Evaluation Checklists Projects, 2013)
Utilization-Focused Evaluation begins with the premise that evaluations should be judged by their utility and actual use; therefore, evaluators should facilitate the evaluation process and design any evaluation with careful consideration of how everything that is done, from beginning to end, will affect use. Use concerns how real people in the real world apply evaluation findings and experience and learn from the evaluation process. The checklist is based on Essentials of Utilization-Focused Evaluation (Patton, 2012, Sage Publications). All references in the checklist to exhibits and menus refer to this book. Step 1 Assess and build program and organizational readiness for utilization-focused evaluation. Step 2 Assess and enhance evaluator readiness and competence to undertake a utilizationfocused evaluation. Step 3 Identify, organize, and engage primary intended users. Step 4 Conduct situation analysis with primary intended users Step 5 Identify primary intended uses by establishing the evaluation’s priority purposes. Step 6 Consider and build in process uses if appropriate. Step 7 Focus priority evaluation questions. Step 8 Check that fundamental areas for evaluation inquiry are being adequately addressed. Step 9 Determine what intervention model or theory of change is being evaluated. Step 10 Negotiate appropriate methods to generate credible findings and support intended use by intended users. Step 11 Make sure intended users understand potential controversies about methods and their implications. Step 12 Simulate use of findings. Step 13 Gather data with ongoing attention to use. Step 14 Organize and present the data for use by primary intended users. Step 15 Prepare an evaluation report to facilitate use and disseminate significant findings to expand influence. Step 16 Follow up with primary intended users to facilitate and enhance use. Step 17 Metaevaluation of use: Be accountable, learn, and improve

Utilization-focused evaluation for agricultural innovation - Patton, Michael Quinn; Horton, D (ILAC Brief, 2009)
Utilization-focused evaluation (UFE) is based on the principle that an evaluation should be judged by its utility. So no matter how technically sound and methodologically elegant, an evaluation is not truly a good evaluation unless the findings are used. UFE is a framework for enhancing the likelihood that evaluation findings will be used and lessons will be learnt from the evaluation process. This Brief, based on the book Utilization-focused evaluation, introduces this approach to evaluation, outlines key steps in the evaluation process, identifies some of the main benefits of UFE, and provides two examples of UFE in the context of programmes aimed at promoting agricultural innovation.

Reflections on if and how our partnership is working in Bangladesh - Paul, Sukanta; Snijder, Mieke; Apgar, Marina (CLARISSA, 2020)
Effective partnership working is crucial to producing quality results, especially when working with complex problems such as the worst forms...

Lessons Learned from Applying the Data Powered Positive Deviance - Pawelke, Andreas; Glücker, Andreas; Albanna, Basma; Boy, Jeremy (DPPD, 2021)
This report presents six learnings from four pilot projects conducted by the Data Powered Positive Deviance (DPPD) initiative, a global collaboration between the GIZ Data Lab, the UNDP Accelerator Labs Network, the University of Manchester Center for Digital Development, and UN Global Pulse Lab Jakarta. The pilots seek out grassroots solutions to development challenges that range from the interaction between livestock farming and deforestation to gender-based violence and insecurity in dense urban environments in Ecuador, Mexico, Niger and Somalia. The learnings relate to the early stages of the DPPD method, originally proposed by Albanna & Heeks [1], and focus mainly on the access to, and use of digital data. They are summarized as follows: 1. Remain flexible in the face of data unavailability 2. Leverage existing partnerships for data access 3. Map and fill know-how gaps early 4. Scale with caution 5. Look at deviance over time 6. Look beyond individual or community practices and behavior The report is written for development practitioners, data analysts, domain experts, and more generally anyone interested in using new data sources and technologies to uncover successful local solutions to development challenges.

Realist Evaluation - Pawson, Ray; Tilley, Nick (2004)
Contents: Introduction 1. The nature of programmes and how they work 2. Basic concepts in the explanation and understanding of programmes 3. Strategies and methods of realist evaluation 4. Realism’s place in the policy cycle: formative, summative and synthetic approaches 5. The nature, presentation and use of findings from realist evaluation 6. Conclusion: strengths, limitations and relationships with other approaches Appendix I – ‘Thinking it through’: an exercise in realist hypothesis making. Appendix II – ‘Varieties of realist evaluation’: pocket illustrations of quantitative, qualitative, formative and synthetic applications. Appendix III – ‘Would it work here?’: a grid to help decide on the feasibility of mounting a programme ‘on your patch’.

Measuring Impact. Making Use of the Portfolio: Organizational Learning at USAID - Peabody, Shawn (Measuring Impact, 2015)
This technical analysis explores previous and ongoing social learning efforts, best practices, challenges, and lessons in USAID as a foundation for improving the implementation and design of the Agency’s forestry and biodiversity programs. This analysis is particularly relevant as the Bureau of Economic Growth, Education and the Environment’s Office of Forestry and Biodiversity (E3/FAB) begins to develop a Cross-Mission Learning Program under the Measuring Impact initiative, a five-year activity to promote the adoption of best practices in the USAID program cycle among Missions using biodiversity funds. The findings reported in this document can help inform the overall design and structure of the Learning Program and provide insight into possible challenges and best practices.

Leveraging Experimental Evaluations for Understanding Causal Mechanisms - Peck, Laura R. (New Directions for Evaluation, 2020)
Experimental evaluations—especially when grounded in theory-based impact evaluation—can provide insights into the mechanisms that generate program impacts. This chapter details variants of experimental evaluation designs and also analytic strategies that leverage experimental evaluation data to learn about causal mechanisms. The design variants are poised to illuminate causal mechanisms related to program implementation and the contribution of selected components of multifaceted programs. The analysis strategies lend themselves to illuminating causal mechanisms related to participants’ responses to program components as well as to the contributions of selected program components themselves. The chapter offers an example from one, theory-based impact evaluation, which embedded both design and analytic strategies to examine the extent to which specific program components and participant experiences might be identified as causal mechanisms. The particular value in using this theory-based experimental strategy is that the results are rigorous and potentially highly relevant to policy and practice.

Evaluability assessment for Impact Evaluation: Guidance, checklists and decision support - Peersman, Greet; Guijt, Irene; Pasanen, Tiina (ODI, 2015)
This guidance note focuses on the utility of, and guidance for, evaluability assessment before undertaking an impact evaluation. The primary audience for this guidance note is evaluators conducting an evaluability assessment for impact evaluation. The secondary audience is people commissioning or managing an evaluability assessment for impact evaluation, as well as funders of an impact evaluation. Sections one and two provide an overview of evaluability assessment and how it can be used for impact evaluation. Section three provides guidance for planning to undertake an evaluability assessment for impact evaluation. This is informative for all intended users of the guidance note. Section four includes checklists and decision support for evaluability assessments. The checklist is geared to those conducting the evaluability assessment and can be adapted to suit a particular context or purpose. The decision support provides those conducting an evaluability assessment with evidence-based recommendations for impact evaluation funders and commissioners, about whether, when and how to proceed with the evaluation. Sections five and six provide guidance on what to do after the assessment is concluded, and offer lessons learned from evaluability assessments in practice.

When and how to develop an impact-oriented monitoring and evaluation system - Peersman, Greet; Rogers, Patricia; Guijt, Irene; Hearn, Simon; Pasanen, Tiina; Buffardi, Anne (2016)
This guidance note focuses on: • what an impact-oriented monitoring and evaluation system entails • why an organisation may want to establish such a system • when integrating an impact-orientation into an monitoring and evaluation system is most useful • what should be considered in developing the monitoring and evaluation system, or in tweaking an existing system, to become more impact-focused. The primary audience for this guidance note is internal and external monitoring and evaluation advisors involved in designing and implementing, and/or assessing monitoring and evaluation systems to include a focus on impact. It will also be useful for senior management of organisations who need to know how best to plan for a sustainable monitoring and evaluation system that supports impact assessment or to adapt an existing system to incorporate an impact perspective.

Crowdsourcing citizen feedback on district development in Ghana using interactive voice response surveys - Pegus, Ciana-Marie (Making All Voices Count Practice Paper, 2016)
Keywords: IMPORTANT

When Does ICT-Enabled Citizen Voice Lead to Government Responsiveness? - Peixoto, Tiago; Fox, Jonathan (IDS Bulletin, 2016)

Citizen engagement: emerging digital technologies create new risks and value - Peixoto, Tiago; Steinberg, Tom (World Bank, 2019)
The recent rapid evolution of digital technologies has been changing behaviors and expectations in countries around the world. These shifts make it the right time to pose the key question this paper explores: Will digital technologies, both those that are already widespread and those that are still emerging, have substantial impacts on the way citizens engage and the ways through which power is sought, used, or contested? The authors address this question both to mitigate some of the World Bank’s operational risks, and to initiate a conversation with peers about how those risks might require policy shifts. The overall framing question also is being explored in case theapproaches to citizen engagement advocated by the World Bank are changing and may require different advice for client countries. Despite the lower technology penetration levels in developing countries, their more malleable governance contexts may be more influenced by the effects of emerging technologies than older states with greater rigidity. Digitally influencedcitizen engagement is, in short, one of those “leapfrog” areas in which developing nations may exploit technologies before the wealthier parts of the world. But countries can leapfrog to worse futures, not just better ones. This paper explores what technology might mean for engagement, makespredictions, and offers measures for governments to consider.

MLE or MEL in adaptive programming? - Pellini, Arnaldo (Knowledge counts, 2018)

Working Politically: A story of Change about the contribution of research evidence to the new Village Law in Indonesia - Pellini, Arnaldo; Angelina, Maesy; Purnawati, Endah (Austrialian Community Development and Civil Society Strenghtening Scheme (ACCESS), 2014)
On 18 December 2013, the Indonesian House of Representatives passed the new Village Law, a vote that was the culmination of a journey that had started in 2007. This Story of Change takes the passing of the Village Law as its starting point and describes the relative influence that research-based evidence, produced by the Institute for Research and Empowerment (IRE), has had at critical junctions of the legislative process. This Story of Change concludes that good quality, research-based evidence is necessary but not sufficient to influence policy-making processes. Researchers and research organisations need to think and work politically to achieve their influencing goals and to adapt to changes in local circumstances.

Doing Development Differently at Scale - Pellini, Arnaldo; Karetji, Petrarca C.; Soekadis, Ade (Knowledge, Politics and Policymaking in Indonesia, 2018)
In recent decades there has been an increasing recognition that politics and political institutions matter for development. There is also a much greater interest in contextually grounded approaches. This has stemmed from an acknowledgement that purely technocratic approaches to development often result in failure because they do not take into account the nature of political institutions. Nor do they take account of the context in a particular developing country and the interests and incentives of powerful national actors. Policy processes are embedded in specific social, political and organisational contexts. Approaches that focus on implementing universal best practices in evidence-informed policymaking are unlikely to be successful. Instead, what is needed is an approach that takes the local context as the starting point for understanding what issues are relevant to policymakers and developing contextually appropriate solutions. The authors of this chapter reflect on the management approaches and systems that may be required to enable and support large-scale development programmes to be flexible and adaptive to the local policy context and circumstances. The chapter argues that such programmes struggle to adopt adaptive management principle and that to do that at scale requires some changes in the way such programmes are managed.
Keywords: Adaptive Management; Doing development differently; Evidence-informed policymaking; Indonesia; Knowledge sector; Thinking and working politically

A Political Economy Analysis Framework for EdTech Evidence Uptake - Pellini, Arnaldo; Nicolai, Susan; Magee, Arran; Sharp, Sam; Wilson, Sam (The EdTech Hub, 2021)

On understanding software agility: A social complexity point of view - Pelrine, Joseph (E:CO, 2011)
Over the last decade, the field of so-called Agile software development has grown to be a major force in the socio-economic arena of delivering quality software on time, on budget, and on spec. The acceleration in changing needs brought on by the rise in popularity of the Internet has helped push Agile practices far beyond their original boundaries, and possibly into domains where their application is not the optimal solution to the problems at hand. The question of where Agile software development practices and techniques make sense, and where are they out of place, is a valid one. It can be addressed by looking at software development as a complex endeavour, and using tools and techniques from the Cynefin method and other models of social complexity.
Keywords: IMPORTANT

A Theory of Change of citizen participation: an update - Peña-López, Ismael (ICTlogy, 2020)
when it was reduced to a subsidiary internal service lacking all kind of political attributions. The work done in those years had been formidable, but too many things had passed since, especially the 15M Spanish Indignados Movement, the raise of technopolitics… and the raise of populism and fascism all across Europe. We urgently needed a theoretical framework in which to substantiate our political strategy, so I came up with a Theory of Change of citizen participation (see Figure 1) which defined four expected impacts of our political action: 1. Efficiency, efficacy and legitimacy of public decisions improves. 2. Populism has decreased in institutions and the public sphere. 3. Citizens understand the complexity of public decision-making. 4. Citizen participation and political engagement clearly shifts towards a technopolitical paradigm. These impacts were expected to be achieved after some outcomes resulting from some outputs grouped in five programmes: 1. Programme of citizen participation. 2. Programme of internal participation. 3. Programme of collaboration. 4. Programme of intermediaries, facilitators and infomediaries. 5. Programme of e-participation, e-voting and technopolitics. 20 months after, the Theory of Change of Citizen Participation has worked quite well. But it does have some limitations, especially at the operational level-which is what the whole thing was about, to help in putting some order in our daily work.

Citizen participation and the rise of the open source city in Spain - Peña-López, Ismael (MAVC & IT for Change, 2017)
Research on the Information Society, the Digital Divide and Information and Communication Technologies for development

Gestión de la complejidad para el impacto sistémico: respuestas a entornos VUCA y BANI - Peña-López, Ismael (SociedadRed, 2023)
Research on the Information Society, the Digital Divide and Information and Communication Technologies for development

ICT4D Blog » A citizen participation ecosystem - Peña-López, Ismael (ICTlogy, 2019)
Research on the Information Society, the Digital Divide and Information and Communication Technologies for development

Review of current system & portfolio practices in the development sector: Organizational… - Pereverza, Kateryna (Medium, 2024)
Building on interviews with colleagues in development organizations, donor & partner governments and foundations, this article explores the adoption of system and portfolio-based approaches in the development sector focusing on organizational motivations and challenges, changes in the network of the development sector actors, and types of collaborative experiments initiated around the novel approach. The aim is to take steps towards understanding how this type of approach for system change can be institutionalized in the sector and become one of the main ways of operating for delivering transformative impact. Summary While there is a broad consensus that linear, fragmented, projectized ways of doing development & public policy are not a match for the type of context we live in today, there is less consensus about what alternatives look like. In this space, system and portfolio approaches have emerged during the last few years and a new field is beginning to slowly emerge. Currently, there is a diversity in interpretations of system and portfolio-based approaches by the development sector organizations. This is connected to different logics and theories of transformative change in complex systems. Also, the intent with the adoption of the novel approach can vary along the dimension of organizational efficiency vs place-based transformations. Organizations engage with portfolio-based approaches with different intents and in different roles. For example, OECD OPSI takes forward the innovation portfolio management for partners; the European Commission funds place-based portfolios e.g. within the M4EG program; while UNDP facilitates place-based and thematic portfolios for system transformations. When it comes to the motivations of development sector partners to experiment with system and portfolio-based approaches, these include increased effectiveness & efficiency of the use of funds for place-based transformations; response to the post-colonial critique; creation of visibility as a frontrunner; the potential of joint risk-taking; the opportunity of collective learning from an entire portfolio (coherence & dynamic evolution); and directionality added to internal and external activities. Challenges encountered by partners engaged in the experimentation with system and portfolio-based approaches are connected to resources and time constraints that emerge from the prevailing logics & constraints of current organizational structures premised on fragmentation & projectization; organizational leadership dynamics; the influence of dominant organizational culture and mindsets; the difficulties of adopting the ‘portfolio language’/jargon; prevailing funding structures and cycles; and a shortage of both time and capacity to engage with numerous local partners. The structure of the development sector ecosystem is undergoing changes driven by a rapidly changing context. The experimentation with system and portfolio-based approaches has led to new actors joining the space, such as innovative consultancies developing the logic of transformative change in systems and facilitating portfolios on the ground, and the emergence of new roles and relationships, for example, ‘learning partners’. Since 2020, three noticeable types of collaborative experiments emerged in the development sector ecosystem: systemic investments; thematic and place-based portfolio interventions; and monitoring, evaluation and learning (MEL) for the context of system change. This study revealed areas for further exploration and analysis. One is a need to co-create institutional structures to enable a co-ownership of place-based portfolios by several international partners as well as local stakeholders. Another area connects to the need to create and implement spaces and tools for collective learning from portfolios. Finally, there is a need to understand what mechanisms can enable the institutionalization of novel approaches on the level of the entire sector and their interconnection with organizational changes needed. This study is based on the interviews with representatives of ten development sector organizations conducted in spring 2023 and insights shared within portfolio practitioner gatherings and the M&E Sandbox webinars run during 2023. Introduction: The article emphasizes the importance of (re)framing challenges in portfolio-based approaches for steering system transformations. It acknowledges that the way we frame challenges significantly influences the solutions we develop, and therefore, it’s crucial to address the right problems rather than wasting resources on incorrect issues. Study Insights: The insights come from a study initiated in autumn 2021, focusing on how framing processes unfold, what needs to be framed, the open-ended nature of framing, and the roles international organizations like UNDP can play in developing inclusive and system transformation-oriented framings of urban challenges. Key Points: The study involved interviews with experts from the UNDP Istanbul and Thailand Regional hubs. It explored the experimentation with the portfolio approach in collaboration with local and national governments and partners. The article presents insights related to the framing process and highlights open questions for researchers and practitioners. System Transformations: System transformations require a deep understanding beyond surface-level issues, as existing systems often limit our imagination and prevent us from exploring the root causes of complex societal challenges. Portfolio Approaches: The article references the collaboration between the Swedish Strategic Innovation Programme “Viable Cities” and the Strategic Innovation Unit of UNDP. They focus on portfolio-based approaches for urban low-carbon transitions in Sweden, targeting the mission: “Climate-neutral cities 2030 with a good life for all within planetary boundaries.” Conclusion: The blog post concludes by highlighting the need for further research and practice in developing effective portfolio approaches to address complex societal challenges.

Grow - Scale - Impact: How to help inclusive business achieve scale - Pérez Castro, Andrea A.; Tora, Krisztina (GIZ, 2016)

Cost–Benefit Analysis (CBA) or the Highway? An Alternative Road to Investigating the Value for Money of International Development Research - Peterson, Heidi (The European Journal of Development Research, 2023)
Research for development (R4D) funding is increasingly expected to demonstrate value for money (VfM). However, the dominance of positivist approaches to evaluating VfM, such as cost-benefit analysis, do not fully account for the complexity of R4D funds and risk undermining efforts to contribute to transformational development. This paper posits an alternative approach to evaluating VfM, using the UK’s Global Challenges Research Fund and the Newton Fund as case studies. Based on a constructivist approach to valuing outcomes, this approach applies a collaboratively developed rubric-based peer review to a sample of projects. This is more appropriate for the complexity of R4D interventions, particularly when considering uncertain and emergent outcomes over a long timeframe. This approach could be adapted to other complex interventions, demonstrating that our options are not merely “CBA or the highway” and there are indeed alternative routes to evaluating VfM.

Discussing Adaptive Approaches for Development Programmes - Pett, Jamie (Medium - LearnAdapt, 2020)
What can adaptive approaches from other sectors teach development practitioners about dealing with uncertainty?

Four ways development practitioners can borrow from private sector adaptive approaches - Pett, Jamie (LearnAdapt, 2020)
Agile, lean startup and human-centred design can be an answer — if you’re asking the right questions

Navigating adaptive approaches for development programmes - Pett, Jamie (Working Paper, 2020)
This working paper compares six of the most prominent adaptive approaches to emerge over the past two decades. Three come from the world of innovation, largely in the private sector (agile, lean startup and human-centred design), and three from the global development sector (thinking and working politically, forms of adaptive management and problem-driven iterative adaptation). While all of these approaches are valuable when used in the right context, practitioners may be perplexed by the multiplicity of methods and jargon. This paper aims to address some of this confusion by mapping where these approaches have come from and showing how they can be applied across the adaptive programme cycle. Armed with this knowledge, practitioners might experiment with different combinations and sequences of adaptive approaches according to the kind of problem and context faced. In turn, this may help us move beyond a siloed view of approaches linked to innovation, adaptive management or more politically smart ways of working. Key messages: • Adaptive approaches have emerged in several sectors, including software development, product and service design, technology startups and international development. • Adaptive approaches can help practitioners counteract misplaced certainty. By talking to potential users, understanding institutions, interests and ideas and investigating the root causes of a problem, practitioners applying these approaches can illuminate the underlying nature of the problem and context. • Rather than building a whole solution straight away, these approaches commonly encourage practitioners to start small and use structured cycles of testing and learning. There is scope to further consider how different approaches can be better brought together and combined. • Adaptive approaches in development provide a wider range of options for what to create and facilitate – not only products or services, but also forms of collective action. There are also alternative ways to think about scale – considering how others might take up an idea and looking for leverage, rather than quantity.

The Long and Short of It: Responding to immediate needs while pursuing long-term goals - Pett, Jamie (LearnAdapt, 2020)
How can we balance our need to respond to a crisis with our long-term goals for systemic change?

Evidence, hierarchies, and typologies: horses for courses - Petticrew, M; Roberts, H (Journal of epidemiology and community health, 2003)
Debate is ongoing about the nature and use of evidence in public health decision making, and there seems to be an emerging consensus that the "hierarchy of evidence" may be difficult to apply in other settings. It may be unhelpful however to simply abandon the hierarchy without having a framework or guide to replace it. One such framework is discussed. This is based around a matrix, and emphasises the need to match research questions to specific types of research. This emphasis on methodological appropriateness, and on typologies rather than hierarchies of evidence may be helpful in organising and appraising public health evidence.
Keywords: Decision Making; Policy making; Public Health; Public policy

Power Analysis: A Practical Guide - Pettit, Jethro (Sida, 2013)

Power Above and Below the Waterline: Bridging Political Economy and Power Analysis - Pettit, Jethro; Mejía Acosta, Andrés (IDS Bulletin, 2014)

Barriers and enablers to climate change adaptation in hierarchical governance systems: the case of Vietnam - Phuong, Le Thi Hong; Biesbroek, G. Robbert; Wals, Arjen E. J. (Journal of Environmental Policy & Planning, 2018)
Governments fulfil important roles in increasing the adaptive capacity of local communities to respond to climate change impacts, particularly in developing countries. Existing studies on how governments enable and constrain the ways in which local level communities learn and build their adaptive capacity, however, generally adopt network or market-oriented types of governance. However, the most vulnerable regions to climate change impact in the world are generally governed through hierarchical policy systems. This research aims to understand how the hierarchical policy system in Vietnam creates enables and/or constrains the policy capacity of policy actors to contribute to effective climate change adaptation. We conducted interviews (n = 26) with key actors at multiple levels of government. Our findings show the importance of clear legal institutions, available financing for implementing policies, and the training of governmental staff, particularly at district and commune levels where the policy capacities are generally too low to deal with climate change impacts. We conclude that any efforts to support local actors (i.e. smallholder farmers) should include investments in policy capacity to ensure uptake and upscaling of adaptation actions more broadly.

Using a social learning configuration to increase Vietnamese smallholder farmers’ adaptive capacity to respond to climate change - Phuong, Le Thi Hong; Wals, Arjen; Sen, Le Thi Hoa; Hoa, Nguyen Quoc; Lu, Phan Van; Biesbroek, Robbert (Local Environment, 2018)
Social learning is crucial for local smallholder farmers in developing countries to improve their adaptive capacity and to adapt to the current and projected impacts of climate change. While it is widely acknowledged that social learning is a necessary condition for adaptation, few studies have systematically investigated under which conditions particular forms of social learning are most successful in improving adaptive capacity of the most vulnerable groups. This study aims to design, implement and evaluate a social learning configuration in a coastal community in Vietnam. We make use of various methods during four workshop-based interventions with local smallholder farmers: interviews with key farmers and commune leaders, farmer-to-farmer learning, participatory observations and focus group discussions. The methods for evaluation of social learning configuration include in-depth interviews, focus group discussions and structured survey interviews. Our findings show that the social learning configuration used in this study leads to an increased problem ownership, an enhanced knowledge-base with regard to climate change impacts and production adaptation options, improved ability to see connections and interdependencies and finally, strengthened relationships and social cohesion. The results suggest that increased social learning in the community leads to increase in adaptive capacity of smallholder farmers and improves both their economic and environmental sustainability. We discuss the key lessons for designing learning configurations that can successfully enhance adaptive capacity and smallholder farmers’ agency and responsiveness to the challenges posed by climate change impacts.
Keywords: Adaptive capacity; Social learning; Vietnamese smallholder farmers

The interplay between social learning and adaptive capacity in climate change adaptation: A systematic review - Phuong, Thi Hong Le; Biesbroek, G. Robbert; Wals, Arjen E.J. (NJAS - Wageningen Journal of Life Sciences, 2017)
Successful implementation climate change adaptation depends to a large extent on the capabilities of individuals, organizations, and communities to create and mobilize the adaptive capacity (AC) of their socio-ecological system. Creating and mobilizing AC is a continuous process that requires social learning (SL). Although rich with empirical cases, the literature theorizing and empirically investigating the relationship between AC and SL is highly fragmented. This paper aims to critically examine the peer-reviewed literature that focusses on SL and AC in the context of climate change adaptation (CCA). Special attention is paid to the interplay between the two. Understanding this interplay can help improve our understanding of how CCA takes place in practice and advances theoretical debates on CCA. Systematic review methods are used to analyse 43 papers (1997–2016). Our findings reveal three perspectives that each play an important role in different contexts: an AC-focused perspective, a SL-focused perspective, and a hybrid perspective. These differences in conceptualizations of the relationship between SL and AC may seem trivial at first, but they have consequences for the design of learning-based interventions aimed at helping communities respond to climate change. It appears that such interventions need to be preceded by an analysis of the climate change context in order to decide whether to emphasize AC, SL or both simultaneously.
Keywords: Adaptive capacity; Climate change adaptation; Social learning; Systematic review

What is democratic evaluation? - Picciotto, R. (Evaluation Connections, 2013)
This article from Robert Picciotto provides an overview of democratic evaluation, particularly with reference to its use in the European Union context. "Can evaluation strengthen the democratic process? Speciﬁcally can it help ﬁll the democratic deﬁcit (limited transparency; weak bureaucratic accountability, etc.) often ascribed to the European Union? First and foremost, evaluators operating in the European space should be committed to the ethical and democratic values of the European project and the promotion of its social inclusion and cohesion ideals. But two other sets of challenges must also be met: those that relate to evaluation in democracy and those that relate to democracy in evaluation. The former has to do with the macro positioning of evaluation processes in society. The latter has to do with the evaluation approaches and methods used in deliberative democratic decision making processes." (Picciotto, 2013)

Context-monitoring for adaptive management - Pickwick, Sarah (World Vision, 2022)

Adaptive Management: Responding to the evolving needs of PRIME's complex systems - Picon, Diana; Wild, Lorenz (Mercy Corps, 2015)

Review of Selected DFAT Facilities: Independent Consultant Report to DFAT, Key Findings - Pieper, Lynn (DFAT, 2018)
For the purposes of this report, a facility is defined as an aid delivery mechanism that provides flexible (adaptive and responsive) services managed in an integrated way. Objectives (or endof-facility outcomes) are specified, but the pathways to deliver them are left unspecified. The facility is a highly relevant model for delivering Australian aid effectively. Achievement of the Sustainable Development Goals depends on flexible approaches that work across sectors and that integrate significant cross-cutting themes such as gender equality and social inclusion. Efficient DFAT management of a $4 billion aid program requires a shift to fewer and larger initiatives, to make best use of limited internal management resources while still enabling policy and program choices that can have real impact. Facilities and other flexible delivery mechanisms offer significant potential for better development results. They can: enable collaborative and responsive partnership approaches to gain traction; allow activities to experiment and adapt based on progress, demand and contextual changes; and provide the opportunity for outcome-focused coherence across sectors, enabling the whole to be more than the sum of the parts. Facilities are not new, but they are different today. Facilities have existed since at least the mid-1990s. What is new is that some facilities today are very large; and the flexibility they enable is increasingly and consciously being used to strengthen links between the technical and the political, for more effective development results. This is a positive trend, but it adds enormously to their complexity and visibility, creates new risks, and has resulted in role confusion (especially in early years of implementation) between DFAT and its contractors.

Decide Madrid: a case study on e-Participation - Pina, Vicente; Torres, Lourdes; Royo, Sonia; García-Rayado, Jaime (XX International AECA Congress, 2019)
This paper analyses the award-winning e-participation initiative of the city council of Madrid, Decide Madrid, to discover the critical success factors at contextual, organizational and individual level. This analysis is carried out with desk research and semi-structured interviews. Results show that the most relevant success factors are the socio-economic context, the commitment of the city council, the method used to recruit the workers and the knowledge of senior managers about citizen participation and ICTs. However, the lack of transparency and poor functioning of some of its participation activities are negatively affecting its performance.

Local First in practice - Pinnington, R (Peace Direct, 2014)

Is DFID Getting Real About Politics?A stocktake of how DFID has adopted a politically-informed approach (2010-2015) - Piron, Laure-Hélène; Baker, Aislin; Savage, Laura; Wiseman, Katie (DFID, 2016)
1. Background This internal stocktake assesses whether DFID is “getting real about politics” - how it is taking power and politics into account in all its operations. Country Poverty Reduction Diagnostics undertaken by DFID teams identify politics as the most frequent barrier to poverty reduction and growth. The UK 2015 Aid Strategy has committed DFID to spending 50% of Official Development Assistance (ODA) in fragile states. This requires a “patient, long-term approach” to addressing barriers to peace and stability which are fundamentally political, rather than purely financial or technical. The stocktake is based on three DFID offices case studies (Democratic Republic of the Congo, Pakistan and Malawi) as well as extensive internal and external consultations between June and December 2015. It provides illustrations of how DFID is evolving but does not systematically offer evidence of development impacts or non-country work, as this would have required a different methodology. 2. What does it mean to take politics into account? Politically-informed approaches are based on a large body of evidence that confirms the importance of institutions and politics for sustainable development. External assistance needs to support locally-led change. Success depends on timing, context, political processes and local actors. Desirable outcomes are hard to achieve and difficult to predict. Politically-informed approaches improve development effectiveness through:  The ‘what’: political goals, using development assistance to shift how power is distributed in the economy and society. The two main elements are: aiming for long term transformation of institutions; and supporting locally-led change processes more likely to be sustainable and successful: locallyowned (i.e. with local salience) and locally-negotiated.  The ‘how’: politically-smart methods, with greater realism and feasibility. The three main elements are: understanding power and politics in a specific context in order to identify opportunities and barriers for change; influencing and stakeholder management skills; and proactive risk management. To influence DFID operations, a politically-informed approach needs to be iterative, not one-off. The explicit understanding of context, whether formal setpiece studies or more routine analysis, should inform policy and programme decisions, from high level strategic choices, to day-to-day implementation, for both international policy and country support. This is a dynamic process: as the context evolves and lessons are learned about what works, analyses and decisions are updated. These are the principles behind the ‘flexible and adaptive’ agenda.

Twenty years of UK governance programmes in Nigeria - Piron, Laure-Hélène; Cummings, Clare; Williams, Gareth; Derbyshire, Helen; Hadley, Sierd (ODI, 2021)
This Flagship report analyses 20 years of governance programmes in Nigeria funded by the UK Department for International Development (DFID) and the Foreign, Commonwealth and Development Office (FCDO) in the North-western states of Jigawa (since 2001), Kano (since 2005) and Kaduna (since 2006), as well as the North-eastern state of Yobe (since 2011). The report’s main research question is whether, how, under what conditions and for whom UK-funded state-level governance programmes in Nigeria have contributed to sustained changes in governance, and related changes in health and education. ... The report concludes with the following recommendations: To international development partners: 1. Invest for the long term – 10 to 20 years – combining support for both state and nonstate actors. 2. Ensure programmes have the strategic-level mandate, managerial capacity and frontline staff skills to pursue politically savvy opportunities. 3. Take PEA to the next level by unpacking causal mechanisms, understanding incentives and designing interventions to make change happen. 4. Give governance programmes the ability to flex between core governance and service delivery issues. 5. Incentivise greater collaboration between governance and sector programmes. 6. Incentivise greater attention to gender, and to social inclusion beyond disability issues, in governance programming. To FCDO: 7. Empower and resource FCDO teams to enable TWP programmes, ensuring decision-making by country teams to respond to local priorities. 8. Re-imagine TWP for FCDO Nigeria, giving implementers the space to operate in TWP ways. 9. Incentivise stronger collaboration between PERL, Lafiya (health programme) and the Partnership for Learning for All in Nigerian Education. 10. Invest in impact data analysis. To partner governments in Nigeria and beyond: 11. Explicitly set out the objectives for which you would like to receive assistance. 12. Use TWP principles to decide how development partners can support your political objectives and the scope for politically-feasible and mutually-beneficial collaboration. 13. Invest in the coordination of development partners. To non-state partners in Nigeria and beyond: 14. Join coalitions to achieve your priorities. 15. Select development partners which can strengthen your skills, not just fund your activities.

What Startups Can Learn from General McChrystal about Combining Strategy and Execution - Pisoni, Adam (First Round, 2015)
Yammer Co-founder Adam Pisoni speaks with General Stanley McChrystal about the lessons technology companies might take away from his new book Team of Teams.

Design thinking: understand - improve - apply - Plattner, Hasso; Meinel, Christoph; Leifer, Larry (Understanding innovation, 2011)
"In this book, the researchers take a system's view that begins with a demand for deep, evidence-based understanding of design thinking phenomena. They continue with an exploration of tools which can help improve the adaptive expertise needed for design thinking. The final part of the book concerns design thinking in information technology and its relevance for business process modeling and agile software development, i.e. real world creation and deployment of products, services, and enterprise systems"--Cover
Keywords: Creative ability in business; Creative thinking; Designer; Industrial management; Innovation; Organizational change; Product development; System design; Technological innovations

Seeing (from) Digital Peripheries: Technology and Transparency in Kenya’s Silicon Savannah - Poggiali, Lisa (Cultural Anthropology, 2016)

Measuring Governance, Advocacy and Power: A Database of Existing Indicators, Tools and Indices - Poirrier, Caroline; Tolmie, Courtney (R4D, 2020)
Measuring Governance, Advocacy, and Power is an excel sheet that brings together existing indicators, tools, and indices that may be useful to practitioners responsible for the measurement of outcomes in the field of governance, advocacy, and power in an easily accessible and filterable format.

Illogical Framework: The Importance of Monitoring and Evaluation in International Development Studies - Pomerantz, Jessica R. (Cornell Policy Review, 2011)
In my brief experience with monitoring and evaluation, I have become convinced that it is critically important both as an international development project component and as a field of academic study. Throughout my academic career at Cornell University, I have, at times, argued that monitoring and evaluation has actually impeded development efforts, but here I wish to amend my opinion. Bad monitoring and evaluation can sabotage development projects and our meaningful interpretation of development impacts; failures can appear to be successes and vice-versa. As a student and practitioner of monitoring and evaluation, I have drawn the conclusions listed below and I submit them for your consideration. • Monitoring and evaluation is a key element of the international development industry applicable to many areas of public administration, domestic and international. • International development failures could be discovered and averted or corrected given proper monitoring and evaluation activities. • Anecdotal evidence from development activities in Afghanistan provides one example of the international community’s lack of attention to monitoring and evaluation concerning an ongoing development catastrophe. • Higher education ought to be filling the monitoring and evaluation knowledge gap but to date is failing to do so.

Winners and Losers in the Global App Economy - Pon, Bryan (2016)

A new bird's eye view on the agile forest - Portman, Henny (PM World Journal, 2020)

A new bird's eye view on the agile forest (2022 update) - Portman, Henny (PM World Journal, 2020)
Some years ago, you could say “Scrum is agile” and ask “is Agile Scrum?” Now we know there is much more flesh on the bones. At this moment there are more than fifty known and less known agile approaches, frameworks or methods available. To get a first impression of the different approaches, I try to bring some structure in the jungle to approaches, methods and frameworks. In Figure 1, I position the best-known agile approaches in a structure. The approaches, frameworks or methods are positioned within the 'One-time programs / projects' sections or within 'Business as usual’ / indefinite, or both. On the other side the approaches, frameworks or methods are clustered around team, product or programme and portfolio level. In the dark blue boxes in Figure 1 we see agile approaches that are only applicable in IT-focused organizations. All other approaches can be used within IT and non-IT-oriented organizations (light blue coloured). I haven’t mapped all the known approaches, frameworks and methods in this figure, and to be honest, I think there is a lot of duplication and probably commercial drivers play a role too to ‘develop’ the next kid on the block without added value in comparison with the existing approaches, frameworks or methods. The team level, including Scrum and Kanban, is applicable in both IT-oriented and non-IT-oriented products and services development and operations. The engineering level focuses specifically on IT-oriented product development. The one-time, temporary projects and programme frameworks and methods are suitable for both IT and non-IT. The permanent umbrella frameworks (both product-targeted and team-targeted) focus specifically on IT and product development and the Culture-targeted approaches help organisations to increase their agility.

Positive Deviance Initiative - Positive Deviance Initiative (2018)

A Pragmatic Approach to Assessing System Change - Posthumus, Hans; Shah, Rachel; Miehlbradt, Alexandra; Kessler, Adam (DCED, 2020)
Planning for and assessing system change is a strategic management issue. It is critical for everything from developing a strategy and designing interventions, to adapting strategy, improving implementation and reporting impact. But many programmes get stuck when it comes to assessing system change. The private sector development field has struggled to agree on an approach that programmes can implement and stakeholders can understand. However some mature programmes are starting to assess system change more effectively. Building on these emerging practices, this paper outlines a process that programmes can use to assess system changes regularly and practically. Two complementary papers: Overview and How to put it into practice The Overview summarises the approach and How to put it into practice provides more detailed implementation guidance, worked examples, and useful tips. The Overview explores how to: develop a system change strategy and intervention plans that lay the groundwork for system change assessment, including how to set system boundaries and how to identify the system changes a programme aims to catalyse assess system changes using both: - an intervention lens focused on changes introduced by specific interventions - a helicopter lens that provides a whole system view By analysing findings from both lenses, programmes can improve their strategy and report on their contribution to system change. How to put it into practice uses two case examples for illustration throughout the paper - PRISMA’s work in the maize system in East Java and Indonesia and S4J’s work in the Vocational Education and Training (VET) system in Albania. It targets practitioners responsible for facilitating and/or assessing system change. The paper explains how to: articulate the system changes that a programme aims to catalyse assess those changes use the results to inform decision making and reporting The approach described in the paper builds on the practices outlined in the DCED Results Measurement Standard. The guidance provided has been designed to be useful to programmes that aim to catalyse system changes whether or not they apply the DCED Standard.

Information needs and policy change - Potter, Stephen; Subrahumanian, Ramya (Sage, 2007)

Critical agency and development: applying Freire and Sen to ICT4D in Zambia and Brazil - Poveda, Sammia; Roberts, Tony (Information Technology for Development, 2017)
(2017). Critical agency and development: applying Freire and Sen to ICT4D in Zambia and Brazil. Information Technology for Development. Ahead of Print. doi: 10.1080/02681102.2017.1328656

Poverty Stoplight - povertystoplight.org (2018)
What is the Poverty Stoplight? The Stoplight is a tool that seeks to activate the potential of families and communities to lift themselves out of poverty. Using a technology platform, it offers a self-assessment survey and intervention model that enables people to develop practical solutions to overcome their specific needs. We work with organizations from different sectors around the world and bring together a network of powerful actors with one clear mission: eliminate poverty worldwide. The Stoplight Community is comprised of government agencies, private companies, small and medium-sized enterprises, non-profits, cooperatives, microfinance organizations, academic institutions, and sports clubs, among others.

Decision-Making and Data Use Landscaping [at DFID] - Powell, Josh; Orton-Vipond, Sarah; Bhatia, Vinisha; Kilroy, Annie (Development Gateway, 2018)
As DFID aims to harness the Data Revolution, ensuring that data1 drive decision-making, public accountability, and the achievement of the Sustainable Development Goals (SDGs), ensuring that systems, processes, and skills for data are aligned with these objectives is paramount. Across sector policy teams, country offices, and various analytical and technical cadres, different strengths and weaknesses, as well as needs and ambitions exist. To inform a strategic approach to data, as framed in its forthcoming Data Roadmap, DFID collaborated with Development Gateway to perform a Decision and Data Use Landscaping study. This report details lessons learned from approximately 60 interviews across 4 DFID country offices, all sector policy teams, senior managers, and various analytical and technical cadres and offices.

Causal mapping for evaluators - Powell, Steve; Copestake, James; Remnant, Fiona (Evaluation, 2023)
Evaluators are interested in capturing how things causally influence one another. They are also interested in capturing how stakeholders think things causally i...

All about behaviour: KAPE, Adaptation and ' Sticky' Institutional Change - Power, Greg (Politically Agile Programming, 2016)
This paper describes Global Partners Governance’s (GPG’s) approach to institutional reform and political change. Developed over the last decade of working in some of the most complex and sensitive political environments with politicians and officials in parliaments, political parties, ministries and local government, it describes the KAPE® (knowledge-application-practice-effect) methodology that we adopt to get ‘sticky’ institutional and behavioural change. Contents 1) Two dimensions of ‘adaptive programmes’: Flexible delivery and getting behaviour change 2) Enabling Change: KAPE and The Logic of Institutional Reform (Knowledge-Application-Practice-Effect) - K: Knowledge – Defining the problem and what to do about it - A: Application – Making Systems Work in Practice - P: Practice – Pockets of good practice and establishing ‘the new normal’ - E: Effect – Improved performance and the ‘Ripple Effect’ 3) Measuring Impact: Monitoring and Evolving 4) Conclusion: Behavioural insights, adaptive management and sticky change

Engaging scientists through institutional histories - Prasad, Shambu; Hall, Andrew; Thummuru, Laxmi (ILAC Brief, 2006)
An institutional history is a narrative that records key points about how institutional arrangements – new ways of working – evolve over time creating more effective ways to achieve goals. It can be used to document institutional innovations in projects and to highlight barriers to change. An institutional history draws out and synthesizes lessons for research organizations and partners as well as for others in similar circumstances.

Identifying general trends and patterns in complex systems research: An overview of theoretical and practical implications - Preiser, Rika (Systems Research and Behavioral Science, 2019)
Research on complex systems is becoming more prolific, and there is a need to provide some point of orientation to researchers and practitioners that are drawing on the body of literature that informs the field of complexity research. In this paper, I aim to give an overview of the development of the field and offer some overarching trends and patterns that are recognizable in research on complex systems. The paper then draws on the work to provide six organizing principles of complex systems to inform practical implications and methods for studying and understanding complex systems.
Keywords: complex systems; heuristics; trends and patterns

Combating Wildlife Trafficking Cross-Mission Learning Agenda - Present, Teresa; Peabody, Shawn (USAID, 2017)

Evaluating Complexity. Propositions for improving practice - Preskill, Hallie; Gopal, Srik (FSG, 2014)
9 propositions can help evaluators measure progress on complex social problems.
Keywords: IMPORTANT

Redefining Rigor: Describing Quality Evaluation in Complex, Adaptive Settings - Preskill, Hallie; Lynn, Jewlya (FSG, 2016)
Traditionally, evaluation has focused on understanding whether a program is making progress against pre-determined indicators. In this context, the quality of the evaluation is often measured in part by the “rigor” of the methods and scientific inquiry.

Putting vulnerability to climate change on the map: a review of approaches, benefits, and risks - Preston, Benjamin L.; Yuen, Emma J.; Westaway, Richard M. (Sustainability Science; Dordrecht, 2011)
There is growing demand among stakeholders across public and private institutions for spatially-explicit information regarding vulnerability to climate change at the local scale. However, the challenges associated with mapping the geography of climate change vulnerability are non-trivial, both conceptually and technically, suggesting the need for more critical evaluation of this practice. Here, we review climate change vulnerability mapping in the context of four key questions that are fundamental to assessment design. First, what are the goals of the assessment? A review of published assessments yields a range of objective statements that emphasize problem orientation or decision-making about adaptation actions. Second, how is the assessment of vulnerability framed? Assessments vary with respect to what values are assessed (vulnerability of what) and the underlying determinants of vulnerability that are considered (vulnerability to what). The selected frame ultimately influences perceptions of the primary driving forces of vulnerability as well as preferences regarding management alternatives. Third, what are the technical methods by which an assessment is conducted? The integration of vulnerability determinants into a common map remains an emergent and subjective practice associated with a number of methodological challenges. Fourth, who participates in the assessment and how will it be used to facilitate change? Assessments are often conducted under the auspices of benefiting stakeholders, yet many lack direct engagement with stakeholders. Each of these questions is reviewed in turn by drawing on an illustrative set of 45 vulnerability mapping studies appearing in the literature. A number of pathways for placing vulnerability mapping on a more robust footing are also identified. [PUBLICATION ABSTRACT]
Keywords: Climate change; Mapping; Risk assessment; Sustainable development

A trainer's guide for participatory learning and action - Pretty, Jules; Guijt, Irene (IIED Participatory methodology series, 2002)

How Does Developing Enforcement Capacity Reduce Wildlife Crime? Increasing program effectiveness by understanding common assumptions - Price, Claire; Buff, Jesse; Peabody, Shawn; Present, Tess; Lauck, Liz (USAID, 2018)
"How Does Developing Enforcement Capacity Reduce Wildlife Crime?" summarizes findings from the literature around four key assumptions about capacity building for enforcement and prosecution. Using the experiences and evidence summarized in this brief, program designers and implementing partners should be able to increase the effectiveness and efficiency of their capacity building efforts.

Adaptiveness through ‘simplest tools and practices’ - Prieto Martin, Pedro (Institute of Development Studies, 2017)
In recent years, more and more influential development organisations have been openly recognising the central role that adaptive management capacities – the ability to keep improving strategies and actions as programmes unfold – play for the success of complex interventions. As a result, there has been a rich exchange of ideas and experiences on how to promote adaptiveness in development. But development organisations and professionals are having a hard time translating the many adaptive theories and recommendations into improved practices and outcomes. They are generally failing to adjust the design and operation of programmes to accommodate incipient lessons learnt and the unexpected changes in the context. Nowadays, most programme adaptations still take the form of a change of direction once it is clear that things have gone really wrong. Why is so difficult to put what we know about adaptiveness into practice? And more importantly: what can be done to improve this situation?

How to develop capacity in the international development sector - Prieto Martín, Pedro (CLARISSA, 2022)
In her acclaimed study of effective cross-organizational teamwork, Harvard Business School Professor Amy C. Edmonson concluded that “trying things that...

Participation Schemas: a tool to characterize collaborative participation - Prieto-Martín, Pedro (PDD2014, Contemporary difficulties and future prospects for Participatory and Deliberative Democracy, NewCastle, 2014)

READ ME - HOW TO USE THIS AM LIBRARY - Prieto Martin, Pedro (Institute of Development Studies, 2019)

Adaptive Management in SDC: Challenges and Opportunities - Prieto Martin, Pedro; Apgar, Marina; Hernandez, Kevin (IDS, 2020)
Adaptive management (AM) is a programme management approach that helps international development organisations to become more learning-oriented and more effective in addressing complex development challenges. AM practices have been applied for decades within other sectors as varied as logistics, manufacturing, product design, military strategy, software development and lean enterprise. At its core, AM is not much more than common sense, as it essentially recognises that the solutions to complex and dynamic problems cannot be identified at the outset of a programme but need to emerge throughout the process of implementation as a result of systematic and intentional monitoring and learning. The generic AM process typically involves an iterative cycle of design, implementation, reflection and adaptation activities, supported both by system monitoring and stakeholder involvement to obtain a better understanding of the evolving system and improve how the intervention is managed. A favourable context for AM in development. During recent decades, the international development sector has aimed to increase its results and impact orientation. As a result, a growing number of development organisations and governments have become increasingly aware of the limitations of traditional ‘linear and prescriptive’ programming approaches. They are now recognising the need to handle complexity better, and have begun to adapt their policies and practices to facilitate adaptive approaches. The World Bank, for example, now acknowledges that aid agencies need to increase flexibility of implementation, tolerate greater risk and ambiguity, devolve power from aid providers to aid partners, and avoid simplistic linear schemes for measuring results. Multilateral and bilateral organisations such as the World Bank, the United Kingdom’s (UK) Department for International Development (DFID) and the United States Agency for International Development (USAID) are currently experimenting with adaptive approaches. A multitude of adaptive approaches and communities of practice have emerged that aim to improve the effectiveness of aid, including Collaborating, Learning, and Adapting, Thinking and Working Politically, Doing Development Differently, Market Systems Development, Conflict-Sensitive Programme Management, and Science of Delivery. Since generic AM approaches have existed for decades in other sectors, AM has the potential to act as a neutral ‘bridge language’ that facilitates exchange and learning among the different communities and donors. This report is the result of a learning partnership between the Swiss Agency for Development and Cooperation (SDC) and the Institute of Development Studies (IDS). It assesses the relevance of AM to SDC, how it relates to working practices across SDC, and the key challenges and opportunities for SDC. Its process of elaboration involved a literature review on AM, an exploration of AM approaches from several bilateral donors, a series of 6 interviews with SDC staff and partners working in different countries and thematic domains, and a learning workshop at SDC headquarters (HQ), where staff from several SDC divisions reflected on AM and on how to advance the organisation’s capacity for adaptive programming and learning.

Interview protocol: Most significant turning points - Prieto Martin, Pedro; Faith, Becky (Institute of Development Studies, 2017)
This interview protocol was used for a research project on adaptiveness in technology for governance initiatives in Kenya. For more information, please read the research report at: Prieto Martin, P.; Hernandez, K.; Faith, B. and Ramalingam, B. (2017) Doing Digital Development Differently: Lessons in adaptive management from technology for governance initiatives in Kenya, MAVC Research Report, Brighton: Institute of Development Studies, ids.ac.uk/project/making-all-voices-count
Keywords: IMPORTANT

Doing Digital Development Differently: Lessons in adaptive management from technology for governance initiatives in Kenya - Prieto Martin, Pedro; Hernandez, Kevin; Faith, Becky; Ramalingam, Ben (MAVC Research Report, 2017)
Development projects don’t always work as planned. This has long been acknowledged by those in the sector, and has led to several approaches that seek to solve complex development problems through enabling and encouraging greater adaptiveness and learning within projects (e.g. Doing Development Differently and Problem-Driven Iterative Adaptation). Digital development projects experience many of these issues. Using technology for transparency and accountability (Tech4T&A) projects in Kenya as case studies, our research analysed the many different theoretical approaches to learning and adaptation, and then tested how these play out in reality. Firstly, we conducted an extensive review of the literature on the spectrum of approaches to adaptive learning. The findings were used to develop a framework through which to analyse adaptiveness at the different layers of complexity in projects (e.g. software design and development, programme design and management). The second part of the research consisted of interviews and focus group discussions with participants in Tech4T&A projects in Kenya. Respondents helped us identify the main characteristics of adaptiveness in their projects (e.g. who needs to adapt, and how and when) and the challenges and issues that inhibit projects’ abilities to be adaptive. This process also revealed how accountability interplays with adaptiveness, and considered how better collaboration flows can enable adaptiveness. From our literature review and empirical study, we draw several conclusions for increasing adaptiveness in digital development projects. These include simplifying the many adaptive theories that abound, increasing responsiveness to project beneficiaries and stakeholders, and for stakeholders to keep on experimenting, networking and advocating.
Keywords: IMPORTANT

Caracterizando la participación ciudadana en el marco del Gobierno Abierto - Prieto-Martín, Pedro; Ramírez-Alujas, Álvaro V. (Revista del CLAD. Reforma y Democracia, 2014)
Este artículo profundiza conceptualmente en las dimensiones del Gobierno Abierto, centrándose en el análisis y caracterización de su dimensión participativa. Se propone, en primer lugar, una tipología de actos participativos que muestra el efecto potenciador que las TIC ejercen sobre las distintas formas de participación. En segundo lugar, se presentan los denominados “Esquemas de participación” (EdP), un instrumento conceptual que facilita el análisis y la representación estandarizada de las dimensiones más importantes de la participación. Los EdP proporcionan así, por un lado, un modelo de categorización que extiende nuestra comprensión crítica de la participación y, por otro lado, una herramienta potente y flexible para la comunicación, el diseño y la evaluación de diversos tipos de iniciativas participativas.

Between logframes and theory of change: reviewing debates and a practical experience - Prinsen, Gerard; Nijhof, Saskia (Development in Practice, 2015)
Theory of Change (ToC) is an emerging methodology in the practice of development programmes, often contrasted with the dominant logical framework. This article reviews current debates around ToC before identifying five aspects that are appreciated in practice. It appears that these aspects mostly cover areas where the logical framework is not – or is no longer – meeting the needs of practitioners. Subsequently, the article analyses experiences in ToC training for NGO staff and concludes that ToC can address shortcomings of the logical framework – if only by going back to some of the roots of the logical framework.

How did China create “Directed Improvisation”? - Pritchett, Lant (Building State Capability, 2017)
written by Lant Pritchett Yuen-Yuen Ang, a Professor of Political Science at University of Michigan came to speak at Harvard the other day and I was lucky enough to hear her presentation.  Her most…
Keywords: IMPORTANT

Let’s Take the Con Out of Randomized Control Trials in Development - Pritchett, Lant (CID Faculty Working Paper, 2021)
The enthusiasm for the potential of RCTs in development rests in part on the assumption that the use of the rigorous evidence that emerges from an RCT (or from a small set of studies identified as rigorous in a “systematic” review) leads to the adoption of more effective policies, programs or projects. However, the supposed benefits of using rigorous evidence for “evidence based” policy making depend critically on the extent to which there is external validity. If estimates of causal impact or treatment effects that have internal validity (are unbiased) in one context (where the relevant “context” could be country, region, implementing organization, complementary policies, initial conditions, etc.) cannot be applied to another context then applying evidence that is rigorous in one context may actually reduce predictive accuracy in other contexts relative to simple evidence from that context—even if that evidence is biased (Pritchett and Sandefur 2015). Using empirical estimates from a large number of developing countries of the difference in student learning in public and private schools (just as one potential policy application) I show that commonly made assumptions about external validity are, in the face of the actual observed heterogeneity across contexts, both logically incoherent and empirically unhelpful. Logically incoherent, in that it is impossible to reconcile general claims about external validity of rigorous estimates of causal impact and the heterogeneity of the raw facts about differentials. Empirically unhelpful in that using a single (or small set) of rigorous estimates to apply to all other actually leads to a larger root mean square error of prediction of the “true” causal impact across contexts than just using the estimates from non-experimental data from each country. In the data about private and public schools, under plausible assumptions, an exclusive reliance on the rigorous evidence has RMSE three times worse than using the biased OLS result from each context. In making policy decisions one needs to rely on an understanding of the relevant phenomena that encompasses all of the available evidence.

The Debate about RCTs in Development is over. We won. They lost. - Pritchett, Lant (DRi Seminar, 2018)
There has been a debate in development economics over the last 20 years as some claimed the use of RCTs as a tool for independent impact evaluation would significantly improve development practice and hence development. While right about the methodological claims about the superiority of randomization to produce cleaner estimates of the LATE (local average treatment effect) of projects and programs, this, in and of itself, does not change development practice. All of the five claims needed to sustain a positive model in which RCT/IIE has a major positive impact are demonstrably false. The proponents of RCTs have responded to losing the first round decisively by changing significantly both their claims and their practice.

It's All About MeE: Using Structured Experiential Learning (“e”) to Crawl the Design Space - Pritchett, Lant; Samji, Salimah; Hammer, Jeffrey (Working Paper 322, 2013)
This paper argues that within-project variations in design can serve as their own counterfactual, reducing the incremental cost of evaluation and increasing the direct usefulness of evaluation to implementing agencies. It suggests combining monitoring (‘M’), structured experiential learning (‘e’), and evaluation (‘E’) so as to facilitate innovation and organisational capability building while also providing accountability …

Berkana Two Loop Model: A quick introduction to an accessible model for systems change - Proud, Emma (LinkedIn - Emma Proud, 2023)
A model I've been using a lot recently, to bring systems thinking to life, is the Berkana Two Loop model. It doesn't describe complexity or systems thinking. But it does describe systems change in a way that's simple and oriented to action.

Managing uncertainty when your brain doesn’t like it - Proud, Emma (LearnAdapt, 2020)
Over lockdown there were a lot of tantrums in our household. The tears and tussles were at a dramatic high when the schools closed, and…

Physically distanced adaptive management - Proud, Emma (LearnAdapt, 2020)
Opportunities and challenges for local leadership

Principles for managing in complexity - Proud, Emma (LearnAdapt, 2020)
Written by Toby Lowe and Shaheen Warren (Centre for Public Impact) and Sam Sharp (Overseas Development Institute), with input from Jamie…

Procuring and Managing Adaptively: 5 Case Studies of Adaptive Mechanisms - Pryor, Tony (USAID Learning Lab, USAID, 2018)
Flexibility in program management is essential in all of the countries where USAID works. This is especially true in non-permissive environments (NPEs), where the ability to learn and adapt quickly to changing circumstances can help USAID staff members achieve their desired outcomes.

Adaptive Programming in Fragile, Conflict and Violence-Affected Settings, What Works and Under What Conditions?: The Case of PERL, Nigeria - Punton, Melanie; Burge, Richard (Action for Empowerment and Accountability Research Programme, 2018)
This paper examines adaptive approaches to aid programming in Nigeria. Through field research and desk reviews, we have investigated some of the assertions around the ‘adaptive management and programming’ approach, which has arisen in recent years as a response to critiques of overly rigid, pre-designed, blueprint and linear project plans. This is the second of three case studies in a series which explore if and how adaptive approaches, including rapid learning and planning responses, are particularly relevant and useful for promoting empowerment and accountability in fragile, conflict and violence-affected settings (FCVAS). This case study focuses on PERL (Partnership to Engage, Reform and Learn) in Nigeria, a five-year governance programme (2016-21) with a total budget of £100 million. It aims to promote better service delivery through bringing together government and citizens groups to collectively address governance challenges. PERL is viewed by DFID as the final stage of a 20-year investment, building on learning, experience and partnerships from 15 years of DFID-funded governance programming in Nigeria. It was designed to incorporate learning and adaptation through a ‘living’ theory of change, continuous political economy analysis at different levels, adaptive implementation by location-based delivery teams who are encouraged to be flexible and let partners take the lead, regular learning and reflection, and adaptive resourcing, HR and administrative systems. The case study draws on a conceptual framework (the ‘adaptive triangle’) that looks at three types of adaptation – adaptive management, adaptive programming and adaptive delivery – and the interconnections and tensions between them.
Keywords: A4EA; Adaptive Development; Economy; Fishery

Keeping it Real: Using Mechanisms to Promote Use in the Realist Evaluation of the Building Capacity to Use Research Evidence Program - Punton, Melanie; Vogel, Isabel (New Directions for Evaluation, 2020)
This chapter explores the use of mechanisms within the realist evaluation of the Building Capacity to Use Research Evidence (BCURE) program, a £15.7 million initiative aiming to improve the use of evidence in decision-making in low and middle-income countries. The evaluation was commissioned to establish not just whether BCURE worked but also how and why capacity building can contribute to increased use of evidence in policymaking in the very different contexts in which the program operated. This chapter argues that using mechanisms helped provide nuanced and robust insights into these questions, while also strengthening the usefulness and policy relevance of the evaluation. Drawing primarily on qualitative data, including interviews with more than 500 stakeholders over 3 years, the evaluation explored the mechanisms that promote capacities to use evidence in decision-making, through developing and testing realist context-intervention-mechanism-outcome configurations (CIMOs). Uncovering the value of mechanisms for policy and program learning was not easy, and the chapter sets out some of the thorny challenges faced and how the BCURE evaluation navigated these. Ultimately, the use of mechanisms in the BCURE evaluation helped to generate practical and nuanced insights that fed directly into the design of a £17 million follow-up program. The seven mechanisms uncovered are continuing to inform our own and others’ work on institutional capacity change in a wide range of fields.

Reflections from a Realist Evaluation in Progress: Scaling Ladders and Stitching Theory - Punton, Melanie; Vogel, Isabel; Lloyd, Rob (CDI Practice Paper 18, 2016)
Realist evaluation provides valuable insights into how and why programmes lead to change, and can generate transferable lessons to help practitioners roll out or scale up an intervention. However, as ...

Straws-in-the-wind, Hoops and Smoking Guns: What can Process Tracing Offer to Impact Evaluation? - Punton, Melanie; Welle, Katharina (Centre for Development Impact, 2015)
This CDI Practice Paper by Melanie Punton and Katharina Welle explains the methodological and theoretical foundations of process tracing, and discusses its potential application in international development impact evaluations. It draws on two early applications of process tracing for assessing impact in international development interventions: Oxfam Great Britain (GB)’s contribution to advancing universal health care in Ghana, and the impact of the Hunger and Nutrition Commitment Index (HANCI) on policy change in Tanzania. In a companion to this paper, Practice Paper 10 Annex describes the main steps in applying process tracing and provides some examples of how these steps might be applied in practice.

Engines of Change: What Civic Tech can learn from Social Movements - Purpose; Omidyar Network (Omidyar Network and Purpose, 2016)

Innovation teams and labs. A Practice Guide - Puttick, Ruth (Nesta, 2014)

Building elevators for development mutants - Quaggiotto, Giulio (Disrupt & Innovate, 2016)
I recently had the opportunity to learn about General Mill’s (the US food giant) “emerging brands elevator” program (also known as 301 Inc). Traditionally, General Mills has grown either through mergers and acquisitions, or by building new businesses from the ground up. Increasingly, however, it found that small brands were much faster at innovation, so … Continue reading Building elevators for development mutants

Profiling the international development mutants - Quaggiotto, Giulio (Development Impact and You, 2017)
As part of a new series, we talk to Giulio Quaggiotto about the development ‘mutants’ – innovators working at the edge of the field.

The era of development mutants - Quaggiotto, Giulio (Nesta, 2016)
If you were looking for the cutting edge of the development sector, where would you go these days?

Innovation at UNDP: from weekend sport to daily practice - Quaggiotto, Giulio; Begovic, Mellie (Nesta, 2016)
The experience of working to embed innovation approaches in everyday practice at UNDP's country offices
Keywords: IMPORTANT

Fall in love with the solution, not the problem - Quaggiotto, Giulio; Leurs, Bas; Hazeldine, Shaun (Nesta, 2016)
Three strategies for international development organisations to solve problems without defining them.

Connect. Learn. Adapt. Repeat - Practicing adaptive management in complex, conflict-affected environments: barriers and promising practices - Queen, Emily Forsyth (2018)
To shift aid culture toward adaptive management, we can: - Stay humble, empathetic, and build skills in facilitation and listening. - Focus on working with more flexible foundations or individual donors. Or, work for more rigid donors and help make their policies more flexible. - Get clearer about goals and processes while finding ways to be less specific and more committed to local leadership about pathways to get to the goal. - Resist looking for a magic adaptive management tool and instead get clearer about when teams make what kinds of decisions and matching to tools that support that decision-making. - Validate that everyone’s perspective is a form of data and try out simple ways to document how teams learns and changes. - Broaden and ground the idea of expertise – remembering that, especially when working with folx on the margins, people are the only experts on their own life. Practitioners must also examine how dominant patriarchal, white supremacist, colonial ideas have pushed us toward wanting more control, less flexibility, and less space for equitable decision making.

Snapshot of Adaptive Management in Peacebuilding Programs - Queen, Emily Forsyth; Baumgardner-Zuzik, Jessica; Hume, Elizabeth; Greenberg, Melanie (Alliance for Peacebuilding, 2018)
The Alliance for Peacebuilding developed the report Snapshot of Adaptive Management in Peacebuilding Programs with support from Humanity United. This report examines how existing organizational programs are designing and learning from adaptive management in a conflict effected and fragile states. It further provides practical recommendations for applying adaptive management to peacebuilding programs based on synthesis … Continue reading "Snapshot of Adaptive Management in Peacebuilding Programs"
Keywords: KEY

Use of environmental sensors and sensor networks to develop water and salinity budgets for seasonal wetland real-time water quality management - Quinn, Nigel W. T.; Ortega, Ricardo; Rahilly, Patrick J. A.; Royer, Caleb W. (Environmental Modelling & Software, 2010)
Management of river salt loads in a complex and highly regulated river basin such as the San Joaquin River Basin of California presents significant challenges for current Information Technology. Computer-based numerical models are used as a means of simulating hydrologic processes and water quality within the basin and can be useful tools for organizing Basin data in a structured and readily accessible manner. These models can also be used to extend information derived from environmental sensors within existing monitoring networks to areas outside these systems based on similarity factors – since it would be cost prohibitive to collect data for every channel or pollutant source within the Basin. A common feature of all hydrologic and water quality models is the ability to perform mass balances. This paper describes the use of a number of state-of-the-art sensor technologies that have been deployed to obtain water and salinity mass balances for a 60,000ha tract of seasonally managed wetlands in the San Joaquin River Basin of California. These sensor technologies are being combined with more traditional environmental monitoring techniques to support real-time salinity management (RTSM) in the River Basin. Two of these new technology applications: YSI-Econet (which supports continuous flow and salinity monitoring of surface water deliveries and seasonal wetland drainage); and electromagnetic salinity mapping (a remote sensing technology for mapping soil salinity in the surface soils) – have not previously been reported in the literature. Continuous sensor deployments that experience more widespread use include: weather station sensor arrays – used to estimate wetland pond evaporation and moist soil plant evapotranspiration; high resolution multi-spectral imagery – used to discriminate between and estimate the area of wetland moist soil plant vegetation; and groundwater level sensors – used primarily to estimate seepage losses beneath a wetland pond during flood-up. Important issues associated with quality assurance of continuous data are discussed and the application of a state-of-the-art software product AQUARIUS, which streamlines the process of data error correction and dissemination, is described as an essential element of ensuring successful RTSM implementation in the San Joaquin River Basin.
Keywords: Environmental decision support; Forecasting; Sensor networks; Sensors; Water quality

Agile on steroids through the RenDanHeYi - Quintarelli, Emanuele (Boundaryless, 2023)
This article offers initial insights into the overlap, positioning, shared values, and convergences between RenDanHeYi compared to agile practices.

A Call for Innovation in International Development - R4D (Results for Development, 2015)

What are you afraid of: Collective leadership and its learning implications - Raelin, Joseph A (Management Learning, 2018)
In this provocation, the author attempts to cite the advantages of collective leadership while acknowledging the objections and fears of challengers. Collective leadership is seen as remote because it defies the traditional view of leadership as an individualistic attractive quality that not only protects us but is efficient when applied. Nevertheless, the collective alternative may not only be advisable but required in a connected world featuring a networked economy. The contemporary socio-politico-economic environment requires the contribution of, creativity from, and collaboration among multiple agents providing a dynamic concentration of management and knowledge. If we are to accept and recognize the contribution of a collective leadership, its development would require an entirely different learning model. In particular, collective leadership development occurs as an acute immersion into the practices that are embedded within in situ material–discursive relations—in other words, among people, objects, and their institutions.

MERL Tech State of the Field - The evolution of MERL Tech - Raftree, Linda (MERL Tech, 2020)

Emerging Opportunities: Monitoring and Evaluation in a Tech-Enabled World - Raftree, Linda; Bamberger, Michael (Discussion Paper, 2014)
Monitoring and evaluation practice is using information and communication technologies for more timely data, and more inclusive voice and feedback.

What Is a Case?: Exploring the Foundations of Social Inquiry - Ragin, Charles (Cambridge University Press, 2010)
The concept of the case is a basic feature of social science research and yet many questions about how a case should be defined, selected, and judged are far from settled. The contributors to this volume probe the nature of the case and the ways in which different understandings of the concept affect the conduct and the results of research. The contributions demonstrate that the work of any given researcher is often characterised by some hybrid of these basic approaches, and it is important to understand that most research involves multiple definitions and uses of cases, as both specific empirical phenomena and as general theoretical categories.

Getting Practical With Causal Mechanisms: The application of Process-Tracing Under Real-World Evaluation Constraints - Raimondo, Estelle (New Directions for Evaluation, 2020)
Over the past decade, the field of development evaluation has seen a renewed interest in methodological approaches that can answer compelling causal questions about what works, for whom, and why. Development evaluators have notably started to experiment with Bayesian Process Tracing to unpack, test, and enhance their comprehension of causal mechanisms triggered by development interventions. This chapter conveys one such experience of applying Bayesian Process Tracing to the study of citizen engagement interventions within a conditional cash transfer program under real-world evaluation conditions. The chapter builds on this experience to discuss the benefits, challenges, and potential for the applicability of this approach under real-world evaluation conditions of time, money, and political constraints.

Aid on the Edge of Chaos: Rethinking International Cooperation in a Complex World - Ramalingam, Ben (Oxford University Press, 2014)
It is widely recognised that the foreign aid system - of which every country in the world is a part - is in need of drastic overhaul. There are conflicting opinions as to what should be done. Some call for dramatic increases to achieve longstanding promises. Others bang the drum for cutting it altogether, and suggest putting the fate of poor and vulnerable people in the hands of markets or business. A few argue that what is needed is creative, innovative transformation. The arguments in Aid on the Edge of Chaos are firmly in the third of these categories. In this ground-breaking book, Ben Ramalingam shows that the linear, mechanistic models and assumptions that foreign aid is built on are more at home in early twentieth century industry than in the dynamic, complex world we face today. The reality is that economies and societies are less like machines and more like ecosystems. Aid on the Edge of Chaos explores how thinkers and practitioners in economics, business, and public policy have started to embrace new, ecologically literate approaches to thinking and acting, informed by the ideas of complex adaptive systems research. It showcases insights, experiences, and dramatic results of a growing network of practitioners, researchers, and policy makers who are applying a complexity-informed approach to aid challenges. From transforming approaches to child malnutrition, to rethinking process of macroeconomic growth, from rural Vietnam to urban Columbia, Aid on the Edge of Chaos shows how embracing the ideas of complex systems thinking can help make foreign aid more relevant, more appropriate, more innovative, and more catalytic. It argues that taking on these ideas will be a vital part of the transformation of aid, from a post-WW2 mechanism of resource transfer, to a truly innovative and dynamic form of global cooperation fit for the twenty-first century.

Development innovation: Fad, silo or catalyst? - Ramalingam, Ben (Nesta, 2015)
In the first of a two-part blog piece, Ben Ramalingam, affiliate of the Overseas Development Institute / Institute of Development Studies, discusses the challenges of implementing innovation in to the development sector - from keeping expectations realistic to managing the innovation process and assessing impact and value.

Development innovation: Taking the high road - Ramalingam, Ben (Nesta, 2015)
In part two of his blog on the challenges facing development innovation, Ben Ramalingam suggests how we might take the ‘high road’ to bring about long-lasting and meaningful innovation in the sector.

Herramientas de Conocimiento y Aprendizaje: Una Guía para Organizaciones Humanitarias y de Desarrollo - Ramalingam, Ben (ODI, 2006)
This toolkit presents entry points and references to the wide range of tools and methods that have been used to facilitate improved knowledge and learning in the development and humanitarian sectors.
Keywords: spanish

Learning how to learn: eight lessons for impact evaluations that make a difference - Ramalingam, Ben (ODI, 2011)
This Background Note outlines key lessons on impact evaluations, utilisation-focused evaluations and evidence-based policy. While methodological pluralism is the key to effective impact evaluation in development, the focus here is on the factors that need to be considered for impact evaluations to be used in policy and practice – regardless of the method employed.

Learning to Adapt: Building Adaptive Management as a Core Competency in Development Practice - Ramalingam, Ben (Institute of Development Studies, 2016)
Key messages 1. Development policies and programs increasingly operate in situations of high complexity and uncertainty. 2. There are growing efforts across the sector to design, plan and implement more adaptive responses that are more relevant and appropriate in such contexts 3. At the heart of effective adaptive programming is the capability to gather, interpret and use knowledge, information and data in real-time 4. Strengthening this capability requires a positive enabling environment—including tools and methods, structures and processes, leadership and management and culture and mindsets. 5. For adaptive development to succeed, there needs to be greater attention and investment in both knowledge, information and data systems and in appropriate enabling environments
Keywords: IMPORTANT

Manifesto for Half-Arsed Development Reforms - Ramalingam, Ben (Aid on the Edge of Chaos, 2014)
Keywords: IMPORTANT

Tools for Knowledge and Learning: A Guide for Development and Humanitarian Organisations - Ramalingam, Ben (ODI, 2006)
This toolkit presents entry points and references to the wide range of tools and methods that have been used to facilitate improved knowledge and learning in the development and humanitarian sectors.

Bridging Real-Time Data and Adaptive Management in International Devepment - Case Study Report - Ramalingam, Ben; Barnett, Inka; Valters, Craig; Oppenheimer, Carrie; Levy, Anna; Hernandez, Kevin (IDS, 2017)

Innovation for International Development: Navigating the Paths and Pitfalls - Ramalingam, Ben; Bound, Kirsten (NESTA, 2016)
Experiences, insights and practical advice from over 20 leading practitioners in innovation for international development, brought together in one collection.

Ten Frontier Technologies for International Development - Ramalingam, Ben; Hernandez, Kevin; Prieto Martin, Pedro; Faith, Becky (IDS, 2016)
As new technologies and digital business models reshape economies and disrupt incumbencies, interest has surged in the potential of novel frontier technologies to also contribute to positive changes in international development and humanitarian contexts. Widespread adoption of new technologies is acknowledged as centrally important to achieving the United Nations Sustainable Development Goals by 2030. But while frontier technologies can rapidly address large-scale economic, social or political challenges, they can also involve the displacement of existing technologies and carry considerable uncertainty and risk. Although there have been significant wins bringing the benefits of new technologies to poor consumers through examples such as mobile money or off-grid solar energy, there are many other areas where the applications may not yet have been developed into viable market solutions, or where opportunities have not yet been taken up in development practice.

Exploring the science of complexity: Ideas and implications for development and humanitarian efforts - Ramalingam, Ben; Jones, Harry; Reba, Toussaint; Young, John (Working Paper 285, 2008)
This paper draws on the science of complexity to outline alternative approaches to analysing and understanding problems faced in humanitarian and development work.

From best practice to best fit: understanding and navigating wicked problems in international development - Ramalingam, Ben; Laric, Miguel; Primrose, John (Working Paper, 2014)

Strengthening humanitarian networks: Applying the network functions approach - Ramalingam, Ben; Mendizabal, Enrique; Schenkenberg van Mierop, Ed (Background papers, 2008)
This note offers a simple, flexible and powerful methodology — the network functions approach (NFA) — that can be applied to analyse and strengthen humanitarian and development networks. Based on research undertaken at ODI and elsewhere, the NFA suggests there are six overlapping functions that different networks perform in varying combinations. Through reflection on a network’s current activities and how they relate to each of these functions, the NFA helps those facilitating, acting within or supporting networks to work towards an ‘ideal’ functional mix. The aim is for network strategies to be honed, thinking clarified, activities sharpened and ultimately, humanitarian performance improved.

Learning to change: The case for systemic learning strategies in the humanitarian sector - Ramalingam, Ben; Mitchell, John (ALNAP/ODI, 2022)
This paper presents the case for systemic organisational change in the humanitarian system. The paper firstly shows that that organisational learning has tended to reinforce existing ways of working and has not been able to shift a culture that values action over reflection. As a result, the rest of the paper asks about the most significant changes in the humanitarian sector

5 Principles to Guide Adaptive Leadership - Ramalingam, Ben; Nabarro, David; Oqubuy, Arkebe; Carnall, Dame Ruth; Wild, Leni (Harvard Business Review, 2020)
How you respond to a crisis will have repercussions for years to come. The Covid-19 pandemic is constantly evolving, with leaders facing unpredictability, imperfect information, multiple unknowns, and the need to identify responses quickly — all while recognizing the multi-dimensional (health-related, economic, social, political, cultural) nature of the crisis. Responding to the crisis requires adaptive leadership, which involves what we refer to as the 4 A’s: Anticipation of likely future needs, trends and options. Articulation of these needs to build collective understanding and support for action. Adaptation so that there is continuous learning and the adjustment of responses as necessary. Accountability, including maximum transparency in decision-making processes and openness to challenges and feedback.
Keywords: Crisis management; Leadership

Strengthening the Humanitarian Innovation Ecosystem - Ramalingam, Ben; Rush, Howard; Bessant, John; Marshall, Nicholas; Gray, Bill; Hoffman, Kurt; Bayley, Simon; Warren, Kim (University of Brighton, 2015)

Innovations in international humanitarian action - Ramalingam, Ben; Scriven, Kim; Foley, Conor (8th review of humanitarian action, 2009)

Making adaptive rigour work - Principles and practices for strenghening monitoring, evaluation and learning for adaptive management - Ramalingam, Ben; Wild, Leni; Buffardi, Anne L (ODI/GLAM, 2019)
Adaptive programmes can be accountable, rigorous and high quality in how they use evidence by taking an ’adaptive rigour’ approach. Core development and humanitarian challenges are complex, and require processes of testing, learning and iteration to find solutions – adaptive management offers one approach for this. Yet large bureaucracies and development organisations can have low tolerance for experimentation and learning, and adaptive management can be viewed as an excuse for ‘making things up as you go along’. This briefing from the Global Learning for Adaptive Management (GLAM) initiative argues that adaptive programmes can be accountable, rigorous and high quality in how they use evidence – but this requires rethinking some key assumptions about how they are practised. The paper sets out three key elements of an ‘adaptive rigour’ approach: - Strengthening the quality of monitoring, evaluation and learning data and systems. - Ensuring appropriate investment in monitoring, evaluation and learning across the programme cycle. - Strengthening capacities and incentives to ensure the effective use of evidence and learning as part of decision-making, leading ultimately to improved effectiveness.

Adaptive leadership in the coronavirus response - Ramalingam, Ben; Wild, Leni; Ferrari, Matt (ODI, 2020)
The coronavirus pandemic poses unprecedented challenges to science, policy and the interface between the two. How – and how quickly – policy-makers, practitioners and researchers react to this emerging and complex crisis is making a profound difference to people’s lives and livelihoods (WHO, 2020). But how can we ensure effective collective decision-making on the basis of emerging evidence, changing trends and shifting scientific understanding, all in the face of considerable uncertainty? Recent experience highlights the need for adaptive leadership in national and global responses to the outbreak. This briefing paper sets out key principles for what this might look like, and proposes a roadmap for policy-makers, practitioners and researchers to move towards such an approach as they tackle the unfolding crisis. Key messages Tackling the coronavirus outbreak requires adaptation at operational and leadership levels. Operationally, there is scope to strengthen evidence-based adaptive management practices, to adjust the mix and type of interventions being implemented and learn as we go so as to achieve shared goals. This requires adaptive leadership capacities, being open and transparent about learning, using collective decision-making processes and building trust with communities and individuals.

Utilization focused evaluation: a primer for evaluators - Ramirez, Ricardo; Brodhead, Dal (Southbound, 2013)
Ramírez, R., & Brodhead, D. (2013). Utilization focused evaluation: A primer for evaluators. Penang, Malaysia: Southbound. Retrieved from http://evaluationinpractice.files.wordpress.com/2013/04/ufeenglishprimer... This book, authored by Ricardo Ramírez and Dal Brodhead, is designed to support evaluators and program managers implement Utilization-focused evaluation (UFE). It includes detailed discussion of the 12 steps for implementing UFE and also provides a number of case studies to guide the user. Excerpt "Throughout this Primer we refer to the value of having a mentor to assist an evaluator who is using UFE for the first time. Our collective experiences with UFE indicated having a mentor was, for many UFE participants, an essential support and it reflects how we learned and mentored UFE. Evaluators may use elements of a UFE in their work naturally, for example by engaging users in planning the process or in assisting them in the utilization of findings. This Primer, however, walks the reader through UFE by systematically covering all of the 12 steps. It reflects deeply on the UFE evaluation practice and builds from it." (Ramírez & Brodhead 2013) Contents What is Utilization Focused Evaluation? 1 The UFE Framework Summarized in Steps 3 The DECI Project 6 The UFE Steps Illustrated with Project Examples 9 Step 1 Assessing Program Readiness 10 Step 2 Assessing Evaluators’ Readiness 18 Step 3 Identifying Primary Intended Users 24 Step 4 Situational Analysis 31 Step 5 Identification of Primary Intended Users 38 Step 6 Focusing the Evaluation 44 Step 7 Evaluation Design 54 Step 8 Simulation of Use 58 Step 9 Data Collection 62 Step 10 Data Analysis 65 Step 11 Facilitation of Use 68 Step 12 Meta Evaluation 72 Summary About What Each Step Entails 76 What Benefit Does UFE Bring to Commissioners of Evaluation? 81 Take Away Lessons 83 Postscript 87 The UFE Checklist 89 Case Studies 103 Recommended Reading 110

Accounting for a changing and uncertain climate in planning and policymaking today: lessons for developing countries - Ranger, Nicola; Garbett-Shiels, Su-Lin (Climate and Development, 2012)
Climate change is increasingly altering the pattern of climate-related risks. Developing countries and in particular least developed countries will be among the most severely impacted by climate change. These risks can seem remote in comparison with more immediate threats and needs, but if climate change is not considered upfront in existing planning and policymaking processes today, decision makers risk locking-in future impacts that may prove irreversible or much more costly and difficult to rectify than is necessary. The challenge for planners and policymakers, explored in this paper, is that future climate conditions are deeply uncertain. Decision methods are available to tackle these problems; however, these tend to be data- and resource-intensive and therefore, difficult to routinely apply. Further, a gap in currently available guidance is the explicit link to the adaptation needs of a developing country. We discuss the implications of this development context for the priorities for adaptation and the relative allocation of efforts in adaptation. This paper focuses on the identification of adaptation options and strategies that are robust to the deep uncertainties in future climate risk, culminating in a framework of six building blocks. It takes the perspective of exploring how decisions today might be adjusted to account for uncertain and changing long-term climate risks. We suggest a core principle is to focus on promoting climate-resilient development and increasing long-term adaptive capacity while, crucially, avoiding inflexible decisions that could lock-in future climate risk or foreclose adaptation options.
Keywords: Adaptation; Climate change; Development; Planning; robustness; uncertainty

Problem-driven iterative approaches and wider governance reform - Rao, Sumedh (Helpdesk Research Report 1099, 2014)
Identify evidence which suggests that problem-driven, iterative approaches to public sector reform can deliver more substantial, wider, long-term governance reform.

Adapting lean thinking to market systems development: Principles and practices for donors/funders - Rasmussen, Leanne (Engineers without borders, 2014)
The purpose of this paper is to provide funders and implementers of market systems development (MSD) projects with principles, practices, and structures that enables these projects to thrive. It is based on a book that has sold millions of copies worldwide, and a school of thought taught at institutions such as Harvard Business School and practiced in companies ranging from giants such as Toyota to the most successful tech start-ups in Silicon Valley. Published in 2011, Eric Ries wrote the Lean Startup as a remedy to the countless start-ups that create their own demises by getting started with the wrong goals, the wrong structures, and the wrong processes. He outlines an approach that enables the startup to navigate ambiguity and risk while using resources effectively. The conditions a MSD project faces are remarkably similar to those which a startup business faces; it needs structures and processes that are matched to the unpredictable, complex environment which it is attempting to influence. MSD projects can thus benefit from private-sector thought leadership on how start-ups can situate themselves for success. This paper starts with an overview of lean thinking, a concept that derives from lean manufacturing which is widely accepted as the leading approach to modern manufacturing. It explains how Ries’s Lean Startup adapts these principles, and then it in turn adapts these ideas for market systems development. The following three sections provide an overview of Lean Startup concepts, applying them to the development sector as appropriate. The final section summarizes and makes suggestions on next steps for funders who wish to set MSD projects up for success using a lean approach. Overall, the paper builds a case for how the Lean Startup’s approach can enable MSD projects to work successfully in ambiguity and increase their potential for achieving robust and sustainable results, all while using donor resources more efficiently.
Keywords: Adaptive Development; Lean