Participation and Power in Climate Change Adaptation Policies: Vulnerability in Food Security Programs in Nepal - Nagoda, Sigrid; Nightingale, Andrea J. (World Development, 2017)
The article explores the moments wherein participatory approaches in climate change adaptation (CCA) policies contribute to reinforcing, rather than transforming, the underlying causes of vulnerability. Using the case of food insecure households in the district of Humla in northwestern Nepal, the study demonstrates that the same social and power relations that are driving local vulnerability dynamics, such as caste, gender, and access to social and political networks, also play important roles in shaping the impact of CCA policies. By tracing Nepal’s CCA programs, starting with the local level, through district to international-national level dynamics, the study adds insights into the barriers to exclusion that embed power relations all the way through the chain of policy development. The purpose is to better understand how CCA can perpetuate rather than alleviate the conditions that create differential vulnerability patterns at village level. It raises questions about how whether CCA programs are an adequate response to increasing vulnerability for some of the world’s most marginalized people.

The Clash of the Counter-bureaucracy and Development - Natsios, Andrew (The Center for Global Development, 2010)
In this essay, Andrew Natsios gives a first-hand account of what he finds most hinders USAID—layers of bureaucracy that misguide and derail development work.

Designing for Public Services - Nesta (Nesta/IDEO, 2016)

DIY - Development Impact and You: Practical Tools to Trigger and Support Social Innovation - Nesta (NESTA, 2014)
Keywords: IMPORTANT

Skills, atitudes and behaviours that fuel public innovation - a guide to getting the most from Nesta's Competency Framework for Experimenting and Public Problem Solving - Nesta (Nesta, 2018)

States of Change - Learning programme - Nesta (Nesta, 2018)

Life and death in the App Store - Newton, Casey (The Verge, 2016)
As last year began, the app developer Pixite held its company retreat at a converted Old West movie set outside of Palm Springs. They spent a few days dreaming about the future while eating...

6 Reviewing research evidence - NICE (6. Developing NICE guidelines: the manual, 2012)

A guide to Social Return on Investment - Nicholls, Jeremy; Lawlor, Eilis; Neitzert, Eva; Goodspeed, Tim (The SROI Network, 2012)
There is increasing recognition that we need better ways to account for the social, economic and environmental value that results from our activities. The language varies – ‘impact’, ‘returns’, ‘benefit’, ‘value’ – but the questions around what sort of difference and how much of a difference we are making are the same. Understanding and managing this broader value is becoming increasingly important for the public and private sectors alike. This is true whether it is civil society organisations working to create value, Governments commissioning and investing in activities to create social value, investors seeking to ensure that their investments will make a difference, or private businesses recognising both risk and opportunity in the wider effects of operations. All this means that it is also more important that we have some consistency and a shared language when we talk about value. SROI is the application of a set of principles within a framework that is designed to help bring about that consistency, whilst at the same time recognising that what is of value will be very different for different people in different situations and cultures. The first edition of this guide, which itself built on the work of three earlier SROI guides1, was prepared as part of a three year programme on measuring social value funded in 2008 by the then ‘Office of the Third Sector’ based in the Cabinet Office of the UK Government. This was delivered by a consortium of organisations: the SROI Network, nef (the new economics foundation), Charities Evaluation Services, the National Council for Voluntary Organsations and New Philanthropy Capital. In addition to this programme, the Scottish Government also supported the development of SROI, including a database of indicators to support SROI analysis. The work of the SROI Network now stretches across many different countries and continents, and this second edition of the guide reflects that interest. We have though decided to use only one currency symbol, for reasons of clarity and consistency, and so have continued to use £. However, readers will be able to find examples in various currencies and translations of the Guide on our website. For more information on the developments of SROI, please refer to the SROI Network website: www.thesroinetwork.org

Guidance: Rigor and Reproducibility in Grant Applications | grants.nih.gov - NIH (National Institutes of Health, 2018)

Supporting accountability in fragile settings: A review for the Somalia Implementation and Analysis in Action of Accountability Programme - Nixon, Hamish; Buffardi, Anne; Wales, Joseph; Pasanen, Tiina (ODI, 2017)

From purposeful to meaningful adaptive programming: how about adaptive operations? - Nixon, Nicola (Devpolicy Blog from the Development Policy Centre, 2019)
Embracing adaptive programming approaches but sticking with rigid, traditional systems-based operations? It's time to take a long hard look at all the boring stuff, says Nicola Nixon.

Theory of change in ten steps - Noble, James (NPC, 2019)

The Design of Everyday Things, revised and expanded edition - Norman, Donald A. (MIT Press, 2013)
Even the smartest among us can feel inept as we try to figure out the shower control in a hotel or attempt to navigate an unfamiliar television set or stove. When The Design of Everyday Things was published in 1988, cognitive scientist Don Norman provocatively proposed that the fault lies not in ourselves, but in design that ignores the needs and psychology of people. Fully revised to keep the timeless principles of psychology up to date with ever-changing new technologies, The Design of Everyday Things is a powerful appeal for good design, and a reminder of how -- and why -- some products satisfy while others only disappoint.

Experiential Learning: Fostering lasting behavioural change - Norton, Brittney M. (Prospects practice paper, 4, 2016)
The term ‘experiential learning’ is often mistakenly used to describe any learning experience that involves participatory activities. This paper presents an overview of what true experiential learning is and how this form of learning allows for real-world application - thus promoting lasting behavioural change. Furthermore, it explains how experiential learning was adopted as the methodological core of the Prospects psychosocial programme in Liberia, which builds ‘pre-employment skills’ in 3,500 vulnerable youth aged 15-24 years. The goal of the paper is to provide other youth and education development practitioners with a resource that can be used to strengthen youth and behaviour change programmes through the incorporation of the experiential learning theory. In addition, this paper explains its application in the Prospects programme, before concluding with key considerations of such a design. 1

Working the System: What We’ve Learned About Strengthening Accountability - Nugent, Sarah (Chemonics International, 2018)
When designing a new project, how often do we set out to strengthen a particular actor’s or institution’s ability to “hold government to account?” What does that mean exactly? Maybe the better question to ask is: what is the most effective way to strengthen accountability in our development work? And then, what does it look …

Using adaptive monitoring, evaluation and learning in programme design - Nzegwu, Femi (INASP, 2018)

Adaptive management: What it means for CSOs - O’Donnell, Michael (Bond, 2016)
Keywords: IMPORTANT

Is It Time for a New Paradigm for "Citizen Engagement"? The Role of Context and What the Evidence Tells Us - O'Meally, Simon (People, Spaces, Deliberation, 2013)
The meteoric rise of "citizen engagement" Almost all development agencies promote some form of citizen engagement and accountability, often framed as 'voice', 'demand-side governance', 'demand for good governance' or 'social accountability'. The current World Bank president, Jim Yong Kim, recently put it that, "citizen voice can be pivotal in providing the demand-side pressure on government, service providers, and organizations such as the World Bank that is needed to encourage full and swift response to citizen needs". There has, in turn, been a mushrooming of useful operational guidance on different "tools" for social accountability - i.e. steps, inputs and methodologies - that guide discrete interventions, ranging from citizen score cards to participatory expenditure tracking. One might, however, be forgiven for thinking that some of the debates on citizen engagement need an injection of realism; especially as contextual factors can make or break a "tool's" implementation. A review of experience to date would be one good place to start.

Mapping Context for Social Accountability: A resource paper - O'Meally, Sion C. (The World Bank, 2013)

Using adaptive development to support feminist action - O’Neil, Tam (ODI, 2016)
This paper looks at how adaptive development is being applied by gender programmes and argues that gender and adaptive development communities have much to offer each other.

Doing legal empowerment differently: Learning from pro-poor litigation in Bangladesh - O'Neil, Tam; Valters, Craig; Farid, Cynthia (ODI, 2015)
Marginalised groups and their allies can, and do, use the law and justice systems, including public interest litigation, to improve their access to rights, goods and services.  Yet there is no automatic link between legal action and improved outcomes for poor people.  Where some minimum conditions are met – a progressive legal framework, a sympathetic judiciary and legal advocacy organisation – pro-poor litigation is a potential tool in disputes over rights and resources.  But concrete benefits for poor people also requires state action to enforce progressive rulings.  Much depends on whether claimants, legal activists and state reformers cooperate around shared interests, and whether activists can negotiate power and interest structures to motivate government action to implement rulings.  Effective strategies are those that link litigation with grassroots legal action and other forms of political and social activism and advocacy.  Donors need to fund legal advocacy organisations in ways that enable them to select social issues that are locally relevant and political feasible – and allows activists and reformers to work in politically smart and adaptive ways.

Dynamic gridlock: Adaptive Humanitarian Action in the Democratic Republic of Congo - Obrecht, Alice (ALNAP Country Study, 2018)

Making humanitarian response more flexible: Challenges and questions - Obrecht, Alice; Bourne, S. (ALNAP/ODI, 2018)

More than just luck: Innovation in humanitarian action - Obrecht, Alice; Warner, Alexandra T. (Humanitarian Innovation Fund, 2016)
In South Sudan, a new water treatment system that provides a community with more clean water at a lower cost. In Sierra Leone, a poster explaining how to prevent the...

Evaluating Humanitarian Innovation - Obrecht, Alice; Warner, Alexandra; Dillon, Neil (HIF-ALNAP working paper, 2017)

Doing Development Differently: Who we are, What we’re doing, What we’re learning - ODI (ODI, 2016)
In November 2014, the doing development differently community got together in Harvard to discuss what successful development interventions look like. Two years on, our community is broader than aid. It's broader than donors. It's about all organisations delivering change, producing real solutions to real problems that have real impact. It's about building trust, empowering people and promoting sustainability. Over the past two years, the community has been putting these ideas into practice across the world – being honest about what we are learning, including where we are not getting things right. This document aims to be an entry point for anyone interested in doing development differently. It explores the things that could be stopping you including: You are constrained by a disabling environment. You have strict reporting requirements. People aren't sold on the idea. It's easier to do things the way you've always done it. It's difficult to do in practice. If you are facing one, several, or all of these constraints, this document gives you a place to start. Using 43 case studies from practitoners within donors, governments, implementing organisations and NGOs across the globe, we aim to draw out some key lessons learnt, and give some advice for people considering this approach. These case studies are split across five categories: Swimming against the tide. Working in and with government. Feedback loops and data. Organisational change. Diffusion. Finally, if you want to know more, the document concludes with a suggested list of blogs and reports which explain these concepts in more detail.
Keywords: IMPORTANT

The Methods Lab evaluation toolkit - ODI (2016)
The Methods Lab develops and tests flexible approaches to impact evaluation for interventions that are harder to evaluate because of their complexity. This toolkit brings together analysis, guidance and templates for anyone: planning an impact evaluation designing a monitoring and evaluation system working in a consortium or managing a portfolio of projects

Case studies of results-based management by providers: Canada - OECD (OECD, 2017)

Case studies of results-based management by providers: New Zealand - OECD (OECD, 2017)

Case studies of results-based management by providers: Sweden - OECD (OECD, 2017)

Case studies of results-based management by providers: Switzerland - OECD (OECD, 2017)

Case studies of results-based management by providers: The Netherlands - OECD (OECD, 2017)

Case studies of results-based management by providers: United Kingdom - OECD (OECD, 2017)

Case studies of results-based management by providers: World Bank Group - OECD (OECD, 2017)

DAC - Mid-term Peer Review of Switzaerland - 2016 - OECD (OECD, 2016)

DAC Peer Review of Switzerland - 2013 - OECD (OECD, 2013)

DAC Peer Review of Switzerland - 2019 - OECD (OECD, 2019)

Evaluation of development programmes - OECD - OECD (2019)
Data & research on evaluation of development programmes inc. paris declaration, budget support, multilateral effectiveness, impact evaluation, joint evaluations, governance, aid for trade

OECD Development Assistance Committee Peer Review of Canada - OECD (OECD, 2018)

Systems Approaches to Public Sector Challenges: Working with Change - OECD (OECD, 2017)

Quality standards for development evaluation - OECD; Development Assistance Committee (OECD, 2010)
Based on a broad international consultative process, the DAC Quality Standards for Development Evaluation are a reference guide to good practice in development evaluation. With a view to improving the quality of evaluation processes and products, and facilitating collaboration, this reference guide lays out standards for each phase of a typical evaluation process: from defining purpose, to planning, designing, implementing, reporting, and learning from and using evaluation results.

Minister of International Development and La Francophonie Mandate Letter (November 12, 2015) - Office of the Prime Minister (2015)
Dear Minister:I am honoured that you have agreed to serve Canadians as Minister of International Development and La Francophonie. You will be part of a strong team of ministers led by the Minister of Foreign Affairs.

A Holistic Approach to Evaluating Research - Ofir, Zenda; Schwandt, Thomas; Duggan, Colleen; McLean, Robert (IDRC, 2016)

Open Governance Research Exchange - OGRX (Your hub for quantitative and qualitative research on innovations in governance, 2017)
Your hub for quantitative and qualitative research on innovations in governance.

The limits of agile - can we apply it to policy making? - Ollerhead, Lisa (2015)
Bringing new policy tools and techniques to the UK Government

Evaluating the Design of Behavior Change Interventions: A Case Study of Rhino Horn in Vietnam - Olmedo, Alegría; Sharif, Vian; Milner‐Gulland, E. J. (Conservation Letters, 2018)
Behavioral change interventions are increasingly widely used in conservation. Several projects addressing rhino horn consumption were recently launched in Vietnam. We used key informant interviews, document analysis, and marketing theory to explore their strategies for intervention design, implementation, monitoring, and evaluation. We developed a framework to evaluate whether they followed best practice and identify implementation challenges. Interventions could make greater use of key project design steps, including basing interventions on robust research to understand the behavior in question, identifying the target audience whose behavior interventions aim to change, and developing measures that can provide reliable evidence of success or not. Challenges include the need for law enforcement to complement campaigns; improving cooperation between NGOs; and clearly defining aims of demand-reduction initiatives. Using best practice from other fields and considering demand reduction within the wider context of wildlife, trade policy will help address these challenges.
Keywords: Awareness; Ilegal wildlife trade; Project design; demand reduction; social marketing

Systems Practice - Omidyar Group (The Omidyar Group, 2017)

Using Participatory Statistics to Examine the Impact of Interventions to Eradicate Slavery: Lessons from the Field - Oosterhoff, P.; Bharadwaj, S.; Burns, D.; Raj, A. M.; Nanda, R. B.; Narayanan, P. (2016)
This CDI Practice Paper by Pauline Oosterhoff, Sowmyaa Bharadwaj, Danny Burns, Aruna Mohan Raj, Rituu B. Nanda and Pradeep Narayanan reflects on the use of participatory statistics to assess the impact of interventions to eradicate slavery and bonded labour. It deals with: (1) the challenges of estimating changes in the magnitude of various forms of slavery; (2) the potential of combining participatory approaches with statistical principles to generate robust data for assessing impact of slavery eradication; and (3) the practical and ethical questions in relation to working with people living within a context of modern slavery. The paper draws lessons from the realities of using participatory statistics to support the evaluation of a slavery eradication programme in North India.

Modern Slavery Prevention and Responses in South Asia: An Evidence Map - Oosterhoff, Pauline; Yunus, Raudah; Jensen, Charity; Somerwell, Francis; Pocock, Nicola (DFID, 2018)
The Asia Pacific region has the highest numbers of both slavery and child labour victims in the world. The International Labour Organization (ILO) estimates that 16.5 million and 8.4 million persons were in situations of forced labour and forced marriage respectively in Asia Pacific, among 40.3 million in modern slavery globally (ILO & Walk Free Foundation 2017). Although there is a growing body of research and evaluations on specific sub-sectors and interest in the worst forms of labour exploitation, there has not yet been a systematic scoping or synthesis of studies that would help policymakers understand ‘what works’ to reduce the prevalence of “Modern Slavery”2(MS) in the countries of interest to DFID (India, Nepal, Bangladesh, Pakistan). India alone is estimated to have the largest number of people in modern slavery in the world with nearly 8 million persons living in modern slavery, followed by 3.2 million, 592,000 and 171,000 in Pakistan, Bangladesh and Nepal respectively (Walk Free Foundation 2018). While systematic reviews on interventions to reduce prevalence of trafficking, forced or bonded labour or slavery exist, these have not focused on the target countries of interest to DFID. Previous systematic reviews have focussed on particular types of MS, such as cross border sexual exploitation globally (van der Laan et al. 2011), labour exploitation in Europe (Cockbain et al. 2018) or community based interventions for safer migration programming in low and middle income countries (LMICs) (Zimmerman et al. 2016). Several systematic reviews focus specifically on health needs and post-trafficking care or interventions (Ottisova et al. 2016; Muraya & Fry 2016; Hemmings et al. 2016; Dell et al. 2017), and research methods and tools used in trafficking in persons (TIP) and health research (Cannon et al. 2018; Doherty et al. 2016). A recent global review and evidence map of MS interventions found some interventions in the target countries, but the outcomes were not specified (Bryant & Joudo 2018). In this Evidence Map, we address this gap in evidence for India, Nepal, Bangladesh and Pakistan specifically, scoping the range of Modern Slavery interventions and outcomes for specific target populations (survivors, employers, landlords, services providers, criminal justice officials) and at different levels (individual, community, state). This map is foremost targeted to DFID and its partners in India, Nepal, Bangladesh and Pakistan to inform evidence-based policy making. We hope that it is also useful to academics and practitioners working to address modern slavery, or in intervention areas or locations described.

Adaptive management in wildlife conservation - Organ, John; Decker, D.J.; McDonald, J.E.; Riley, Shawn; Mahoney, S.P. (The Wildlife Society Techniques Manual, 2012)
Adaptive management in wildlife conservation emerged from the wildlife profession's search for better solutions to increasingly complex conservation challenges. Adaptive management is an effective process for wildlife managers to employ to (1) deal with uncertainty in the management system, (2) learn from their management actions, and (3) achieve desired results. Being adaptable or flexible in your management approach is not the same as managing adaptively or conducting adaptive management. Adaptive management requires adhering to a stepwise process and fully executing each step. A critical step is rigorous monitoring and assessment of management interventions. Without this, wildlife managers cannot achieve the essence of adaptive management, which is the explicit goal of learning more about the management system after each management action.

“Shared learning” for building urban climate resilience – experiences from Asian cities - Orleans Reed, Sarah; Friend, Richard; Toan, Vu Canh; Thinphanga, Pakamas; Sutarto, Ratri; Singh, Dilip (Environment and Urbanization, 2013)
This paper considers how resilience thinking and, in particular, its emphasis on learning has been applied in 10 cities in Vietnam, India, Thailand and Indonesia. Applying a “shared learning” approach in the Asian Cities Climate Change Resilience Network (ACCCRN) has helped to create or strengthen networks, build appreciation for complexity and uncertainty among stakeholders, provide a space for deliberating concepts such as vulnerability and resilience, and build knowledge and capacities for stakeholders to engage and represent their own interests. Shared learning approaches face considerable challenges navigating politicized urban environments, in which the nature and value of existing systems − and therefore the value of building resilience − are contested. This article suggests that deliberate, strategic intervention by facilitators may contribute to more transformative change on behalf of equitable, socially just outcomes – and thus cautions against seeing urban climate vulnerability as a technical challenge, or shared learning as a “toolkit” for building resilience.

‘Learning journeys’ for adaptive management – Where does it take us? - Ørnemark, Charlotte (GPSA Note 12, 2016)
Keywords: IMPORTANT

What about the results? Lessons from long-term process support to strengthen results-based management (RBM) for Swedish framework NGOs operating in Western Balkans - Ørnemark, Charlotte (NCG, 2012)
The observations and lessons outlined in this report should be seen as a contribution to the on-going learning and reflections in a wider debate on how to assess and monitor results from support to CSOs involved in complex social change processes (as opposed to those involved in more ‘classical’ service delivery and more linear development assistance). These lessons, though directly emerging from the consultancy to provide long-term RBM support to the FOs and their local CSO partners in Western Balkans, also draws on previous experiences and similar lessons from on-going initiatives. In brief, these lessons call for a certain ‘paradigm shift’ in the way we (as development professionals) regard and support RBM for actors involved in less tangible social change processes in highly contested political spaces. Some of these shifts, outlined in more detail in different sections of this report, are summarised below:  From a linear, aggregated cause-and-effect thinking around RBM to one that is linked to human factors and is embedded in systems,  From pushing the burden of reporting down in the system to a clearer division of labour between donors, recipient governments, intermediary agents like framework NGOs (FOs) and implementing CSOs with each actor using its comparative advantage and appropriate role in the RBM process and for its own learning,  From operating in programming and results frameworks where outcomes and impact are largely assumed to be predictable to setting up systems that deal with uncertainty and that capture emerging result patterns through tracking of gradual changes,  From a project/programme perspective to a focus on institutions and systems as actors and arenas for change, where organisations are enabled to act as change agents towards clearly identified processes of social transformation,  From SMART to REAL results frameworks – although SMART (specific, measurable, achievable, realistic and time-bound) principles are always good to keep in mind for the formulation of goals and indicators, the danger of coming up with fictive measuring frameworks calls for a more systematic incorporation of ‘real’ concerns, including making RBM processes rights-based, embedded in local realities (and empowering for those involved), aligned to national reform efforts and national and international human rights commitments, and learning-oriented for all different levels of operations.

Planning & Navigating Social Change - Tools for Pacific voyagers - Orr, Douglas Epeli; Cavatore, Maria; Aston, Tom (Oxfam, 2019)
Those working to advance social change in the Pacific must understand, work with and respond to the complex and changing relationships and dynamics of power that exist within such networks, and situate their work in the context of decolonisation and self-determination. But the approaches typically used by NGOs to plan our projects and programs have failed to give due attention to these dynamics. The result is that our projects fall short of achieving their promise, despite being technically sound and logical. In this guide, we aim to provide a process and tools that prioritise and draw out the rich, often implicit, knowledge that Pacific Islanders have about our contexts to plan and manage social change initiatives in a manner that values and responds to this ocean of relationships. Drawing from these relationship-based and voyaging traditions, we emphasise the need to: • frame change in terms of change in people; e.g. change in the situation of groups of people and change in terms of the actions of people; • identify a course towards a particular ‘change destination’; • identify the signs to read that will help you determine where you are on that course; • be constantly attentive to the context as you make the journey, and • be ready to adjust your course in response to changes in your context. To support this way of working, we have selected a number of tools from approaches that we believe fit. In particular, we have drawn from the following approaches: • Thinking and Working Politically and other ‘politically aware’ approaches that recognise that the success of a development initiative will be determined by the interplay of relationships of power within a particular context, not just by having the right technical solution; • Outcome mapping because of its focus on people and relationships, and on identifying and supporting needed changes in the behaviour of specific actors in relation to others. This is key, because it’s individuals who deliver services, individuals who decide whether to allocate budget and resources to address a problem, and individuals who make the choice to believe whether a service is relevant to solving their issues; • Adaptive management because of its recognition that development initiatives take place within complex and changing systems and need to adjust to these and that this requires ongoing learning and adaptation.

Monitoring and Adaptive Management - Ossi, Damien (District of Columbia Wildlife Action Plan - 2015, 2015)

VOICES - On speaking, mediation, representation and listening - Oswald, Katy (Think Piece, 2014)

Which Voices Are Heard and By Whom? - Oswald, Katy (Making All Voices Count, 2014)

Knowledge Management strategies and processes in Agile Software Development: a systematic literature review - Ouriques, Raquel Andrade Barros; Wnuk, Krzysztof; Gorschek, Tony; Svensson, Richard Berntsson (CoRR, 2018)
Knowledge-intensive companies that adopt Agile Software Development (ASD) relay on efficient implementation of Knowledge Management (KM) strategies to promotes different Knowledge Processes (KPs) to gain competitive advantage. This study aims to

Outcome Mapping FAQs - Outcome Mapping Learning Community (Outcome Mapping, 2018)
FAQ resource produced by IDRC's Evaluation Unit, with contributions from Kaia Ambrose, Sarah, Earl, Jan Van Ongevalle and Julius Nyangaga. We've added it here to answer some of your OM related questions.

Outcome Harvesting. Understanding social change results in complex circumstances - outcomeharvesting.net (2018)
This website is a source of applications, events and resources to support the development of a community of practitioners of Outcome Harvesting. BetterEvaluation is one of three sister sites of OutcomeHarvesting.net

‘We have the solutions to our own problems’: supporting civil society in Nigeria - Overseas Development Institute (ODI, 2014)
Development assistance programmes that start from careful analysis of what works and what does not in country contexts are still remarkably rare. One reason for this is the lack of well documented examples of ‘politically’ smart aid programming. To this end, this film outlines the experience of the State Accountability and Voice Initiative (SAVI), a DFID-funded programme in Nigeria providing support to civil society, media and governmental organisations.

Illegal Wildlife Trade - Frameworks - Oxford Martin Programme on the Illegal Wildlife Trade (Oxford Martin Programme on the Illegal Wildlife Trade, 2018)
Our frameworks will enable a better understanding of wildlife product consumer preferences and motivations to effectively influence the system.

Organizational Performance Index (OPI) Handbook: A practical guide to the OPI tool for practitioners and development professionals - Pact (Pact, 2015)
This handbook is intended to help development practitioners understand how to use and what can be achieved by Pact’s Organizational Performance Index (OPI) tool. This document will provide practitioners and managers with information on how the Pact OPI works and how best to incorporate it in their country strategy or program.

Guide to Evaluating Participatory Processes - Practical guide - Parés, Marc (2013)
Guide to Evaluating Participatory Processes - Practical guide
Keywords: Academia; Biology; Computer Science; Earth Sciences; Geography; History; Law; Math; Medicine; Philosophy; Physics; Political science; Psychology; Religion; Research; economics; universities

The Politics of Evidence: From Evidence-based Policy to the Good Governance of Evidence - Parkhurst, Justin (Taylor & Francis, 2017)
The Open Access version of this book, available at http://www.tandfebooks.com/, has been made available under a Creative Commons Attribution-Non Commercial-No Derivatives 3.0 license. There has been an enormous increase in interest in the use of evidence for public policymaking, but the vast majority of work on the subject has failed to engage with the political nature of decision making and how this influences the ways in which evidence will be used (or misused) within political areas. This book provides new insights into the nature of political bias with regards to evidence and critically considers what an ‘improved’ use of evidence would look like from a policymaking perspective. Part I describes the great potential for evidence to help achieve social goals, as well as the challenges raised by the political nature of policymaking. It explores the concern of evidence advocates that political interests drive the misuse or manipulation of evidence, as well as counter-concerns of critical policy scholars about how appeals to ‘evidence-based policy’ can depoliticise political debates. Both concerns reflect forms of bias – the first representing technical bias, whereby evidence use violates principles of scientific best practice, and the second representing issue bias in how appeals to evidence can shift political debates to particular questions or marginalise policy-relevant social concerns. Part II then draws on the fields of policy studies and cognitive psychology to understand the origins and mechanisms of both forms of bias in relation to political interests and values. It illustrates how such biases are not only common, but can be much more predictable once we recognise their origins and manifestations in policy arenas. Finally, Part III discusses ways to move forward for those seeking to improve the use of evidence in public policymaking. It explores what constitutes ‘good evidence for policy’, as well as the ‘good use of evidence’ within policy processes, and considers how to build evidence-advisory institutions that embed key principles of both scientific good practice and democratic representation. Taken as a whole, the approach promoted is termed the ‘good governance of evidence’ – a concept that represents the use of rigorous, systematic and technically valid pieces of evidence within decision-making processes that are representative of, and accountable to, populations served.
Keywords: Public policy; Social Policy; Social Services & Welfare

The politics of evidence: from evidence-based policy to the good governance of evidence - Parkhurst, Justin O. (Routledge studies in governance and public policy, 2017)
Introduction -- Evidence-based policymaking : an important first step, and the need to take the next -- Bias and the politics of evidence -- The overt politics of evidence : bias and the pursuit of political interests -- The subtle politics of evidence - the cognitive-political origins of bias -- What is "good evidence for policy"? From hierarchies to appropriate evidence -- What is the "good use of evidence" for policy? -- From evidence-based policy to the good governance of evidence

Are we suffering from obsessive measurement disorder? - Pasanen, Tiina (From Poverty to Power, 2019)
ODI’s Tiina Pasanen argues that more data doesn’t necessarily mean we make better decisions, and sets out some ideas for how we count what counts.

Five tips for evaluating your impact in international development - Pasanen, Tiina (The Guardian, 2015)
How can impact evaluations actually be helpful? These top tips will save development professionals time, energy and money. International development can be messy – with uncertain, complex settings, and multiple partners with different interests, goals and capacities. At the same time, we are under increasing pressure to demonstrate impact. We have to show that our projects have made a real change in people’s lives and that donor or taxpayer money hasn’t been wasted. But impact evaluations, which are often seen as a solution to this, aren’t always used. And in some cases they are unhelpful. So how can we improve the quality of impact evaluations, so that they produce results that are useful? This is where evaluability assessments come in. By asking whether we should evaluate a project – and if so, when and how – these assessments can improve the quality of impact evaluations. Here are five other reasons to assess evaluability before starting an impact evaluation.

Monitoring and evaluation: five reality checks for adaptive management - Pasanen, Tiina (ODI Insight, 2017)

Monitoring And Evaluation Tools and Approaches to Support Adaptive Management - Pasanen, Tiina (GLAM, 2019)

Supporting adaptive management. Monitoring and evaluation tools and approaches - Pasanen, Tiina; Barnett, Inka (ODI, 2019)
Key messages • This working paper introduces a set of monitoring and evaluation (M&E) tools and approaches, discussing their potential usefulness in supporting adaptive management in development and humanitarian programmes. It emphasises adaptive programmes characterised by complex aspects, such as: (1) they are innovative; (2) they have uncertain pathways for change; or (3) they operate in uncertain or unstable environments. • The majority of these tools have been used in international development for many years. However, adaptive management brings additional challenges for monitoring and evaluating programmes, as they require intentional M&E design from the start that is oriented towards both learning and accountability. • All of the tools and approaches introduced in this paper have potential to support learning and adaptation, although in various ways and at different stages of a programme. Some tools can support strategic planning and diagnosing throughout a programme – especially during design and inception – while others can help analyse causal relationships at specific points in a programme. It is important to tailor the approach used for its intended purpose. However, whether learning and adaptation happens depends also on factors other than the choice of M&E methods. • For some of these approaches a considerable body of evidence already exists but, for many, more practical examples and systematic analysis is needed. In addition to building the evidence base concerned with which approaches are suitable for different types of adaptive programmes and why, it is also important to improve understanding of the enabling environmental conditions necessary for the tools and approaches outlined here to facilitate and strengthen evaluative thinking, evidence-informed decision-making and ongoing programme iteration.

How to design a monitoring and evaluation framework for a policy research project - Pasanen, Tiina; Shaxson, Louise (Methods Lab, 2016)
Policy research projects face a number of challenges: policy processes are complex, involve multiple actors and often feature a significant time-lag between research and what may or may not happen as a result of it. To complicate matters further, the scope and scale of policy research projects are increasingly moving away from single research studies towards multi-component, multi-site and multi-sector endeavours. These factors mean that developing an overarching monitoring and evaluation framework can be challenging. But it is not impossible. The guidance note aims to support the first steps in designing and structuring the M&E framework (that is, what aspects or areas of policy research projects to monitor and evaluate, why, when and how). It builds on an M&E framework for policy research projects developed and tested by the Research and Policy in Development (RAPID) programme of the Overseas Development Institute, and draws on case studies of how it has been used by RAPID and other research teams.

The Power of Positive Deviance - Pascale, Richard; Sternin, Jerry; Sternin, Monique (Harvard Business Press, 2010)

The Rapid Evaluation, Action, and Learning (REAL) Approach: A toolkit to measure and refine changes and interventions in health campaigns - PATH (PATH, 2019)
Evaluative Tools for Improving Campaign Effectiveness Public health campaigns aim to control disease or deliver essential health services and products through time-limited and periodic channels. Many countries use campaigns to augment or replace routine service delivery, to target certain populations, or to accelerate progress towards coverage targets. The Root Cause Analysis and Rapid Evaluation, Action, and Learning toolkits present a systematic but flexible approach to identifying the root causes of campaign bottlenecks and then designing, testing, and refining solutions to optimize potential impact. These toolkits respond to a growing need for “fit-for-purpose” rapid-testing, adaptive learning approaches to evaluation and the need for a culture shift toward iterative adaptation and improvement that integrates measurement and evidence-informed decision-making into daily practice. The comprehensive package of toolkits, in addition to the individual Root Cause Analysis toolkit and Rapid Evaluation, Action, and Learning toolkit are available below.

Complexity-Aware Monitoring Discussion Note (Screencast) - Patsalides, Melissa; Britt, Heather (USAID, 2014)
This screencast covers information included in the Complexity-Aware Monitoring Discussion Note. The topic is intended for those seeking cutting-edge solutions to monitoring complex aspects of strategies and projects. Most of the principles and promising approaches discussed here have a significant body of theory and practice behind them, but many have not been used to monitor USAID strategies and projects. We have much to learn about whether and how they can be applied successfully in the Agency. USAID's Office of Learning, Evaluation, and Research in the Bureau of Policy, Planning, and Learning has partnered with different groups through monitoring trials to build a body of evidence regarding which complexity-aware M&E approaches are most effective in the contexts of USAID programming.

Developmental Evaluation: Applying Complexity Concepts to Enhance Innovation and Use - Patton, Michael Quinn (Guilford Press, 2010)
Developmental evaluation (DE) offers a powerful approach to monitoring and supporting social innovations by working in partnership with program decision makers. In this book, eminent authority Michael Quinn Patton shows how to conduct evaluations within a DE framework. Patton draws on insights about complex dynamic systems, uncertainty, nonlinearity, and emergence. He illustrates how DE can be used for a range of purposes: ongoing program development, adapting effective principles of practice to local contexts, generating innovations and taking them to scale, and facilitating rapid response in crisis situations. Students and practicing evaluators will appreciate the book's extensive case examples and stories, cartoons, clear writing style, "closer look" sidebars, and summary tables. Provided is essential guidance for making evaluations useful, practical, and credible in support of social change.See also Developmental Evaluation Exemplars, edited by Michael Quinn Patton, Kate McKegg, and Nan Wehipeihana, which presents 12 in-depth case studies.
Keywords: Evaluation; Measurement; Organizational effectiveness; Project management

Essentials of Utilization-Focused Evaluation - Patton, Michael Quinn (SAGE Publications, 2011)
Based on Michael Quinn Patton's best-selling Utilization-Focused Evaluation, this briefer book provides an overall framework and essential checklist steps for designing and conducting evaluations that actually get used. The new material and innovative graphics present the utilization-focused evaluation process as a complex adaptive system, incorporating current understandings about systems thinking and complexity concepts. The book integrates theory and practice, is based on both research and professional experience, and offers new case examples and cartoons with Patton's signature humor.
Keywords: Reference / Research; Research

Principles-Focused Evaluation: The GUIDE - Patton, Michael Quinn (Guilford Press, 2017)
How can programs and organizations ensure they are adhering to core principles--and assess whether doing so is yielding desired results? From evaluation pioneer Michael Quinn Patton, this book introduces the principles-focused evaluation (P-FE) approach and demonstrates its relevance and application in a range of settings. Patton explains why principles matter for program development and evaluation and how they can serve as a rudder to navigate the uncertainties, turbulence, and emergent challenges of complex dynamic environments. In-depth exemplars illustrate how the unique GUIDE framework is used to determine whether principles provide meaningful guidance (G) and are useful (U), inspiring (I), developmentally adaptable (D), and evaluable (E). User-friendly features include rubrics, a P-FE checklist, firsthand reflections and examples from experienced P-FE practitioners, sidebars and summary tables, and end-of-chapter application exercises.

Utilization-Focused Evaluation (U-FE) Checklist - Patton, Michael Quinn (Evaluation Checklists Projects, 2013)
Utilization-Focused Evaluation begins with the premise that evaluations should be judged by their utility and actual use; therefore, evaluators should facilitate the evaluation process and design any evaluation with careful consideration of how everything that is done, from beginning to end, will affect use. Use concerns how real people in the real world apply evaluation findings and experience and learn from the evaluation process. The checklist is based on Essentials of Utilization-Focused Evaluation (Patton, 2012, Sage Publications). All references in the checklist to exhibits and menus refer to this book. Step 1 Assess and build program and organizational readiness for utilization-focused evaluation. Step 2 Assess and enhance evaluator readiness and competence to undertake a utilizationfocused evaluation. Step 3 Identify, organize, and engage primary intended users. Step 4 Conduct situation analysis with primary intended users Step 5 Identify primary intended uses by establishing the evaluation’s priority purposes. Step 6 Consider and build in process uses if appropriate. Step 7 Focus priority evaluation questions. Step 8 Check that fundamental areas for evaluation inquiry are being adequately addressed. Step 9 Determine what intervention model or theory of change is being evaluated. Step 10 Negotiate appropriate methods to generate credible findings and support intended use by intended users. Step 11 Make sure intended users understand potential controversies about methods and their implications. Step 12 Simulate use of findings. Step 13 Gather data with ongoing attention to use. Step 14 Organize and present the data for use by primary intended users. Step 15 Prepare an evaluation report to facilitate use and disseminate significant findings to expand influence. Step 16 Follow up with primary intended users to facilitate and enhance use. Step 17 Metaevaluation of use: Be accountable, learn, and improve

Measuring Impact. Making Use of the Portfolio: Organizational Learning at USAID - Peabody, Shawn (Measuring Impact, 2015)
This technical analysis explores previous and ongoing social learning efforts, best practices, challenges, and lessons in USAID as a foundation for improving the implementation and design of the Agency’s forestry and biodiversity programs. This analysis is particularly relevant as the Bureau of Economic Growth, Education and the Environment’s Office of Forestry and Biodiversity (E3/FAB) begins to develop a Cross-Mission Learning Program under the Measuring Impact initiative, a five-year activity to promote the adoption of best practices in the USAID program cycle among Missions using biodiversity funds. The findings reported in this document can help inform the overall design and structure of the Learning Program and provide insight into possible challenges and best practices.

Evaluability assessment for Impact Evaluation: Guidance, checklists and decision support - Peersman, Greet; Guijt, Irene; Pasanen, Tiina (ODI, 2015)
This guidance note focuses on the utility of, and guidance for, evaluability assessment before undertaking an impact evaluation. The primary audience for this guidance note is evaluators conducting an evaluability assessment for impact evaluation. The secondary audience is people commissioning or managing an evaluability assessment for impact evaluation, as well as funders of an impact evaluation. Sections one and two provide an overview of evaluability assessment and how it can be used for impact evaluation. Section three provides guidance for planning to undertake an evaluability assessment for impact evaluation. This is informative for all intended users of the guidance note. Section four includes checklists and decision support for evaluability assessments. The checklist is geared to those conducting the evaluability assessment and can be adapted to suit a particular context or purpose. The decision support provides those conducting an evaluability assessment with evidence-based recommendations for impact evaluation funders and commissioners, about whether, when and how to proceed with the evaluation. Sections five and six provide guidance on what to do after the assessment is concluded, and offer lessons learned from evaluability assessments in practice.

When and how to develop an impact-oriented monitoring and evaluation system - Peersman, Greet; Rogers, Patricia; Guijt, Irene; Hearn, Simon; Pasanen, Tiina; Buffardi, Anne (2016)
This guidance note focuses on: • what an impact-oriented monitoring and evaluation system entails • why an organisation may want to establish such a system • when integrating an impact-orientation into an monitoring and evaluation system is most useful • what should be considered in developing the monitoring and evaluation system, or in tweaking an existing system, to become more impact-focused. The primary audience for this guidance note is internal and external monitoring and evaluation advisors involved in designing and implementing, and/or assessing monitoring and evaluation systems to include a focus on impact. It will also be useful for senior management of organisations who need to know how best to plan for a sustainable monitoring and evaluation system that supports impact assessment or to adapt an existing system to incorporate an impact perspective.

Crowdsourcing citizen feedback on district development in Ghana using interactive voice response surveys - Pegus, Ciana-Marie (Making All Voices Count Practice Paper, 2016)
Keywords: IMPORTANT

When Does ICT-Enabled Citizen Voice Lead to Government Responsiveness? - Peixoto, Tiago; Fox, Jonathan (IDS Bulletin, 2016)

Citizen engagement: emerging digital technologies create new risks and value - Peixoto, Tiago; Steinberg, Tom (World Bank, 2019)
The recent rapid evolution of digital technologies has been changing behaviors and expectations in countries around the world. These shifts make it the right time to pose the key question this paper explores: Will digital technologies, both those that are already widespread and those that are still emerging, have substantial impacts on the way citizens engage and the ways through which power is sought, used, or contested? The authors address this question both to mitigate some of the World Bank’s operational risks, and to initiate a conversation with peers about how those risks might require policy shifts. The overall framing question also is being explored in case theapproaches to citizen engagement advocated by the World Bank are changing and may require different advice for client countries. Despite the lower technology penetration levels in developing countries, their more malleable governance contexts may be more influenced by the effects of emerging technologies than older states with greater rigidity. Digitally influencedcitizen engagement is, in short, one of those “leapfrog” areas in which developing nations may exploit technologies before the wealthier parts of the world. But countries can leapfrog to worse futures, not just better ones. This paper explores what technology might mean for engagement, makespredictions, and offers measures for governments to consider.

MLE or MEL in adaptive programming? - Pellini, Arnaldo (Knowledge counts, 2018)

Working Politically: A story of Change about the contribution of research evidence to the new Village Law in Indonesia - Pellini, Arnaldo; Angelina, Maesy; Purnawati, Endah (Austrialian Community Development and Civil Society Strenghtening Scheme (ACCESS), 2014)
On 18 December 2013, the Indonesian House of Representatives passed the new Village Law, a vote that was the culmination of a journey that had started in 2007. This Story of Change takes the passing of the Village Law as its starting point and describes the relative influence that research-based evidence, produced by the Institute for Research and Empowerment (IRE), has had at critical junctions of the legislative process. This Story of Change concludes that good quality, research-based evidence is necessary but not sufficient to influence policy-making processes. Researchers and research organisations need to think and work politically to achieve their influencing goals and to adapt to changes in local circumstances.

Doing Development Differently at Scale - Pellini, Arnaldo; Karetji, Petrarca C.; Soekadis, Ade (Knowledge, Politics and Policymaking in Indonesia, 2018)
In recent decades there has been an increasing recognition that politics and political institutions matter for development. There is also a much greater interest in contextually grounded approaches. This has stemmed from an acknowledgement that purely technocratic approaches to development often result in failure because they do not take into account the nature of political institutions. Nor do they take account of the context in a particular developing country and the interests and incentives of powerful national actors. Policy processes are embedded in specific social, political and organisational contexts. Approaches that focus on implementing universal best practices in evidence-informed policymaking are unlikely to be successful. Instead, what is needed is an approach that takes the local context as the starting point for understanding what issues are relevant to policymakers and developing contextually appropriate solutions. The authors of this chapter reflect on the management approaches and systems that may be required to enable and support large-scale development programmes to be flexible and adaptive to the local policy context and circumstances. The chapter argues that such programmes struggle to adopt adaptive management principle and that to do that at scale requires some changes in the way such programmes are managed.
Keywords: Adaptive Management; Doing development differently; Evidence-informed policymaking; Indonesia; Knowledge sector; Thinking and working politically

On understanding software agility: A social complexity point of view - Pelrine, Joseph (E:CO, 2011)
Over the last decade, the field of so-called Agile software development has grown to be a major force in the socio-economic arena of delivering quality software on time, on budget, and on spec. The acceleration in changing needs brought on by the rise in popularity of the Internet has helped push Agile practices far beyond their original boundaries, and possibly into domains where their application is not the optimal solution to the problems at hand. The question of where Agile software development practices and techniques make sense, and where are they out of place, is a valid one. It can be addressed by looking at software development as a complex endeavour, and using tools and techniques from the Cynefin method and other models of social complexity.
Keywords: IMPORTANT

Citizen participation and the rise of the open source city in Spain - Peña-López, Ismael (MAVC & IT for Change, 2017)
Research on the Information Society, the Digital Divide and Information and Communication Technologies for development

ICT4D Blog » A citizen participation ecosystem - Peña-López, Ismael (ICTlogy, 2019)
Research on the Information Society, the Digital Divide and Information and Communication Technologies for development

Grow - Scale - Impact: How to help inclusive business achieve scale - Pérez Castro, Andrea A.; Tora, Krisztina (GIZ, 2016)

Power Analysis: A Practical Guide - Pettit, Jethro (Sida, 2013)

Power Above and Below the Waterline: Bridging Political Economy and Power Analysis - Pettit, Jethro; Mejía Acosta, Andrés (IDS Bulletin, 2014)

Barriers and enablers to climate change adaptation in hierarchical governance systems: the case of Vietnam - Phuong, Le Thi Hong; Biesbroek, G. Robbert; Wals, Arjen E. J. (Journal of Environmental Policy & Planning, 2018)
Governments fulfil important roles in increasing the adaptive capacity of local communities to respond to climate change impacts, particularly in developing countries. Existing studies on how governments enable and constrain the ways in which local level communities learn and build their adaptive capacity, however, generally adopt network or market-oriented types of governance. However, the most vulnerable regions to climate change impact in the world are generally governed through hierarchical policy systems. This research aims to understand how the hierarchical policy system in Vietnam creates enables and/or constrains the policy capacity of policy actors to contribute to effective climate change adaptation. We conducted interviews (n = 26) with key actors at multiple levels of government. Our findings show the importance of clear legal institutions, available financing for implementing policies, and the training of governmental staff, particularly at district and commune levels where the policy capacities are generally too low to deal with climate change impacts. We conclude that any efforts to support local actors (i.e. smallholder farmers) should include investments in policy capacity to ensure uptake and upscaling of adaptation actions more broadly.

Using a social learning configuration to increase Vietnamese smallholder farmers’ adaptive capacity to respond to climate change - Phuong, Le Thi Hong; Wals, Arjen; Sen, Le Thi Hoa; Hoa, Nguyen Quoc; Lu, Phan Van; Biesbroek, Robbert (Local Environment, 2018)
Social learning is crucial for local smallholder farmers in developing countries to improve their adaptive capacity and to adapt to the current and projected impacts of climate change. While it is widely acknowledged that social learning is a necessary condition for adaptation, few studies have systematically investigated under which conditions particular forms of social learning are most successful in improving adaptive capacity of the most vulnerable groups. This study aims to design, implement and evaluate a social learning configuration in a coastal community in Vietnam. We make use of various methods during four workshop-based interventions with local smallholder farmers: interviews with key farmers and commune leaders, farmer-to-farmer learning, participatory observations and focus group discussions. The methods for evaluation of social learning configuration include in-depth interviews, focus group discussions and structured survey interviews. Our findings show that the social learning configuration used in this study leads to an increased problem ownership, an enhanced knowledge-base with regard to climate change impacts and production adaptation options, improved ability to see connections and interdependencies and finally, strengthened relationships and social cohesion. The results suggest that increased social learning in the community leads to increase in adaptive capacity of smallholder farmers and improves both their economic and environmental sustainability. We discuss the key lessons for designing learning configurations that can successfully enhance adaptive capacity and smallholder farmers’ agency and responsiveness to the challenges posed by climate change impacts.
Keywords: Adaptive capacity; Social learning; Vietnamese smallholder farmers

The interplay between social learning and adaptive capacity in climate change adaptation: A systematic review - Phuong, Thi Hong Le; Biesbroek, G. Robbert; Wals, Arjen E.J. (NJAS - Wageningen Journal of Life Sciences, 2017)
Successful implementation climate change adaptation depends to a large extent on the capabilities of individuals, organizations, and communities to create and mobilize the adaptive capacity (AC) of their socio-ecological system. Creating and mobilizing AC is a continuous process that requires social learning (SL). Although rich with empirical cases, the literature theorizing and empirically investigating the relationship between AC and SL is highly fragmented. This paper aims to critically examine the peer-reviewed literature that focusses on SL and AC in the context of climate change adaptation (CCA). Special attention is paid to the interplay between the two. Understanding this interplay can help improve our understanding of how CCA takes place in practice and advances theoretical debates on CCA. Systematic review methods are used to analyse 43 papers (1997–2016). Our findings reveal three perspectives that each play an important role in different contexts: an AC-focused perspective, a SL-focused perspective, and a hybrid perspective. These differences in conceptualizations of the relationship between SL and AC may seem trivial at first, but they have consequences for the design of learning-based interventions aimed at helping communities respond to climate change. It appears that such interventions need to be preceded by an analysis of the climate change context in order to decide whether to emphasize AC, SL or both simultaneously.
Keywords: Adaptive capacity; Climate change adaptation; Social learning; Systematic review

What is democratic evaluation? - Picciotto, R. (Evaluation Connections, 2013)
This article from Robert Picciotto provides an overview of democratic evaluation, particularly with reference to its use in the European Union context. "Can evaluation strengthen the democratic process? Speciﬁcally can it help ﬁll the democratic deﬁcit (limited transparency; weak bureaucratic accountability, etc.) often ascribed to the European Union? First and foremost, evaluators operating in the European space should be committed to the ethical and democratic values of the European project and the promotion of its social inclusion and cohesion ideals. But two other sets of challenges must also be met: those that relate to evaluation in democracy and those that relate to democracy in evaluation. The former has to do with the macro positioning of evaluation processes in society. The latter has to do with the evaluation approaches and methods used in deliberative democratic decision making processes." (Picciotto, 2013)

Adaptive Management: Responding to the evolving needs of PRIME's complex systems - Picon, Diana; Wild, Lorenz (Mercy Corps, 2015)

Review of Selected DFAT Facilities: Independent Consultant Report to DFAT, Key Findings - Pieper, Lynn (DFAT, 2018)
For the purposes of this report, a facility is defined as an aid delivery mechanism that provides flexible (adaptive and responsive) services managed in an integrated way. Objectives (or endof-facility outcomes) are specified, but the pathways to deliver them are left unspecified. The facility is a highly relevant model for delivering Australian aid effectively. Achievement of the Sustainable Development Goals depends on flexible approaches that work across sectors and that integrate significant cross-cutting themes such as gender equality and social inclusion. Efficient DFAT management of a $4 billion aid program requires a shift to fewer and larger initiatives, to make best use of limited internal management resources while still enabling policy and program choices that can have real impact. Facilities and other flexible delivery mechanisms offer significant potential for better development results. They can: enable collaborative and responsive partnership approaches to gain traction; allow activities to experiment and adapt based on progress, demand and contextual changes; and provide the opportunity for outcome-focused coherence across sectors, enabling the whole to be more than the sum of the parts. Facilities are not new, but they are different today. Facilities have existed since at least the mid-1990s. What is new is that some facilities today are very large; and the flexibility they enable is increasingly and consciously being used to strengthen links between the technical and the political, for more effective development results. This is a positive trend, but it adds enormously to their complexity and visibility, creates new risks, and has resulted in role confusion (especially in early years of implementation) between DFAT and its contractors.

Decide Madrid: a case study on e-Participation - Pina, Vicente; Torres, Lourdes; Royo, Sonia; García-Rayado, Jaime (XX International AECA Congress, 2019)
This paper analyses the award-winning e-participation initiative of the city council of Madrid, Decide Madrid, to discover the critical success factors at contextual, organizational and individual level. This analysis is carried out with desk research and semi-structured interviews. Results show that the most relevant success factors are the socio-economic context, the commitment of the city council, the method used to recruit the workers and the knowledge of senior managers about citizen participation and ICTs. However, the lack of transparency and poor functioning of some of its participation activities are negatively affecting its performance.

Local First in practice - Pinnington, R (Peace Direct, 2014)

Is DFID Getting Real About Politics?A stocktake of how DFID has adopted a politically-informed approach (2010-2015) - Piron, Laure-Hélène; Baker, Aislin; Savage, Laura; Wiseman, Katie (DFID, 2016)
1. Background This internal stocktake assesses whether DFID is “getting real about politics” - how it is taking power and politics into account in all its operations. Country Poverty Reduction Diagnostics undertaken by DFID teams identify politics as the most frequent barrier to poverty reduction and growth. The UK 2015 Aid Strategy has committed DFID to spending 50% of Official Development Assistance (ODA) in fragile states. This requires a “patient, long-term approach” to addressing barriers to peace and stability which are fundamentally political, rather than purely financial or technical. The stocktake is based on three DFID offices case studies (Democratic Republic of the Congo, Pakistan and Malawi) as well as extensive internal and external consultations between June and December 2015. It provides illustrations of how DFID is evolving but does not systematically offer evidence of development impacts or non-country work, as this would have required a different methodology. 2. What does it mean to take politics into account? Politically-informed approaches are based on a large body of evidence that confirms the importance of institutions and politics for sustainable development. External assistance needs to support locally-led change. Success depends on timing, context, political processes and local actors. Desirable outcomes are hard to achieve and difficult to predict. Politically-informed approaches improve development effectiveness through:  The ‘what’: political goals, using development assistance to shift how power is distributed in the economy and society. The two main elements are: aiming for long term transformation of institutions; and supporting locally-led change processes more likely to be sustainable and successful: locallyowned (i.e. with local salience) and locally-negotiated.  The ‘how’: politically-smart methods, with greater realism and feasibility. The three main elements are: understanding power and politics in a specific context in order to identify opportunities and barriers for change; influencing and stakeholder management skills; and proactive risk management. To influence DFID operations, a politically-informed approach needs to be iterative, not one-off. The explicit understanding of context, whether formal setpiece studies or more routine analysis, should inform policy and programme decisions, from high level strategic choices, to day-to-day implementation, for both international policy and country support. This is a dynamic process: as the context evolves and lessons are learned about what works, analyses and decisions are updated. These are the principles behind the ‘flexible and adaptive’ agenda.

What Startups Can Learn from General McChrystal about Combining Strategy and Execution - Pisoni, Adam (First Round, 2015)
Yammer Co-founder Adam Pisoni speaks with General Stanley McChrystal about the lessons technology companies might take away from his new book Team of Teams.

Design thinking: understand - improve - apply - Plattner, Hasso; Meinel, Christoph; Leifer, Larry (Understanding innovation, 2011)
"In this book, the researchers take a system's view that begins with a demand for deep, evidence-based understanding of design thinking phenomena. They continue with an exploration of tools which can help improve the adaptive expertise needed for design thinking. The final part of the book concerns design thinking in information technology and its relevance for business process modeling and agile software development, i.e. real world creation and deployment of products, services, and enterprise systems"--Cover
Keywords: Creative ability in business; Creative thinking; Designer; Industrial management; Innovation; Organizational change; Product development; System design; Technological innovations

Seeing (from) Digital Peripheries: Technology and Transparency in Kenya’s Silicon Savannah - Poggiali, Lisa (Cultural Anthropology, 2016)

Illogical Framework: The Importance of Monitoring and Evaluation in International Development Studies - Pomerantz, Jessica R. (Cornell Policy Review, 2011)
In my brief experience with monitoring and evaluation, I have become convinced that it is critically important both as an international development project component and as a field of academic study. Throughout my academic career at Cornell University, I have, at times, argued that monitoring and evaluation has actually impeded development efforts, but here I wish to amend my opinion. Bad monitoring and evaluation can sabotage development projects and our meaningful interpretation of development impacts; failures can appear to be successes and vice-versa. As a student and practitioner of monitoring and evaluation, I have drawn the conclusions listed below and I submit them for your consideration. • Monitoring and evaluation is a key element of the international development industry applicable to many areas of public administration, domestic and international. • International development failures could be discovered and averted or corrected given proper monitoring and evaluation activities. • Anecdotal evidence from development activities in Afghanistan provides one example of the international community’s lack of attention to monitoring and evaluation concerning an ongoing development catastrophe. • Higher education ought to be filling the monitoring and evaluation knowledge gap but to date is failing to do so.

Winners and Losers in the Global App Economy - Pon, Bryan (2016)

Positive Deviance Initiative - Positive Deviance Initiative (2018)

Critical agency and development: applying Freire and Sen to ICT4D in Zambia and Brazil - Poveda, Sammia; Roberts, Tony (Information Technology for Development, 2017)
(2017). Critical agency and development: applying Freire and Sen to ICT4D in Zambia and Brazil. Information Technology for Development. Ahead of Print. doi: 10.1080/02681102.2017.1328656

Poverty Stoplight - povertystoplight.org (2018)
What is the Poverty Stoplight? The Stoplight is a tool that seeks to activate the potential of families and communities to lift themselves out of poverty. Using a technology platform, it offers a self-assessment survey and intervention model that enables people to develop practical solutions to overcome their specific needs. We work with organizations from different sectors around the world and bring together a network of powerful actors with one clear mission: eliminate poverty worldwide. The Stoplight Community is comprised of government agencies, private companies, small and medium-sized enterprises, non-profits, cooperatives, microfinance organizations, academic institutions, and sports clubs, among others.

Decision-Making and Data Use Landscaping [at DFID] - Powell, Josh; Orton-Vipond, Sarah; Bhatia, Vinisha; Kilroy, Annie (Development Gateway, 2018)
As DFID aims to harness the Data Revolution, ensuring that data1 drive decision-making, public accountability, and the achievement of the Sustainable Development Goals (SDGs), ensuring that systems, processes, and skills for data are aligned with these objectives is paramount. Across sector policy teams, country offices, and various analytical and technical cadres, different strengths and weaknesses, as well as needs and ambitions exist. To inform a strategic approach to data, as framed in its forthcoming Data Roadmap, DFID collaborated with Development Gateway to perform a Decision and Data Use Landscaping study. This report details lessons learned from approximately 60 interviews across 4 DFID country offices, all sector policy teams, senior managers, and various analytical and technical cadres and offices.

All about behaviour: KAPE, Adaptation and ' Sticky' Institutional Change - Power, Greg (Politically Agile Programming, 2016)
This paper describes Global Partners Governance’s (GPG’s) approach to institutional reform and political change. Developed over the last decade of working in some of the most complex and sensitive political environments with politicians and officials in parliaments, political parties, ministries and local government, it describes the KAPE® (knowledge-application-practice-effect) methodology that we adopt to get ‘sticky’ institutional and behavioural change. Contents 1) Two dimensions of ‘adaptive programmes’: Flexible delivery and getting behaviour change 2) Enabling Change: KAPE and The Logic of Institutional Reform (Knowledge-Application-Practice-Effect) - K: Knowledge – Defining the problem and what to do about it - A: Application – Making Systems Work in Practice - P: Practice – Pockets of good practice and establishing ‘the new normal’ - E: Effect – Improved performance and the ‘Ripple Effect’ 3) Measuring Impact: Monitoring and Evolving 4) Conclusion: Behavioural insights, adaptive management and sticky change

Identifying general trends and patterns in complex systems research: An overview of theoretical and practical implications - Preiser, Rika (Systems Research and Behavioral Science, 2019)
Research on complex systems is becoming more prolific, and there is a need to provide some point of orientation to researchers and practitioners that are drawing on the body of literature that informs the field of complexity research. In this paper, I aim to give an overview of the development of the field and offer some overarching trends and patterns that are recognizable in research on complex systems. The paper then draws on the work to provide six organizing principles of complex systems to inform practical implications and methods for studying and understanding complex systems.
Keywords: complex systems; heuristics; trends and patterns

Combating Wildlife Trafficking Cross-Mission Learning Agenda - Present, Teresa; Peabody, Shawn (USAID, 2017)

Evaluating Complexity. Propositions for improving practice - Preskill, Hallie; Gopal, Srik (FSG, 2014)
9 propositions can help evaluators measure progress on complex social problems.
Keywords: IMPORTANT

Redefining Rigor: Describing Quality Evaluation in Complex, Adaptive Settings - Preskill, Hallie; Lynn, Jewlya (FSG, 2016)
Traditionally, evaluation has focused on understanding whether a program is making progress against pre-determined indicators. In this context, the quality of the evaluation is often measured in part by the “rigor” of the methods and scientific inquiry.

Putting vulnerability to climate change on the map: a review of approaches, benefits, and risks - Preston, Benjamin L.; Yuen, Emma J.; Westaway, Richard M. (Sustainability Science; Dordrecht, 2011)
There is growing demand among stakeholders across public and private institutions for spatially-explicit information regarding vulnerability to climate change at the local scale. However, the challenges associated with mapping the geography of climate change vulnerability are non-trivial, both conceptually and technically, suggesting the need for more critical evaluation of this practice. Here, we review climate change vulnerability mapping in the context of four key questions that are fundamental to assessment design. First, what are the goals of the assessment? A review of published assessments yields a range of objective statements that emphasize problem orientation or decision-making about adaptation actions. Second, how is the assessment of vulnerability framed? Assessments vary with respect to what values are assessed (vulnerability of what) and the underlying determinants of vulnerability that are considered (vulnerability to what). The selected frame ultimately influences perceptions of the primary driving forces of vulnerability as well as preferences regarding management alternatives. Third, what are the technical methods by which an assessment is conducted? The integration of vulnerability determinants into a common map remains an emergent and subjective practice associated with a number of methodological challenges. Fourth, who participates in the assessment and how will it be used to facilitate change? Assessments are often conducted under the auspices of benefiting stakeholders, yet many lack direct engagement with stakeholders. Each of these questions is reviewed in turn by drawing on an illustrative set of 45 vulnerability mapping studies appearing in the literature. A number of pathways for placing vulnerability mapping on a more robust footing are also identified. [PUBLICATION ABSTRACT]
Keywords: Climate change; Mapping; Risk assessment; Sustainable development

How Does Developing Enforcement Capacity Reduce Wildlife Crime? Increasing program effectiveness by understanding common assumptions - Price, Claire; Buff, Jesse; Peabody, Shawn; Present, Tess; Lauck, Liz (USAID, 2018)
"How Does Developing Enforcement Capacity Reduce Wildlife Crime?" summarizes findings from the literature around four key assumptions about capacity building for enforcement and prosecution. Using the experiences and evidence summarized in this brief, program designers and implementing partners should be able to increase the effectiveness and efficiency of their capacity building efforts.

Adaptiveness through ‘simplest tools and practices’ - Prieto Martin, Pedro (Institute of Development Studies, 2017)
In recent years, more and more influential development organisations have been openly recognising the central role that adaptive management capacities – the ability to keep improving strategies and actions as programmes unfold – play for the success of complex interventions. As a result, there has been a rich exchange of ideas and experiences on how to promote adaptiveness in development. But development organisations and professionals are having a hard time translating the many adaptive theories and recommendations into improved practices and outcomes. They are generally failing to adjust the design and operation of programmes to accommodate incipient lessons learnt and the unexpected changes in the context. Nowadays, most programme adaptations still take the form of a change of direction once it is clear that things have gone really wrong. Why is so difficult to put what we know about adaptiveness into practice? And more importantly: what can be done to improve this situation?

Participation Schemas: a tool to characterize collaborative participation - Prieto-Martín, Pedro (PDD2014, Contemporary difficulties and future prospects for Participatory and Deliberative Democracy, NewCastle, 2014)

READ ME - HOW TO USE THIS AM LIBRARY - Prieto Martin, Pedro (Institute of Development Studies, 2019)

Interview protocol: Most significant turning points - Prieto Martin, Pedro; Faith, Becky (Institute of Development Studies, 2017)
This interview protocol was used for a research project on adaptiveness in technology for governance initiatives in Kenya. For more information, please read the research report at: Prieto Martin, P.; Hernandez, K.; Faith, B. and Ramalingam, B. (2017) Doing Digital Development Differently: Lessons in adaptive management from technology for governance initiatives in Kenya, MAVC Research Report, Brighton: Institute of Development Studies, ids.ac.uk/project/making-all-voices-count
Keywords: IMPORTANT

Doing Digital Development Differently: Lessons in adaptive management from technology for governance initiatives in Kenya - Prieto Martin, Pedro; Hernandez, Kevin; Faith, Becky; Ramalingam, Ben (MAVC Research Report, 2017)
Development projects don’t always work as planned. This has long been acknowledged by those in the sector, and has led to several approaches that seek to solve complex development problems through enabling and encouraging greater adaptiveness and learning within projects (e.g. Doing Development Differently and Problem-Driven Iterative Adaptation). Digital development projects experience many of these issues. Using technology for transparency and accountability (Tech4T&A) projects in Kenya as case studies, our research analysed the many different theoretical approaches to learning and adaptation, and then tested how these play out in reality. Firstly, we conducted an extensive review of the literature on the spectrum of approaches to adaptive learning. The findings were used to develop a framework through which to analyse adaptiveness at the different layers of complexity in projects (e.g. software design and development, programme design and management). The second part of the research consisted of interviews and focus group discussions with participants in Tech4T&A projects in Kenya. Respondents helped us identify the main characteristics of adaptiveness in their projects (e.g. who needs to adapt, and how and when) and the challenges and issues that inhibit projects’ abilities to be adaptive. This process also revealed how accountability interplays with adaptiveness, and considered how better collaboration flows can enable adaptiveness. From our literature review and empirical study, we draw several conclusions for increasing adaptiveness in digital development projects. These include simplifying the many adaptive theories that abound, increasing responsiveness to project beneficiaries and stakeholders, and for stakeholders to keep on experimenting, networking and advocating.
Keywords: IMPORTANT

Caracterizando la participación ciudadana en el marco del Gobierno Abierto - Prieto-Martín, Pedro; Ramírez-Alujas, Álvaro V. (Revista del CLAD. Reforma y Democracia, 2014)
Este artículo profundiza conceptualmente en las dimensiones del Gobierno Abierto, centrándose en el análisis y caracterización de su dimensión participativa. Se propone, en primer lugar, una tipología de actos participativos que muestra el efecto potenciador que las TIC ejercen sobre las distintas formas de participación. En segundo lugar, se presentan los denominados “Esquemas de participación” (EdP), un instrumento conceptual que facilita el análisis y la representación estandarizada de las dimensiones más importantes de la participación. Los EdP proporcionan así, por un lado, un modelo de categorización que extiende nuestra comprensión crítica de la participación y, por otro lado, una herramienta potente y flexible para la comunicación, el diseño y la evaluación de diversos tipos de iniciativas participativas.

Between logframes and theory of change: reviewing debates and a practical experience - Prinsen, Gerard; Nijhof, Saskia (Development in Practice, 2015)
Theory of Change (ToC) is an emerging methodology in the practice of development programmes, often contrasted with the dominant logical framework. This article reviews current debates around ToC before identifying five aspects that are appreciated in practice. It appears that these aspects mostly cover areas where the logical framework is not – or is no longer – meeting the needs of practitioners. Subsequently, the article analyses experiences in ToC training for NGO staff and concludes that ToC can address shortcomings of the logical framework – if only by going back to some of the roots of the logical framework.

How did China create “Directed Improvisation”? - Pritchett, Lant (Building State Capability, 2017)
written by Lant Pritchett Yuen-Yuen Ang, a Professor of Political Science at University of Michigan came to speak at Harvard the other day and I was lucky enough to hear her presentation.  Her most…
Keywords: IMPORTANT

The Debate about RCTs in Development is over. We won. They lost. - Pritchett, Lant (DRi Seminar, 2018)
There has been a debate in development economics over the last 20 years as some claimed the use of RCTs as a tool for independent impact evaluation would significantly improve development practice and hence development. While right about the methodological claims about the superiority of randomization to produce cleaner estimates of the LATE (local average treatment effect) of projects and programs, this, in and of itself, does not change development practice. All of the five claims needed to sustain a positive model in which RCT/IIE has a major positive impact are demonstrably false. The proponents of RCTs have responded to losing the first round decisively by changing significantly both their claims and their practice.

It's All About MeE: Using Structured Experiential Learning (“e”) to Crawl the Design Space - Pritchett, Lant; Samji, Salimah; Hammer, Jeffrey (Working Paper 322, 2013)
This paper argues that within-project variations in design can serve as their own counterfactual, reducing the incremental cost of evaluation and increasing the direct usefulness of evaluation to implementing agencies. It suggests combining monitoring (‘M’), structured experiential learning (‘e’), and evaluation (‘E’) so as to facilitate innovation and organisational capability building while also providing accountability …

Procuring and Managing Adaptively: 5 Case Studies of Adaptive Mechanisms - Pryor, Tony (USAID Learning Lab, USAID, 2018)
Flexibility in program management is essential in all of the countries where USAID works. This is especially true in non-permissive environments (NPEs), where the ability to learn and adapt quickly to changing circumstances can help USAID staff members achieve their desired outcomes.

Adaptive Programming in Fragile, Conflict and Violence-Affected Settings, What Works and Under What Conditions?: The Case of PERL, Nigeria - Punton, Melanie; Burge, Richard (Action for Empowerment and Accountability Research Programme, 2018)
This paper examines adaptive approaches to aid programming in Nigeria. Through field research and desk reviews, we have investigated some of the assertions around the ‘adaptive management and programming’ approach, which has arisen in recent years as a response to critiques of overly rigid, pre-designed, blueprint and linear project plans. This is the second of three case studies in a series which explore if and how adaptive approaches, including rapid learning and planning responses, are particularly relevant and useful for promoting empowerment and accountability in fragile, conflict and violence-affected settings (FCVAS). This case study focuses on PERL (Partnership to Engage, Reform and Learn) in Nigeria, a five-year governance programme (2016-21) with a total budget of £100 million. It aims to promote better service delivery through bringing together government and citizens groups to collectively address governance challenges. PERL is viewed by DFID as the final stage of a 20-year investment, building on learning, experience and partnerships from 15 years of DFID-funded governance programming in Nigeria. It was designed to incorporate learning and adaptation through a ‘living’ theory of change, continuous political economy analysis at different levels, adaptive implementation by location-based delivery teams who are encouraged to be flexible and let partners take the lead, regular learning and reflection, and adaptive resourcing, HR and administrative systems. The case study draws on a conceptual framework (the ‘adaptive triangle’) that looks at three types of adaptation – adaptive management, adaptive programming and adaptive delivery – and the interconnections and tensions between them.
Keywords: A4EA; Adaptive Development; Economy; Fishery

Reflections from a Realist Evaluation in Progress: Scaling Ladders and Stitching Theory - Punton, Melanie; Vogel, Isabel; Lloyd, Rob (CDI Practice Paper 18, 2016)
Realist evaluation provides valuable insights into how and why programmes lead to change, and can generate transferable lessons to help practitioners roll out or scale up an intervention. However, as ...

Straws-in-the-wind, Hoops and Smoking Guns: What can Process Tracing Offer to Impact Evaluation? - Punton, Melanie; Welle, Katharina (Centre for Development Impact, 2015)
This CDI Practice Paper by Melanie Punton and Katharina Welle explains the methodological and theoretical foundations of process tracing, and discusses its potential application in international development impact evaluations. It draws on two early applications of process tracing for assessing impact in international development interventions: Oxfam Great Britain (GB)’s contribution to advancing universal health care in Ghana, and the impact of the Hunger and Nutrition Commitment Index (HANCI) on policy change in Tanzania. In a companion to this paper, Practice Paper 10 Annex describes the main steps in applying process tracing and provides some examples of how these steps might be applied in practice.

Engines of Change: What Civic Tech can learn from Social Movements - Purpose; Omidyar Network (Omidyar Network and Purpose, 2016)

Innovation teams and labs. A Practice Guide - Puttick, Ruth (Nesta, 2014)

Building elevators for development mutants - Quaggiotto, Giulio (Disrupt & Innovate, 2016)
I recently had the opportunity to learn about General Mill’s (the US food giant) “emerging brands elevator” program (also known as 301 Inc). Traditionally, General Mills has grown either through mergers and acquisitions, or by building new businesses from the ground up. Increasingly, however, it found that small brands were much faster at innovation, so … Continue reading Building elevators for development mutants

Profiling the international development mutants - Quaggiotto, Giulio (Development Impact and You, 2017)
As part of a new series, we talk to Giulio Quaggiotto about the development ‘mutants’ – innovators working at the edge of the field.

The era of development mutants - Quaggiotto, Giulio (Nesta, 2016)
If you were looking for the cutting edge of the development sector, where would you go these days?

Innovation at UNDP: from weekend sport to daily practice - Quaggiotto, Giulio; Begovic, Mellie (Nesta, 2016)
The experience of working to embed innovation approaches in everyday practice at UNDP's country offices
Keywords: IMPORTANT

Fall in love with the solution, not the problem - Quaggiotto, Giulio; Leurs, Bas; Hazeldine, Shaun (Nesta, 2016)
Three strategies for international development organisations to solve problems without defining them.

Connect. Learn. Adapt. Repeat - Practicing adaptive management in complex, conflict-affected environments: barriers and promising practices - Queen, Emily Forsyth (2018)
To shift aid culture toward adaptive management, we can: - Stay humble, empathetic, and build skills in facilitation and listening. - Focus on working with more flexible foundations or individual donors. Or, work for more rigid donors and help make their policies more flexible. - Get clearer about goals and processes while finding ways to be less specific and more committed to local leadership about pathways to get to the goal. - Resist looking for a magic adaptive management tool and instead get clearer about when teams make what kinds of decisions and matching to tools that support that decision-making. - Validate that everyone’s perspective is a form of data and try out simple ways to document how teams learns and changes. - Broaden and ground the idea of expertise – remembering that, especially when working with folx on the margins, people are the only experts on their own life. Practitioners must also examine how dominant patriarchal, white supremacist, colonial ideas have pushed us toward wanting more control, less flexibility, and less space for equitable decision making.

Snapshot of Adaptive Management in Peacebuilding Programs - Queen, Emily Forsyth; Baumgardner-Zuzik, Jessica; Hume, Elizabeth; Greenberg, Melanie (Alliance for Peacebuilding, 2018)
The Alliance for Peacebuilding developed the report Snapshot of Adaptive Management in Peacebuilding Programs with support from Humanity United. This report examines how existing organizational programs are designing and learning from adaptive management in a conflict effected and fragile states. It further provides practical recommendations for applying adaptive management to peacebuilding programs based on synthesis … Continue reading "Snapshot of Adaptive Management in Peacebuilding Programs"
Keywords: KEY

Use of environmental sensors and sensor networks to develop water and salinity budgets for seasonal wetland real-time water quality management - Quinn, Nigel W. T.; Ortega, Ricardo; Rahilly, Patrick J. A.; Royer, Caleb W. (Environmental Modelling & Software, 2010)
Management of river salt loads in a complex and highly regulated river basin such as the San Joaquin River Basin of California presents significant challenges for current Information Technology. Computer-based numerical models are used as a means of simulating hydrologic processes and water quality within the basin and can be useful tools for organizing Basin data in a structured and readily accessible manner. These models can also be used to extend information derived from environmental sensors within existing monitoring networks to areas outside these systems based on similarity factors – since it would be cost prohibitive to collect data for every channel or pollutant source within the Basin. A common feature of all hydrologic and water quality models is the ability to perform mass balances. This paper describes the use of a number of state-of-the-art sensor technologies that have been deployed to obtain water and salinity mass balances for a 60,000ha tract of seasonally managed wetlands in the San Joaquin River Basin of California. These sensor technologies are being combined with more traditional environmental monitoring techniques to support real-time salinity management (RTSM) in the River Basin. Two of these new technology applications: YSI-Econet (which supports continuous flow and salinity monitoring of surface water deliveries and seasonal wetland drainage); and electromagnetic salinity mapping (a remote sensing technology for mapping soil salinity in the surface soils) – have not previously been reported in the literature. Continuous sensor deployments that experience more widespread use include: weather station sensor arrays – used to estimate wetland pond evaporation and moist soil plant evapotranspiration; high resolution multi-spectral imagery – used to discriminate between and estimate the area of wetland moist soil plant vegetation; and groundwater level sensors – used primarily to estimate seepage losses beneath a wetland pond during flood-up. Important issues associated with quality assurance of continuous data are discussed and the application of a state-of-the-art software product AQUARIUS, which streamlines the process of data error correction and dissemination, is described as an essential element of ensuring successful RTSM implementation in the San Joaquin River Basin.
Keywords: Environmental decision support; Forecasting; Sensor networks; Sensors; Water quality

A Call for Innovation in International Development - R4D (Results for Development, 2015)

What are you afraid of: Collective leadership and its learning implications - Raelin, Joseph A (Management Learning, 2018)
In this provocation, the author attempts to cite the advantages of collective leadership while acknowledging the objections and fears of challengers. Collective leadership is seen as remote because it defies the traditional view of leadership as an individualistic attractive quality that not only protects us but is efficient when applied. Nevertheless, the collective alternative may not only be advisable but required in a connected world featuring a networked economy. The contemporary socio-politico-economic environment requires the contribution of, creativity from, and collaboration among multiple agents providing a dynamic concentration of management and knowledge. If we are to accept and recognize the contribution of a collective leadership, its development would require an entirely different learning model. In particular, collective leadership development occurs as an acute immersion into the practices that are embedded within in situ material–discursive relations—in other words, among people, objects, and their institutions.

Emerging Opportunities: Monitoring and Evaluation in a Tech-Enabled World - Raftree, Linda; Bamberger, Michael (Discussion Paper, 2014)
Monitoring and evaluation practice is using information and communication technologies for more timely data, and more inclusive voice and feedback.

What Is a Case?: Exploring the Foundations of Social Inquiry - Ragin, Charles (Cambridge University Press, 2010)
The concept of the case is a basic feature of social science research and yet many questions about how a case should be defined, selected, and judged are far from settled. The contributors to this volume probe the nature of the case and the ways in which different understandings of the concept affect the conduct and the results of research. The contributions demonstrate that the work of any given researcher is often characterised by some hybrid of these basic approaches, and it is important to understand that most research involves multiple definitions and uses of cases, as both specific empirical phenomena and as general theoretical categories.

Aid on the Edge of Chaos: Rethinking International Cooperation in a Complex World - Ramalingam, Ben (Oxford University Press, 2014)
It is widely recognised that the foreign aid system - of which every country in the world is a part - is in need of drastic overhaul. There are conflicting opinions as to what should be done. Some call for dramatic increases to achieve longstanding promises. Others bang the drum for cutting it altogether, and suggest putting the fate of poor and vulnerable people in the hands of markets or business. A few argue that what is needed is creative, innovative transformation. The arguments in Aid on the Edge of Chaos are firmly in the third of these categories. In this ground-breaking book, Ben Ramalingam shows that the linear, mechanistic models and assumptions that foreign aid is built on are more at home in early twentieth century industry than in the dynamic, complex world we face today. The reality is that economies and societies are less like machines and more like ecosystems. Aid on the Edge of Chaos explores how thinkers and practitioners in economics, business, and public policy have started to embrace new, ecologically literate approaches to thinking and acting, informed by the ideas of complex adaptive systems research. It showcases insights, experiences, and dramatic results of a growing network of practitioners, researchers, and policy makers who are applying a complexity-informed approach to aid challenges. From transforming approaches to child malnutrition, to rethinking process of macroeconomic growth, from rural Vietnam to urban Columbia, Aid on the Edge of Chaos shows how embracing the ideas of complex systems thinking can help make foreign aid more relevant, more appropriate, more innovative, and more catalytic. It argues that taking on these ideas will be a vital part of the transformation of aid, from a post-WW2 mechanism of resource transfer, to a truly innovative and dynamic form of global cooperation fit for the twenty-first century.

Development innovation: Fad, silo or catalyst? - Ramalingam, Ben (Nesta, 2015)
In the first of a two-part blog piece, Ben Ramalingam, affiliate of the Overseas Development Institute / Institute of Development Studies, discusses the challenges of implementing innovation in to the development sector - from keeping expectations realistic to managing the innovation process and assessing impact and value.

Development innovation: Taking the high road - Ramalingam, Ben (Nesta, 2015)
In part two of his blog on the challenges facing development innovation, Ben Ramalingam suggests how we might take the ‘high road’ to bring about long-lasting and meaningful innovation in the sector.

Learning how to learn: eight lessons for impact evaluations that make a difference - Ramalingam, Ben (ODI, 2011)
This Background Note outlines key lessons on impact evaluations, utilisation-focused evaluations and evidence-based policy. While methodological pluralism is the key to effective impact evaluation in development, the focus here is on the factors that need to be considered for impact evaluations to be used in policy and practice – regardless of the method employed.

Learning to Adapt: Building Adaptive Management as a Core Competency in Development Practice - Ramalingam, Ben (Institute of Development Studies, 2016)
Key messages 1. Development policies and programs increasingly operate in situations of high complexity and uncertainty. 2. There are growing efforts across the sector to design, plan and implement more adaptive responses that are more relevant and appropriate in such contexts 3. At the heart of effective adaptive programming is the capability to gather, interpret and use knowledge, information and data in real-time 4. Strengthening this capability requires a positive enabling environment—including tools and methods, structures and processes, leadership and management and culture and mindsets. 5. For adaptive development to succeed, there needs to be greater attention and investment in both knowledge, information and data systems and in appropriate enabling environments
Keywords: IMPORTANT

Manifesto for Half-Arsed Development Reforms - Ramalingam, Ben (Aid on the Edge of Chaos, 2014)
Keywords: IMPORTANT

Bridging Real-Time Data and Adaptive Management in International Devepment - Case Study Report - Ramalingam, Ben; Barnett, Inka; Valters, Craig; Oppenheimer, Carrie; Levy, Anna; Hernandez, Kevin (IDS, 2017)

Innovation for International Development: Navigating the Paths and Pitfalls - Ramalingam, Ben; Bound, Kirsten (NESTA, 2016)
Experiences, insights and practical advice from over 20 leading practitioners in innovation for international development, brought together in one collection.

Ten Frontier Technologies for International Development - Ramalingam, Ben; Hernandez, Kevin; Prieto Martin, Pedro; Faith, Becky (IDS, 2016)
As new technologies and digital business models reshape economies and disrupt incumbencies, interest has surged in the potential of novel frontier technologies to also contribute to positive changes in international development and humanitarian contexts. Widespread adoption of new technologies is acknowledged as centrally important to achieving the United Nations Sustainable Development Goals by 2030. But while frontier technologies can rapidly address large-scale economic, social or political challenges, they can also involve the displacement of existing technologies and carry considerable uncertainty and risk. Although there have been significant wins bringing the benefits of new technologies to poor consumers through examples such as mobile money or off-grid solar energy, there are many other areas where the applications may not yet have been developed into viable market solutions, or where opportunities have not yet been taken up in development practice.

From best practice to best fit: understanding and navigating wicked problems in international development - Ramalingam, Ben; Laric, Miguel; Primrose, John (Working Paper, 2014)

Strengthening the Humanitarian Innovation Ecosystem - Ramalingam, Ben; Rush, Howard; Bessant, John; Marshall, Nicholas; Gray, Bill; Hoffman, Kurt; Bayley, Simon; Warren, Kim (University of Brighton, 2015)

Making adaptive rigour work - Principles and practices for strenghening monitoring, evaluation and learning for adaptive management - Ramalingam, Ben; Wild, Leni; Buffardi, Anne L (ODI/GLAM, 2019)
Adaptive programmes can be accountable, rigorous and high quality in how they use evidence by taking an ’adaptive rigour’ approach. Core development and humanitarian challenges are complex, and require processes of testing, learning and iteration to find solutions – adaptive management offers one approach for this. Yet large bureaucracies and development organisations can have low tolerance for experimentation and learning, and adaptive management can be viewed as an excuse for ‘making things up as you go along’. This briefing from the Global Learning for Adaptive Management (GLAM) initiative argues that adaptive programmes can be accountable, rigorous and high quality in how they use evidence – but this requires rethinking some key assumptions about how they are practised. The paper sets out three key elements of an ‘adaptive rigour’ approach: - Strengthening the quality of monitoring, evaluation and learning data and systems. - Ensuring appropriate investment in monitoring, evaluation and learning across the programme cycle. - Strengthening capacities and incentives to ensure the effective use of evidence and learning as part of decision-making, leading ultimately to improved effectiveness.

Utilization focused evaluation: a primer for evaluators - Ramirez, Ricardo; Brodhead, Dal (Southbound, 2013)
Ramírez, R., & Brodhead, D. (2013). Utilization focused evaluation: A primer for evaluators. Penang, Malaysia: Southbound. Retrieved from http://evaluationinpractice.files.wordpress.com/2013/04/ufeenglishprimer... This book, authored by Ricardo Ramírez and Dal Brodhead, is designed to support evaluators and program managers implement Utilization-focused evaluation (UFE). It includes detailed discussion of the 12 steps for implementing UFE and also provides a number of case studies to guide the user. Excerpt "Throughout this Primer we refer to the value of having a mentor to assist an evaluator who is using UFE for the first time. Our collective experiences with UFE indicated having a mentor was, for many UFE participants, an essential support and it reflects how we learned and mentored UFE. Evaluators may use elements of a UFE in their work naturally, for example by engaging users in planning the process or in assisting them in the utilization of findings. This Primer, however, walks the reader through UFE by systematically covering all of the 12 steps. It reflects deeply on the UFE evaluation practice and builds from it." (Ramírez & Brodhead 2013) Contents What is Utilization Focused Evaluation? 1 The UFE Framework Summarized in Steps 3 The DECI Project 6 The UFE Steps Illustrated with Project Examples 9 Step 1 Assessing Program Readiness 10 Step 2 Assessing Evaluators’ Readiness 18 Step 3 Identifying Primary Intended Users 24 Step 4 Situational Analysis 31 Step 5 Identification of Primary Intended Users 38 Step 6 Focusing the Evaluation 44 Step 7 Evaluation Design 54 Step 8 Simulation of Use 58 Step 9 Data Collection 62 Step 10 Data Analysis 65 Step 11 Facilitation of Use 68 Step 12 Meta Evaluation 72 Summary About What Each Step Entails 76 What Benefit Does UFE Bring to Commissioners of Evaluation? 81 Take Away Lessons 83 Postscript 87 The UFE Checklist 89 Case Studies 103 Recommended Reading 110

Accounting for a changing and uncertain climate in planning and policymaking today: lessons for developing countries - Ranger, Nicola; Garbett-Shiels, Su-Lin (Climate and Development, 2012)
Climate change is increasingly altering the pattern of climate-related risks. Developing countries and in particular least developed countries will be among the most severely impacted by climate change. These risks can seem remote in comparison with more immediate threats and needs, but if climate change is not considered upfront in existing planning and policymaking processes today, decision makers risk locking-in future impacts that may prove irreversible or much more costly and difficult to rectify than is necessary. The challenge for planners and policymakers, explored in this paper, is that future climate conditions are deeply uncertain. Decision methods are available to tackle these problems; however, these tend to be data- and resource-intensive and therefore, difficult to routinely apply. Further, a gap in currently available guidance is the explicit link to the adaptation needs of a developing country. We discuss the implications of this development context for the priorities for adaptation and the relative allocation of efforts in adaptation. This paper focuses on the identification of adaptation options and strategies that are robust to the deep uncertainties in future climate risk, culminating in a framework of six building blocks. It takes the perspective of exploring how decisions today might be adjusted to account for uncertain and changing long-term climate risks. We suggest a core principle is to focus on promoting climate-resilient development and increasing long-term adaptive capacity while, crucially, avoiding inflexible decisions that could lock-in future climate risk or foreclose adaptation options.
Keywords: Adaptation; Climate change; Development; Planning; robustness; uncertainty

Problem-driven iterative approaches and wider governance reform - Rao, Sumedh (Helpdesk Research Report 1099, 2014)
Identify evidence which suggests that problem-driven, iterative approaches to public sector reform can deliver more substantial, wider, long-term governance reform.

Adapting lean thinking to market systems development: Principles and practices for donors/funders - Rasmussen, Leanne (Engineers without borders, 2014)
The purpose of this paper is to provide funders and implementers of market systems development (MSD) projects with principles, practices, and structures that enables these projects to thrive. It is based on a book that has sold millions of copies worldwide, and a school of thought taught at institutions such as Harvard Business School and practiced in companies ranging from giants such as Toyota to the most successful tech start-ups in Silicon Valley. Published in 2011, Eric Ries wrote the Lean Startup as a remedy to the countless start-ups that create their own demises by getting started with the wrong goals, the wrong structures, and the wrong processes. He outlines an approach that enables the startup to navigate ambiguity and risk while using resources effectively. The conditions a MSD project faces are remarkably similar to those which a startup business faces; it needs structures and processes that are matched to the unpredictable, complex environment which it is attempting to influence. MSD projects can thus benefit from private-sector thought leadership on how start-ups can situate themselves for success. This paper starts with an overview of lean thinking, a concept that derives from lean manufacturing which is widely accepted as the leading approach to modern manufacturing. It explains how Ries’s Lean Startup adapts these principles, and then it in turn adapts these ideas for market systems development. The following three sections provide an overview of Lean Startup concepts, applying them to the development sector as appropriate. The final section summarizes and makes suggestions on next steps for funders who wish to set MSD projects up for success using a lean approach. Overall, the paper builds a case for how the Lean Startup’s approach can enable MSD projects to work successfully in ambiguity and increase their potential for achieving robust and sustainable results, all while using donor resources more efficiently.
Keywords: Adaptive Development; Lean

Retrospective ‘Outcome Harvesting’: Generating robust insights about a global voluntary environmental network - Rassmann, Kornelia; Smith, Richard; Mauremootoo, John; Wilson-Grau, Ricardo (Better Evaluation, 2013)

Strategic framework for mainstreaming citizen engagement in World Bank Group operations : engaging with citizens for improved results - Rebolledo, Miguel; Seyedian, Aaron; Peixoto, Tiago; Hernandez, Zenaida; Zakhour, Jad; Mahmood, Syed A.; Masud, Harika; Manroth, Astrid; Hamad, Qays (The World Bank, 2014)
The objective of this strategic framework is to mainstream citizen engagement in World Bank Group (WBG)-supported policies, programs, projects, and advisory services and analytics to improve their development results and within the scope of these operations, contribute to building sustainable national systems for citizen engagement with governments and the private sector. This framework will capture the diverse experiences, assess lessons learned, and outline methods and entry points to provide a more systematic and results-focused approach for the WBG. Progress toward this objective will be assessed using indicators included in program, project, and corporate results frameworks. The WBG strategy incorporates citizen engagement, including beneficiary feedback, specifically in its treatment of inclusion, which entails empowering citizens to participate in the development process and integrating citizen voice in development programs as key accelerators to achieving results. This framework builds on stocktaking and lessons learned from WBG-financed operations across regions and sectors. A key lesson is the importance of country context, government ownership, and clear objectives for citizen engagement. The approach to mainstreaming citizen engagement in WBG-supported operations is guided by five principles: 1) it is results-focused; 2) it involves engaging throughout the operational cycle; 3) it seeks to strengthen country systems; 4) it is context-specific; and 5) it is gradual. Under the right circumstances, citizen engagement can contribute to achieving development outcomes in support of the goals the WBG aims to support through all of the operations it funds: eradicating extreme poverty and boosting shared prosperity in a sustainable manner.

Using Data to Influence Government Decisions. Opportunities and Challenges for User-Centered Design to Improve Governance Data Impact - Reboot (Reboot & OSF, 2015)

Implementing Innovation: A User's Manual for Open Government Programs - Reboot Design (2015)
This guide draws from our experience around the world helping government reformers achieve real change. It is a practical resource for anyone working to implement an open government initiative.

What is Social Learning? - Reed, Mark; Evely, Anna; Cundill, Georgina; Fazey, Ioan; Glass, Jayne; Laing, Adele; Newig, Jens; Parrish, Brad; Prell, Christina; Raymond, Chris; Stringer, Lindsay (Ecology and Society, 2010)
Reed, M. S., A. C. Evely, G. Cundill, I. Fazey, J. Glass, A. Laing, J. Newig, B. Parrish, C. Prell, C. Raymond, and L. C. Stringer. 2010. What is social learning? Ecology and Society 15(4): r1. https://doi.org/10.5751/ES-03564-1504r01

The Truth of the Work: Theories of Change in a changing world - Reeler, Doug; van Blerk, Rubert (The Community Development Resource Association, 2017)

Conjunctions: Introducing Cultural Participation as a Transdisciplinary Project - Reestorff, Camilla Møhring; Fabian, Louise; Fritsch, Jonas; Stage, Carsten; Stephensen, Jan Løhmann (Conjunctions. Transdisciplinary Journal of Cultural Participation, 2014)
In the introduction of Conjunction: Transdisciplinary Journal of Cultural Participation we introduce 1) the aim of the journal, 2) the journal’s conception of transdisciplinarity as an important precondition for understanding contemporary processes and dilemmas of participation, 3) important trajectories in the existing literature on participation that focus on participation as linked to technological changes, to democratic processes of transferring power, and to complex social situations calling for analytical and evaluative frameworks able to grasp multiplicity and competing interests, and 4) the theme and articles of the this special issue: cultural participation and citizenship.
Keywords: Cultural participation; Participatory citizenship; Power dynamics; democracy; transdisciplinarity

Exploring ways to reconcile accountability and learning in the evaluation of niche experiments - Regeer, Barbara J.; de Wildt-Liesveld, Renée; van Mierlo, Barbara; Bunders, Joske F. G. (Evaluation, 2016)
While evaluation is seen as a mechanism for both accountability and learning, it is not self-evident that the evaluation of niche experiments focuses on both accountability and learning at the same time. Tensions exist between the accountability-oriented needs of funders and the learning needs of managers of niche experiments. This article explores the differences in needs and expectations of funders and managers in terms of upwards, downwards and internal accountability. The article shows that as the multi-stakeholder contexts in which niche experiments take place give rise to various requirements, tensions in evaluation are essentially a specific manifestation of tensions between niche experiments and their multiple contexts. Based on our findings, an adjusted accountability framework is proposed, including several strategies that can reconcile a learning approach with accountability needs in niche experiments aiming to change current practices in a more sustainable direction.

Confronting the Contradiction - An exploration into the dual purpose of accountability and learning in aid evaluation - Reinertsen, Hilde; Bjørkdahl, Kristian; McNeill, Desmond (Rapport 2017:06, 2017)
Keywords: IMPORTANT

Integrating modelling and smart sensors for environmental and human health - Reis, Stefan; Seto, Edmund; Northcross, Amanda; Quinn, Nigel W. T.; Convertino, Matteo; Jones, Rod L.; Maier, Holger R.; Schlink, Uwe; Steinle, Susanne; Vieno, Massimo; Wimberly, Michael C. (Environmental Modelling & Software, 2015)
Sensors are becoming ubiquitous in everyday life, generating data at an unprecedented rate and scale. However, models that assess impacts of human activities on environmental and human health, have typically been developed in contexts where data scarcity is the norm. Models are essential tools to understand processes, identify relationships, associations and causality, formalize stakeholder mental models, and to quantify the effects of prevention and interventions. They can help to explain data, as well as inform the deployment and location of sensors by identifying hotspots and areas of interest where data collection may achieve the best results. We identify a paradigm shift in how the integration of models and sensors can contribute to harnessing ‘Big Data’ and, more importantly, make the vital step from ‘Big Data’ to ‘Big Information’. In this paper, we illustrate current developments and identify key research needs using human and environmental health challenges as an example.
Keywords: Big data; Environmental health; Environmental sensors; Integrated modelling; Population health

(Breaking) The Iron Triangle of Evaluation - Reynolds, Martin (IDS Bulletin, 2015)
Keywords: Evaluation; impact

Models for Foresight Use in International Development - Rhydderch, Alun (IDS Bulletin, 2016)
This article sets out the components of the foresight approach that has been adopted by many governments in the developed world, and identifies elements of this 'dominant' approach that may hinder its uptake in developing countries. Instead, it suggests that a less rigid, more exploratory and normative approach may be better suited to many developing country contexts. With reference to the writings and practice of the creator of 'la prospective', Gaston Berger, it argues for an attitude that combines bold and inclusive thinking about how to create better futures with the pragmatic engagement with political and administrative systems that can help bring these about.

Does Foreign Aid Really Work? An Updated Assessment - Riddell, Roger C. (Social Science Research Network, 2014)
This paper draws widely from the book Does foreign aid really work?, building on that discussion to provide an updated answer to the question based on recent evidence and contemporary debates on aid effectiveness. It starts with a brief discussion (Section 2) of the question: ‘does emergency aid work?’ This is important to the wider debate for two linked reasons. Firstly, the harshest critics of development aid are all supportive of emergency aid, with a number calling for its expansion in spite of evidence of major weaknesses and failures. Secondly, the sharp historical distinction made between emergency and development aid is becoming increasingly strained, as more emergency aid is being used a year or more after emergencies strike to rebuild lives and restore livelihoods, while more development aid is used to directly save lives.The rest of the paper focuses exclusively on development aid. Section 3 provides a rapid overview of the evidence of the impact of individual aid projects, including those of Non-Governmental Organisations (NGOs). The picture is overwhelmingly positive: well over 75% of projects meet their immediate objectives and impact has improved, even though sustaining benefits remains a challenge and there continue to be aid failures. Section 4 reviews the evidence of the wider and long-term impact of aid at the sectoral and country level, including a brief discussion of academic studies on aid and growth. Though there are still major gaps in our knowledge, the quality of the data is improving. However, there is little firm, quantitative evidence to show the specific contribution that aid has made to help achieve the Millennium Development Goals (MDGs), which are a central purpose of many current donor programmes (Section 4.2). More widely, there is plenty of evidence to suggest that aid has contributed positively to both growth and wider development at the sector and country level, though some studies suggest aid has had little or no impact. Overall, the contribution that aid makes to aggregate development is lower than the public has been led to believe (Sections 4.1, 4.3. and 4.4). Assessments of the overall impact of the aid provided by NGOs are rare, but a recent study suggests it has been positive (Section 4.5). More attention is now given to the issues of corruption in aid. Although there is evidence of aid funds being used for corrupt purposes and of aid worsening corruption, on balance it remains a minor issue. Aid has had significant successes in helping the poor affected by corruption (Section 4.6). The second half of the paper shifts focus to the large gap between what aid has done and what it might do. Section 5 discusses a range of inefficiencies within and across the aid system and their costs in terms of reduced impact, including the way aid is allocated, its growing complexity, and the volatility and unpredictability of aid flows (Section 5.1 and 5.2). These inefficiencies place in a different light the evidence of aid’s overall positive impact. The paper looks at the different initiatives that have been mounted to begin to address these weaknesses and failures, including the 2005 Paris Declaration, and discusses why donors have failed to honour the promises they have made to change the ways that they give aid.In Section 5.3, the paper argues that the most critical debate about whether aid works concerns the assessment of whether the short-term, immediate and extensive benefits that aid undoubtedly brings are outweighed by the direct and indirect systemic problems that it risks creating or accentuating. As increasingly over the last decade donors have channelled more of their aid into short-term, quick-impact projects, assessing the wider negative systemic effects of aid has become even more important. Some recent studies suggest that aid’s systemic problems are large and growing, narrowing the gap between aid’s harshest critics and broader analyses of aid impact. Against the backdrop of already too many proposals of how to make aid work better, Section 6 lays out nine concrete proposals for doing this: deepening knowledge of local contexts; ensuring short-term uses of aid are consistent with and supportive of long-term development, and that all aid is more closely related to overall recipient development goals and processes; helping build local capacities for recipients to be able to coordinate aid better; moving from rhetoric to reality in learning lessons from aid; using aid to help the poor in middle income countries; reducing volatility in aid at the country level; encouraging budget support by addressing donor-country political concerns; and rethinking ways of communicating about aid. Section 7 concludes. It suggests that, paradoxically, aid’s impact may well have been harmed by focussing too narrowly on trying to make short-term aid work better, and that the main focus of attention needs to widen to assess how aid can contribute more to a recipient’s own development goals. Additionally, donors need to help build the capacity of developing countries and developing country scholars to enable them to play a bigger role in helping to answer the question of whether aid works; unsettling though their assessments may be.

The Science in Adaptive Management - Ripley, Matt; Jaccard, Sabine (ILO, 2016)
‘Adaptive management’ is all the rage in international development circles. But to avoid yet another buzzword – we need to learn from the experience of natural resource science.
Keywords: IMPORTANT

Making Sense of ‘Messiness’. Monitoring and measuring change in market systems: a practitioner's perspective - Ripley, Matthew; Nippard, Daniel (2014)

Adaptive management: where are we now? - Rist, Lucy; Campbell, Bruce M.; Frost, Peter (Environmental Conservation, 2013)
Adaptive management (AM) emerged in the literature in the mid-1970s in response both to a realization of the extent of uncertainty involved in management, and a frustration with attempts to use modelling to integrate knowledge and make predictions. The term has since become increasingly widely used in scientific articles, policy documents and management plans, but both understanding and application of the concept is mixed. This paper reviews recent literature from conservation and natural resource management journals to assess diversity in how the term is used, highlight ambiguities and consider how the concept might be further assessed. AM is currently being used to describe many different management contexts, scales and locations. Few authors define the term explicitly or describe how it offers a means to improve management outcomes in their specific management context. Many do not adhere to the idea as it was originally conceived, despite citing seminal work. Significant confusion exists over the distinction between active and passive approaches. Over half of the studies reporting to implement AM claimed to have done so successfully, yet none quantified specific benefits, or costs, in relation to possible alternatives. Similarly those studies reporting to assess the approach did so only in relation to specific models and their parameterizations; none assessed the benefits or costs of AM in the field. AM is regarded by some as an effective and well-established framework to support the management of natural resources, yet by others as a concept difficult to realize and fraught with implementation challenges; neither of these observations is wholly accurate. From a scientific and technical perspective many practical questions remain; in particular real-world assessments of the value of experimentation within a management framework, as well as of identified challenges and pathologies, are needed. Further discussion and systematic assessment of the approach is required, together with greater attention to its definition and description, enabling the assessment of new approaches to managing uncertainty, and AM itself.
Keywords: Adaptive Management; Decision making; Natural resource management; conservation; experimental management; uncertainty

Hospicing The Old - Robinson, Cassie (Stewarding Loss, 2019)
In 2010 I was introduced to the Berkana Institutes’s Two Loop model, and I come back to it again and again. As I’ve moved across different…

Thinking and Working Politically through Applied Political Economy Analysis. A guide for practitioners - Rocha Menocal, Alina; Cassidy, Marc; Swift, Sarah; Jacobstein, David; Rothblum, Corinne; Tservil, Ilona (USAID, 2018)
Thinking and Working Politically (TWP) through Applied Political Economy Analysis (PEA). A guide for practitioners Have you ever done everything right in a development program — followed every technical best practice — but still missed the mark? When this happens, it often relates to factors in the context beyond any external development actor’s ability to control.

Overcoming Premature Evaluation - Roche, Chris (2016)
Guest post from Chris Roche on practical ways of introducing adaptive management, learning from failure etc in aid programmes

Simplicity, Accountability and Relationships: Three ways to ensure MEL supports Adaptive Management - Roche, Chris (From Poverty to Power, 2018)
Based on a recent discussion in Manila, Chris Roche reflects on how Monitoring, Evaluation and Learning can better support 'adaptive programming'.

Monitoring and evaluation for adaptive programming - Roche, Chris; Kelly, Linda (Devpolicy Blog from the Development Policy Centre, 2018)
Chris Roche and Linda Kelly with six take-aways on what is being tried and learnt in setting up monitoring and evaluation frameworks for adaptive programs.

Why Diverse Teams Are Smarter - Rock, David; Halvorson, Heidi Grant (Harvard Business Review, 2016)
Research shows they’re more successful in three important ways.

Social Learning and Natural Resource Management: The Emergence of Three Research Perspectives - Rodela, Romina (Ecology and Society, 2011)
Rodela, R. 2011. Social learning and natural resource management: the emergence of three research perspectives. Ecology and Society 16(4): 30. https://doi.org/10.5751/ES-04554-160430

Beyond enforcement: communities, governance, incentives and sustainable use in combating wildlife crime. Symposium Report. - Roe, Dilys (IIED, 2015)

Making the Most of Mess: Reliability and Policy in Today's Management Challenges - Roe, Emery (Duke University Press Books, 2013)
In Making the Most of Mess, Emery Roe emphasizes that policy messes cannot be avoided or cleaned up; they need to be managed. He shows how policymakers and other professionals can learn these necessary skills from control operators who manage large critical infrastructures such as water supplies, telecommunications systems, and electricity grids. The ways in which they prevent major accidents and failures offer models for policymakers and other professionals to manage the messes they face.Throughout, Roe focuses on the global financial mess of 2008 and its ongoing aftermath, showing how mismanagement has allowed it to morph into other national and international messes. More effective management is still possible for this and many other policy messes but that requires better recognition of patterns and formulation of scenarios, as well as the ability to translate pattern and scenario into reliability. Developing networks of professionals who respond to messes is particularly important. Roe describes how these networks enable the avoidance of bad or worse messes, take advantage of opportunities resulting from messes, and address societal and professional challenges. In addition to finance, he draws from a wide range of case material in other policy arenas. Roe demonstrates that knowing how to manage policy messes is the best approach to preventing crises.

Managing Policy Mess - Roe, Emery (Governance in Turbulent Times, 2016)
What are the conditions for political development and decay, and the likelihood of sustained political order? What are the limits of established rule as we know it? How much stress can systems tackle before they reach some kind of limit? How do governments tackle enduring ambiguity and uncertainty in their systems and environments? These are some of the big questions of our time. Governance in turbulent times may serve as a stress-test of well-known ways of governing in the 21st century. Governance in Turbulent Times discusses this pertinent challenge and suggests how governments and organizations cope with and live with turbulence. The book explores how organizations and institutions respond to precipitous, conflicting, and novel-in short, turbulent-governance challenges. This book is a comprehensive and ground-breaking endeavor to understand how governance systems respond to turbulent challenges, and how turbulent times provide excellent opportunities to investigate the sustainability of governance systems. The book illustrates how politics, administrative scale and complexity, uncertainty, and time constraints can collide to produce turbulence. Building on prior work in organization theory and political science, we argue that turbulence refers to four properties related to the interaction of demands for action: variability, consistency, expectation, and unpredictability. Turbulence occurs where the interaction of demands is experienced as highly variable, inconsistent, unexpected, and/or unpredictable.
Keywords: Comparative Politics; International Relations; Political Process; Political science; Public Affairs & Administration

Policy messes and their management - Roe, Emery (Policy Sciences, 2016)
This paper presents a framework for better managing policy messes and draws implications for bad and good mess management in policy analysis and management. The framework has three foci: (1) the cognitive space in which policy messes develop, particularly in terms of gaps between macro-designers and micro-operators; (2) the unique domain of competence within that space where professionals manage the resulting messes by virtue of their skills in recognizing system-wide patterns, formulating locally specific contingency scenarios and translating both patterns and scenarios in highly reliable services; and (3) the ability of those mess and reliability professionals to be reliable in their domain and with these skills by maneuvering across different performance modes as conditions dictate—just-in-case, just-on-time, just-for-now or just-this-way.

Bridging the Gap: Synthesising Evidence from Secondary Quantitative and Primary Qualitative Data - Roelen, Keetie (CDI Practice Paper, 2016)

How to Make ‘Cash Plus’ Work: Linking Cash Transfers to Services and Sectors - Roelen, Keetie; Devereux, Stephen; Abdulai, Abdul-Gafaru; Martorano, Bruno; Palermo, Tia; Ragno, Luigi Peter (UNICEF Office of Research, 2017)
The broad-ranging benefits of cash transfers are now widely recognized. However, the evidence base highlights that they often fall short in achieving longer-term and second-order impacts related to nutrition, learning outcomes and morbidity. In recognition of these limitations, several ‘cash plus’ initiatives have been introduced, whereby cash transfers are combined with one or more types of complementary support. This paper aims to identify key factors for successful implementation of these increasingly popular ‘cash plus’ programmes, based on (i) a review of the emerging evidence base of ‘cash plus’ interventions and (ii) an examination of three case studies, namely, Chile Solidario in Chile, IN-SCT in Ethiopia and LEAP in Ghana. The analysis was guided by a conceptual framework proposing a menu of ‘cash plus’ components. The assessment of three case studies indicated that effective implementation of ‘cash plus’ components has indeed contributed to greater impacts of the respective programmes. Such initiatives have thereby addressed some of the non-financial and structural barriers that poor people face and have reinforced the positive effects of cash transfer programmes. In design of such programmes, further attention should be paid to the constraints faced by the most vulnerable and how such constraints can be overcome. We conclude with recommendations regarding the provision of complementary support and cross-sectoral linkages based on lessons learned from the case studies. More research is still needed on the impact of the many variations of ‘cash plus’ programming, including evidence on the comparative roles of individual ‘plus’ components, as well as the knowledge, attitudes and behaviour pathways which influence these impacts.

Theory of Change - Rogers, Patricia (Methodological Briefs: Impact Evaluation, 2014)

The politics of public sector performance: pockets of effectiveness in developing countries - Roll, Michael (Routledge research in comparative politics, 2014)

Citizen-generated data and sustainable development - Rono-Bett, Karen; Kenei, Steve (DevInit and DRT, 2017)
The potential of citizen-generated data in the sustainable development agenda

Complexity, Institutions and Public Policy: Agile Decision-making in a Turbulent World - Room, Graham (Edward Elgar Pub, 2011)
Graham Room argues that conventional approaches to the conceptualization and measurement of social and economic change are unsatisfactory. As a result, researchers are ill-equipped to offer policy advice. This book offers a new analytical approach, combining complexity science and institutionalism. It also provides tools for policy makers in turbulent times. Part 1 is concerned with the conceptualization of socio-economic change. It integrates complexity science and institutionalism into a coherent ontology of social and policy dynamics. Part 2 is concerned with models and measurement. It combines some of the principal approaches developed in complexity analysis with models and methods drawn from mainstream social and political science. Part 3 offers empirical applications to public policy: the dynamics of social exclusion; the social dimension of knowledge economies; the current financial and economic crisis. These are supplemented by a toolkit for the practice of 'agile policy making'. This is a stimulating, provocative and highly original book. It will appeal to academics and students in social and policy studies and to a wide range of scholars in other disciplines where complexity science is already well-developed. It will also be of major interest for decision makers coping with complex and turbulent policy terrains.Contents: Preface 1. Introduction Part I: Concepts 2. The Complexity Paradigm 3. Complex Adaptive Systems 4. The Economy as a Complex Adaptive System 5. Institutional Settings and Architectures 6. Institutional Dynamics 7. The Struggle for Positional Advantage 8. Conceptualising Social Dynamics Part II: Methods 9. Attractors and Orbits in Dynamic Systems 10. Patterns in Time and Space 11. Connections and Networks 12. Mobility on Social Landscapes 13. Towards a Generic Methodology Part III: Policies 14. Agile Policy-Making 15. Poverty and Social Exclusion 16. Social Dynamics of the Knowledge Economy 17. Global Turbulence and Crisis Postscript: Tools for Policy-Makers References Index

Supporting Learning? Exploring the relationship between grantee learning and grantmaking practice in the transparency and accountability sector - Ross, Jenny (INTRAC, 2015)
Learning is fundamental to work on transparency and accountability in complex environments. But how can funding practices best support learning?

State of the Field Review: Fiscal Transparency and Accountability - Rudiger, Anja (International Budget Partnership, 2018)

Evaluation of the market systems development approach: Lessons for expanded use and adaptive management at Sida Volume I: Evaluation Report - Ruffer, Tim; Bailey, Helen; Dahlgren, Stefan; Spaven, Patrick; Winters, Mark (Evaluation of the market systems development approach, 2018)
This report presents the findings of an evaluation of Sida’s management of the market systems development (MSD) approach. It aims to inform thinking on how Sida can best manage its growing portfolio of MSD programs. Beyond this, it provides insights relevant to Sida’s wider support to complex and adaptive programs. The evaluation identified several factors that affect Sida’s ability to ensure that conducive conditions are in place for effective MSD programs and good development programming more generally. Sida’s relatively flexible framework of rules, guidelines and systems for project management provide the space needed for staff to innovate and manage adaptively. But for this to happen consistently and effectively, Sida needs to invest more deliberately in building the capacity of its staff in relevant areas. In addition, leadership and incentives are key to shaping a culture of active experimentation and learning. This needs to be supported with clearer guidance for those involved in the design and appraisal of MSD projects; and strengthened oversight of project performance, including through adjustments to Sida’s contracts and funding agreements.