Naturalizing sensemaking - Snowden, Dave (Informed by Knowledge: Expert Performance in Complex Situations, 2010)
The focus of this book is on how experts adapt to complexity, synthesize and interpret information in context, and transform or "fuse" disparate items of information into coherent knowledge. The chapters examine these processes across experts (e.g. global leaders, individuals in extreme environments, managers, police officers, pilots, commanders, doctors, inventors), across contexts (e.g. space and space analogs, corporate organizations, command and control, crisis and crowd management, air traffic control, the operating room, product development), and for both individual and team performance. Successful information integration is a key factor in the success of diverse endeavors, including team attempts to climb Mt. Everest, crowd control in the Middle East, and remote drilling operations. This volume is divided into four sections, each with a specific focus on an area of expert performance, resulting in a text that covers a wide range of useful information. These sections present well-researched discussions, such as: the management of complex situations in various fields and decision contexts; technological and training approaches to facilitate knowledge management by individual experts and expert teams; new or neglected perspectives in expert decision making; and the importance of ‘modeling’ expert performance through techniques and frameworks such as Cognitive Task Analysis, computational architectures based on the notion of causal belief mapping such as ‘Convince Me,’ or the data/frame model of sensemaking. The volume provides essential reading for researchers and practitioners of Naturalistic Decision Making and those who study Expertise; Organizational and Cognitive Psychologists; and researchers and students in Business and Engineering.

External Review of the Building Effective and Accessible Markets (BEAM) programme - Snedden, Ewan; Freer, Gordon; Moffatt, Michelle (WYG International, 2016)

Ten years of Outcome Mapping adaptations and support - Smith, Richard; Mauremootoo, John; Rassmann, Kornelia (OM Resources: Key Community Documents, 2012)
An analysis of how and where Outcome Mapping has been applied, how users have experienced OM and the support options available and required for its use. This research was commissioned by the OMLC Stewards.

10 years of Outcome Mapping - Smith, R; Mauremootoo, John; Ambrose, K.; Hearn, Simon (2012)
This webinar from the Outcome Mapping Learning Community (OMLC) presents the key findings from research conducted into the extent of Outcome Mapping use and the support required for its implementation.

Chris Argyris: theories of action, double-loop learning and organizational learning - Smith, Mark K. (The encyclopedia of pedagogy and informal education, 2013)
Chris Argyris: theories of action, double-loop learning and organizational learning. The work of Chris Argyris (1923-2013) has influenced thinking about the relationship of people and organizations…

40 Agile Methods in 40 Minutes - Smith, Craig (Scrum Australia 2014, 2014)
My presentation from Scrum Australia 2014 called “40 Agile Methods in 40 Minutes” is available on Slideshare. With 73% of the world using Scrum as their predominant Agile method, this session will …

40 Agile Methods in 40 Minutes - Smith, Craig (YOW! Australia Software Development Conference, 2015)

I know what I know (but how do I know what I don’t?) - Smit, Debbie; de Lanerolle, Indra; Braam, Tamara; Byrne, Deborah; Legong, Gontse (MAVC, 2017)
An important support function of Making All Voices Count South Africa is to design, plan and facilitate community of practice gatherings for sustained learning and sharing across Making All Voices Count grantees. This report aims to capture the content of a one-day Making All Voices Count South African Community of Practice (CoP) Meeting held in November 2016. The South African MAVC CoP has been running for three years and has met between two and four times a year. It is a space for MAVC grantees and others working to foster innovation in the fields of transparency and accountability, to share experiences and knowledge, and collaborate in learning and improving work.

Built to improve: Leveraging realtime M&E for adaptive youth employment programming - Skelton, John (Prospects practice paper, 2, 2016)
Recognising that aid and development programming takes place in complex contexts, Mercy Corps is increasingly seeking to understand how best to manage programs which iterate, adapt and respond to the consistently evolving settings in which we work. This brief Practice Paper provides some examples of what adaptive management looks like in practice on the Prospects youth employment program in Liberia. It does not seek to function as a manual or set of guidelines, but simply provides some practical examples and insights into how a youth employment program governed by principles of adaptive management operates.

Real World SAFe – Leapfrogging a successful waterfall company into Scaled Agile - Skarin, Mattias (CRISP, 2017)

Top-down vs. Bottom-up Hierarchy: Or, How to Design a Self-Managed Organization - Sisney, Lex (Organizational Physics - Systems thinking for breakthrough business performance, 2016)
Top-down vs. Bottom-up Hierarchy: Or, How to Design a Self-Managed Organization

Routledge international handbook of participatory design - Simonsen, Jesper; Robertson, Toni (Routledge, 2013)
"Participatory Design is about the direct involvement of people in the co-design of the technologies they use. Its central concern is how collaborative design processes can be driven by the participation of the people affected by the technology designed. Embracing a diverse collection of principles and practices aimed at making technologies, tools, environments, businesses, and social institutions more responsive to human needs, the International Handbook of Participatory Design is a state-of-the-art reference handbook for the subject. The Handbook brings together a multidisciplinary and international group of highly recognized and experienced experts to present an authoritative overview of the field and its history and discuss contributions and challenges of the pivotal issues in Participatory Design, including heritage, ethics, ethnography, methods, tools and techniques and community involvement. The book also highlights three large-scale case studies which show how Participatory Design has been used to bring about outstanding changes in different organisations. The book shows why Participatory Design is an important, highly relevant and rewarding area for research and practice. It will be an invaluable resource for students, researchers, scholars and professionals in Participatory Design"--

Digital Democracy: The Tools Transforming Political Engagement - Simon, Julie; Bass, Theo; Boelman, Victoria; Mulgan, Geoff (NESTA, 2017)
This paper shares lessons from Nesta’s research into some of the pioneering innovations in digital democracy which are taking place across Europe and beyond

Beyond Logframes - Simister, Nigel (Intrac, 2017)
The logical framework, otherwise known as a logframe, is a commonly used planning tool. Initially designed for use within simple projects, it is increasingly being applied to complex programmes and organisations. In these cases the logical framework has limitations. There are several options that can help overcome these limitations. These include expanding the logframe, and using multiple logframes.

How Adaptive Management is challenging the monitoring and evaluation of complex programmes - Simister, Nigel (INTRAC, 2018)
By Nigel Simister Adaptive management is a broad approach designed to support development or humanitarian programmes in complex or uncertain …

In pursuit of the deep and sustained shifts necessary for adaptive management - Simister, Nigel (INTRAC, 2018)
The final blog in Nigel Simister's series on adaptive management and the M&E of complex projects and programmes.

Summarising portfolio change: results frameworks at organisational level - Simister, Nigel (M&E Paper 10, 2016)

Learning to ADAPT: monitoring and evaluation approaches in climate change adaptation and disaster risk reduction – challenges, gaps and ways forward - Silva Villanueva, Paula (SRC Discussion Paper, 2011)
This working paper is a methodological contribution to the emerging debate on monitoring and evaluation (M&E) in the context of climate change adaptationand disaster risk reduction. Effectively managing disaster risk is critical for adapting to the impacts of climate change, however disasters risk reduction M&E practice may be limited in capturing progress towards adaptation. The unique nature of adaptation to climate change calls for experience-based learning M&E processes for discovering the key insights into adaptive capacity and its links to adaptation processes, and to risk and vulnerability reduction at large. The ADAPT guiding principles and indicators set the foundations towards this end.

Achieving reforms in oligarchical democracies: the role of leadership and coalitions in the Philippines - Sidel, John T. (Research Papers (27), 2014)
This paper examines the role of developmental leadership in two major reforms introduced in the Philippines in 2012: the passage into law in December of excise tax reform which significantly raised taxes on cigarettes and alcohol – generally referred to as the Sin Tax Reform – and, in July, the re-registration of voters in the Autonomous Region of Muslim Mindanao (ARMM). These reforms have a) strengthened government finances and healthcare; and b) improved the quality of elections and promoted good governance and conflict resolution in the southern Philippines. Key points: These reforms were not achieved exclusively through the executive leadership of Philippine President Benigno Aquino III, this paper argues. A broader form of developmental leadership was critical to their passage into legislation and their subsequent implementation, made up of reform coalitions that incorporated elements of government, the legislature, and civil society. While these coalitions were diverse and flexible in their form and composition, their core strength came from established advocacy groups and experienced activists. These groups and activists used highly labour-intensive, specialized and complex forms of mobilization. The success of these reform coalitions, it is argued, has implications for economic and governance reform in the developing world, particularly in systems characterised by oligarchical democracy, where competition for elected office is closely linked to the entrenched interests of business and industry.

Amateur hour: CfC's 'surprising' success in addressing school congestion in the philippines - Sidel, John T. (Coalitions for Change, 2017)
The international development community has increasingly embraced the idea that finding durable solutions to complex development problems requires new ways of working that move beyond industry norms. This paper makes an important contribution to the current debate by outlining an innovative monitoring system called Strategy Testing (ST). This is the third paper in the Working Politically in Practice paper series, launched together with the Australian Department of Foreign Affairs and Trade.
Keywords: IMPORTANT

Improving the Evaluability of INGO Empowerment and Accountability Programmes - Shutt, Cathy; McGee, Rosemary (CDI Practice Paper 1, 2013)

Towards an Alternative Development Management Paradigm? - Shutt, Cathy (Rapport 2016:07, 2016)
Demonstrating results has been a concern in international development cooperation ever since it was started and in recent years there has been an increased focus on achieving and reporting on “results”. Despite the fact that everyone involved in development cooperation wants to make a difference there has been a growing criticism from practitioners about the “results agenda” based on a concern that the approaches used are not fit for purpose. In the EBA-report, Cathy Shutt, at the University of Sussex, scrutinizes the recent critical debate about results based management, and the main arguments and motives behind the criticism. She shows that the debate is not only a matter of obsessive measurement and reporting of meaningless numbers for political accountability, but also a matter of problematic assumptions and how we think about development, evidence and learning. In the report, Shutt also explores what could be learned from those who are not just critiquing results based management approaches but also proposing alternatives. Are these new alternatives an answer to the criticism?
Keywords: IMPORTANT

Conflict Aid Goes “Lean” - Shinkle, Whitney (Stanford Social Innovation Review, 2017)
Iterative design methods are essential to development work—even (or especially) in regions marked by war and violence.

How DFID can better manage complexity in development programming - Sharp, Samuel; Valters, Craig; Whitty, Brendan (ODI, 2019)
The United Kingdom’s (UK) Department for International Development (DFID) is an ambitious government department that is committed to reducing poverty and conflict overseas. Many of the issues on which DFID works are complex; whether focused on climate change, gender equality, health or other priorities, simple solutions rarely exist. And to tackle these complex challenges, DFID staff must interact with unpredictable systems of political, organisational and individual behaviours and incentives. There is a risk that complex problems spur more complicated programmes; that the complexity of development challenges is addressed through designing programmes with too many projects and implementers. While there may be valid reasons for this, too many of these complicated programmes will overburden staff. This briefing note is the outcome of an ongoing process within DFID to confront these issues and answer the question: how can DFID design and manage programmes to address complex development challenges without creating too much staff workload? Key messages DFID deals with complex problems, which require flexible systems to support testing, learning and adaptation. • Complex problems do not necessarily require workload-heavy delivery structures, but simpler approaches depend on delivery partners’ experience and competence. • However, programmes that tackle complex problems do require more hands-on engagement and face more workload from inflexible compliance requirements. We suggest that DFID and similar agencies: • Pay closer attention to delivery options in programme design, making use of existing options where possible and, where not, fostering local organisations through long-term investments. • Encourage programme designers to articulate how ambition matches resources and consider ‘good enough’ design. • Reduce the burden of compliance by cultivating partner autonomy, reassessing results and valuefor-money requirements and promoting more flexible contracting and procurement. • Plan and prioritise management resources across a portfolio of programmes to make sure they can be focused in the right areas, where the complexity of the problem requires greater engagement.

USAID/Colombia Introduces Political Economy Analysis to Better Adapt Programming to Local Contexts - Sharp, Preston (CLA Case Competition, 2015)

Exploring new territories for evaluation - Sharp, Cathy (Collective Leadership for Scotland, 2018)
Drawing on action research, this paper recasts evaluation as ‘action inquiry’, an embedded evaluative learning practice that can help navigate complexity when enacting collective leadership. It is offered as an invitation to inquiry amongst a reasonably well-informed audience of policy makers and practitioners who work in and for public services. It will particularly interest those who provide research, evaluation and facilitation support, and those seeking to develop a more relational approach to research and evaluation. Action inquiry is a model of practising change together in environments where ‘nothing is clear, and everything keeps changing’ that significantly challenges the prevailing discourse on evaluation. Action inquiry can be wrapped around and enmeshed within initiatives and programmes that work with complexity - anywhere where success will depend on the quality of relationships that can be developed. The paper reviews some important interrelated concepts that underpin the ideals of collective leadership and public service reform and which confront deeply embedded traditional notions of leadership, expertise and participation. These offer important challenges to ideas about how change happens and recognise that relationships are at the heart of practising change. This warrants a re-examination of the high expectations of evidence-based or informed practice. Collective leadership makes new demands of evidence as it rests on help to determine ‘wise actions’ in real-life situations. This confronts the practical reality of how to work together in conditions often expressed as ‘dynamic’ or ‘turbulent’ and the added human complexities of power, emotions and relationships; too often these elements are denied or avoided aspects of a change process. Facilitated action inquiry makes these elements part of the conversations, in the midst of ‘work-as-we-are-doing-it’, to increase areas of choice for individuals and a group as a whole. Within public policy, there is an overwhelming amount of evidence about ‘what works’ yet change seems to be stubborn and slow. The paper explores some of the deep-rooted vestiges of a ‘hierarchy of evidence’ and assumptions about standardisation and generalisability that act as a ‘barrier to transformation’. These include the narrow framing of what counts as evidence and consequent relegation of community perspectives, lived experience and practice-based evidence. The paper acknowledges the clear appetite for different approaches to evaluation, especially those that better reflect deeply held values and avoid creating a culture of ‘gaming’, rooted in fear of failure and loss of funding, at the expense of learning. The need for new forms of developmental evaluative thinking, collaborative inquiry and action research to create embedded learning is well overdue. Action inquiry is a desirable and necessary response to the complex situations and challenges of human services and recognises the essentialness of knowledge co-production. It is a model of co-creation at every stage and endorses the idea that people learn from participation in evaluation and by testing theories of change through action. Action inquiry builds on the idea of inquiry, or a moment-to-moment awareness and quality of attention and draws from several elements of action research practice. It sees inquiry as an intervention in itself, one that furthermore, explicitly seeks to enhance the probability of the success of a programme, focus on learning, the collaborative development of practice-based knowledge and positive relationships. The paper highlights the importance of building inquiry into living systems, the role of facilitation, systemic inquiry, and evaluative thinking. It proposes an expansion of ideas of appreciation as a relational and collaborative practice that is a driver of emergence. Social recognition that acknowledges someone’s social value to the community and implies mutual moral obligations to cooperation and participation is particularly crucial in a work context that requires successful coordination and multiple contributions to achieve results across hierarchies of position, professional rank and sectors. Hence, appreciation goes beyond the idea of positivity to include social recognition, valuing more explicit forms of inquiry, building participants’ aspirations to design new social systems and acting in new ways to embed change. In developing this discussion, the paper contributes to emerging dialogues about the need for a model of ‘5th generation evaluation’. Such a model would be based on the idea that appreciative and challenging inquiry that is contextual, relational and open-minded will create better opportunities for change and development. The paper sets out some ‘provocative propositions’ that can help us to navigate this terrain, perhaps of a fledgling ‘5th generation approach’ to inquiry. Facilitated action inquiry can hold the key to developing both new knowledge and an adaptive, collaborative and improvisational skill-set, able to respond in new ways to systemic and complex issues on the ground. It’s common to hear the expression ‘it’s all about relationships’ and it is clearly time to shift our focus to relationships; not relationships as ‘things’, but as co-created and dynamic relational processes in which we are embedded. In this way we can bring new qualities to our talking to each other about our various and shared visions of a better future.

Exploring new territories for evaluation - Provocative propositions - Sharp, Cathy (2018)

Glossary of Knowledge Management - Serrat, Olivier (Knowledge Solutions: Tools, Methods, and Approaches to Drive Organizational Performance, 2017)
The knowledge management discipline can be cryptic. These Knowledge Solutions define its most common concepts in simple terms.
Keywords: Definitions; Knowledge management; Taxonomy; Terminology

What We Do: Getting Beyond Statistics - Sense Guide (Sensemaker narrative monitoring, 2018)
Our approach Understanding daily life reality with micro-narratives Narratives and more specifically micro-narratives are a fundamental and ancient way by which humans interpret their experience and make decisions. SenseMaker® provides the ability to capture and understand those narratives. Through the web or app environment the software allows the capture of pictures, recordings and writing in various combinations to reflect how the respondents are making sense of the world. Reducing cognitive bias with self signification In a patented method, the respondent then interprets their own story into a series of abstract constructs (‘signifiers’) that feel more like a game than a survey, but allow profound meaning to emerge. This interpretation adds layers of meaning rather than simply interpreting the story and provides quantitative data to detect visual patterns among stories. The patterns are linked back to the original material enabling a deeper dive into individual stories. If the researcher first look for patterns in the metadata using statistical or visual tools, he or she is less likely to be biased by content and prematurely converge on an interpretation. Evidence based insights that enables action The output of SenseMaker® is statistical data backed up by explanatory narrative. This means that advocacy is an integral part of the system. Numbers on their own appear objective but are not persuasive; anecdotes on their own may be persuasive but are not objective. SenseMaker® puts the two together and provides a powerful means of persuasion. It also enables action. Instead of saying “How do we create a culture of X?” we say “How do we create more stories like this and fewer stories like that?” Then, as actions are initiated, we see the impact in real time.

Evaluation for equitable development results - Segone, M (UNICEF, 2011)
This document is made up of a range of Evaluation Working Papers (EWP) focused on evaluation for equitable development. Put together by evaluation specialists they present strategic evaluation findings, lessons learned and innovative approaches and methodologies. Part 1:Evaluation and equity Evaluation to accelerate progress towards equity, social justice and human rights 2 Human rights and gender equality in evaluation 13 When human rights is the starting point for evaluation 25 Strengthening Equity- focused evaluations through insights from feminist theory and approaches 39 Decolonizing evaluation in a developing world. Implications and cautions for Equity-focused evaluations 59 Part 2: Methodological implications for Equity-focused evaluations Methodological issues to design and implement equity-focused evaluations 86 Developmental evaluation for Equity-focused evaluations 102 Systems thinking and Equity-focused evaluations 115 Methodological challenges in using programme theory to evaluate pro-poor and equity-focused programmes by Patricia Rogers, Royal Melbourne Institute of Technology University and Richard Hummelbrunner, Independent consultant 142 Case Study and equity in Evaluationby Saville Kushner, University of the West of Englan 172 Values-Engaged Evaluationsby Jennifer Greene, University of Illinois at Urbana-Champaign 192 Part 3: Examples of Equity-focused evaluations Evaluating the contribution of UNDP to equity-focused public policies in Brazil and China 210 Using a human rights approach to evaluate ILO’s discrimination strategy 222 CONEVAL experience in evaluating interventions for Indigenous populations in Mexico 244 UNICEF supported evaluations with elements of equity-focused evaluations 258

Managing and Adapting a development program: Lessons from PRISMA - Seely, Kevin (PRISMA, 2019)
This case study is part of the AIP-Rural Learning Series. Funders and implementers of international development programs largely agree that adaptive management is industry best practice. Most development experts also broadly agree on what ‘adaptive management’ means. In this case study, we use a common definition of ‘adaptive management’, including the following features: Flexibility. Implementers create, modify and drop interventions when circumstances change or new information emerges, in order to tailor strategies to context and maximise impact. Purposeful experimentation. Implementers test different activities at the same time, monitoring them to learn what will achieve the desired impact. Since 2015, the number of publications calling for and praising adaptive management in development programs has grown fast. These publications give us an idea of the state of adaptive management in development programs. They reveal progress in some areas, such as tailoring interventions to local context and using evidence for decision-making. Yet they also reveal multiple, ongoing, real-world constraints to adaptive management. This case study explores how one development program, PRISMA, has avoided and overcome some of these constraints, whilst continuing to battle others. Four major constraints revealed in the adaptive management literature are discussed in this case study: Programs are designed in ways that make it hard to adapt interventions, target regions or sectors based on new learning or changing circumstances. Programs struggle to create an organisational culture that encourages learning and adapting. When staffing programs, recruiters prioritise sectoral expertise and length of experience over adaptive managerial competence. Program managers face pressures to spend their budgets predictably and before their program ends. Learning and testing take time and cost little, so managers feel pressure to deprioritise them. The first of these constraints is explored in the next section, ‘Design’, which looks at how the selection of target sectors and performance targets affects PRISMA implementers’ ability to manage adaptively. The third section, ‘Organisational Culture’, shares lessons on encouraging staff to test and improve interventions. Section 4, ‘Staffing’, looks at hiring adaptive managers, and freeing up their time to improve interventions. The final section summarises this case study’s key lessons. Throughout this paper, findings are based on 16 in-depth interviews with PRISMA’s junior, mid-level and senior staff, and its funders. To contextualise these findings, the authors reviewed the ‘adaptive management’ literature, and drew on their experience with development programs worldwide.

Adaptive Programming: The “Space of Possibility” - Seavey, Sara Mizuta (USAID Learning Lab, 2016)
Sara Mizuta Seavey, M.A., is a Senior Program Officer at FHI360 for the Mobile Solutions Technical Assistance and Research (mSTAR) project, where she works with USAID’s Digital Development Team to conceive, design, and test how real-time data systems can enable a more adaptive and participatory approach to development. She is passionate about systems thinking and participatory approaches to international development. What does space travel have to do with international development?

Dispatch on Switzerland's International Cooperation 2017–2020: Key points in brief - SDC (Federal Department of Foreign Affairs FDFA, 2016)
The “Dispatch on Switzerland’s International Cooperation, 2017–2020”, is a report to the Swiss Parliament outlining the strategic priorities proposed by the Federal Council in this area. It includes framework credits for each of the five policy instruments used by Switzerland to implement its international cooperation strategy. These are implemented by the Swiss Agency for Development and Cooperation (SDC) of the Federal Department of Foreign Affairs (FDFA), the Human Security Division (HSD) of the FDFA’s Directorate of Political Affairs, and the State Secretariat for Economic Affairs (SECO) of the Federal Department of Economic Affairs, Education and Research (EAER).

Guidelines for the Monitoring System for Development-Related Changes (MERV) - SDC (SDC, 2013)

Peacebuilding and Statebuilding Strategy for SDC’s work in fragile and conflict contexts - SDC (Swiss Agency for Development and Cooperation SDC, 2015)

Swiss Cooperation Strategy Pakistan, 2017-2019 - SDC (2016)

Was bedeutet “Staying Engaged” im Kontext der aktuellen Krise in Nicaragua? Überlegungen und Strategien der ALAK (Internal memo) - SDC (SDC, 2018)
«Mitigation» (im Sinne von Notlinderung) und «Transformation» sind die zwei Schlagworte, an welchen sich die Neuausrichtung des Portfolios in Nicaragua über die nächsten Monate orientieren wird. Ziel ist es, dass die Schweiz ihr über die letzten Jahrzehnte akquiriertes politisches Kapital und ihre Expertise nutzt, um zu einer Linderung der Auswirkung der Krise für die Bevölkerung beizutragen und mithilft die Grundlagen für eine Transformation der politischen Kultur und eine langfristig angelegte Demokratisierung zu schaffen.

The 2015 State of Scrum Report: How the world is applying the most popular Agile approach to projects - Scrum Alliance (Scrum Alliance, 2015)

Strategy Deployment and Spotify Rhythm - Scotl, Karl (AvailAgility, 2016)
Last month, Henrik Kniberg posted slides from a talk he gave at Agile Sverige on something called Spotify Rhythm - "Spotify’s current approach to getting aligned as a company". While looking through the material, it struck me that what he was describing was a form of Strategy Deployment. This interpretation is based purely on those slides - I haven't had a chance yet to explore this more deeply with Henrik or anyone else from Spotify. I hope I will do some day, but given that caveat, here's how I currently understand the approach in terms of the X-Matrix Model.

What is Uncertainty and Why Does it Matter? - Scoones, Ian (STEPS Working Paper 105, 2019)
Uncertainty defines our times. Whether it is in relation to climate change, disease outbreaks, financial volatility, natural disasters or political settlements, every media headline seems to assert that things are uncertain, and increasingly so. Uncertainty, where we do not know the probabilities of either likelihoods or outcomes, is different to risk, the implications of which are explored in this paper through five different ways of thinking about uncertainty, derived from highly diverse literatures encompassing societal, political, cultural, practice and individual perspectives. The paper continues by examining how these perspectives relate to four domains: finance and banking; critical infrastructures; disease outbreaks and climate change; natural hazards and disasters. Reflecting on these experiences, the paper argues that embracing uncertainty raises some fundamental challenges. It means questioning simple, linear perspectives on modernity and progress. It means rethinking expertise and including diverse knowledges in deliberations about the future. It means understanding how uncertainties emerge in social, political and economic contexts, and how uncertainties affect different people, depending on class, gender, race, age and other dimensions of social difference. And, if uncertainty is not reducible to probabilistic risk, it means a radically different approach to governance; one that rejects control-oriented, technocratic approaches in favour of more tentative, adaptive, hopeful and caring responses. The paper concludes by asking whether we can learn from those who live with and from uncertainty – including pastoralists in marginal settings – as part of a wider conversation about embracing uncertainties to meet the challenges of our turbulent world.

The Scrum Guide - Schwaber, Ken; Sutherland, Jeff (scrum.org, 2016)

Making design core to the agile process: a look into how we built Salesforce's Lightning Experience - Schoen, Ian (Medium, 2016)
A look into how we built Salesforce’s Lightning Experience

Set-theoretic methods for the social sciences: a guide to qualitative comparative analysis - Schneider, Carsten Q.; Wagemann, Claudius (Strategies for social inquiry, 2012)
"Qualitative Comparative Analysis (QCA) and other set-theoretic methods distinguish themselves from other approaches to the study of social phenomena by using sets and the search for set relations. In virtually all social science fields, statements about social phenomena can be framed in terms of set relations, and using set-theoretic methods to investigate these statements is therefore highly valuable. This book guides readers through the basic principles of set theory and then on to the applied practices of QCA. It provides a thorough understanding of basic and advanced issues in set-theoretic methods together with tricks of the trade, software handling and exercises. Most arguments are introduced using examples from existing research. The use of QCA is increasing rapidly and the application of set-theory is both fruitful and still widely misunderstood in current empirical comparative social research. This book provides an invaluable guide to these methods for researchers across the social sciences"--
Keywords: Comparative method; Mathematical models; Qualitative Methode; Set theory; Social science; Vergleichende Forschung

Independent Evaluation of SDC’s Results-Based Management System with a Focus on Poverty Reduction - Schmidt, Martin; Palenberg, Markus; Vähämäki, Janet (SDC, 2017)
The SDC's results-based management system ensures that the processes and instruments used for the design and implementation of programs and projects contribute to achieving the desired results (outputs, outcomes and impact). The independent evaluation examines the extent to which these processes and tools have fostered the results culture within the organization, and improved competencies for results-based management decisions, learning and communication. In addition to the findings, conclusions and recommendations, the evaluation report includes the management response of SDC’s Directorate.

(Draft) Managing to Adapt - Analysing Adaptive Management for Planning, Monitoring, Evaluation, and Learning (draft) - Schlingheider, Annika; Pellfolk, Erica; Maneo, Gabriele; Desai, Harsh (Oxfam Research Reports, 2017)
Adaptive management is at the heart of ‘doing development differently’ (Wild et al., 2016). Whether it is here to stay depends on how much it is mainstreamed into existing development programming by donors and implementers alike, especially in planning, monitoring, evaluation, and learning (PMEL) cycles. In this report, we find that mainstreaming adaptive management in PMEL involves three strategies: 1. planning for flexibility; 2. developing locally owned monitoring and evaluation (M&E); and 3. creating an enabling environment for learning. Adopting these strategies contributes to virtuous cycles of PMEL

ING’s agile transformation - Schlatmann, Bart (McKinsey Quarterly, 2017)
Two senior executives from the global bank describe their recent journey.

Qualitative comparative analysis: A valuable approach to add to the evaluator’s toolbox? Lessons from recent applications - Schatz, Florian; Welle, Katharina (CDI Practice Paper, 2016)

Scaled Agile Framework – SAFe for Lean Software and System Engineering 4.5 - Scaled Agile (2017)

Introduction to SAVI's way of working - State Accountability and Voice Initiative (SAVI) - SAVI (DFID, 2015)
We are piloting a new approach to identifying and supporting our partners.  This breaks with convention in many ways in order to improve impact.  SAVI Approach Papers summarise key aspects of SAVI’s way of working. They explain what we do, and why – and link to relevant tools and frameworks. Our approach is summarized in...
Keywords: Practice

Nigeria Public Sector Accountability and Governance Programme - Business Case - SAVI (DFID, 2016)
In May 2016, SAVI transitioned into a successor programme – the ‘Engaged Citizens Pillar’ (ECP) of a wider DFID-funded governance reform programme ‘The Partnership to Engage, Reform and Learn’ (PERL). ECP is managed by the same service provider, Palladium, and the same core management team as SAVI, and many of the SAVI front line staff...

SAVI Approach Paper 1: Core Values - SAVI (London, 2015)
Governance reform is about government and citizens working together in more responsive, inclusive and accountable ways for the benefit of citizens. More responsive, inclusive and accountable attitudes and behaviour on the part of government and non-government stakeholders are the critical factors which lead to meaningful reform processes, and replicate and sustain reforms beyond the lifetime...

SAVI Approach Paper 10: Engaging State Governments - SAVI (DFID, 2015)
SAVI as a programme does not directly work with state governments – but we work in close conjunction with sister programmes who are supporting state governments on governance and sector reforms. SAVI supports non-government and SHoA partners to play their part in promoting more responsive, inclusive and accountable state governance delivering better services for citizens....

SAVI Approach Paper 11: Managing and Staffing an Adaptive Citizen Engagement Programme - SAVI (DFID, 2016)
SAVI, a DFID funded programme implemented by Palladium, is an adaptive programme putting learning and adaptation at the centre of all decision-making. In relation to management and staffing SAVI has established the following approaches to facilitate this: – An adaptive approach to programme management: Full time strategic technical leadership, that works closely with and complements...

SAVI Approach Paper 12: Managing Programme Finances to Support Adaptive, Locally-led Processes of Citizen Engagement in Governance - SAVI (DFID, 2016)
SAVI, is seeking to support processes of citizen engagement in governance in ways that are effective in influencing reform, and that are able to take on a life of their own without continuing donor support. SAVI is also an adaptive programme, putting learning and adaptation at the centre of all decision-making. Money is used and...

SAVI Approach Paper 13: Measuring Value for Money and using Value for Money Analysis - SAVI (DFID, 2016)
SAVI has established its own framework for assessing Value for Money in annual performance – in relation to expenditure, economy, efficiency, effectiveness and equity. Routine tracking and analysis of expenditure and economy ensure that inputs are supplied and services delivered to partners in line with SAVI’s core values, whilst also meeting DFID requirements and competing...

SAVI Approach Paper 14: Learning, Adaptation and Communications - SAVI (DFID, 2016)
SAVI, a DFID funded programme implemented by Palladium, is an adaptive programme, aiming to put learning and adaptation at the centre of all decision-making. Learning and adaptation takes place in SAVI at three levels: the work of partners; the work of SAVI delivery teams; and the enabling environment of the programme as a whole. Achieving...

SAVI Approach Paper 15: Introduction to PERL: the SAVI Successor Programme - SAVI (DFID, 2016)
In May 2016, SAVI transitioned into a successor programme – the ‘Engaged Citizens Pillar’ (ECP) of a wider DFID-funded governance reform programme ‘The Partnership to Engage, Reform and Learn’ (PERL). ECP is managed by the same service provider, Palladium, and the same core management team as SAVI, and many of the SAVI front line staff...

SAVI Approach Paper 2: Programme Design - SAVI (DFID, 2015)
SAVI supports citizen engagement in governance through a facilitated partnership approach, in contrast to the usual approach of grants to civil society organisations (CSOs). The overall aim is to facilitate and support working relationships and processes of reform that are home-grown, self-sustaining and, after initial engagement, not dependent on external support. Our way of working...

SAVI Approach Paper 3: Theory of Change - SAVI (DFID, 2015)
SAVI’s theory of change is a simple, practical guide that staff and partners use to plan and to monitor change – as well as to reflect on and enhance their own effectiveness. It sets out broad stages of attitude and behaviour change over time to facilitate effective citizen engagement in governance processes, systems and structures....

SAVI Approach Paper 4: Thinking and Working Politically - Supporting partners and staff through a participatory apporach to political economy analysis - SAVI (DFID, 2015)
Thinking and acting politically is central to the SAVI programme. We support staff and partners to analyse the power relations that shape change in their state, and to use this knowledge to inform their decision-making. This includes decisions made by SAVI state teams relating to the issues and partners they engage with and support, and...

SAVI Approach Paper 5: Defining and Measuring Results - SAVI (DFID, 2015)
SAVI aims to facilitate replicable and sustainable processes of citizen engagement in governance. The programme in each state is locally defined, flexible and adaptive, and results are not predictable in advance. Standardised monitoring tools are not applicable, and consequently we have evolved our tools and frameworks during the programme through processes of learning by doing....

SAVI Approach Paper 6: Engaging Civil Society - SAVI (DFID, 2015)
SAVI state teams provide support to CS groups to become more effective agents of citizens’ voice and public accountability, through a variety of mutually reinforcing interventions. These include: hands-on support to demonstration civil society Advocacy Partnerships (APs) facilitating working partnerships between civil society APs, SHoAs, and the media brokering working relationships between all of these...

SAVI Approach Paper 7: Promoting Gender Equality and Social Inclusion - SAVI (DFID, 2015)
We promote attention to gender equality and social inclusion in all of our engagement with CS groups, the media and SHoAs, and in all of the issues and processes they work on. In all the states we work in, we also support partners to focus on some issues and form some partnerships and networks which...

SAVI Approach Paper 8: Engaging the Media - SAVI (DFID, 2015)
The overall aim of SAVI engagement with the media is for media representation of citizens’ interests to become normal, and play its part in helping state governments to be more responsive, inclusive and accountable to their citizens. Media partners – individual media personnel and selected media houses – are supported to be more effective agents...

SAVI Approach Paper 9: Engaging State Houses of Assembly - SAVI (DFID, 2015)
The overall aim of SAVI engagement with SHoAs is to promote lasting reforms that are not dependent on external funding and which promote an increasingly more responsive and accountable relationship between Nigerian state governments and their citizens. SHoAs are supported to be more effective agents of citizen voice and public accountability, demanding better performance from...

State Accountability and Voice Initiative (SAVI) - Approach Paper Series - SAVI (DFID, 2016)
We are piloting a new approach to identifying and supporting our partners.  This breaks with convention in many ways in order to improve impact. Our approach is summarized in the Introduction to SAVI’s way of working. We have broken our approach down into key components – depicted in the form of our ‘knowledge tree’ –...
Keywords: IMPORTANT

“Cash Plus” programmes for children - Save the Children (Resource Paper, 2017)
This paper provides a resource on “Cash Plus” interventions for children. These interventions combine household cash transfers with complementary, context-relevant interventionsn in order to address the multiple drivers of childhood deprivations and generate more powerful impacts for children across Save the Children’s Breakthroughs, in both development and humanitarian contexts.

Using Systemigrams in Problem Definition: A Case Study in Maritime Resilience for Homeland Security - Sauser, Brian; Mansouri, Mo; Omer, Mayada (Journal of Homeland Security and Emergency Management, 2011)
The concept of resilience has been used among a diversity of fields with a myriad of definitions. A systems approach to discovering the essence of resilience could aid in understanding its concept and provide opportunities to distinguish its systemic characteristics that can be identified, planned, or analyzed regardless of the field in which it is considered. In order to begin executing such an approach, a comprehensive study of the literature on the topic and circumstances in which resilience has been used or referred to in several fields of studies is necessary. In this paper, the conceptual common denominator of resilience is identified based on the results of a literature survey and according to the way it has been used in different contexts. This basis leads to the objective of this effort, and that is to recognize the systemic characteristics of resilience and demonstrate the defining of a universally accepted definition. Using a soft systems methodology and a supporting systemic diagramming technique entitled Systemigrams, we present the formulation of defining resilience in maritime homeland security. In this effort, many stakeholders contributed their thoughts and concerns on the meaning and operational use of resilience through a repetitive process to formulate a definition of the term. The result of this methodic approach is a general definition of resilience that has been refined in a collaborative environment via the application the Boardman Soft Systems Methodology.

Probes, toolkits and prototypes: three approaches to making in codesigning - Sanders, Elizabeth B.-N.; Stappers, Pieter Jan (CoDesign, 2014)
The role of making in the design process has been growing, taking on new forms and involving new players over the past 10 years. Where we once primarily saw designers using making to give shape to the future, today we can see designers and non-designers working together, using making as a way to make sense of the future. In this paper, we describe the landscape of design research and practice at the end of 2013 with special attention to the role of making across these perspectives: approach (cultural probes, generative toolkits and design prototypes), mindset (designing for people and designing with people), focus in time (the world as it is, the near future and the speculative future) as well as variations in design intent (provoking, engaging and serving).

Perspectives on Participation in Design - Sanders, Elizabeth (Wer gestaltet die Gestaltung? Praxis, Theorie und Geschichte des partizipatorischen Designs, 2013)
Keywords: Design

Caja de Herramientas PDIA - Un enfoque “hazlo tú mismo” para resolver problemas complejos - Samji, Salimah; Andrews, Matt; Pritchett, Lant; Woolcock, MIchael (Center for International Development at Harvard University, 2018)
(Spanish version of he PDIA toolkit) The PDIAtoolkit is designed to guide you through the process of solving complex problems which requires working in teams. We call it a Do-it-Yourself (DIY) kit, where the ‘you’ is a committed team of 4-6 people mobilized to work together to solve a complex problem that cannot be solved by one person.

PDIA Toolkit - A DIY Approach to Solving Complex Problems - Samji, Salimah; Andrews, Matt; Pritchett, Lant; Woolcock, MIchael (Center for International Development at Harvard University, 2018)
The PDIAtoolkit is designed to guide you through the process of solving complex problems which requires working in teams. We call it a Do-it-Yourself (DIY) kit, where the ‘you’ is a committed team of 4-6 people mobilized to work together to solve a complex problem that cannot be solved by one person.

PDIA Course: Alumni are already practicing what they learned - Samji, Salimah (Building State Capability, 2017)
written by Salimah Samji We offered 4 free PDIA online courses between November 2015 and June 2016. They were well received and 365 people, living in 56 countries, successfully completed the course…

PDIA Notes 2: Learning to Learn - Samji, Salimah (Building State Capability, 2016)
written by Peter Harrington After over two years of working with the government of Albania, and as we embark on a new project to work with the government of Sri Lanka, we at the Building State Capa…

Miradi Measures Dashboard Demo - Salzer, D (2013)
Live demonstration (view in HD) of a new online measures dashboard site (http://miradi.sitkatech.com) that allows users of the Miradi Adaptive Management software (http://miradi.org) to share the progress they are making towards achieving their desired results

Has Kenya’s ICT revolution triggered more citizen participation? - Salome, Nyambura (MAVC Research Briefing, 2016)
Lessons from policy in Ghana, Kenya, South Africa and Tanzania
Keywords: IMPORTANT

How to create an agile organization - survey - Salo, Olli; Ahlbäck, Karin; Fahrbach, Clemens; Murarka, Monica (McKinsey & Co., 2017)
Eighteen practices for organizational agility The survey asked respondents about a series of specific actions that underlie each of the 18 practices (9 of them stable, and 9 dynamic) of organizational agility; all of the practices are summarized in the table below. To rate respondents’ organizations, we asked how frequently their performance units engaged in each action that supports a given practice.

Help! I’m hiring new staff and I want them to work adaptively. A guide to hiring adaptive employees. - Salib, Monalisa (USAID LEARN, 2019)
If you are involved in hiring, this tool can help you increase your chances of selecting staff members skilled in adaptive management. It will help answer the questions: Which competencies should I recruit for in order to hire more adaptive employees? Which desired qualifications should I incorporate into position descriptions to attract adaptive employees? Which interview questions should I ask to screen for adaptive employee competencies? You should use this tool when you decide to hire a new individual - whether that be a direct hire, contractor, or consultant - or when you are changing an individual’s current position description or scope of work. Specifically, this tool will help you determine which qualifications are most important for the position and offers interview questions you can use with candidates. --- You’re in luck! We just released a new guide for hiring adaptive employees. This visual and user-friendly tool will help increase your chances of selecting staff skilled in adaptive management. Here are the details: What do you mean by “work adaptively” or an “adaptive employee”? An adaptive employee is an individual who, in collaboration with relevant stakeholders, systematically acquires and uses knowledge to make decisions and adjustments in their work in order to achieve greater impact. Why focus on adaptive employees? Why does it matter? With industries, technologies, and organizations in a near constant state of flux, leaders are recognizing the importance of adaptability as a critical capacity. Because of this, it’s not surprising that a recent LinkedIn report found that adaptability was the most important soft skill hiring managers were screening for. In addition, evidence shows that teams that apply more data-driven and adaptive leadership practices perform better than those that focus less on these practices. It makes sense, then, that managers want to recruit more team members who are skilled in adaptive management - it helps achieve results. Who should use this tool? Anyone involved in hiring, responsible for developing or approving position descriptions, participating in interviews, and/or approving new hires. When should I use this tool? Use it as soon as you decide to hire a new staff person or when you’re adjusting existing position descriptions. What does the tool help me do? It will help you think through: Which competencies should I recruit for in order to hire more adaptive employees? Which desired qualifications should I incorporate into position descriptions to attract adaptive employees? (You can even copy and paste qualification language from the tool into scopes of work!) Which interview questions should I ask to screen for adaptive employee competencies? (You can copy and paste interview questions into your screening, interview, and reference check protocols!) Has the tool been tested? Yes, staff within USAID’s Global Development Lab and on the USAID LEARN contract tested the tool, providing feedback on the minimum viable product and subsequent versions. Testers confirmed finding the tool useful, and gave it a 9/10 score when asked if they would recommend it to colleagues. Some specific feedback from testers included: "The tool prompts deeper thought on what you might need and want in a job candidate" and "All [of the interview questions I used from the tool] worked well." What about the existing staff? How can they gain greater skills in adaptive management? Existing USAID staff and implementing partners can access online training in collaborating, learning, and adapting (CLA) and find resources in the CLA Toolkit. USAID staff also have access to an internal CLA community of practice and five-day, in-person CLA training.

“There’s no money for that.” Three Ways to Resource Collaborating, Learning, and Adapting - Salib, Monalisa (USAID Learning Lab, 2016)
This blog is the third in an ongoing series exploring the components of USAID's CLA Framework. Here is the first blog on organizational culture and the second on effective learning.

The Internet and Democracy Building in Lusophone African Countries - Salgado, Dr Susana (Ashgate Publishing, Ltd., 2014)
This timely book fills an important gap in the literature on the influence of the Internet and new media in Portuguese speaking African countries. Based on extensive field work throughout the region the author examines the influence of the Internet in the transition to democracy in Africa, and asks whether there are new possibilities for popular activism to emerge from evolving communication environments and media systems.The book analyses the different forms of democracy, the concept of development, and addresses the debate about the relationship between democracy and development and explores the influence of the media in the democratization process, the promises that digital media bring to this process and to development and the implications of the African digital divide. In certain countries in this region democracy and independent news media are in their infancy but are starting to take hold, giving an excellent opportunity to observe the dynamics of civil society and the influence of increased freedom, new voting powers and new media in particular. The book offers important insights into the roles and functions that the media in general, and the Internet in particular, can perform in the creation of a more democratic society, as well as in empowering and educating citizens in democratic values.
Keywords: Business & Economics / Industries / Computers & Information Technology; History / Africa / General; Political Science / Political Ideologies / Democracy

From analysis to action: operationalising learning and adaptation in Savings at the Frontier - Salehi, Yusef (2016)
Keywords: IMPORTANT

Defining the burden of proof in conservation - Salafsky, Nick; Redford, Kent H. (Biological Conservation, 2013)
Conservationists often must take action in the face of uncertainty about the costs and benefits of different options. Although this uncertainty can be paralyzing when the stakes are high, there is obviously a cost to inaction as well as action, and decision makers need to be encouraged to act when appropriate. Many other fields of human endeavor such as law, medicine, and public safety have formally developed the “burden and standards of proof” that decision makers have to meet in choosing to take action. In this paper, we review the standards developed in these other fields to help define a similar framework for conservation. Specifically we propose that a conservation decision maker must assume the burden of proof when there is a decision to act that substantially affects others, in which the decision maker has professional standing, where there is not immediate urgency, and where there is some, but not complete certainty about the outcomes of acting versus not acting. Once these initial tests have been met, in situations in which the decision maker is more worried about the consequences of not acting, then a relatively low standard of proof is required for taking action. If the decision maker is concerned with the consequences of acting in error, but the action is relatively reversible, then a medium standard of proof is required. And finally, if there are concerns about the consequences of acting in error, but the action is relatively irreversible, then a high standard of proof is required.
Keywords: Biodiversity; Burden of proof; Decision theory; Standard of proof; Uncertainty

Defining and using evidence in conservation practice - Salafsky, Nick; Boshoven, Judith; Burivalova, Zuzana; Dubois, Natalie S.; Gomez, Andres; Johnson, Arlyne; Lee, Aileen; Margoluis, Richard; Morrison, John; Muir, Matthew; Pratt, Stephen C.; Pullin, Andrew S.; et al. (Conservation Science and Practice, 2019)
There is growing interest in evidence-based conservation, yet there are no widely accepted standard definitions of evidence, let alone guidance on how to use it in the context of conservation and natural resource management practice. In this paper, we first draw on insights of evidence-based practice from different disciplines to define evidence as being the “relevant information used to assess one or more hypotheses related to a question of interest.” We then construct a typology of different kinds of information, hypotheses, and evidence and show how these different types can be used in different steps of conservation practice. In particular, we distinguish between specific evidence used to assess project hypotheses and generic evidence used to assess generic hypotheses. We next build on this typology to develop a decision tree to support practitioners in how to appropriately use available specific and generic evidence in a given conservation situation. Finally, we conclude with a discussion of how to better promote and enable evidence-based conservation in both projects and across the discipline of conservation. Our hope is that by understanding and using evidence better, conservation can both become more effective and attract increased support from society.
Keywords: Adaptive Management; Evidence-based conservation; Evidence-based practice; Natural resource management; Project management; biodiversity; environmental evidence

Doing things differently: Rethinking monitoring and evaluation to understand change - Saferworld (Saferworld, 2016)
Doing things differently: Rethinking monitoring and evaluation to understand change Learning paper Over the past four years, Saferworld has put in place a way of monitoring, evaluating and learning from our work focused on behaviour and relationship change. This paper outlines the process we have gone through to adapt, embed, and embrace an approach inspired by Outcome Mapping and Outcome Harvesting (OH). Key benefits of our monitoring, evaluation and learning approach are: It is simple, but promotes complex discussion and analysis. It allows conflict- and gender-sensitivity to be built into programmes; it promotes analysis of and adaptation to context. Bringing front-line staff and partners into wider conversations with others substantially increases cross-organisational learning.

Volume 3: Participatory Monitoring, Evaluation, and Learning in Complex Adaptive Environments - SACE (SACE Program, 2018)
This volume is the last in a series of papers about systems approaches in complex environments, which includes the use of the collective impact model to address large-scale social problems, and the application of participant-driven MEL techniques across 17 networks of civil society organizations. It is based on the experiences of Root Change and Chemonics, two development partners working on a USAID civic engagement project in Nigeria (2013-2018), as well as hundreds of civil society organization partners. This is the final paper in the series and aims to highlight how the adaptation of participatory monitoring, evaluation, and learning techniques (e.g., most significant change, outcome mapping, and outcome harvesting) evolved and ultimately empowered cluster members. The first paper in the series presented a brief introduction to systems approaches in advocacy settings, the SACE theory of change, and the scope of Root Change’s work as technical lead on capacity building and measurement. The second volume aims to address the innovative use of the advocacy strategy matrix, adapted from work by the Center for Evaluation Innovation, for collective impact and the Collective Impact Model, an approach that engages multiple players in working together to solve complex social problems.

Towards ‘Targeted Systems Change’ - Rye, Sam (Fieldnotes by Sam Rye, 2019)
Modelling and communicating how to shift systems

Causal Loop Diagrams: Little Known Analytical Tool | iSixSigma - Rushing, William (2010)
The causal loop diagram is an analytical tool that is seldom used in Six Sigma but nonetheless is still very valuable. It is a foundational tool used in system dynamics, a method of analysis used to develop an understanding of complex systems.

Doing the Transparent State: open government data as performance indicators - Ruppert, Evelyn; others (A World of Indicators: The production of knowledge and justice in an interconnected world, 2013)

An Introduction to Adaptive Management for Threatened and Endangered Species - Runge, Michael C. (Journal of Fish and Wildlife Management, 2011)
Management of threatened and endangered species would seem to be a perfect context for adaptive management. Many of the decisions are recurrent and plagued by uncertainty, exactly the conditions that warrant an adaptive approach. But although the potential of adaptive management in these settings has been extolled, there are limited applications in practice. The impediments to practical implementation are manifold and include semantic confusion, institutional inertia, misperceptions about the suitability and utility, and a lack of guiding examples. In this special section of the Journal of Fish and Wildlife Management, we hope to reinvigorate the appropriate application of adaptive management for threatened and endangered species by framing such management in a decision-analytical context, clarifying misperceptions, classifying the types of decisions that might be amenable to an adaptive approach, and providing three fully developed case studies. In this overview paper, I define terms, review the past application of adaptive management, challenge perceived hurdles, and set the stage for the case studies which follow.

Parliament and the people - Rumbul, Rebecca; Moulder, Gemma; Parsons, Alex (2018)
How digital technologies are shaping democratic information flow in Sub-Saharan Africa

Evaluation of the market systems development approach: Lessons for expanded use and adaptive management at Sida Volume I: Evaluation Report - Ruffer, Tim; Bailey, Helen; Dahlgren, Stefan; Spaven, Patrick; Winters, Mark (Evaluation of the market systems development approach, 2018)
This report presents the findings of an evaluation of Sida’s management of the market systems development (MSD) approach. It aims to inform thinking on how Sida can best manage its growing portfolio of MSD programs. Beyond this, it provides insights relevant to Sida’s wider support to complex and adaptive programs. The evaluation identified several factors that affect Sida’s ability to ensure that conducive conditions are in place for effective MSD programs and good development programming more generally. Sida’s relatively flexible framework of rules, guidelines and systems for project management provide the space needed for staff to innovate and manage adaptively. But for this to happen consistently and effectively, Sida needs to invest more deliberately in building the capacity of its staff in relevant areas. In addition, leadership and incentives are key to shaping a culture of active experimentation and learning. This needs to be supported with clearer guidance for those involved in the design and appraisal of MSD projects; and strengthened oversight of project performance, including through adjustments to Sida’s contracts and funding agreements.

Evaluation of the market systems development approach: Lessons for expanded use and adaptive management at Sida Volume II: Case studies - Ruffer, Tim; Bailey, Helen; Dahlgren, Stefan; Spaven, Patrick; Winters, Mark (Evaluation of the market systems development approach, 2018)
This report presents the findings of an evaluation of Sida’s management of the market systems development (MSD) approach. It aims to inform thinking on how Sida can best manage its growing portfolio of MSD programs. Beyond this, it provides insights relevant to Sida’s wider support to complex and adaptive programs. The evaluation identified several factors that affect Sida’s ability to ensure that conducive conditions are in place for effective MSD programs and good development programming more generally. Sida’s relatively flexible framework of rules, guidelines and systems for project management provide the space needed for staff to innovate and manage adaptively. But for this to happen consistently and effectively, Sida needs to invest more deliberately in building the capacity of its staff in relevant areas. In addition, leadership and incentives are key to shaping a culture of active experimentation and learning. This needs to be supported with clearer guidance for those involved in the design and appraisal of MSD projects; and strengthened oversight of project performance, including through adjustments to Sida’s contracts and funding agreements.

State of the Field Review: Fiscal Transparency and Accountability - Rudiger, Anja (International Budget Partnership, 2018)

Supporting Learning? Exploring the relationship between grantee learning and grantmaking practice in the transparency and accountability sector - Ross, Jenny (INTRAC, 2015)
Learning is fundamental to work on transparency and accountability in complex environments. But how can funding practices best support learning?

Complexity, Institutions and Public Policy: Agile Decision-making in a Turbulent World - Room, Graham (Edward Elgar Pub, 2011)
Graham Room argues that conventional approaches to the conceptualization and measurement of social and economic change are unsatisfactory. As a result, researchers are ill-equipped to offer policy advice. This book offers a new analytical approach, combining complexity science and institutionalism. It also provides tools for policy makers in turbulent times. Part 1 is concerned with the conceptualization of socio-economic change. It integrates complexity science and institutionalism into a coherent ontology of social and policy dynamics. Part 2 is concerned with models and measurement. It combines some of the principal approaches developed in complexity analysis with models and methods drawn from mainstream social and political science. Part 3 offers empirical applications to public policy: the dynamics of social exclusion; the social dimension of knowledge economies; the current financial and economic crisis. These are supplemented by a toolkit for the practice of 'agile policy making'. This is a stimulating, provocative and highly original book. It will appeal to academics and students in social and policy studies and to a wide range of scholars in other disciplines where complexity science is already well-developed. It will also be of major interest for decision makers coping with complex and turbulent policy terrains.Contents: Preface 1. Introduction Part I: Concepts 2. The Complexity Paradigm 3. Complex Adaptive Systems 4. The Economy as a Complex Adaptive System 5. Institutional Settings and Architectures 6. Institutional Dynamics 7. The Struggle for Positional Advantage 8. Conceptualising Social Dynamics Part II: Methods 9. Attractors and Orbits in Dynamic Systems 10. Patterns in Time and Space 11. Connections and Networks 12. Mobility on Social Landscapes 13. Towards a Generic Methodology Part III: Policies 14. Agile Policy-Making 15. Poverty and Social Exclusion 16. Social Dynamics of the Knowledge Economy 17. Global Turbulence and Crisis Postscript: Tools for Policy-Makers References Index

Citizen-generated data and sustainable development - Rono-Bett, Karen; Kenei, Steve (DevInit and DRT, 2017)
The potential of citizen-generated data in the sustainable development agenda

The politics of public sector performance: pockets of effectiveness in developing countries - Roll, Michael (Routledge research in comparative politics, 2014)

Theory of Change - Rogers, Patricia (Methodological Briefs: Impact Evaluation, 2014)

How to Make ‘Cash Plus’ Work: Linking Cash Transfers to Services and Sectors - Roelen, Keetie; Devereux, Stephen; Abdulai, Abdul-Gafaru; Martorano, Bruno; Palermo, Tia; Ragno, Luigi Peter (UNICEF Office of Research, 2017)
The broad-ranging benefits of cash transfers are now widely recognized. However, the evidence base highlights that they often fall short in achieving longer-term and second-order impacts related to nutrition, learning outcomes and morbidity. In recognition of these limitations, several ‘cash plus’ initiatives have been introduced, whereby cash transfers are combined with one or more types of complementary support. This paper aims to identify key factors for successful implementation of these increasingly popular ‘cash plus’ programmes, based on (i) a review of the emerging evidence base of ‘cash plus’ interventions and (ii) an examination of three case studies, namely, Chile Solidario in Chile, IN-SCT in Ethiopia and LEAP in Ghana. The analysis was guided by a conceptual framework proposing a menu of ‘cash plus’ components. The assessment of three case studies indicated that effective implementation of ‘cash plus’ components has indeed contributed to greater impacts of the respective programmes. Such initiatives have thereby addressed some of the non-financial and structural barriers that poor people face and have reinforced the positive effects of cash transfer programmes. In design of such programmes, further attention should be paid to the constraints faced by the most vulnerable and how such constraints can be overcome. We conclude with recommendations regarding the provision of complementary support and cross-sectoral linkages based on lessons learned from the case studies. More research is still needed on the impact of the many variations of ‘cash plus’ programming, including evidence on the comparative roles of individual ‘plus’ components, as well as the knowledge, attitudes and behaviour pathways which influence these impacts.

Bridging the Gap: Synthesising Evidence from Secondary Quantitative and Primary Qualitative Data - Roelen, Keetie (CDI Practice Paper, 2016)

Making the Most of Mess: Reliability and Policy in Today's Management Challenges - Roe, Emery (Duke University Press Books, 2013)
In Making the Most of Mess, Emery Roe emphasizes that policy messes cannot be avoided or cleaned up; they need to be managed. He shows how policymakers and other professionals can learn these necessary skills from control operators who manage large critical infrastructures such as water supplies, telecommunications systems, and electricity grids. The ways in which they prevent major accidents and failures offer models for policymakers and other professionals to manage the messes they face.Throughout, Roe focuses on the global financial mess of 2008 and its ongoing aftermath, showing how mismanagement has allowed it to morph into other national and international messes. More effective management is still possible for this and many other policy messes but that requires better recognition of patterns and formulation of scenarios, as well as the ability to translate pattern and scenario into reliability. Developing networks of professionals who respond to messes is particularly important. Roe describes how these networks enable the avoidance of bad or worse messes, take advantage of opportunities resulting from messes, and address societal and professional challenges. In addition to finance, he draws from a wide range of case material in other policy arenas. Roe demonstrates that knowing how to manage policy messes is the best approach to preventing crises.

Managing Policy Mess - Roe, Emery (Governance in Turbulent Times, 2016)
What are the conditions for political development and decay, and the likelihood of sustained political order? What are the limits of established rule as we know it? How much stress can systems tackle before they reach some kind of limit? How do governments tackle enduring ambiguity and uncertainty in their systems and environments? These are some of the big questions of our time. Governance in turbulent times may serve as a stress-test of well-known ways of governing in the 21st century. Governance in Turbulent Times discusses this pertinent challenge and suggests how governments and organizations cope with and live with turbulence. The book explores how organizations and institutions respond to precipitous, conflicting, and novel-in short, turbulent-governance challenges. This book is a comprehensive and ground-breaking endeavor to understand how governance systems respond to turbulent challenges, and how turbulent times provide excellent opportunities to investigate the sustainability of governance systems. The book illustrates how politics, administrative scale and complexity, uncertainty, and time constraints can collide to produce turbulence. Building on prior work in organization theory and political science, we argue that turbulence refers to four properties related to the interaction of demands for action: variability, consistency, expectation, and unpredictability. Turbulence occurs where the interaction of demands is experienced as highly variable, inconsistent, unexpected, and/or unpredictable.
Keywords: Comparative Politics; International Relations; Political Process; Political science; Public Affairs & Administration

Policy messes and their management - Roe, Emery (Policy Sciences, 2016)
This paper presents a framework for better managing policy messes and draws implications for bad and good mess management in policy analysis and management. The framework has three foci: (1) the cognitive space in which policy messes develop, particularly in terms of gaps between macro-designers and micro-operators; (2) the unique domain of competence within that space where professionals manage the resulting messes by virtue of their skills in recognizing system-wide patterns, formulating locally specific contingency scenarios and translating both patterns and scenarios in highly reliable services; and (3) the ability of those mess and reliability professionals to be reliable in their domain and with these skills by maneuvering across different performance modes as conditions dictate—just-in-case, just-on-time, just-for-now or just-this-way.

Beyond enforcement: communities, governance, incentives and sustainable use in combating wildlife crime. Symposium Report. - Roe, Dilys (IIED, 2015)

Social Learning and Natural Resource Management: The Emergence of Three Research Perspectives - Rodela, Romina (Ecology and Society, 2011)
Rodela, R. 2011. Social learning and natural resource management: the emergence of three research perspectives. Ecology and Society 16(4): 30. https://doi.org/10.5751/ES-04554-160430

Why Diverse Teams Are Smarter - Rock, David; Halvorson, Heidi Grant (Harvard Business Review, 2016)
Research shows they’re more successful in three important ways.

Monitoring and evaluation for adaptive programming - Roche, Chris; Kelly, Linda (Devpolicy Blog from the Development Policy Centre, 2018)
Chris Roche and Linda Kelly with six take-aways on what is being tried and learnt in setting up monitoring and evaluation frameworks for adaptive programs.

Overcoming Premature Evaluation - Roche, Chris (2016)
Guest post from Chris Roche on practical ways of introducing adaptive management, learning from failure etc in aid programmes

Simplicity, Accountability and Relationships: Three ways to ensure MEL supports Adaptive Management - Roche, Chris (From Poverty to Power, 2018)
Based on a recent discussion in Manila, Chris Roche reflects on how Monitoring, Evaluation and Learning can better support 'adaptive programming'.

Thinking and Working Politically through Applied Political Economy Analysis. A guide for practitioners - Rocha Menocal, Alina; Cassidy, Marc; Swift, Sarah; Jacobstein, David; Rothblum, Corinne; Tservil, Ilona (USAID, 2018)
Thinking and Working Politically (TWP) through Applied Political Economy Analysis (PEA). A guide for practitioners Have you ever done everything right in a development program — followed every technical best practice — but still missed the mark? When this happens, it often relates to factors in the context beyond any external development actor’s ability to control.

Hospicing The Old - Robinson, Cassie (Stewarding Loss, 2019)
In 2010 I was introduced to the Berkana Institutes’s Two Loop model, and I come back to it again and again. As I’ve moved across different…

Adaptive management: where are we now? - Rist, Lucy; Campbell, Bruce M.; Frost, Peter (Environmental Conservation, 2013)
Adaptive management (AM) emerged in the literature in the mid-1970s in response both to a realization of the extent of uncertainty involved in management, and a frustration with attempts to use modelling to integrate knowledge and make predictions. The term has since become increasingly widely used in scientific articles, policy documents and management plans, but both understanding and application of the concept is mixed. This paper reviews recent literature from conservation and natural resource management journals to assess diversity in how the term is used, highlight ambiguities and consider how the concept might be further assessed. AM is currently being used to describe many different management contexts, scales and locations. Few authors define the term explicitly or describe how it offers a means to improve management outcomes in their specific management context. Many do not adhere to the idea as it was originally conceived, despite citing seminal work. Significant confusion exists over the distinction between active and passive approaches. Over half of the studies reporting to implement AM claimed to have done so successfully, yet none quantified specific benefits, or costs, in relation to possible alternatives. Similarly those studies reporting to assess the approach did so only in relation to specific models and their parameterizations; none assessed the benefits or costs of AM in the field. AM is regarded by some as an effective and well-established framework to support the management of natural resources, yet by others as a concept difficult to realize and fraught with implementation challenges; neither of these observations is wholly accurate. From a scientific and technical perspective many practical questions remain; in particular real-world assessments of the value of experimentation within a management framework, as well as of identified challenges and pathologies, are needed. Further discussion and systematic assessment of the approach is required, together with greater attention to its definition and description, enabling the assessment of new approaches to managing uncertainty, and AM itself.
Keywords: Adaptive Management; Decision making; Natural resource management; conservation; experimental management; uncertainty

Making Sense of ‘Messiness’. Monitoring and measuring change in market systems: a practitioner's perspective - Ripley, Matthew; Nippard, Daniel (2014)

The Science in Adaptive Management - Ripley, Matt; Jaccard, Sabine (ILO, 2016)
‘Adaptive management’ is all the rage in international development circles. But to avoid yet another buzzword – we need to learn from the experience of natural resource science.
Keywords: IMPORTANT

Does Foreign Aid Really Work? An Updated Assessment - Riddell, Roger C. (Social Science Research Network, 2014)
This paper draws widely from the book Does foreign aid really work?, building on that discussion to provide an updated answer to the question based on recent evidence and contemporary debates on aid effectiveness. It starts with a brief discussion (Section 2) of the question: ‘does emergency aid work?’ This is important to the wider debate for two linked reasons. Firstly, the harshest critics of development aid are all supportive of emergency aid, with a number calling for its expansion in spite of evidence of major weaknesses and failures. Secondly, the sharp historical distinction made between emergency and development aid is becoming increasingly strained, as more emergency aid is being used a year or more after emergencies strike to rebuild lives and restore livelihoods, while more development aid is used to directly save lives.The rest of the paper focuses exclusively on development aid. Section 3 provides a rapid overview of the evidence of the impact of individual aid projects, including those of Non-Governmental Organisations (NGOs). The picture is overwhelmingly positive: well over 75% of projects meet their immediate objectives and impact has improved, even though sustaining benefits remains a challenge and there continue to be aid failures. Section 4 reviews the evidence of the wider and long-term impact of aid at the sectoral and country level, including a brief discussion of academic studies on aid and growth. Though there are still major gaps in our knowledge, the quality of the data is improving. However, there is little firm, quantitative evidence to show the specific contribution that aid has made to help achieve the Millennium Development Goals (MDGs), which are a central purpose of many current donor programmes (Section 4.2). More widely, there is plenty of evidence to suggest that aid has contributed positively to both growth and wider development at the sector and country level, though some studies suggest aid has had little or no impact. Overall, the contribution that aid makes to aggregate development is lower than the public has been led to believe (Sections 4.1, 4.3. and 4.4). Assessments of the overall impact of the aid provided by NGOs are rare, but a recent study suggests it has been positive (Section 4.5). More attention is now given to the issues of corruption in aid. Although there is evidence of aid funds being used for corrupt purposes and of aid worsening corruption, on balance it remains a minor issue. Aid has had significant successes in helping the poor affected by corruption (Section 4.6). The second half of the paper shifts focus to the large gap between what aid has done and what it might do. Section 5 discusses a range of inefficiencies within and across the aid system and their costs in terms of reduced impact, including the way aid is allocated, its growing complexity, and the volatility and unpredictability of aid flows (Section 5.1 and 5.2). These inefficiencies place in a different light the evidence of aid’s overall positive impact. The paper looks at the different initiatives that have been mounted to begin to address these weaknesses and failures, including the 2005 Paris Declaration, and discusses why donors have failed to honour the promises they have made to change the ways that they give aid.In Section 5.3, the paper argues that the most critical debate about whether aid works concerns the assessment of whether the short-term, immediate and extensive benefits that aid undoubtedly brings are outweighed by the direct and indirect systemic problems that it risks creating or accentuating. As increasingly over the last decade donors have channelled more of their aid into short-term, quick-impact projects, assessing the wider negative systemic effects of aid has become even more important. Some recent studies suggest that aid’s systemic problems are large and growing, narrowing the gap between aid’s harshest critics and broader analyses of aid impact. Against the backdrop of already too many proposals of how to make aid work better, Section 6 lays out nine concrete proposals for doing this: deepening knowledge of local contexts; ensuring short-term uses of aid are consistent with and supportive of long-term development, and that all aid is more closely related to overall recipient development goals and processes; helping build local capacities for recipients to be able to coordinate aid better; moving from rhetoric to reality in learning lessons from aid; using aid to help the poor in middle income countries; reducing volatility in aid at the country level; encouraging budget support by addressing donor-country political concerns; and rethinking ways of communicating about aid. Section 7 concludes. It suggests that, paradoxically, aid’s impact may well have been harmed by focussing too narrowly on trying to make short-term aid work better, and that the main focus of attention needs to widen to assess how aid can contribute more to a recipient’s own development goals. Additionally, donors need to help build the capacity of developing countries and developing country scholars to enable them to play a bigger role in helping to answer the question of whether aid works; unsettling though their assessments may be.

Models for Foresight Use in International Development - Rhydderch, Alun (IDS Bulletin, 2016)
This article sets out the components of the foresight approach that has been adopted by many governments in the developed world, and identifies elements of this 'dominant' approach that may hinder its uptake in developing countries. Instead, it suggests that a less rigid, more exploratory and normative approach may be better suited to many developing country contexts. With reference to the writings and practice of the creator of 'la prospective', Gaston Berger, it argues for an attitude that combines bold and inclusive thinking about how to create better futures with the pragmatic engagement with political and administrative systems that can help bring these about.

(Breaking) The Iron Triangle of Evaluation - Reynolds, Martin (IDS Bulletin, 2015)
Keywords: Evaluation; impact

Integrating modelling and smart sensors for environmental and human health - Reis, Stefan; Seto, Edmund; Northcross, Amanda; Quinn, Nigel W. T.; Convertino, Matteo; Jones, Rod L.; Maier, Holger R.; Schlink, Uwe; Steinle, Susanne; Vieno, Massimo; Wimberly, Michael C. (Environmental Modelling & Software, 2015)
Sensors are becoming ubiquitous in everyday life, generating data at an unprecedented rate and scale. However, models that assess impacts of human activities on environmental and human health, have typically been developed in contexts where data scarcity is the norm. Models are essential tools to understand processes, identify relationships, associations and causality, formalize stakeholder mental models, and to quantify the effects of prevention and interventions. They can help to explain data, as well as inform the deployment and location of sensors by identifying hotspots and areas of interest where data collection may achieve the best results. We identify a paradigm shift in how the integration of models and sensors can contribute to harnessing ‘Big Data’ and, more importantly, make the vital step from ‘Big Data’ to ‘Big Information’. In this paper, we illustrate current developments and identify key research needs using human and environmental health challenges as an example.
Keywords: Big data; Environmental health; Environmental sensors; Integrated modelling; Population health

Confronting the Contradiction - An exploration into the dual purpose of accountability and learning in aid evaluation - Reinertsen, Hilde; Bjørkdahl, Kristian; McNeill, Desmond (Rapport 2017:06, 2017)
Keywords: IMPORTANT

Exploring ways to reconcile accountability and learning in the evaluation of niche experiments - Regeer, Barbara J.; de Wildt-Liesveld, Renée; van Mierlo, Barbara; Bunders, Joske F. G. (Evaluation, 2016)
While evaluation is seen as a mechanism for both accountability and learning, it is not self-evident that the evaluation of niche experiments focuses on both accountability and learning at the same time. Tensions exist between the accountability-oriented needs of funders and the learning needs of managers of niche experiments. This article explores the differences in needs and expectations of funders and managers in terms of upwards, downwards and internal accountability. The article shows that as the multi-stakeholder contexts in which niche experiments take place give rise to various requirements, tensions in evaluation are essentially a specific manifestation of tensions between niche experiments and their multiple contexts. Based on our findings, an adjusted accountability framework is proposed, including several strategies that can reconcile a learning approach with accountability needs in niche experiments aiming to change current practices in a more sustainable direction.

Conjunctions: Introducing Cultural Participation as a Transdisciplinary Project - Reestorff, Camilla Møhring; Fabian, Louise; Fritsch, Jonas; Stage, Carsten; Stephensen, Jan Løhmann (Conjunctions. Transdisciplinary Journal of Cultural Participation, 2014)
In the introduction of Conjunction: Transdisciplinary Journal of Cultural Participation we introduce 1) the aim of the journal, 2) the journal’s conception of transdisciplinarity as an important precondition for understanding contemporary processes and dilemmas of participation, 3) important trajectories in the existing literature on participation that focus on participation as linked to technological changes, to democratic processes of transferring power, and to complex social situations calling for analytical and evaluative frameworks able to grasp multiplicity and competing interests, and 4) the theme and articles of the this special issue: cultural participation and citizenship.
Keywords: Cultural participation; Participatory citizenship; Power dynamics; democracy; transdisciplinarity

The Truth of the Work: Theories of Change in a changing world - Reeler, Doug; van Blerk, Rubert (The Community Development Resource Association, 2017)

What is Social Learning? - Reed, Mark; Evely, Anna; Cundill, Georgina; Fazey, Ioan; Glass, Jayne; Laing, Adele; Newig, Jens; Parrish, Brad; Prell, Christina; Raymond, Chris; Stringer, Lindsay (Ecology and Society, 2010)
Reed, M. S., A. C. Evely, G. Cundill, I. Fazey, J. Glass, A. Laing, J. Newig, B. Parrish, C. Prell, C. Raymond, and L. C. Stringer. 2010. What is social learning? Ecology and Society 15(4): r1. https://doi.org/10.5751/ES-03564-1504r01

Implementing Innovation: A User's Manual for Open Government Programs - Reboot Design (2015)
This guide draws from our experience around the world helping government reformers achieve real change. It is a practical resource for anyone working to implement an open government initiative.

Using Data to Influence Government Decisions. Opportunities and Challenges for User-Centered Design to Improve Governance Data Impact - Reboot (Reboot & OSF, 2015)

Strategic framework for mainstreaming citizen engagement in World Bank Group operations : engaging with citizens for improved results - Rebolledo, Miguel; Seyedian, Aaron; Peixoto, Tiago; Hernandez, Zenaida; Zakhour, Jad; Mahmood, Syed A.; Masud, Harika; Manroth, Astrid; Hamad, Qays (The World Bank, 2014)
The objective of this strategic framework is to mainstream citizen engagement in World Bank Group (WBG)-supported policies, programs, projects, and advisory services and analytics to improve their development results and within the scope of these operations, contribute to building sustainable national systems for citizen engagement with governments and the private sector. This framework will capture the diverse experiences, assess lessons learned, and outline methods and entry points to provide a more systematic and results-focused approach for the WBG. Progress toward this objective will be assessed using indicators included in program, project, and corporate results frameworks. The WBG strategy incorporates citizen engagement, including beneficiary feedback, specifically in its treatment of inclusion, which entails empowering citizens to participate in the development process and integrating citizen voice in development programs as key accelerators to achieving results. This framework builds on stocktaking and lessons learned from WBG-financed operations across regions and sectors. A key lesson is the importance of country context, government ownership, and clear objectives for citizen engagement. The approach to mainstreaming citizen engagement in WBG-supported operations is guided by five principles: 1) it is results-focused; 2) it involves engaging throughout the operational cycle; 3) it seeks to strengthen country systems; 4) it is context-specific; and 5) it is gradual. Under the right circumstances, citizen engagement can contribute to achieving development outcomes in support of the goals the WBG aims to support through all of the operations it funds: eradicating extreme poverty and boosting shared prosperity in a sustainable manner.

Retrospective ‘Outcome Harvesting’: Generating robust insights about a global voluntary environmental network - Rassmann, Kornelia; Smith, Richard; Mauremootoo, John; Wilson-Grau, Ricardo (Better Evaluation, 2013)

Adapting lean thinking to market systems development: Principles and practices for donors/funders - Rasmussen, Leanne (Engineers without borders, 2014)
The purpose of this paper is to provide funders and implementers of market systems development (MSD) projects with principles, practices, and structures that enables these projects to thrive. It is based on a book that has sold millions of copies worldwide, and a school of thought taught at institutions such as Harvard Business School and practiced in companies ranging from giants such as Toyota to the most successful tech start-ups in Silicon Valley. Published in 2011, Eric Ries wrote the Lean Startup as a remedy to the countless start-ups that create their own demises by getting started with the wrong goals, the wrong structures, and the wrong processes. He outlines an approach that enables the startup to navigate ambiguity and risk while using resources effectively. The conditions a MSD project faces are remarkably similar to those which a startup business faces; it needs structures and processes that are matched to the unpredictable, complex environment which it is attempting to influence. MSD projects can thus benefit from private-sector thought leadership on how start-ups can situate themselves for success. This paper starts with an overview of lean thinking, a concept that derives from lean manufacturing which is widely accepted as the leading approach to modern manufacturing. It explains how Ries’s Lean Startup adapts these principles, and then it in turn adapts these ideas for market systems development. The following three sections provide an overview of Lean Startup concepts, applying them to the development sector as appropriate. The final section summarizes and makes suggestions on next steps for funders who wish to set MSD projects up for success using a lean approach. Overall, the paper builds a case for how the Lean Startup’s approach can enable MSD projects to work successfully in ambiguity and increase their potential for achieving robust and sustainable results, all while using donor resources more efficiently.
Keywords: Adaptive Development; Lean

Problem-driven iterative approaches and wider governance reform - Rao, Sumedh (Helpdesk Research Report 1099, 2014)
Identify evidence which suggests that problem-driven, iterative approaches to public sector reform can deliver more substantial, wider, long-term governance reform.

Accounting for a changing and uncertain climate in planning and policymaking today: lessons for developing countries - Ranger, Nicola; Garbett-Shiels, Su-Lin (Climate and Development, 2012)
Climate change is increasingly altering the pattern of climate-related risks. Developing countries and in particular least developed countries will be among the most severely impacted by climate change. These risks can seem remote in comparison with more immediate threats and needs, but if climate change is not considered upfront in existing planning and policymaking processes today, decision makers risk locking-in future impacts that may prove irreversible or much more costly and difficult to rectify than is necessary. The challenge for planners and policymakers, explored in this paper, is that future climate conditions are deeply uncertain. Decision methods are available to tackle these problems; however, these tend to be data- and resource-intensive and therefore, difficult to routinely apply. Further, a gap in currently available guidance is the explicit link to the adaptation needs of a developing country. We discuss the implications of this development context for the priorities for adaptation and the relative allocation of efforts in adaptation. This paper focuses on the identification of adaptation options and strategies that are robust to the deep uncertainties in future climate risk, culminating in a framework of six building blocks. It takes the perspective of exploring how decisions today might be adjusted to account for uncertain and changing long-term climate risks. We suggest a core principle is to focus on promoting climate-resilient development and increasing long-term adaptive capacity while, crucially, avoiding inflexible decisions that could lock-in future climate risk or foreclose adaptation options.
Keywords: Adaptation; Climate change; Development; Planning; robustness; uncertainty

Utilization focused evaluation: a primer for evaluators - Ramirez, Ricardo; Brodhead, Dal (Southbound, 2013)
Ramírez, R., & Brodhead, D. (2013). Utilization focused evaluation: A primer for evaluators. Penang, Malaysia: Southbound. Retrieved from http://evaluationinpractice.files.wordpress.com/2013/04/ufeenglishprimer... This book, authored by Ricardo Ramírez and Dal Brodhead, is designed to support evaluators and program managers implement Utilization-focused evaluation (UFE). It includes detailed discussion of the 12 steps for implementing UFE and also provides a number of case studies to guide the user. Excerpt "Throughout this Primer we refer to the value of having a mentor to assist an evaluator who is using UFE for the first time. Our collective experiences with UFE indicated having a mentor was, for many UFE participants, an essential support and it reflects how we learned and mentored UFE. Evaluators may use elements of a UFE in their work naturally, for example by engaging users in planning the process or in assisting them in the utilization of findings. This Primer, however, walks the reader through UFE by systematically covering all of the 12 steps. It reflects deeply on the UFE evaluation practice and builds from it." (Ramírez & Brodhead 2013) Contents What is Utilization Focused Evaluation? 1 The UFE Framework Summarized in Steps 3 The DECI Project 6 The UFE Steps Illustrated with Project Examples 9 Step 1 Assessing Program Readiness 10 Step 2 Assessing Evaluators’ Readiness 18 Step 3 Identifying Primary Intended Users 24 Step 4 Situational Analysis 31 Step 5 Identification of Primary Intended Users 38 Step 6 Focusing the Evaluation 44 Step 7 Evaluation Design 54 Step 8 Simulation of Use 58 Step 9 Data Collection 62 Step 10 Data Analysis 65 Step 11 Facilitation of Use 68 Step 12 Meta Evaluation 72 Summary About What Each Step Entails 76 What Benefit Does UFE Bring to Commissioners of Evaluation? 81 Take Away Lessons 83 Postscript 87 The UFE Checklist 89 Case Studies 103 Recommended Reading 110

Making adaptive rigour work - Principles and practices for strenghening monitoring, evaluation and learning for adaptive management - Ramalingam, Ben; Wild, Leni; Buffardi, Anne L (ODI/GLAM, 2019)
Adaptive programmes can be accountable, rigorous and high quality in how they use evidence by taking an ’adaptive rigour’ approach. Core development and humanitarian challenges are complex, and require processes of testing, learning and iteration to find solutions – adaptive management offers one approach for this. Yet large bureaucracies and development organisations can have low tolerance for experimentation and learning, and adaptive management can be viewed as an excuse for ‘making things up as you go along’. This briefing from the Global Learning for Adaptive Management (GLAM) initiative argues that adaptive programmes can be accountable, rigorous and high quality in how they use evidence – but this requires rethinking some key assumptions about how they are practised. The paper sets out three key elements of an ‘adaptive rigour’ approach: - Strengthening the quality of monitoring, evaluation and learning data and systems. - Ensuring appropriate investment in monitoring, evaluation and learning across the programme cycle. - Strengthening capacities and incentives to ensure the effective use of evidence and learning as part of decision-making, leading ultimately to improved effectiveness.

Strengthening the Humanitarian Innovation Ecosystem - Ramalingam, Ben; Rush, Howard; Bessant, John; Marshall, Nicholas; Gray, Bill; Hoffman, Kurt; Bayley, Simon; Warren, Kim (University of Brighton, 2015)

From best practice to best fit: understanding and navigating wicked problems in international development - Ramalingam, Ben; Laric, Miguel; Primrose, John (Working Paper, 2014)

Ten Frontier Technologies for International Development - Ramalingam, Ben; Hernandez, Kevin; Prieto Martin, Pedro; Faith, Becky (IDS, 2016)
As new technologies and digital business models reshape economies and disrupt incumbencies, interest has surged in the potential of novel frontier technologies to also contribute to positive changes in international development and humanitarian contexts. Widespread adoption of new technologies is acknowledged as centrally important to achieving the United Nations Sustainable Development Goals by 2030. But while frontier technologies can rapidly address large-scale economic, social or political challenges, they can also involve the displacement of existing technologies and carry considerable uncertainty and risk. Although there have been significant wins bringing the benefits of new technologies to poor consumers through examples such as mobile money or off-grid solar energy, there are many other areas where the applications may not yet have been developed into viable market solutions, or where opportunities have not yet been taken up in development practice.

Innovation for International Development: Navigating the Paths and Pitfalls - Ramalingam, Ben; Bound, Kirsten (NESTA, 2016)
Experiences, insights and practical advice from over 20 leading practitioners in innovation for international development, brought together in one collection.

Bridging Real-Time Data and Adaptive Management in International Devepment - Case Study Report - Ramalingam, Ben; Barnett, Inka; Valters, Craig; Oppenheimer, Carrie; Levy, Anna; Hernandez, Kevin (IDS, 2017)

Aid on the Edge of Chaos: Rethinking International Cooperation in a Complex World - Ramalingam, Ben (Oxford University Press, 2014)
It is widely recognised that the foreign aid system - of which every country in the world is a part - is in need of drastic overhaul. There are conflicting opinions as to what should be done. Some call for dramatic increases to achieve longstanding promises. Others bang the drum for cutting it altogether, and suggest putting the fate of poor and vulnerable people in the hands of markets or business. A few argue that what is needed is creative, innovative transformation. The arguments in Aid on the Edge of Chaos are firmly in the third of these categories. In this ground-breaking book, Ben Ramalingam shows that the linear, mechanistic models and assumptions that foreign aid is built on are more at home in early twentieth century industry than in the dynamic, complex world we face today. The reality is that economies and societies are less like machines and more like ecosystems. Aid on the Edge of Chaos explores how thinkers and practitioners in economics, business, and public policy have started to embrace new, ecologically literate approaches to thinking and acting, informed by the ideas of complex adaptive systems research. It showcases insights, experiences, and dramatic results of a growing network of practitioners, researchers, and policy makers who are applying a complexity-informed approach to aid challenges. From transforming approaches to child malnutrition, to rethinking process of macroeconomic growth, from rural Vietnam to urban Columbia, Aid on the Edge of Chaos shows how embracing the ideas of complex systems thinking can help make foreign aid more relevant, more appropriate, more innovative, and more catalytic. It argues that taking on these ideas will be a vital part of the transformation of aid, from a post-WW2 mechanism of resource transfer, to a truly innovative and dynamic form of global cooperation fit for the twenty-first century.

Development innovation: Fad, silo or catalyst? - Ramalingam, Ben (Nesta, 2015)
In the first of a two-part blog piece, Ben Ramalingam, affiliate of the Overseas Development Institute / Institute of Development Studies, discusses the challenges of implementing innovation in to the development sector - from keeping expectations realistic to managing the innovation process and assessing impact and value.

Development innovation: Taking the high road - Ramalingam, Ben (Nesta, 2015)
In part two of his blog on the challenges facing development innovation, Ben Ramalingam suggests how we might take the ‘high road’ to bring about long-lasting and meaningful innovation in the sector.

Learning how to learn: eight lessons for impact evaluations that make a difference - Ramalingam, Ben (ODI, 2011)
This Background Note outlines key lessons on impact evaluations, utilisation-focused evaluations and evidence-based policy. While methodological pluralism is the key to effective impact evaluation in development, the focus here is on the factors that need to be considered for impact evaluations to be used in policy and practice – regardless of the method employed.

Learning to Adapt: Building Adaptive Management as a Core Competency in Development Practice - Ramalingam, Ben (Institute of Development Studies, 2016)
Key messages 1. Development policies and programs increasingly operate in situations of high complexity and uncertainty. 2. There are growing efforts across the sector to design, plan and implement more adaptive responses that are more relevant and appropriate in such contexts 3. At the heart of effective adaptive programming is the capability to gather, interpret and use knowledge, information and data in real-time 4. Strengthening this capability requires a positive enabling environment—including tools and methods, structures and processes, leadership and management and culture and mindsets. 5. For adaptive development to succeed, there needs to be greater attention and investment in both knowledge, information and data systems and in appropriate enabling environments
Keywords: IMPORTANT

Manifesto for Half-Arsed Development Reforms - Ramalingam, Ben (Aid on the Edge of Chaos, 2014)
Keywords: IMPORTANT

What Is a Case?: Exploring the Foundations of Social Inquiry - Ragin, Charles (Cambridge University Press, 2010)
The concept of the case is a basic feature of social science research and yet many questions about how a case should be defined, selected, and judged are far from settled. The contributors to this volume probe the nature of the case and the ways in which different understandings of the concept affect the conduct and the results of research. The contributions demonstrate that the work of any given researcher is often characterised by some hybrid of these basic approaches, and it is important to understand that most research involves multiple definitions and uses of cases, as both specific empirical phenomena and as general theoretical categories.

Emerging Opportunities: Monitoring and Evaluation in a Tech-Enabled World - Raftree, Linda; Bamberger, Michael (Discussion Paper, 2014)
Monitoring and evaluation practice is using information and communication technologies for more timely data, and more inclusive voice and feedback.

What are you afraid of: Collective leadership and its learning implications - Raelin, Joseph A (Management Learning, 2018)
In this provocation, the author attempts to cite the advantages of collective leadership while acknowledging the objections and fears of challengers. Collective leadership is seen as remote because it defies the traditional view of leadership as an individualistic attractive quality that not only protects us but is efficient when applied. Nevertheless, the collective alternative may not only be advisable but required in a connected world featuring a networked economy. The contemporary socio-politico-economic environment requires the contribution of, creativity from, and collaboration among multiple agents providing a dynamic concentration of management and knowledge. If we are to accept and recognize the contribution of a collective leadership, its development would require an entirely different learning model. In particular, collective leadership development occurs as an acute immersion into the practices that are embedded within in situ material–discursive relations—in other words, among people, objects, and their institutions.

A Call for Innovation in International Development - R4D (Results for Development, 2015)

Use of environmental sensors and sensor networks to develop water and salinity budgets for seasonal wetland real-time water quality management - Quinn, Nigel W. T.; Ortega, Ricardo; Rahilly, Patrick J. A.; Royer, Caleb W. (Environmental Modelling & Software, 2010)
Management of river salt loads in a complex and highly regulated river basin such as the San Joaquin River Basin of California presents significant challenges for current Information Technology. Computer-based numerical models are used as a means of simulating hydrologic processes and water quality within the basin and can be useful tools for organizing Basin data in a structured and readily accessible manner. These models can also be used to extend information derived from environmental sensors within existing monitoring networks to areas outside these systems based on similarity factors – since it would be cost prohibitive to collect data for every channel or pollutant source within the Basin. A common feature of all hydrologic and water quality models is the ability to perform mass balances. This paper describes the use of a number of state-of-the-art sensor technologies that have been deployed to obtain water and salinity mass balances for a 60,000ha tract of seasonally managed wetlands in the San Joaquin River Basin of California. These sensor technologies are being combined with more traditional environmental monitoring techniques to support real-time salinity management (RTSM) in the River Basin. Two of these new technology applications: YSI-Econet (which supports continuous flow and salinity monitoring of surface water deliveries and seasonal wetland drainage); and electromagnetic salinity mapping (a remote sensing technology for mapping soil salinity in the surface soils) – have not previously been reported in the literature. Continuous sensor deployments that experience more widespread use include: weather station sensor arrays – used to estimate wetland pond evaporation and moist soil plant evapotranspiration; high resolution multi-spectral imagery – used to discriminate between and estimate the area of wetland moist soil plant vegetation; and groundwater level sensors – used primarily to estimate seepage losses beneath a wetland pond during flood-up. Important issues associated with quality assurance of continuous data are discussed and the application of a state-of-the-art software product AQUARIUS, which streamlines the process of data error correction and dissemination, is described as an essential element of ensuring successful RTSM implementation in the San Joaquin River Basin.
Keywords: Environmental decision support; Forecasting; Sensor networks; Sensors; Water quality

Snapshot of Adaptive Management in Peacebuilding Programs - Queen, Emily Forsyth; Baumgardner-Zuzik, Jessica; Hume, Elizabeth; Greenberg, Melanie (Alliance for Peacebuilding, 2018)
The Alliance for Peacebuilding developed the report Snapshot of Adaptive Management in Peacebuilding Programs with support from Humanity United. This report examines how existing organizational programs are designing and learning from adaptive management in a conflict effected and fragile states. It further provides practical recommendations for applying adaptive management to peacebuilding programs based on synthesis … Continue reading "Snapshot of Adaptive Management in Peacebuilding Programs"
Keywords: KEY

Connect. Learn. Adapt. Repeat - Practicing adaptive management in complex, conflict-affected environments: barriers and promising practices - Queen, Emily Forsyth (2018)
To shift aid culture toward adaptive management, we can: - Stay humble, empathetic, and build skills in facilitation and listening. - Focus on working with more flexible foundations or individual donors. Or, work for more rigid donors and help make their policies more flexible. - Get clearer about goals and processes while finding ways to be less specific and more committed to local leadership about pathways to get to the goal. - Resist looking for a magic adaptive management tool and instead get clearer about when teams make what kinds of decisions and matching to tools that support that decision-making. - Validate that everyone’s perspective is a form of data and try out simple ways to document how teams learns and changes. - Broaden and ground the idea of expertise – remembering that, especially when working with folx on the margins, people are the only experts on their own life. Practitioners must also examine how dominant patriarchal, white supremacist, colonial ideas have pushed us toward wanting more control, less flexibility, and less space for equitable decision making.

Fall in love with the solution, not the problem - Quaggiotto, Giulio; Leurs, Bas; Hazeldine, Shaun (Nesta, 2016)
Three strategies for international development organisations to solve problems without defining them.

Innovation at UNDP: from weekend sport to daily practice - Quaggiotto, Giulio; Begovic, Mellie (Nesta, 2016)
The experience of working to embed innovation approaches in everyday practice at UNDP's country offices
Keywords: IMPORTANT

Building elevators for development mutants - Quaggiotto, Giulio (Disrupt & Innovate, 2016)
I recently had the opportunity to learn about General Mill’s (the US food giant) “emerging brands elevator” program (also known as 301 Inc). Traditionally, General Mills has grown either through mergers and acquisitions, or by building new businesses from the ground up. Increasingly, however, it found that small brands were much faster at innovation, so … Continue reading Building elevators for development mutants

Profiling the international development mutants - Quaggiotto, Giulio (Development Impact and You, 2017)
As part of a new series, we talk to Giulio Quaggiotto about the development ‘mutants’ – innovators working at the edge of the field.

The era of development mutants - Quaggiotto, Giulio (Nesta, 2016)
If you were looking for the cutting edge of the development sector, where would you go these days?

Innovation teams and labs. A Practice Guide - Puttick, Ruth (Nesta, 2014)

Engines of Change: What Civic Tech can learn from Social Movements - Purpose; Omidyar Network (Omidyar Network and Purpose, 2016)

Straws-in-the-wind, Hoops and Smoking Guns: What can Process Tracing Offer to Impact Evaluation? - Punton, Melanie; Welle, Katharina (Centre for Development Impact, 2015)
This CDI Practice Paper by Melanie Punton and Katharina Welle explains the methodological and theoretical foundations of process tracing, and discusses its potential application in international development impact evaluations. It draws on two early applications of process tracing for assessing impact in international development interventions: Oxfam Great Britain (GB)’s contribution to advancing universal health care in Ghana, and the impact of the Hunger and Nutrition Commitment Index (HANCI) on policy change in Tanzania. In a companion to this paper, Practice Paper 10 Annex describes the main steps in applying process tracing and provides some examples of how these steps might be applied in practice.

Reflections from a Realist Evaluation in Progress: Scaling Ladders and Stitching Theory - Punton, Melanie; Vogel, Isabel; Lloyd, Rob (CDI Practice Paper 18, 2016)
Realist evaluation provides valuable insights into how and why programmes lead to change, and can generate transferable lessons to help practitioners roll out or scale up an intervention. However, as ...

Adaptive Programming in Fragile, Conflict and Violence-Affected Settings, What Works and Under What Conditions?: The Case of PERL, Nigeria - Punton, Melanie; Burge, Richard (Action for Empowerment and Accountability Research Programme, 2018)
This paper examines adaptive approaches to aid programming in Nigeria. Through field research and desk reviews, we have investigated some of the assertions around the ‘adaptive management and programming’ approach, which has arisen in recent years as a response to critiques of overly rigid, pre-designed, blueprint and linear project plans. This is the second of three case studies in a series which explore if and how adaptive approaches, including rapid learning and planning responses, are particularly relevant and useful for promoting empowerment and accountability in fragile, conflict and violence-affected settings (FCVAS). This case study focuses on PERL (Partnership to Engage, Reform and Learn) in Nigeria, a five-year governance programme (2016-21) with a total budget of £100 million. It aims to promote better service delivery through bringing together government and citizens groups to collectively address governance challenges. PERL is viewed by DFID as the final stage of a 20-year investment, building on learning, experience and partnerships from 15 years of DFID-funded governance programming in Nigeria. It was designed to incorporate learning and adaptation through a ‘living’ theory of change, continuous political economy analysis at different levels, adaptive implementation by location-based delivery teams who are encouraged to be flexible and let partners take the lead, regular learning and reflection, and adaptive resourcing, HR and administrative systems. The case study draws on a conceptual framework (the ‘adaptive triangle’) that looks at three types of adaptation – adaptive management, adaptive programming and adaptive delivery – and the interconnections and tensions between them.
Keywords: A4EA; Adaptive Development; Economy; Fishery

Procuring and Managing Adaptively: 5 Case Studies of Adaptive Mechanisms - Pryor, Tony (USAID Learning Lab, USAID, 2018)
Flexibility in program management is essential in all of the countries where USAID works. This is especially true in non-permissive environments (NPEs), where the ability to learn and adapt quickly to changing circumstances can help USAID staff members achieve their desired outcomes.

It's All About MeE: Using Structured Experiential Learning (“e”) to Crawl the Design Space - Pritchett, Lant; Samji, Salimah; Hammer, Jeffrey (Working Paper 322, 2013)
This paper argues that within-project variations in design can serve as their own counterfactual, reducing the incremental cost of evaluation and increasing the direct usefulness of evaluation to implementing agencies. It suggests combining monitoring (‘M’), structured experiential learning (‘e’), and evaluation (‘E’) so as to facilitate innovation and organisational capability building while also providing accountability …

How did China create “Directed Improvisation”? - Pritchett, Lant (Building State Capability, 2017)
written by Lant Pritchett Yuen-Yuen Ang, a Professor of Political Science at University of Michigan came to speak at Harvard the other day and I was lucky enough to hear her presentation.  Her most…
Keywords: IMPORTANT

The Debate about RCTs in Development is over. We won. They lost. - Pritchett, Lant (DRi Seminar, 2018)
There has been a debate in development economics over the last 20 years as some claimed the use of RCTs as a tool for independent impact evaluation would significantly improve development practice and hence development. While right about the methodological claims about the superiority of randomization to produce cleaner estimates of the LATE (local average treatment effect) of projects and programs, this, in and of itself, does not change development practice. All of the five claims needed to sustain a positive model in which RCT/IIE has a major positive impact are demonstrably false. The proponents of RCTs have responded to losing the first round decisively by changing significantly both their claims and their practice.

Between logframes and theory of change: reviewing debates and a practical experience - Prinsen, Gerard; Nijhof, Saskia (Development in Practice, 2015)
Theory of Change (ToC) is an emerging methodology in the practice of development programmes, often contrasted with the dominant logical framework. This article reviews current debates around ToC before identifying five aspects that are appreciated in practice. It appears that these aspects mostly cover areas where the logical framework is not – or is no longer – meeting the needs of practitioners. Subsequently, the article analyses experiences in ToC training for NGO staff and concludes that ToC can address shortcomings of the logical framework – if only by going back to some of the roots of the logical framework.

Caracterizando la participación ciudadana en el marco del Gobierno Abierto - Prieto-Martín, Pedro; Ramírez-Alujas, Álvaro V. (Revista del CLAD. Reforma y Democracia, 2014)
Este artículo profundiza conceptualmente en las dimensiones del Gobierno Abierto, centrándose en el análisis y caracterización de su dimensión participativa. Se propone, en primer lugar, una tipología de actos participativos que muestra el efecto potenciador que las TIC ejercen sobre las distintas formas de participación. En segundo lugar, se presentan los denominados “Esquemas de participación” (EdP), un instrumento conceptual que facilita el análisis y la representación estandarizada de las dimensiones más importantes de la participación. Los EdP proporcionan así, por un lado, un modelo de categorización que extiende nuestra comprensión crítica de la participación y, por otro lado, una herramienta potente y flexible para la comunicación, el diseño y la evaluación de diversos tipos de iniciativas participativas.

Doing Digital Development Differently: Lessons in adaptive management from technology for governance initiatives in Kenya - Prieto Martin, Pedro; Hernandez, Kevin; Faith, Becky; Ramalingam, Ben (MAVC Research Report, 2017)
Development projects don’t always work as planned. This has long been acknowledged by those in the sector, and has led to several approaches that seek to solve complex development problems through enabling and encouraging greater adaptiveness and learning within projects (e.g. Doing Development Differently and Problem-Driven Iterative Adaptation). Digital development projects experience many of these issues. Using technology for transparency and accountability (Tech4T&A) projects in Kenya as case studies, our research analysed the many different theoretical approaches to learning and adaptation, and then tested how these play out in reality. Firstly, we conducted an extensive review of the literature on the spectrum of approaches to adaptive learning. The findings were used to develop a framework through which to analyse adaptiveness at the different layers of complexity in projects (e.g. software design and development, programme design and management). The second part of the research consisted of interviews and focus group discussions with participants in Tech4T&A projects in Kenya. Respondents helped us identify the main characteristics of adaptiveness in their projects (e.g. who needs to adapt, and how and when) and the challenges and issues that inhibit projects’ abilities to be adaptive. This process also revealed how accountability interplays with adaptiveness, and considered how better collaboration flows can enable adaptiveness. From our literature review and empirical study, we draw several conclusions for increasing adaptiveness in digital development projects. These include simplifying the many adaptive theories that abound, increasing responsiveness to project beneficiaries and stakeholders, and for stakeholders to keep on experimenting, networking and advocating.
Keywords: IMPORTANT

Interview protocol: Most significant turning points - Prieto Martin, Pedro; Faith, Becky (Institute of Development Studies, 2017)
This interview protocol was used for a research project on adaptiveness in technology for governance initiatives in Kenya. For more information, please read the research report at: Prieto Martin, P.; Hernandez, K.; Faith, B. and Ramalingam, B. (2017) Doing Digital Development Differently: Lessons in adaptive management from technology for governance initiatives in Kenya, MAVC Research Report, Brighton: Institute of Development Studies, ids.ac.uk/project/making-all-voices-count
Keywords: IMPORTANT

Adaptiveness through ‘simplest tools and practices’ - Prieto Martin, Pedro (Institute of Development Studies, 2017)
In recent years, more and more influential development organisations have been openly recognising the central role that adaptive management capacities – the ability to keep improving strategies and actions as programmes unfold – play for the success of complex interventions. As a result, there has been a rich exchange of ideas and experiences on how to promote adaptiveness in development. But development organisations and professionals are having a hard time translating the many adaptive theories and recommendations into improved practices and outcomes. They are generally failing to adjust the design and operation of programmes to accommodate incipient lessons learnt and the unexpected changes in the context. Nowadays, most programme adaptations still take the form of a change of direction once it is clear that things have gone really wrong. Why is so difficult to put what we know about adaptiveness into practice? And more importantly: what can be done to improve this situation?

Participation Schemas: a tool to characterize collaborative participation - Prieto-Martín, Pedro (PDD2014, Contemporary difficulties and future prospects for Participatory and Deliberative Democracy, NewCastle, 2014)

READ ME - HOW TO USE THIS AM LIBRARY - Prieto Martin, Pedro (Institute of Development Studies, 2019)

How Does Developing Enforcement Capacity Reduce Wildlife Crime? Increasing program effectiveness by understanding common assumptions - Price, Claire; Buff, Jesse; Peabody, Shawn; Present, Tess; Lauck, Liz (USAID, 2018)
"How Does Developing Enforcement Capacity Reduce Wildlife Crime?" summarizes findings from the literature around four key assumptions about capacity building for enforcement and prosecution. Using the experiences and evidence summarized in this brief, program designers and implementing partners should be able to increase the effectiveness and efficiency of their capacity building efforts.

Putting vulnerability to climate change on the map: a review of approaches, benefits, and risks - Preston, Benjamin L.; Yuen, Emma J.; Westaway, Richard M. (Sustainability Science; Dordrecht, 2011)
There is growing demand among stakeholders across public and private institutions for spatially-explicit information regarding vulnerability to climate change at the local scale. However, the challenges associated with mapping the geography of climate change vulnerability are non-trivial, both conceptually and technically, suggesting the need for more critical evaluation of this practice. Here, we review climate change vulnerability mapping in the context of four key questions that are fundamental to assessment design. First, what are the goals of the assessment? A review of published assessments yields a range of objective statements that emphasize problem orientation or decision-making about adaptation actions. Second, how is the assessment of vulnerability framed? Assessments vary with respect to what values are assessed (vulnerability of what) and the underlying determinants of vulnerability that are considered (vulnerability to what). The selected frame ultimately influences perceptions of the primary driving forces of vulnerability as well as preferences regarding management alternatives. Third, what are the technical methods by which an assessment is conducted? The integration of vulnerability determinants into a common map remains an emergent and subjective practice associated with a number of methodological challenges. Fourth, who participates in the assessment and how will it be used to facilitate change? Assessments are often conducted under the auspices of benefiting stakeholders, yet many lack direct engagement with stakeholders. Each of these questions is reviewed in turn by drawing on an illustrative set of 45 vulnerability mapping studies appearing in the literature. A number of pathways for placing vulnerability mapping on a more robust footing are also identified. [PUBLICATION ABSTRACT]
Keywords: Climate change; Mapping; Risk assessment; Sustainable development

Redefining Rigor: Describing Quality Evaluation in Complex, Adaptive Settings - Preskill, Hallie; Lynn, Jewlya (FSG, 2016)
Traditionally, evaluation has focused on understanding whether a program is making progress against pre-determined indicators. In this context, the quality of the evaluation is often measured in part by the “rigor” of the methods and scientific inquiry.

Evaluating Complexity. Propositions for improving practice - Preskill, Hallie; Gopal, Srik (FSG, 2014)
9 propositions can help evaluators measure progress on complex social problems.
Keywords: IMPORTANT

Combating Wildlife Trafficking Cross-Mission Learning Agenda - Present, Teresa; Peabody, Shawn (USAID, 2017)

Identifying general trends and patterns in complex systems research: An overview of theoretical and practical implications - Preiser, Rika (Systems Research and Behavioral Science, 2019)
Research on complex systems is becoming more prolific, and there is a need to provide some point of orientation to researchers and practitioners that are drawing on the body of literature that informs the field of complexity research. In this paper, I aim to give an overview of the development of the field and offer some overarching trends and patterns that are recognizable in research on complex systems. The paper then draws on the work to provide six organizing principles of complex systems to inform practical implications and methods for studying and understanding complex systems.
Keywords: complex systems; heuristics; trends and patterns

All about behaviour: KAPE, Adaptation and ' Sticky' Institutional Change - Power, Greg (Politically Agile Programming, 2016)
This paper describes Global Partners Governance’s (GPG’s) approach to institutional reform and political change. Developed over the last decade of working in some of the most complex and sensitive political environments with politicians and officials in parliaments, political parties, ministries and local government, it describes the KAPE® (knowledge-application-practice-effect) methodology that we adopt to get ‘sticky’ institutional and behavioural change. Contents 1) Two dimensions of ‘adaptive programmes’: Flexible delivery and getting behaviour change 2) Enabling Change: KAPE and The Logic of Institutional Reform (Knowledge-Application-Practice-Effect) - K: Knowledge – Defining the problem and what to do about it - A: Application – Making Systems Work in Practice - P: Practice – Pockets of good practice and establishing ‘the new normal’ - E: Effect – Improved performance and the ‘Ripple Effect’ 3) Measuring Impact: Monitoring and Evolving 4) Conclusion: Behavioural insights, adaptive management and sticky change

Decision-Making and Data Use Landscaping [at DFID] - Powell, Josh; Orton-Vipond, Sarah; Bhatia, Vinisha; Kilroy, Annie (Development Gateway, 2018)
As DFID aims to harness the Data Revolution, ensuring that data1 drive decision-making, public accountability, and the achievement of the Sustainable Development Goals (SDGs), ensuring that systems, processes, and skills for data are aligned with these objectives is paramount. Across sector policy teams, country offices, and various analytical and technical cadres, different strengths and weaknesses, as well as needs and ambitions exist. To inform a strategic approach to data, as framed in its forthcoming Data Roadmap, DFID collaborated with Development Gateway to perform a Decision and Data Use Landscaping study. This report details lessons learned from approximately 60 interviews across 4 DFID country offices, all sector policy teams, senior managers, and various analytical and technical cadres and offices.

Poverty Stoplight - povertystoplight.org (2018)
What is the Poverty Stoplight? The Stoplight is a tool that seeks to activate the potential of families and communities to lift themselves out of poverty. Using a technology platform, it offers a self-assessment survey and intervention model that enables people to develop practical solutions to overcome their specific needs. We work with organizations from different sectors around the world and bring together a network of powerful actors with one clear mission: eliminate poverty worldwide. The Stoplight Community is comprised of government agencies, private companies, small and medium-sized enterprises, non-profits, cooperatives, microfinance organizations, academic institutions, and sports clubs, among others.

Critical agency and development: applying Freire and Sen to ICT4D in Zambia and Brazil - Poveda, Sammia; Roberts, Tony (Information Technology for Development, 2017)
(2017). Critical agency and development: applying Freire and Sen to ICT4D in Zambia and Brazil. Information Technology for Development. Ahead of Print. doi: 10.1080/02681102.2017.1328656

Positive Deviance Initiative - Positive Deviance Initiative (2018)

Winners and Losers in the Global App Economy - Pon, Bryan (2016)

Illogical Framework: The Importance of Monitoring and Evaluation in International Development Studies - Pomerantz, Jessica R. (Cornell Policy Review, 2011)
In my brief experience with monitoring and evaluation, I have become convinced that it is critically important both as an international development project component and as a field of academic study. Throughout my academic career at Cornell University, I have, at times, argued that monitoring and evaluation has actually impeded development efforts, but here I wish to amend my opinion. Bad monitoring and evaluation can sabotage development projects and our meaningful interpretation of development impacts; failures can appear to be successes and vice-versa. As a student and practitioner of monitoring and evaluation, I have drawn the conclusions listed below and I submit them for your consideration. • Monitoring and evaluation is a key element of the international development industry applicable to many areas of public administration, domestic and international. • International development failures could be discovered and averted or corrected given proper monitoring and evaluation activities. • Anecdotal evidence from development activities in Afghanistan provides one example of the international community’s lack of attention to monitoring and evaluation concerning an ongoing development catastrophe. • Higher education ought to be filling the monitoring and evaluation knowledge gap but to date is failing to do so.

Seeing (from) Digital Peripheries: Technology and Transparency in Kenya’s Silicon Savannah - Poggiali, Lisa (Cultural Anthropology, 2016)

Design thinking: understand - improve - apply - Plattner, Hasso; Meinel, Christoph; Leifer, Larry (Understanding innovation, 2011)
"In this book, the researchers take a system's view that begins with a demand for deep, evidence-based understanding of design thinking phenomena. They continue with an exploration of tools which can help improve the adaptive expertise needed for design thinking. The final part of the book concerns design thinking in information technology and its relevance for business process modeling and agile software development, i.e. real world creation and deployment of products, services, and enterprise systems"--Cover
Keywords: Creative ability in business; Creative thinking; Designer; Industrial management; Innovation; Organizational change; Product development; System design; Technological innovations

What Startups Can Learn from General McChrystal about Combining Strategy and Execution - Pisoni, Adam (First Round, 2015)
Yammer Co-founder Adam Pisoni speaks with General Stanley McChrystal about the lessons technology companies might take away from his new book Team of Teams.

Is DFID Getting Real About Politics?A stocktake of how DFID has adopted a politically-informed approach (2010-2015) - Piron, Laure-Hélène; Baker, Aislin; Savage, Laura; Wiseman, Katie (DFID, 2016)
1. Background This internal stocktake assesses whether DFID is “getting real about politics” - how it is taking power and politics into account in all its operations. Country Poverty Reduction Diagnostics undertaken by DFID teams identify politics as the most frequent barrier to poverty reduction and growth. The UK 2015 Aid Strategy has committed DFID to spending 50% of Official Development Assistance (ODA) in fragile states. This requires a “patient, long-term approach” to addressing barriers to peace and stability which are fundamentally political, rather than purely financial or technical. The stocktake is based on three DFID offices case studies (Democratic Republic of the Congo, Pakistan and Malawi) as well as extensive internal and external consultations between June and December 2015. It provides illustrations of how DFID is evolving but does not systematically offer evidence of development impacts or non-country work, as this would have required a different methodology. 2. What does it mean to take politics into account? Politically-informed approaches are based on a large body of evidence that confirms the importance of institutions and politics for sustainable development. External assistance needs to support locally-led change. Success depends on timing, context, political processes and local actors. Desirable outcomes are hard to achieve and difficult to predict. Politically-informed approaches improve development effectiveness through:  The ‘what’: political goals, using development assistance to shift how power is distributed in the economy and society. The two main elements are: aiming for long term transformation of institutions; and supporting locally-led change processes more likely to be sustainable and successful: locallyowned (i.e. with local salience) and locally-negotiated.  The ‘how’: politically-smart methods, with greater realism and feasibility. The three main elements are: understanding power and politics in a specific context in order to identify opportunities and barriers for change; influencing and stakeholder management skills; and proactive risk management. To influence DFID operations, a politically-informed approach needs to be iterative, not one-off. The explicit understanding of context, whether formal setpiece studies or more routine analysis, should inform policy and programme decisions, from high level strategic choices, to day-to-day implementation, for both international policy and country support. This is a dynamic process: as the context evolves and lessons are learned about what works, analyses and decisions are updated. These are the principles behind the ‘flexible and adaptive’ agenda.

Local First in practice - Pinnington, R (Peace Direct, 2014)

Decide Madrid: a case study on e-Participation - Pina, Vicente; Torres, Lourdes; Royo, Sonia; García-Rayado, Jaime (XX International AECA Congress, 2019)
This paper analyses the award-winning e-participation initiative of the city council of Madrid, Decide Madrid, to discover the critical success factors at contextual, organizational and individual level. This analysis is carried out with desk research and semi-structured interviews. Results show that the most relevant success factors are the socio-economic context, the commitment of the city council, the method used to recruit the workers and the knowledge of senior managers about citizen participation and ICTs. However, the lack of transparency and poor functioning of some of its participation activities are negatively affecting its performance.

Review of Selected DFAT Facilities: Independent Consultant Report to DFAT, Key Findings - Pieper, Lynn (DFAT, 2018)
For the purposes of this report, a facility is defined as an aid delivery mechanism that provides flexible (adaptive and responsive) services managed in an integrated way. Objectives (or endof-facility outcomes) are specified, but the pathways to deliver them are left unspecified. The facility is a highly relevant model for delivering Australian aid effectively. Achievement of the Sustainable Development Goals depends on flexible approaches that work across sectors and that integrate significant cross-cutting themes such as gender equality and social inclusion. Efficient DFAT management of a $4 billion aid program requires a shift to fewer and larger initiatives, to make best use of limited internal management resources while still enabling policy and program choices that can have real impact. Facilities and other flexible delivery mechanisms offer significant potential for better development results. They can: enable collaborative and responsive partnership approaches to gain traction; allow activities to experiment and adapt based on progress, demand and contextual changes; and provide the opportunity for outcome-focused coherence across sectors, enabling the whole to be more than the sum of the parts. Facilities are not new, but they are different today. Facilities have existed since at least the mid-1990s. What is new is that some facilities today are very large; and the flexibility they enable is increasingly and consciously being used to strengthen links between the technical and the political, for more effective development results. This is a positive trend, but it adds enormously to their complexity and visibility, creates new risks, and has resulted in role confusion (especially in early years of implementation) between DFAT and its contractors.

Adaptive Management: Responding to the evolving needs of PRIME's complex systems - Picon, Diana; Wild, Lorenz (Mercy Corps, 2015)

What is democratic evaluation? - Picciotto, R. (Evaluation Connections, 2013)
This article from Robert Picciotto provides an overview of democratic evaluation, particularly with reference to its use in the European Union context. "Can evaluation strengthen the democratic process? Speciﬁcally can it help ﬁll the democratic deﬁcit (limited transparency; weak bureaucratic accountability, etc.) often ascribed to the European Union? First and foremost, evaluators operating in the European space should be committed to the ethical and democratic values of the European project and the promotion of its social inclusion and cohesion ideals. But two other sets of challenges must also be met: those that relate to evaluation in democracy and those that relate to democracy in evaluation. The former has to do with the macro positioning of evaluation processes in society. The latter has to do with the evaluation approaches and methods used in deliberative democratic decision making processes." (Picciotto, 2013)

The interplay between social learning and adaptive capacity in climate change adaptation: A systematic review - Phuong, Thi Hong Le; Biesbroek, G. Robbert; Wals, Arjen E.J. (NJAS - Wageningen Journal of Life Sciences, 2017)
Successful implementation climate change adaptation depends to a large extent on the capabilities of individuals, organizations, and communities to create and mobilize the adaptive capacity (AC) of their socio-ecological system. Creating and mobilizing AC is a continuous process that requires social learning (SL). Although rich with empirical cases, the literature theorizing and empirically investigating the relationship between AC and SL is highly fragmented. This paper aims to critically examine the peer-reviewed literature that focusses on SL and AC in the context of climate change adaptation (CCA). Special attention is paid to the interplay between the two. Understanding this interplay can help improve our understanding of how CCA takes place in practice and advances theoretical debates on CCA. Systematic review methods are used to analyse 43 papers (1997–2016). Our findings reveal three perspectives that each play an important role in different contexts: an AC-focused perspective, a SL-focused perspective, and a hybrid perspective. These differences in conceptualizations of the relationship between SL and AC may seem trivial at first, but they have consequences for the design of learning-based interventions aimed at helping communities respond to climate change. It appears that such interventions need to be preceded by an analysis of the climate change context in order to decide whether to emphasize AC, SL or both simultaneously.
Keywords: Adaptive capacity; Climate change adaptation; Social learning; Systematic review

Using a social learning configuration to increase Vietnamese smallholder farmers’ adaptive capacity to respond to climate change - Phuong, Le Thi Hong; Wals, Arjen; Sen, Le Thi Hoa; Hoa, Nguyen Quoc; Lu, Phan Van; Biesbroek, Robbert (Local Environment, 2018)
Social learning is crucial for local smallholder farmers in developing countries to improve their adaptive capacity and to adapt to the current and projected impacts of climate change. While it is widely acknowledged that social learning is a necessary condition for adaptation, few studies have systematically investigated under which conditions particular forms of social learning are most successful in improving adaptive capacity of the most vulnerable groups. This study aims to design, implement and evaluate a social learning configuration in a coastal community in Vietnam. We make use of various methods during four workshop-based interventions with local smallholder farmers: interviews with key farmers and commune leaders, farmer-to-farmer learning, participatory observations and focus group discussions. The methods for evaluation of social learning configuration include in-depth interviews, focus group discussions and structured survey interviews. Our findings show that the social learning configuration used in this study leads to an increased problem ownership, an enhanced knowledge-base with regard to climate change impacts and production adaptation options, improved ability to see connections and interdependencies and finally, strengthened relationships and social cohesion. The results suggest that increased social learning in the community leads to increase in adaptive capacity of smallholder farmers and improves both their economic and environmental sustainability. We discuss the key lessons for designing learning configurations that can successfully enhance adaptive capacity and smallholder farmers’ agency and responsiveness to the challenges posed by climate change impacts.
Keywords: Adaptive capacity; Social learning; Vietnamese smallholder farmers

Barriers and enablers to climate change adaptation in hierarchical governance systems: the case of Vietnam - Phuong, Le Thi Hong; Biesbroek, G. Robbert; Wals, Arjen E. J. (Journal of Environmental Policy & Planning, 2018)
Governments fulfil important roles in increasing the adaptive capacity of local communities to respond to climate change impacts, particularly in developing countries. Existing studies on how governments enable and constrain the ways in which local level communities learn and build their adaptive capacity, however, generally adopt network or market-oriented types of governance. However, the most vulnerable regions to climate change impact in the world are generally governed through hierarchical policy systems. This research aims to understand how the hierarchical policy system in Vietnam creates enables and/or constrains the policy capacity of policy actors to contribute to effective climate change adaptation. We conducted interviews (n = 26) with key actors at multiple levels of government. Our findings show the importance of clear legal institutions, available financing for implementing policies, and the training of governmental staff, particularly at district and commune levels where the policy capacities are generally too low to deal with climate change impacts. We conclude that any efforts to support local actors (i.e. smallholder farmers) should include investments in policy capacity to ensure uptake and upscaling of adaptation actions more broadly.

Power Above and Below the Waterline: Bridging Political Economy and Power Analysis - Pettit, Jethro; Mejía Acosta, Andrés (IDS Bulletin, 2014)

Power Analysis: A Practical Guide - Pettit, Jethro (Sida, 2013)

Grow - Scale - Impact: How to help inclusive business achieve scale - Pérez Castro, Andrea A.; Tora, Krisztina (GIZ, 2016)

Citizen participation and the rise of the open source city in Spain - Peña-López, Ismael (MAVC & IT for Change, 2017)
Research on the Information Society, the Digital Divide and Information and Communication Technologies for development