Holistic Flexibility for Deploying Systems Thinking as a Cognitive Skill - Chowdhury, Rajneesh (Systemic Practice and Action Research, 2023)
Considering Systems Thinking (ST) as a cognitive skill can create greater acceptability of and openness to the discipline from practitioners and researchers outside operations research and management science. Rather than associating ST with frameworks and methodologies, ST as a cognitive skill can help popularize and democratize the discipline. This paper highlights how the conceptual lens of Holistic Flexibility can help practitioners deploy ST as a cognitive skill without the application of any traditional systems methodology. Holistic Flexibility is defined as the dynamic interplay between a state of mind that has the ability to absorb systemic complexity and a state of practice that has the ability to embrace flexibility, both in intent and in form. Through two case-studies, discussions in this paper highlight how Holistic Flexibility can serve as a conceptual lens for systems practitioners. The case-studies demonstrate the importance of a practitioner’s ability to seamlessly manage and work with multiple variables, stakeholders, and factors to deliver responsible outcomes with the aid of learning loops. The main contribution of this paper lies in the case-studies and analyses presented that provide use cases for Holistic Flexibility in ST, which will help address recent calls in the discipline for ST to be considered as a cognitive skill.

Strengthen Your Team’s CLA Practices: Introducing the CLA Maturity Tool for USAID Implementers - Social Impact , Inc (2023)
The Collaborating, Learning & Adapting (CLA) Maturity Tool has been used by USAID operating units since 2015. The tool enables staff to self-assess their current CLA practice and plan ways to improve their internal and external collaboration, organizational learning efforts, and adaptive management practices. Recently, SI built on this resource to create the CLA Maturity Tool for implementing partners (IPs) to provide a version that speaks more directly to the experience of international and local organizations. Learn more here.

Introducing the CLA Maturity Tool for Implementing Partners - Salib, Monalisa; Ziegler, Jessica (Social Impact, 2023)
On December 13, 2022, Social Impact hosted a webinar in our Evidence for Impact series, "Strengthen Your Team's CLA Practices: Introducing the CLA Maturity Tool for USAID Implementers." Through the session, we shared more about the origin and history of the USAID tool upon which this version is based, why and how we updated

A Better Approach to After-Action Reviews - Fletcher, Angus; Cline, Preston B.; Hoffman, Matthew (Harvard Business Review, 2023)
In the decades since the Army created the After Action Review (AAR), businesses have embraced the practice as a way of learning from both failure and success. But all too often the practice gets reduced to nothing more than a pro forma exercise. The authors of this article describe the history and philosophy of the original AAR, debunk three myths about the practice that impede its proper use, and finally suggest three improvements that can help business leaders make the most of it.
Keywords: Collaboration and teams; Crisis management; Project management

Innovative M&E from the Sandbox and beyond - Vester, Søren; Tran, Samuel (Medium, 2023)
In this blog we are sharing a digest of some of the many useful and innovative monitoring, evaluation and learning resources and efforts that have come through the M&E Sandbox in 2022. A lot of these resources have been shared by our community in response to the overwhelmingly positive feedback from the launch of the Sandbox (please keep them coming!). We hope you find it useful. We have grouped these efforts and resources under six broad questions: - How do we measure systems transformation? - How do we know if we are on track? - How do we rethink complexity and independence in evaluation? - Why, how and for whom do we measure? - How do we generate insights and learn? - How do we make decisions and adapt?

Two Loops Guide - Colchester, Joss (Si Network, 2023)
The Two Loop model is a nonlinear theory of change based upon the ideas of living systems created by Margaret Wheatley & Deborah Frieze. It provides us with insight into the simultaneous growth and decline process that are underway within a system during a transition period. In this way, it provides a heuristic for us to better try and recognize and connect the past to the future during a change process. It likewise gives us the possibility to try and come to some consensus about where we might be as a group in this process of change and the best actions to take at different stages. This guide will be of relevance for anyone involved in a complex organizational change process.

Understanding Political Economy Analysis and Thinking and Working Politically - Whaites, Alan; Piron, Laure-Hélène; Menocal, Alina Rocha; Teskey, Graham (FCDO and TWP CoP, 2023)
This guide is adapted from work by the UK Foreign, Commonwealth and Development Office (FCDO) with inputs from members of the Thinking and Working Politically Community of Practice (TWP CoP). It outlines how to understand and use a set of analytical tools that are collectively known as Political Economy Analysis (PEA). The guide aims to equip practitioners to act in an informed manner, given that development objectives are invariably politically complex, and entail engaging with counterparts’ political incentives and preferences. The guide summarises different types of tools – from very light-touch to more in-depth approaches – and provides advice on how development professionals can decide what is most appropriate in a given context, with illustrations based on the experiences of teams working on these issues. This guide will help development professionals and others to make use of PEA and to apply it to their own specific needs. The first part of the guide offers a general picture of the approach. The second part provides more specific guidance for those who are tasked with deploying a PEA. Contents --> Main audience What is PEA, its role and purpose (Section 2) --> General information for all readers The main elements of PEA (Section 3) Thinking and Working Politically (Section 4) --> Core information for teams planning and using PEA How to ensure quality (Section 5) --> Essential reading for those directly responsible for a PEA Important concepts and terminology (Annex) --> General information for all readers

Celebrating Adaptive Delivery: A View from the Frontline in Myanmar - Barnes, Katrina; Lonsdale, Jane (IDS Working Paper, 2023)
The conversation on adaptive management has grown fast amongst development actors. These conversations often focus on designing, commissioning, and managing large-scale development programmes. Exactly how this impacts the frontline, the implementers, and day-to-day project delivery is still being debated. Yet, perspectives drawn directly from practice are often largely missing within these debates. This paper is written by two development practitioners. Through this paper, we reflect on the difference between adaptive management and adaptive delivery, and how this interacts with risk and aid accountability, particularly in contexts of fragility. Drawing on examples of Oxfam in Myanmar work and our personal insights in relation to delivering programming across humanitarian, peace-building, and development, we suggest that in complex, conflict-affected, and highly political environments adaptive delivery already happens far more regularly than is currently recognised, as a necessity to get activities delivered. However, it happens despite the system, not because of it, and is therefore often hidden and carried out ‘under the radar’ rather than celebrated as a success in difficult environments. This paper was written in 2019, before the military seized control of Myanmar in February 2021. Whilst it draws on examples from pre-2021 Myanmar to illustrate real life cases, it is a contribution to a broader global debate on adaptive management in practice, specifically in fragile contexts. This is not specifically aimed at practitioners working in Myanmar at present, who are now working in a protracted crisis. This paper makes tangible recommendations on steps that donors, international non-governmental organisations, local staff, and partners could take to promote a system of encouraging and celebrating adaptability in programme delivery in fragile contexts.

Don't Lead Alone: Think Like a System, Act Like a Network, Lead Like a Movement! - Justis, Cleveland; Student, Daniel (Fast Company Press, 2023)
Think. Act. Lead.It seems simple enough. But understanding your desired impact and how it fits into a larger picture, connecting your work to others and finding new collaborators, and bringing those collaborators together and moving them in a unified direction is never easy.Governments, businesses, and nonprofits all have unique approaches and ideas that many of us learn through our work. Yet, we rarely consider the skills needed to create and maintain the partnerships between them. Most of us learn those skills through trial, error, and often, failure. Worse, we typically stay in our self-reinforcing silos, sharing perspectives and frustrations with like-minded people, limiting our vision of what our work can become. By partnering with other sectors, we combine and adapt approaches to solve complex problems, and leaders in any industry can create large-scale change.Cleveland Justis and Daniel Student share a road map for effective partnerships that increase impact and profitability. Using real-life examples and practice exercises, the authors teach how to acquire and use skills to solve complex problems and propel your organization forward by combining a multitude of perspectives, split into three sections:Think Like a SystemAct Like a NetworkLead Like a MovementIt’s time to get out of our silos. Don’t lead alone.

How Big Things Get Done: The Surprising Factors Behind Every Successful Project, from Home Renovations to Space Exploration - Flyvbjerg, Bent; Gardner, Dan (Macmillan, 2023)
World expert Bent Flyvbjerg and bestselling author Dan Gardner reveal the secrets to successfully planning and delivering ambitious projects on any scale.Nothing is more inspiring than a big vision that becomes a triumphant new reality. Think of how Apple’s iPod went from a project with a single employee to an enormously successful product launch in eleven months. But such successes are the exception. Consider how London’s Crossrail project delivered five years late and billions over budget. More modest endeavours, whether launching a small business, organizing a conference, or just finishing a work project on time, also commonly fail. Why?Understanding what distinguishes the triumphs from the failures has been the life’s work of Oxford professor Bent Flyvbjerg. In How Big Things Get Done, he identifies the errors that lead projects to fail, and the research-based principles that will make yours succeed:- Understand your odds. If you don’t know them, you won’t win.- Plan slow, act fast. Getting to the action quick feels right. But it’s wrong.- Think right to left. Start with your goal, then identify the steps to get there.- Find your Lego. Big is best built from small.- Master the unknown unknowns. Most think they can’t, so they fail. Flyvbjerg shows how you can.Full of vivid examples ranging from the building of the Sydney Opera House to the making of the latest Pixar blockbusters, How Big Things Get Done reveals how to get any ambitious project done – on time and on budget.

The Big Con: How the Consulting Industry Weakens our Businesses, Infantilizes our Governments and Warps our Economies - Mazzucato, Mariana; Collington, Rosie (Allen Lane, 2023)
There is an entrenched relationship between the consulting industry and the way business and government are managed today which must change.Mariana Mazzucato and Rosie Collington show that our economies' reliance on companies such as McKinsey, Boston Consulting Group, Bain & Company, PwC, Deloitte, KPMG and EY stunts innovation, obfuscates corporate and political accountability and impedes our collective mission of halting climate breakdown.The 'Big Con' describes the confidence trick the consulting industry performs in contracts with hollowed-out and risk-averse governments and shareholder value-maximizing firms. It grew from the 1980s and 1990s in the wake of reforms by both the neoliberal right and Third Way progressives, and it thrives on the ills of modern capitalism, from financialization and privatization to the climate crisis. It is possible because of the unique power that big consultancies wield through extensive contracts and networks - as advisors, legitimators and outsourcers - and the illusion that they are objective sources of expertise and capacity. To make matters worse, our best and brightest graduates are often redirected away from public service into consulting. In all these ways, the Big Con weakens our businesses, infantilizes our governments and warps our economies.Mazzucato and Collington expertly debunk the myth that consultancies always add value to the economy. With a wealth of original research, they argue brilliantly for investment and collective intelligence within all organizations and communities, and for a new system in which public and private sectors work innovatively for the common good. We must recalibrate the role of consultants and rebuild economies and governments that are fit for purpose.

Reflections on Ten Years of USAID’s Experience with Political Economy Analysis and Thinking and Working Politically - Brinkerhoff, Derick; Cassidy, Marc (TWP CoP, 2023)
The global Thinking and Working Politically (TWP) Community of Practice (CoP), the Washington DC TWP CoP, and USAID organised a webinar on 5 December 2022 to take stock of how USAID and its partners have used PEA to inform programme strategy, design and implementation, and support TWP. This paper synthesises the key points arising from the webinar, including observations on the impacts, opportunities, challenges, and prospects for PEA/TWP to become more deeply adopted and sustained as a development methodology and approach across sectors. It starts by defining key concepts. It then highlights insights from the discussions of the impact of the application of PEA and TWP principles across sectors. The paper concludes by looking at progress achieved to date, as well as constraints and opportunities to increase the uptake of both thinking and working politically in USAID-sponsored programming.

Adaptive Evaluation: A Complexity-based approach to Systematic Learning for Innovation and Scaling in Development - Gokhale, Siddhant; Walton, Michael (CID Faculty Working Paper, 2023)
Nearly all challenges in international development tend to be complex because they depend on constantly evolving human behaviour, systems, and contexts, involving multiple actors, entities, and processes. As a result, both the discovery and scaling of innovations to address challenges in development often involve changes in system behaviour or even system-level transformation. This is rarely a linear process over time and can result in unexpected outcomes. Existing evaluation techniques commonly used in international development, including Randomized Control Trials (RCT) and quasi-experimental methods, are good at assessing specific effects of interventions but are not designed for the change processes inherent to innovation and scaling within a system. There is a need to reconstruct how we use existing measurement tools, techniques, and methodologies so that they capture the complexity of the environment in which an intervention or change occurs. We introduce Adaptive Evaluation, designed to learn at various levels of complexity while supporting the transformation needed to foster sustainable change. An Adaptive Evaluation uses three main approaches to work with complex questions—systems diagnosis, theorybased assessment of change processes, and iterative designs. An Adaptive Evaluation typically builds hypotheses from field-based interactions, emphasizes learning over testing, advocates open-mindedness with techniques, and appreciates the value of dialogue and participation in navigating complex processes. It can use RCT or similar techniques to analyse specific processes within a system or a development cycle, but these are embedded in a broader approach to assessment and interpretation. It is designed to be flexible and adjust to shifting contexts. Finally, an Adaptive Evaluation can be applied at any stage in a complex intervention's lifecycle, from the interpretation of the system and change processes to rapid experimentation, prototyping, and testing of select interventions, and then adaptation to different settings for impact at scale. This paper provides the theoretical basis for an Adaptive Evaluation—the main approaches, core ideology, process, and applications.

Glass Half Full: Civic Space and Contestation in Bangladesh, Sri Lanka and Nepal - Surie, Mandakini D.; Saluja, Sumaya; Nixon, Nicola (The Asia Foundation, 2023)
The past decade has witnessed a surge of interest in and concern over the global trend toward democratic regression. In South Asia, regulatory and institutional frameworks have become increasingly restrictive, curbing the ability of citizens and civil society organizations to occupy and use civic spaces to organize, express themselves, and participate in decisions that affect the lives of people whose interests they serve. Of course, this is only one half of the story. The other half is how citizens, collectives, and organizations adapt by carving out spaces where they can maintain—or even expand—the boundaries of their engagement in local and national civic spaces. In this paper, we examine how civic spaces are evolving in three South Asian countries—Bangladesh, Sri Lanka, and Nepal—drawing on the experiences and perspectives of civil society representatives. The paper concludes with recommendations for donors and development practitioners on how they can make relevant investments that will strengthen civic spaces and support democratic resilience in the region.

The Secret Sauce of Development Professionals: Tools for Assessing TOR Potential to Source Scalable Learning Interventions - Barton, Adam (Research on Improving Systems of Education (RISE), 2023)
Terms of reference (TORs) play an outsized role in driving scalable educational programming. These procurement documents shape, constrain, and signal programme priorities and possibilities. Successful funders and implementers across the globe hold rich processual knowledge about this documentation, which they use to draft and assess TORs. This project explores such best-practice knowledge around TOR review, seeking to support the design and implementation of educational programmes that can improve learning at scale in developing contexts.

Process Tracing Innovations in Practice: Finding the Middle Path - Aston, Thomas; Wadeson, Alix (CDI Practice Paper, 2023)
Evaluation practitioners in the international development sector have given considerable attention in recent years to process tracing as a method for evaluating impact, including discussion of how to assess the relative importance of causal factors. Despite the increasing interest, there is a relative dearth of examples of practical learning and evidence of applying process tracing in practice. This CDI Practice Paper draws on comparative learning from applying three different types of process tracing in international development initiatives. It argues in favour of a ‘middle path’ of applying evidence tests and rubrics to structure evaluative judgements rather than formal Bayesian updating or looser forms of process tracing. It also calls attention to the potential added value of taking a participatory approach, offering practical recommendations for how to do this effectively.

The Last 10 Per Cent: Why the World Needs a Leaner, More Innovative and Pragmatic Development Sector, Today - Harper, Erica (Routledge India, 2023)
Criticism that the development sector has not delivered in terms of eliminating extreme poverty, fast-tracking growth and preventing conflict, is neither new nor surprising. In fact, it may be the one thing that scholars, donors and practitioners agree on. While many of these concerns are valid, this book makes a case that the sector is closer to unlocking the gates to more effective and efficient development outcomes than is popularly believed. Specifically, it argues that by overturning a few myths, making better use of evidence and employing some different rules, practitioners, policy specialists and donors can foster the changes in the development architecture that are needed to reach the 10 percent of the world’s population still living in extreme poverty.Engaging, provocative and clear sighted, the book provides insight into interventions around democratic governance, refugee response, counterterrorism, gender mainstreaming, environmental protection and private sector engagement. It is instructive reading for professionals across the development sector, think tanks and NGOs.

Berkana Two Loop Model: A quick introduction to an accessible model for systems change - Proud, Emma (LinkedIn - Emma Proud, 2023)
A model I've been using a lot recently, to bring systems thinking to life, is the Berkana Two Loop model. It doesn't describe complexity or systems thinking. But it does describe systems change in a way that's simple and oriented to action.

Revealing the Relational Mechanisms of Research for Development Through Social Network Analysis - Apgar, Marina; Fournie, Guillaume; Haesler, Barbara; Higdon, Grace Lyn; Kenny, Leah; Oppel, Annalena; Pauls, Evelyn; Smith, Matthew; Snijder, Mieke; Vink, Daan; Hossain, Mazeda (The European Journal of Development Research, 2023)
Achieving impact through research for development programmes (R4D) requires engagement with diverse stakeholders across the research, development and policy divides. Understanding how such programmes support the emergence of outcomes, therefore, requires a focus on the relational aspects of engagement and collaboration. Increasingly, evaluation of large research collaborations is employing social network analysis (SNA), making use of its relational view of causation. In this paper, we use three applications of SNA within similar large R4D programmes, through our work within evaluation of three Interidsiplinary Hubs of the Global Challenges Research Fund, to explore its potential as an evaluation method. Our comparative analysis shows that SNA can uncover the structural dimensions of interactions within R4D programmes and enable learning about how networks evolve through time. We reflect on common challenges across the cases including navigating different forms of bias that result from incomplete network data, multiple interpretations across scales, and the challenges of making causal inference and related ethical dilemmas. We conclude with lessons on the methodological and operational dimensions of using SNA within monitoring, evaluation and learning (MEL) systems that aim to support both learning and accountability.

Evaluating Research for Development: Innovation to Navigate Complexity - Apgar, Marina; Snijder, Mieke; Higdon, Grace Lyn; Szabo, Sylvia (The European Journal of Development Research, 2023)
Large publicly funded programmes of research continue to receive increased investment as interventions aiming to produce impact for the world’s poorest and most marginalized populations. At this intersection of research and development, research is expected to contribute to complex processes of societal change. Embracing a co-produced view of impact as emerging along uncertain causal pathways often without predefined outcomes calls for innovation in the use of complexity-aware approaches to evaluation. The papers in this special issue present rich experiences of authors working across sectors and geographies, employing methodological innovation and navigating power as they reconcile tensions. They illustrate the challenges with (i) evaluating performance to meet accountability demands while fostering learning for adaptation; (ii) evaluating prospective theories of change while capturing emergent change; (iii) evaluating internal relational dimensions while measuring external development outcomes; (iv) evaluating across scales: from measuring local level end impact to understanding contributions to systems level change. Taken as a whole, the issue illustrates how the research for development evaluation field is maturing through the experiences of a growing and diverse group of researchers and evaluators as they shift from using narrow accountability instruments to appreciating emergent causal pathways within research for development.

Theory of Change in complex Research for Development programmes: challenges and solutions from the Global Challenges Research Fund - Chapman, Sarah; Boodhoo, Adiilah; Duffy, Carren; Goodman, Suki; Michalopoulou, Maria (The European Journal of Development Research, 2023)
The United Kingdom Research and Innovation (UKRI) Global Challenges Research Fund (GCRF) aimed to address global challenges to achieve the United Nations (UN) Sustainable Development Goals through 12 interdisciplinary research hubs. This research documents key lessons learned around working with Theory of Change (ToC) to guide Monitoring, Evaluation and Learning (MEL) within these complex research for development hubs. Interviews and document reviews were conducted in ten of the research hubs. The results revealed that only one hub invested in an explicit visual system mapping approach, and that funder timelines, budget constraints and issues with capacity and expertise limited the application of these approaches across all hubs. In contrast, many hubs attempted to deal with visual complexity by means of ether constructing multiple, nested ToCs, or a conscious simplification of complexity through reducing their ToC towards a straightforward and uncomplicated chain model or spherical model. While the former approach had some value, most hubs struggled to find capacity to support the full articulation of nested ToCs. In contrast, the latter approach resulted in ToCs which lacked detail or mechanism articulation, but which nevertheless were often ‘fit for purpose’ in ensuring effective communication and coherence across diverse stakeholders and sub-projects. We conclude that in instances where the reporting, funding and management cycles of complex research for development programmes cannot be adapted to properly support learning-based approaches to ToC development, imposing simplicity in the ToC might be fit for purpose. This might also be preferable to more complex visual approaches that are only partially realised.

Cost–Benefit Analysis (CBA) or the Highway? An Alternative Road to Investigating the Value for Money of International Development Research - Peterson, Heidi (The European Journal of Development Research, 2023)
Research for development (R4D) funding is increasingly expected to demonstrate value for money (VfM). However, the dominance of positivist approaches to evaluating VfM, such as cost-benefit analysis, do not fully account for the complexity of R4D funds and risk undermining efforts to contribute to transformational development. This paper posits an alternative approach to evaluating VfM, using the UK’s Global Challenges Research Fund and the Newton Fund as case studies. Based on a constructivist approach to valuing outcomes, this approach applies a collaboratively developed rubric-based peer review to a sample of projects. This is more appropriate for the complexity of R4D interventions, particularly when considering uncertain and emergent outcomes over a long timeframe. This approach could be adapted to other complex interventions, demonstrating that our options are not merely “CBA or the highway” and there are indeed alternative routes to evaluating VfM.

How are Research for Development Programmes Implementing and Evaluating Equitable Partnerships to Address Power Asymmetries? - Snijder, Mieke; Steege, Rosie; Callander, Michelle; Wahome, Michel; Rahman, M. Feisal; Apgar, Marina; Theobald, Sally; Bracken, Louise J.; Dean, Laura; Mansaray, Bintu; Saligram, Prasanna; Garimella, Surekha; et al. (The European Journal of Development Research, 2023)
The complexity of issues addressed by research for development (R4D) requires collaborations between partners from a range of disciplines and cultural contexts. Power asymmetries within such partnerships may obstruct the fair distribution of resources, responsibilities and benefits across all partners. This paper presents a cross-case analysis of five R4D partnership evaluations, their methods and how they unearthed and addressed power asymmetries. It contributes to the field of R4D partnership evaluations by detailing approaches and methods employed to evaluate these partnerships. Theory-based evaluations deepened understandings of how equitable partnerships contribute to R4D generating impact and centring the relational side of R4D. Participatory approaches that involved all partners in developing and evaluating partnership principles ensured contextually appropriate definitions and a focus on what partners value.

Laying the Foundations for Impact: Lessons from the GCRF Evaluation - Vogel, Isabel; Barnett, Chris (The European Journal of Development Research, 2023)
Research for development (R4D) aims to make a tangible difference to development challenges, but these effects typically take years to emerge. Evaluation (especially impact evaluation) often takes place before there is evidence of development impact. In this paper, we focus on opportunities for assessing the potential for impact at earlier stages in the research and innovation process. We argue that such a focus can help research programme managers and evaluators learn about the pre-conditions for impact and adjust accordingly. Using the Global Challenges Research Fund (GCRF) as a large-scale case of R4D evaluation, we identify and explore some of the building blocks that can increase impact potential. Guided by GCRF’s theory of change, we explore emerging evidence that highlights the importance of ways of working that supports positioning for impact. We conclude by drawing out a unifying construct around standards of development excellence; to sit alongside notions of scientific excellence for research intended to have an impact. Standards can help programme managers, researchers and evaluators learn and adapt to increase the likelihood of impact.

Building Capacity for Strategic Innovation: an Emerging Competency Framework for Portfolio Work - Begovic, Milica; Colville, Jenniffer; Quaggiotto, Giulio; Vester, Søren; Naatujuna, Deborah; Oprunenco, Alex; Sadiku; Lejla; Uusikyla, Ida; Uriartt, Simone; Zorouali, Kawtar (Medium, 2023)
In a previous post we reflected on how a key learning from our “project to portfolio” journey to date is that it is ultimately about mustering the organizational will to transform. If the early days of our innovation work were about demonstrating results quickly and creating space for experimentation, now the challenge is of a different order. Eventually this means helping UNDP transition to a different value proposition and business model, as eloquently articulated by Gerd Trogemann: “No matter how well conceived and relevant in their own right, projects tend to pursue single point rather than systemic solutions, limit strategic space and the ability to adapt continuously and to connect the dots systemically. Systemic solutions need adaptive ways of working, strategic space, iterative learning, and radical collaboration.”

A new pathway: how can funders support meaningful monitoring, evaluation, and learning practice in the field? - Blog post on Better Evaluation - Acevedo, Andrea; Colnar, Megan (2023)
How can donors and grantees work together to create effective monitoring, evaluation, and learning (MEL) practices that drive field-wide transformation?

Participatory action research - Cornish, Flora; Breton, Nancy; Moreno-Tabarez, Ulises; Delgado, Jenna; Rua, Mohi; de-Graft Aikins, Ama; Hodgetts, Darrin (Nature Reviews Methods Primers, 2023)
Participatory action research (PAR) is an approach to research that prioritizes the value of experiential knowledge for tackling problems caused by unequal and harmful social systems, and for envisioning and implementing alternatives. PAR involves the participation and leadership of those people experiencing issues, who take action to produce emancipatory social change, through conducting systematic research to generate new knowledge. This Primer sets out key considerations for the design of a PAR project. The core of the Primer introduces six building blocks for PAR project design: building relationships; establishing working practices; establishing a common understanding of the issue; observing, gathering and generating materials; collaborative analysis; and planning and taking action. We discuss key challenges faced by PAR projects, namely, mismatches with institutional research infrastructure; risks of co-option; power inequalities; and the decentralizing of control. To counter such challenges, PAR researchers may build PAR-friendly networks of people and infrastructures; cultivate a critical community to hold them to account; use critical reflexivity; redistribute powers; and learn to trust the process. PAR’s societal contribution and methodological development, we argue, can best be advanced by engaging with contemporary social movements that demand the redressingl of inequities and the recognition of situated expertise.

The adoption of innovation in international development organisations: lessons for development co-operation - Kumpf, Benjamin; Jhunjhunwala, Parnika (OECD Development Co-operation Working Papers, 2023)
Addressing 21st century development challenges requires investments in innovation, including the use of new approaches and technologies. Currently, many development organisations prioritise investments in isolated innovation pilots that leverage a specific approach or technology rather than pursuing a strategic approach to expand the organisation’s toolbox with innovations that have proven their comparative advantage over what is currently used. This Working Paper addresses this challenge of adopting innovations. How can development organisations institutionalise a new way of working, bringing what was once novel to the core of how business is done? Analysing successful adoption efforts across five DAC agencies, the paper lays out a proposed process for the adoption of innovations. The paper features five case-studies and concludes with a set of lessons and recommendations for policy makers on innovation management generally, and adoption of innovation in particular.

USAID ADS 201 - Operational Policy for the Program Cycle (Update 05/22/2023) - USAID (USAID, 2023)
The Program Cycle is USAID’s operational model for planning, delivering, assessing, and adapting development programming in a given region or country to advance U.S. foreign policy. It encompasses guidance and procedures for: 1) Making strategic decisions at the regional or country level about programmatic areas of focus and associated resources; 2) Designing supportive projects and/or activities to implement these strategic plans; and 3) Learning from performance monitoring, evaluations, and other relevant sources of information to make course corrections as needed and inform future programming.

CLARISSA Cash Plus: Innovative Social Protection in Bangladesh - Roelen, Keetie; Howard, Neil; Afroze, Jiniya; Aktar, Afrin; Ton, Giel; Huq, Lopita (CLARISSA Design Note, 2023)
Social protection, and cash transfers especially, have been found to have many positive impacts on families’ lives and are now widely recognised as a cornerstone of any prosperous, fair society. The CLARISSA Cash Plus intervention is an innovative social protection scheme for tackling social ills, including the worst forms of child labour (WFCL). Combining community mobilisation, case work and cash transfers, it aims to support people in a poor neighbourhood in Dhaka to build their individual, family, and group capacities to meet their needs. An increase in capacities is expected to lead to a corresponding decrease in deprivation and community-identified social issues that negatively affect wellbeing, including WFCL.

Participation for Humanitarian Innovation - Background Paper - Smith, Amy; Thompson, Martha; Saida Benhayoune; Crespo Cardona, Omar (Elrha, 2023)
A resource designed to help organisations, teams and individuals manage innovation journeys responsibly and successfully. We have partnered with MIT D-Lab to develop a new resource to drive greater diversity and inclusion within project design and implementation. The Participation for Humanitarian Innovation (PfHI) toolkit sets out a robust approach to setting expectations for and monitoring the degree of participation within research and innovation projects for, with, and by people affected by crisis. The PfHI toolkit is composed of five tools: Opportunity Adviser: Identify and prioritise the desired benefits of participation while weighing the potential barriers. Participation Matrix: Agree on the precise degree of participation to target at a given project stage. Resource Navigator: Select tools and processes to address the needs of stakeholders, the project objectives and context. Quality Guidance: Ensure engagements are delivered to the highest standard. Assessment Matrix: Collectively evaluate the degree of participation achieved during an activity or project phase, learn and adapt. By applying these tools before, during and after a research and/or innovation project, implementors can ensure that stakeholders/end-users are included and participating at the highest possible degree. We encourage users of the PfHI toolkit to consider how else to integrate the tools into existing practices. For instance, the Evaluation Matrix could be used to supplement existing MEAL activities to: Establish baselines to measure changes in participation over time. Track the degree of participation/engagement of stakeholders. Regularly assess the degree of participation. Seek feedback from participants about the degree of their participation. Monitor the progress of activities related to stakeholder engagement. Our Participation for Humanitarian Innovation toolkit represents our ongoing commitment to responsible research and innovation across our portfolio of grants and for the humanitarian sector more broadly. We hope you will consider downloading and using the toolkit on your next project.

Participation for Humanitarian Innovation - Toolkit - Smith, Amy; Thompson, Martha; Saida Benhayoune; Crespo Cardona, Omar (Elrha, 2023)
A resource designed to help organisations, teams and individuals manage innovation journeys responsibly and successfully. We have partnered with MIT D-Lab to develop a new resource to drive greater diversity and inclusion within project design and implementation. The Participation for Humanitarian Innovation (PfHI) toolkit sets out a robust approach to setting expectations for and monitoring the degree of participation within research and innovation projects for, with, and by people affected by crisis. The PfHI toolkit is composed of five tools: Opportunity Adviser: Identify and prioritise the desired benefits of participation while weighing the potential barriers. Participation Matrix: Agree on the precise degree of participation to target at a given project stage. Resource Navigator: Select tools and processes to address the needs of stakeholders, the project objectives and context. Quality Guidance: Ensure engagements are delivered to the highest standard. Assessment Matrix: Collectively evaluate the degree of participation achieved during an activity or project phase, learn and adapt. By applying these tools before, during and after a research and/or innovation project, implementors can ensure that stakeholders/end-users are included and participating at the highest possible degree. We encourage users of the PfHI toolkit to consider how else to integrate the tools into existing practices. For instance, the Evaluation Matrix could be used to supplement existing MEAL activities to: Establish baselines to measure changes in participation over time. Track the degree of participation/engagement of stakeholders. Regularly assess the degree of participation. Seek feedback from participants about the degree of their participation. Monitor the progress of activities related to stakeholder engagement. Our Participation for Humanitarian Innovation toolkit represents our ongoing commitment to responsible research and innovation across our portfolio of grants and for the humanitarian sector more broadly. We hope you will consider downloading and using the toolkit on your next project.

Mechanisms of Techno-Moral Change: A Taxonomy and Overview - Danaher, John; Sætra, Henrik Skaug (Ethical Theory and Moral Practice, 2023)
The idea that technologies can change moral beliefs and practices is an old one. But how, exactly, does this happen? This paper builds on an emerging field of inquiry by developing a synoptic taxonomy of the mechanisms of techno-moral change. It argues that technology affects moral beliefs and practices in three main domains: decisional (how we make morally loaded decisions), relational (how we relate to others) and perceptual (how we perceive situations). It argues that across these three domains there are six primary mechanisms of techno-moral change: (i) adding options; (ii) changing decision-making costs; (iii) enabling new relationships; (iv) changing the burdens and expectations within relationships; (v) changing the balance of power in relationships; and (vi) changing perception (information, mental models and metaphors). The paper also discusses the layered, interactive and second-order effects of these mechanisms.

How to Build Movements with Cyclical Patterns in Mind - Non Profit News | Nonprofit Quarterly - Algoso, Dave; Guerzovich, Florencia; Gattoni, Soledad (Nonprofit Quarterly, 2023)
The world changes too much for anyone who is invested in social change work to imagine that this work is linear and predictable. Opportunities come and go, whether caused by a pandemic or political shifts. This much most social movement leaders and activists intuitively understand. But what can be done with this realization? How might movement groups better prepare for moments of opportunity? We want to explore how we can create the changes we want to see by responding to the changes that are outside our control.

When Complex is as Simple as it Gets: Guide for Recasting Policy and Management in the Anthropocene - Roe, Emery (IDS Working Paper, 2023)
Many readers recognise and understand that complex is about as simple as it gets for major policy and management. This guide is for those unwilling in the Anthropocene to shrink back into the older platitudes about ‘keep it simple’ and ‘not to worry, we’ll scale up the analysis later on’. This guide offers key concepts, methods, counternarratives, and analogies that recast major policy and management issues in ways that do not deny their complexity but help render them more tractable for action.

Evaluative reasoning in complexity - King, Julian (Evaluation and Value for Investment, 2023)

Value for Investment: Application and Insights. Youth Primary Mental Health and Addictions Evaluation - King, Julian; Crocket, Alicia; Field, Adrian (Dovetail Consulting, 2023)
What’s in the guide? This new document provides: An overview of the VfI approach, including the conceptual principles and processes underpinning it A worked example of the approach in action: evaluating the Youth Primary Mental Health and Addictions initiative in New Zealand Transferrable learning for others considering the use of the VfI approach.

Practitioners Guidance to Assessing Systems Change: Co-Authors Preview - Gover, Dun; Nasution, Zulka; Okutu, David; Bolder, Meghan; Henao, Lina (USAID, 2023)
Check out this video to see what’s inside our new resource: Practitioners' Guidance to Assessing Systems Change, developed by MEL Managers for MEL Managers. (Check out the Guidance here https://bit.ly/MSPMELClinics.) Hear from the authors about which parts they love the most and how this guide challenges MEL managers to assess systems change as an ongoing aspect of implementation, generating feedback that teams need to better understand and catalyze change, for more impact.

Guide to Innovation Partnerships – A United Nations Ghide - Halse, Michelle; Ginsberg-Fletcher, Gabriella; Balbi, Luisa (UN Global Pulse, 2023)

Applying Adaptive Management in a Fragile Context – Case Study - Lonsdale, Jane; Green, Duncan; Robertson, Kelly (DT Global, 2023)
DT Global is proud to introduce our new Guidance Note: Practical Introduction to Adaptive Management There is a growing consensus around adaptive management as an effective (even necessary) approach when programs are tackling complex development problems. While there is no standard definition of adaptive management, there is general agreement that such programs need to routinely engage with and respond to program context; constantly test what works in that context; and adjust approaches, plans, and activities based on continuous learning. However, there remains a more limited body of evidence about what this looks like in practice—the enabling conditions, systems, resourcing, skills, and attitudes to effectively operationalise adaptive management. There is also limited guidance around when adaptive management is required, and to what extent—both critical and often overlooked considerations when planning for successful adaptive management. This Guidance Note draws together lessons and good practice in adaptive management from across DT Global’s diverse portfolio of donor-funded programs. It outlines our conceptual framework for adaptive management, with practical guidance on how it can be applied by our program teams. It is also designed to help our teams distinguish adaptive management from good (non adaptive) project management, consider when adaptive management is most useful on a program, and how adaptive a program (or part of a program) should be.

Applied political economy analysis for human rights programs and campaigns: A guide for practitioners (second edition) - Pact (Pact, 2023)
This updated guide provides practical guidance to practitioners in the human rights sector and beyond on how to integrate Applied Political Economy Analysis

Knowledge Retention and Transfer (KRT) - Model overview presentation - USAID (USAID, 2023)
What problem is the KRT model trying to solve? For workforces that experience continuous staff turnover, the lack of systematic knowledge transfer can often lead to: - Loss of programmatic momentum, - Duplication of efforts and frustration, and - Wasted time and resources. The Knowledge Retention and Transfer (KRT) model provides tools, processes, and practices to individuals, teams, offices, and organizations to improve knowledge handover, which in turn improves efficiency and programmatic and operational learning.

Theory of change software - Better Evaluation (2023)
There are a number of options when it comes to using software to help create a logic model. These range from generic word processing tools (Word, Powerpoint, or their Google Doc or Mac equivalents), to software that has been specifically tailored for visualising Theories of Change, like TOCO or Miradi. You should consider what resources you have to invest in software, both in terms of cost and in time to learn and use the features. If you only have a short timeframe and have simple needs, then a basic tool may suit you better than some of the more complex software available. It's important to investigate a few options and see what is going to be best for you.

Fast Forwarding to Systems Maps of Corruption: Getting to Usable Analysis More Quickly - Woodrow, Peter (CJL, 2023)
For AC practitioners, systems mapping is essential but also difficult. To lessen obstacles, Peter Woodrow proposes a scaffolded approach. An Experiment in “Fast Forwarding” Drawing upon our near-decade work on corruption, we recently decided to try an experiment: we would present “common patterns” of corruption as tentative models to adapt and add to—rather than try to teach people to do systems mapping from scratch. In this teaching experiment, each common pattern would function as a kind of “scaffolding” or framework that participants can build on to generate a cogent systems map as the basis for identifying possible points of intervention and subsequent program planning.

Using a ‘Partnership Rubric’ in Participatory Evaluations - Snijder, Mieke; Hicks, Jacky; Paul, Sukanta; Arulanantham, Amit; Apgar, Marina; Afroze, Jiniya; Karki, Shanta; Mareschal, Sophie; Prieto Martín, Pedro; Uddin, Forhad; Veitch, Helen (CLARISSA Learning Note, 2023)
Programmes that aim to tackle complex societal issues, such as the worst forms of child labour, require rich partnerships that bring together different perspectives. CLARISSA’s consortium partnership adopts an empowerment approach to the interventions we deliver and our ways of working together. Part of this approach involves ongoing reflection and learning about how we work together in our partnership, and how this can be adapted if needed. This learning note focuses on a method used in CLARISSA to both reflect on and strengthen how we work in partnerships – the partnership rubric. We found that using the rubric flexibly was key to mitigating some of the challenges of such a complex consortium. This included using it in different sizes of forum, with different levels of preparation. Periodically adapting it for country context and as new partners came on board also helped ensure a shared sense of our preferred ways of working as the project progressed.

Agile on steroids through the RenDanHeYi - Quintarelli, Emanuele (Boundaryless, 2023)
This article offers initial insights into the overlap, positioning, shared values, and convergences between RenDanHeYi compared to agile practices.

Does the whole add more than the SUM of its parts? - Guerzovich, Florencia (Medium, 2023)
This is a series about Monitoring, Evaluating and Learning (MEL) whether sets of interventions/portfolios are adding more together than each one would produce on their own. In post 1, I pointed to coherence, the new OECD-DAC evaluation criteria as a way to bridge the ambition of bringing bigger change with the MEL world. In post 2, I shared 3 of 4 practical lessons I’ve learned in experimenting with MEL systems and exercises that focus explicitly on interactions of interventions/portfolios. In the third post, I bring Paul Pierson’s groundbreaking argument for social science to MEL. Paraphrasing, most contemporary MEL takes a “snapshot” view of interventions and portfolios, distorting their effects and meaning by ripping them from their temporal context. Instead, we should place in time interventions/portfolios with the ambition to add more than the sum of the part by constructing MEL systems looking at “moving pictures” rather than taking snapshots.

Different kinds of rubrics - King, Julian (Evaluation and Value for Money, 2023)
Generic, analytic, holistic, orbic, asterisc, escalieric, cephalopodic

Lasting Roots: Naatal Mbay and the Integrated Finance Model in Senegal - Fowler, Ben; Courbois, Laura (MSP Ex-Post Study, 2023)
This report addresses the well-recognized evidence gap1 on the longer-term impacts created by marketdriven programming; specifically, programming influenced by market systems development (MSD) principles. It does so by presenting the findings of an ex-post study conducted three and a half years after the close of USAID’s Feed the Future Senegal Naatal Mbay Activity (hereafter Naatal Mbay) in 2019. It examines the scale and sustainability of changes resulting from Naatal Mbay’s introduction of an integrated finance model (IFM) – described in Error! Reference source not found. below – in the domestic rice sector. This study is one in a series of ex-post evaluations that are being conducted between 2023-2026 on USAID-funded MSD interventions around the world. This study focused on four questions, noted below in Figure 1. These were addressed using a mix of desk research, 122 key informant interviews with market actors and other stakeholders remotely and in Senegal, focus group discussions with 26 rice producers networks in Senegal, and a validation workshop with USAID/Senegal, implementing partner staff and market actors. Findings were analyzed leveraging the Disrupting System Dynamics (DSD) framework (see Figure 4 in the body of the report) as an analytical tool for understanding systems change.

Three out of Four Practical Lessons about right fitting MEL for Additive Effects - Guerzovich, Florencia (Medium, 2023)
This is a series about Monitoring, Evaluating and Learning (MEL) whether sets of interventions/portfolios are adding more together than each one would produce on their own. In post 1, I pointed to coherence, the new OECD-DAC evaluation criteria as a way to bridge the ambition of bringing bigger change with the MEL world. In post 2, I shared 3 of 4 practical lessons I’ve learned in experimenting with MEL systems and exercises that focus explicitly on interactions of interventions/portfolios. In the third post, I bring Paul Pierson’s groundbreaking argument for social science to MEL. Paraphrasing, most contemporary MEL takes a “snapshot” view of interventions and portfolios, distorting their effects and meaning by ripping them from their temporal context. Instead, we should place in time interventions/portfolios with the ambition to add more than the sum of the part by constructing MEL systems looking at “moving pictures” rather than taking snapshots.

From mouthset to mindset shifts in co-creating systems change - Blomkamp, Emma; Snow, Thea; Burkett, Ingrid (Griffith Centre for Systems Innovation, 2023)

Adaptive Management Across Project Cycles: Look into Coherence in Time - Guerzovich, Florencia (Medium, 2023)
This is a series about Monitoring, Evaluating and Learning (MEL) whether sets of interventions/portfolios are adding more together than each one would produce on their own. In post 1, I pointed to coherence, the new OECD-DAC evaluation criteria as a way to bridge the ambition of bringing bigger change with the MEL world. In post 2, I shared 3 of 4 practical lessons I’ve learned in experimenting with MEL systems and exercises that focus explicitly on interactions of interventions/portfolios. In the third post, I bring Paul Pierson’s groundbreaking argument for social science to MEL. Paraphrasing, most contemporary MEL takes a “snapshot” view of interventions and portfolios, distorting their effects and meaning by ripping them from their temporal context. Instead, we should place in time interventions/portfolios with the ambition to add more than the sum of the part by constructing MEL systems looking at “moving pictures” rather than taking snapshots. ...

Gestión de la complejidad para el impacto sistémico: respuestas a entornos VUCA y BANI - Peña-López, Ismael (SociedadRed, 2023)
Research on the Information Society, the Digital Divide and Information and Communication Technologies for development

Applying the River of Life Method to Support Reflection and Learning in Terre des hommes Nepal - Karki, Shanta; Giri, Roju; Neupane, Sudarshan; Snijder, Mieke; Apgar, Marina (CLARISSA Learning Note, 2023)
The RoL method is a visual narrative method that helps people tell stories of the past, present, and future. Individuals can use this method to introduce themselves in a fun and descriptive way. A group can use it to understand and reflect on the past and imagine the future of a project. Besides, it can also be used to build a shared view of a process over time while acknowledging different and perhaps contradictory perspectives. The method uses drawings rather than text, making it useful in groups that do not share a common language. Metaphors from a river are used to explore aspects of a story – such as whirlpools depicting challenges or lakes suggesting a sense of calm etc. When used in a group, it is an active method, engaging people in the process of storytelling and listening through visualising their experiences and using metaphors to explore in depth. In CLARISSA, we adapted the RoL method to document our collective understanding of the story of implementation of the programme as part of the programme’s monitoring, evaluation and learning component. The purpose was to surface the details of our process of the systemic Action Research that we are undertaking with children in the worst forms of child labour and business owners. We used the same river metaphors as is often applied when the method is used with individuals.

Is the problem driven iterative adaptation approach (PDIA) a panacea for public financial management reform? Evidence from six African countries - Lawson, Andrew; Harris, Jamelia (World Development Perspectives, 2023)
This article assesses the application of the problem driven iterative adaptation (PDIA) approach to public financial management reform in six African countries. It draws on primary data collected using a mix of interviews, overt participation observations and a short survey. PDIA responds to shortcomings in orthodox approaches to reform and technical assistance in developing countries. It stresses local solutions to local problems, achieved through experimentation and adaptation. The principles of PDIA are appealing, but its empirical assessment is in its infancy. This study aims to fill part of this gap. Findings show that PDIA delivers results in the short-term, particularly in cases where there is an influential authorising agent and dedicated team. Progress was less forthcoming for reforms that required high level political buy-in from senior officials. The approach does exceptionally well to develop staff capability, transferable skills, and local empowerment to solve local problems, thus potentially benefitting future reforms.

Fractal approaches to scaling transformations to sustainability - O’Brien, Karen; Carmona, Rosario; Gram-Hanssen, Irmelin; Hochachka, Gail; Sygna, Linda; Rosenberg, Milda (Ambio, 2023)
Responses to sustainability challenges are not delivering results at the scale and speed called for by science, international agreements, and concerned citizens. Yet there is a tendency to underestimate the large-scale impacts of small-scale, local, and contextualized actions, and particularly the role of individuals in scaling transformations. Here, we explore a fractal approach to scaling sustainability transformations based on “universal values.” Universal values are proposed as intrinsic characteristics that connect humans and nature in a coherent, acausal way. Drawing on the Three Spheres of Transformation framework, we consider how enacting universal values can generate fractal-like patterns of sustainability that repeat recursively across scales. Fractal approaches shift the focus from scaling through “things” (e.g., technologies, behaviors, projects) to scaling through a quality of agency based on values that apply to all. We discuss practical steps involved in fractal approaches to scaling transformations to sustainability, provide examples, and conclude with questions for future research.
Keywords: Agency; Fractals; Relational paradigms; Scaling; Three Spheres of Transformation; Universal values

Sense-making with stakeholders and rubrics - King, Julian (Evaluation and Value for Investment, 2023)
Principles and processes

Causal mapping for evaluators - Powell, Steve; Copestake, James; Remnant, Fiona (Evaluation, 2023)
Evaluators are interested in capturing how things causally influence one another. They are also interested in capturing how stakeholders think things causally i...

The curious case of the realist trial: Methodological oxymoron or unicorn? - Nielsen, Steffen Bohni; Jaspers, Sofie Østergaard; Lemire, Sebastian (Evaluation, 2023)
Realist evaluation and experimental designs are both well-established approaches to evaluation. Over the past 10 years, realist trials—evaluations purposefully combining realist evaluation and experimental designs—have emerged. Informed by a comprehensive review of published realist trials, this article examines to what extent and how realist trials align with quality standards for realist evaluations and randomized controlled trials and to what extent and how the realist and trial aspects of realist trials are integrated. We identified only few examples that met high-quality standards for both experimental and realist studies and that merged the two designs.

Foreign Aid and Its Unintended Consequences - Koch, Dirk-Jan (Routledge, 2023)
Foreign aid and international development frequently bring with it a range of unintended consequences, both negative and positive. This book delves into these consequences, providing a fresh and comprehensive guide to understanding and addressing them. The book starts by laying out a theoretical framework based on complexity thinking, before going on to explore the ten most prevalent kinds of unintended effects of foreign aid: backlash effects, conflict effects, migration and resettlement effects, price effects, marginalization effects, behavioural effects, negative spillover effects, governance effects, environmental effects, and ripple effects. Each chapter revolves around a set of concrete case studies, analysing the mechanisms underpinning the unintended effects and proposing ways in which policymakers, practitioners, and evaluators can tackle negative side effects and maximize positive side effects. The book also includes personal testimonies, a succinct overview of unintended effects, and suggestions for further reading. Providing a clear overview of what side effects to anticipate when planning, executing, and evaluating aid, this book will be an important resource for students, development practitioners, and policymakers alike.

Why are effective feedback mechanisms in cash transfers so important? - Jannat, Mahiratul (CLARISSA, 2023)
The CLARISSA Social Protection Intervention was set us as an innovative social policy intervention for tackling social ills, with a...

Participatory evaluation design and sense-making - making meaning through co-constructed lenses - King, Julian; Elliott, Kerry; Hollingsworth, Hilary (Australian Evaluation Society International Evaluation Conference, 2023)
Based on a presentation at Australian Evaluation Society International Evaluation Conference, Brisbane, 28 September 2023. The importance of participatory approaches to evaluation is well-understood. Involving the right people and balancing voices at the stakeholder table can help to make evaluations more valid, credible, useful, and actually used. Not all evaluations, however, are done in participatory ways. This presentation shares the experience of three evaluators in two recent projects with clients and stakeholders who were unaccustomed to participatory evaluation. In both cases, new ways of working were introduced, requiring participants to come into the evaluation tent, collaborate and learn with us. Engagement processes deliberately focused on incrementally building deep stakeholder input into evaluation design - including key evaluation questions, context analysis, program theory, rubrics, methods of data gathering, and reporting formats. The resulting evaluation frameworks represent an agreed, shared set of lenses for making sense of the evidence. Through this process, relationships and trust were cemented, stakeholders understood the evaluation process and endorsed the resulting frameworks. The client and sector have become more interested, more engaged stakeholders and more savvy evaluation participants and consumers. As the evaluations progress, the evaluation design builds in ongoing stakeholder involvement in reviewing emergent footprints of system change and making evaluative judgements. Participatory approaches are many, varied, and flexible. We will share our reflections and generalisable learning from the participatory approaches we applied in these evaluations. We will invite the audience to reflect on their own practices and experiences, and provoke them to consider whether participatory evaluation is 'just' a methodological option or a foundational element of good evaluation.

How adaptive M&E from the peace sector can help demonstrate the value of aid - Kratzer, Sebastian (2023)
Over the last decade, the peace sector has been developing and adapting Monitoring and Evaluation (M&E) systems and tools to fit their contexts and ways of working. This evolution may hold some insights for the aid community in how to go beyond more traditional, backwards-looking M&E to navigate today’s volatile, interest-based world of politics and aid.

Changing theories - Aston, Thomas (Medium, 2023)
You may, or may not, be surprised to hear that many theories of change lack what we might generally understand as a theory.

Storytelling and Evaluation: Can Pathways be the Theory Missing in Explanatory Narratives? - Guerzovich, Florencia (Medium, 2023)
Florencia Guerzovich

Anatomy of Social Accountability 3.0: A working framework for moving forward - Guerzovich, Florencia; Aston, Tom (Florianopolis, SC, Brazil: Grupo Politeia, ESAG, University of the State of Santa Catarina and act4delivery, 2023)
This note presents the key components of a potential next generation of social accountability programming. The key aspects outlined emerge from a review of over 150 cases as well as tacit knowledge from dozens of practitioners (Guerzovich and Aston, 2023).The main thread of social accountability 3.0 and what distinguishes it from previous generations is a focus on its contribution towards more responsive systems and accountable social contracts. In particular, social accountability should be considered as an operational means to rework social contracts.Over the last two decades, the field has been defined by a supposed dichotomy between minimalist (i.e., “widget” based) approach or a maximalist (“go big or go home”) approaches. Yet, most practice sits somewhere between these extremes. The paper presents various options derived from practice; practitioners should choose whichever best fits their own particular circumstances.Those hoping to improve and expand their practice with a systems-lens may benefit from considering how they approach the key elements of social accountability 3.0.
Keywords: accountability; development; responsiveness; social accountability; social contract; systems

Hidden Potential: The Science of Achieving Greater Things - Grant, Adam (WH Allen, 2023)
#1 New York Times bestseller "This brilliant book will shatter your assumptions about what it takes to improve and succeed. I wish I could go back in time and gift it to my younger self. It would've helped me find a more joyful path to progress."-Serena Williams, 23-time Grand Slam singles tennis championThe #1 New York Times bestselling author of Think Again illuminates how we can elevate ourselves and others to unexpected heights.We live in a world that’s obsessed with talent. We celebrate gifted students in school, natural athletes in sports, and child prodigies in music. But admiring people who start out with innate advantages leads us to overlook the distances we ourselves can travel. We can all improve at improving. And when opportunity doesn’t knock, there are ways to build a door.Hidden Potential offers a new framework for raising aspirations and exceeding expectations. Adam Grant weaves together groundbreaking evidence, surprising insights, and vivid storytelling that takes us from the classroom to the boardroom, the playground to the Olympics, and underground to outer space. He shows that progress depends less on how hard you work than how well you learn. Growth is not about the genius you possess – it’s about the character you develop. Grant explores how to build the character skills and motivational structures to realize our own potential, and how to design systems that create opportunities for those who have been underrated and overlooked.This book reveals how anyone can rise to achieve greater things. The true measure of your potential is not the height of the peak you’ve reached, but how far you’ve climbed to get there.

Challenge-led Innovation Workbook. Organising for Systems Innovation at Scale - Burkett, Ingrid (Griffith Centre for Systems Innovation, 2023)
Organising for Systems Innovation at Scale Our team at Griffith Centre for Systems Innovation have been experimenting with and evolving a Challenge-led Innovation Approach (based on Mission-oriented approaches developed by Mariana Mazzucato at UCL IIPP and others internationally). We are using this approach to guide the way we work internally and engage with our systems innovation partners. We’ve facilitated intensive Re:Treats, worked with government bodies, businesses and civic organisations, and engaged deeply with others exploring this work. We have a bias for developing and testing HOW such approaches could be applied to respond to both local and global challenges rather than getting too caught up in the what and why of such approaches. We decided to openly share our learnings and thinking to date in this workbook, to spark conversations and innovation in both practice and thinking amongst those exploring how we work, and to learn together to address complex systems and challenges. We see this booklet as a first step in a longer learning journey. In it we share an overview of: the principles and processes that sparked our evolution to a Challenge-led Innovation framework (from Mission-oriented). examples of our learnings from other system innovators who are experimenting. an adaptable process to help guide the learning journey. learning tools and canvases to catalyse thinking, practice, and further adaptations. Part One sets out some foundations we’ve identified as important to Challenge-led Innovation. If you want to jump straight into the mapping process, we suggest you skip to Part Two. The final section, Part Three, focuses on what we have learnt about the conditions needed and how to get started on a Challenge-led initiative.

Scaling Up Development Impact - Guerrero, Isabel; Gokhale, Siddhant; Fahsbender, Jossie (Bowker, 2023)
Today, nearly one billion people lack electricity, over three billion lack clean water, and 750 million lack basic literacy skills. Many of these challenges could be solved with existing solutions, and technology enables us to reach the last mile like never before. Yet, few solutions attain the necessary scale to match the size of these challenges. Scaling Up Development Impact offers an analytical framework, a set of practical tools, and adaptive evaluation techniques to accompany the scaling process. It presents rich organizational experiences that showcase real-world journeys toward increased impact. From the people from IMAGOgg.org

Life Stories From Children Working in Bangladesh’s Leather Sector and its Neighbourhoods: Told and Analysed by Children - Sayem, Mashrique; Sayed, Sayma; Maksud, A. K. M.; Reaz Hossain, Khandaker; Afroze, Jiniya; Burns, Danny; Raw, Anna; Hacker, Elizabeth (CLARISSA Research and Evidence Paper, 2023)
CLARISSA (Child Labour: Action-Research-Innovation in South and South-Eastern Asia) has a participatory and child-centred approach that supports children to gather evidence, analyse it themselves and generate solutions to the problems they identify. The life story collection and collective analysis processes supported children engaged in the worst forms of child labour in Bangladesh to share and analyse their life stories. Over 400 life stories were collected from children who worked in the leather supply chain, or who lived and worked in leather sector neighbourhoods. Using causal mapping, 53 children who were engaged in or had experience of the worst forms of child labour collectively analysed the data. This resulted in children’s life stories becoming the evidence base for revealing macro‑level system dynamics that drive the worst forms of child labour. This paper is a record of the children’s analysis of the life stories and key themes they identified, which formed the basis of a series of seven child-led Participatory Action Research groups.

3 things that are enabling the UNDP’s shift to portfolios - Begovic, Milica (Medium, 2023)
By Millie Begovic, Head of the UNDP Strategic Innovation Unit

How AI Fits into Lean Six Sigma - Holweg, Matthias; Davenport, Thomas H.; Snyder, Ken (Harvard Business Review, 2023)
AI already is being used in some areas of process improvement, and the usage of this technology — including generative AI — promises to grow. That’s because it can perform tasks faster and much less expensively than humans alone. But it will never fully replace people — and that poses management challenges.

Insights for Influence: Understanding Impact Pathways in Crisis Response - Clark, Louise; Carpenter, Jo; Taylor, Joe (Institute of Development Studies, 2023)
The Covid-19 Responses for Equity (CORE) programme was a three-year initiative funded by the Canadian International Development Research Centre (IDRC) that brought together 20 projects from across the global South to understand the socioeconomic impacts of the Covid-19 pandemic, improve existing responses, and generate better policy options for recovery. The research covered 42 countries across Africa, Asia, Latin America, and the Middle East to understand the ways in which the pandemic affected the most vulnerable people and regions, and deepened existing vulnerabilities. Research projects covered a broad range of themes, including macroeconomic policies for support and recovery; supporting essential economic activity and protecting informal businesses, small producers, and women workers; and promoting democratic governance to strengthen accountability, social inclusion, and civil engagement. The Institute of Development Studies (IDS) provided knowledge translation (KT) support to CORE research partners to maximise the learning generated across the research portfolio and deepen engagement with governments, civil society, and the scientific community. As part of this support, the IDS KT team worked with CORE project teams to reconstruct and reflect on their impact pathways to facilitate South-South knowledge exchange on effective strategies for research impact, and share learning on how the CORE cohort has influenced policy and delivered change. This report presents an overview of these impact pathways and the lessons learnt from a selection of the projects chosen to represent the diversity of approaches to engage policymakers, civil society, and the media to generate and share evidence of the effect of the pandemic on diverse vulnerable groups.

Unconditional cash support sparks transformation: A story from CLARISSA Social Protection Intervention in Dhaka - Aktar, Afrin; Debnath, Raj; Kashfi, Sharmin (CLARISSA, 2023)
The CLARISSA Social Protection (SP) intervention provided six months of unconditional cash transfers to every household in the Gojmohol neighbourhood,...

Disentangling Government Responses: How Do We Know When Accountability Work is Gaining Traction? - Fox, Jonathan; Halloran, Brendan; Fölscher, Alta; McGee, Rosie (Working Paper, 2024)
Advocacy for public accountability aims to produce certain reactions from government officials or service providers. However, the reactions can be many and diverse, and it is not always clear to advocates how to interpret them and decide on next steps—whether to intensify efforts or back off; continue the same strategy or make adjustments. This paper presents a framework to help accountability advocates and practitioners interpret government reactions to their efforts and move forward appropriately. The framework arises from learning and reflection in the context of the International Budget Partnership (IBP)’s Strengthening Public Accountability with Results and Knowledge (SPARK) program. SPARK seeks to bolster the collective agency of marginalized communities and coalitions to advance democratic and equitable fiscal governance systems that channel public resources to services that address the priority needs of these historically excluded groups. The paper: · unpacks the broad umbrella category of government responses to citizen-led accountability initiatives, discerning within it three overlapping categories of responses, responsiveness, and accountable responsiveness (RRA), and pinpointing what distinguishes them; · maps out the public financial management (PFM) system as the critical conduit for any public service to progress from non-response through these three categories, reflecting the terrain and priorities of the SPARK program and IBP more broadly. Looking at RRA in the context of this PFM map offers key insights for the twin tasks of tracing and unblocking bottlenecks in resource management systems that hinder responsiveness and accountability, as well as demanding specific reforms that make the fiscal governance system more open to citizen engagement; and · discusses how the RRA framework can be used to take stock of partial or incremental responses and assess whether change is moving towards accountable governance, by reference to the SPARK program and other IBP efforts. The RRA framework supports accountability-claiming practice by facilitating close attention to the actions of and influences on key government stakeholders, careful interpretation of these, and consequential adjustments to advocacy strategies. Its implications for monitoring, evaluating, and learning (MEL) from practice are also significant. MEL has been evolving in the accountability field in line with the thinking and practical challenges behind the development of the RRA framework. The framework adds to a range of emerging approaches that support the application theories of change, evidence-based adaptive programming, and politically aware systems thinking in the complex task of building accountable responsiveness towards constituencies historically excluded from public goods and services, and ultimately equitable development outcomes from an intersectional perspective.
Keywords: Development studies; Public policy; Social policy

Scaling up Social Accountability in Complex Governance Systems: A Relational Approach for Evidencing Sustainability - Guerzovich, Florencia; Wadeson, Alix (World Bank, 2024)
When social accountability interventions scale up and their sustainability depends on the interactions of many agents and system components, related results are rarely observable at the end of an intervention. The 2019 OECD Development Assistance Committee’s (OECD DAC) revamped evaluations criteria for assessing sustainability acknowledges that such results are often emergent, and should be monitored and evaluated with this in mind. It therefore emphasizes a turn towards assessing complex processes prospectively. It also asks evaluations to consider how likely it is that these results are evident at the time they are monitored or evaluated. However,the social accountability field continues to have gaps regarding doing this effectively in practice. This paper presents and provides evidence from testing an innovative operational approach that has promising potential to support this aim - a sequential, relational rubric. This approach can support practitioners to monitor, evaluate and learn about the causal processes of scale up of social accountability interventions with an eye towards sustainability i.e., considering prospective sustainability. It is grounded in systems thinking, co-production and social learning theory, as well as links with collective governance and social contract theory for development. Evidence yielded from the authors’ testing of this approach on a sample of diverse projects from the Global Partnership for Social Accountability (GPSA) program revealed that the alleged ‘absence of evidence’ dilemma of social accountability scale up is due to ill-fitting concepts and methods for assessment. It challenges existing assumptions and findings that claim that social accountabilityprocesses do not scale and are unsustainable. The authors propose that by using fit-for-purpose concepts and methods with a focus on social learning and compromise – also called a ‘resonance pathway to scale’ which this paper discusses in detail – it is possible to observe loosely coordinated scale up processes at work in many (but not all) social accountability interventions and identify tangible evidence of prospective sustainability. An important caveat is that these processes, the outcomes they generate, and the corresponding evidence often look qualitatively different than the original intervention design and predictions for scale-up at that point in time. This is because the process of deliberation and compromise inherent to social accountability work in dynamic local systems introduces changes and new conditions for uptake by diverse actors in the public sector, civil society, and donor institutions. The paper concludes that even relatively small-scale localized projects of three to five years with budgets of less than one million USD, across different contexts and sectors can produce processes and outcomes which contribute to many forms of sustainability, including via scaleup.Furthermore, the cross-fertilization of learning and aggregation of results for scale-up across projects within and beyond the GPSA (and other programs) can help monitoring evaluation and learning (MEL) and social accountability practitioners alike to deliver on a program’s mandate. Doing so can also create new knowledge for the wider social accountability field that siloed interventions, lacking suitable concepts and methods for assessing scale-up and prospective sustainability, often fail to produce. The paper ends with recommendations for taking forward this approach and the associated benefits, implications and required investments.

Breaking free from the theory of change straight jacket - Apgar, Marina; Snijder, Mieke; Paul, Sukanta (CLARISSA, 2024)
The CLARISSA Social Protection Intervention was set us as an innovative social policy intervention for tackling social ills, with a...

Social Accountability 3.0: Engaging Citizens to Increase Systemic Responsiveness. Background Paper - Guerzovich, Florencia; Aston, Tom (Grupo Politeia, ESAG, UDESC and act4delivery, 2024)
Social accountability has become increasingly important to development programming since the 2004 World Development Report spearheaded its first generation of programming. In 2016, Thomas Carothers synthesized global experts’ perspectives on what became known as the second generation. With an evolving global context, new research and evaluations and tacit knowledge on how those assumptions played out in practice, the time is ripe for asking what the evidence since 2004 can tell us about what the social accountability’s next generation might look like? This meta-analysis takes stock of that knowledge through a systems lens and presents a possible “social accountability 3.0.” It draws on a database of 157 cases of initiatives implemented by different civil society organizations around the world and the lived experience of practitioners. Scholarly debates present practitioners’ options in binary terms, contrasting 1.0 minimalist and 2.0 maximalist options. The review finds that this binary is inaccurate and misleading; it masks practitioners’ diverse interpretations of 2.0 and range of feasibility considerations. Instead, work in the last decade looks more like a mosaic. By opening the analysis to the broader range of evidence, and reflecting more critically on scholars’ assumptions, a more adaptable middle path for social accountability 3.0 emerges. Its key outcomes include contributing to systemic responsiveness and an operational mechanism to rework the social contract.

Addressing the evaluation-of-sustainability-paradox: A relational rubric for evidencing… - Guerzovich, Florencia (Medium, 2024)
Florencia Guerzovich and Alix Wadeson

The Radical How - Greenway, Andrew; Loosemore, Tom (Nesta and Public Digital, 2024)
Any mission-focused government should be well equipped to define, from day one, what outcomes it wants to bring about. But radically changing what the government does is only part of the challenge. We also need to change how government does things. The usual methods, we argue in this paper, are too prone to failure and delay. There’s a different approach to public service organisation, one based on multidisciplinary teams, starting with citizen needs, and scaling iteratively by testing assumptions. We’ve been arguing in favour of it for years now, and the more it gets used, the more we see success and timely delivery. We think taking a new approach makes it possible to shift government from an organisation of programmes and projects, to one of missions and services. It offers even constrained administrations an opportunity to improve their chances of delivering outcomes, reducing risk, saving money, and rebuilding public trust. The Radical How in a nutshell The struggles and shortcomings of delivering in government are well rehearsed. Many of the root causes that make it tough have been restated several times over several decades. But what to do? We believe the government can and should change how it delivers, by: organising around multidisciplinary teams embracing incremental, feedback-driven iteration focusing more on outcomes. The Radical How is a change of mindset as much as a change in organisation. It promotes methods and processes that have been shown to work, multiple times, at scale. They are the default ways of working for many of the world’s most successful companies. However, the occasions where they have been deployed are rare in government. These occasions have come about thanks to exceptional leaders, exceptional circumstances, or both. We think they’d make a big difference if they became the norm, rather than the exception. We also think that without them, mission oriented government will not become a reality. New policy ideas will remain just that, rather than translating into profound improvements to society. Central to this approach is the widespread adoption of internet-era ways of working. This paper explains both those and our thinking in more detail, with reference to real examples.

Doing weeknotes - What weeknotes are, how weeknotes work, and how to start writing weeknotes of your own - Turnbull, Giles (Use the Human Voice, 2024)
Doing weeknotes brings together various things I’ve written about weeknotes in different places. This text expands on things I wrote in The agile comms handbook, as well as various blog posts. Quite a lot of it is brand new. - Weeknotes for beginners - Why write weeknotes - The weeknotes rules - Weeknotes within the corporate environment - What weeknotes can bring about - Examples of good weeknotes - How to write weeknotes - Weeknotes tips and tricks - Further reading

Partnering with communities to co-design humanitarian health strategies: A SeeChange CommunityFirst Framework for implementation in MSF projects - Chapela Trillo, Violeta; Farber, Jessica (MSF, 2034)
The CommunityFirst Framework is intended to be implemented by field teams at MSF. The theoretical aspects and evidence presented on the importance of community engagement are intended for all MSF staff seeking to learn more about why and how to shift the way we work with communities as humanitarians. We believe this guideline, and other tools like it (including OCA’s Person-Centred Approach Guidance07, and MSF Vienna Evaluation Unit’s Guidance for Involving Communities08), to be an important contribution to the growing movement of communities and humanitarian actors who are pushing for changes in the humanitarian system that translate to dignity, health, justice, equity and self-determination for communities around the world. Specifically, the CommunityFirst Framework is intended to guide MSF teams to co-design health strategies with communities, throughout all stages of the project cycle, for exploratory missions, projects that are just opening, projects that have been running for some time, or those that are closing. At the time of publication, the CommunityFirst Framework has been tested in pilot projects in: (1) Madre de Dios, Peru (MSF OCP, August 2022), (2) Tonkolili, Sierra Leone (MSF OCA, November 2022) and (3) Anzoátegui, Venezuela (MSF OCB, February 2023) The experiences from these pilots (feedback from teams, implementation results, adaptations to each context, etc.) have informed the adaptation of the Framework. CommunityFirst builds on existing community engagement work inside MSF and contributes a practical framework for co-designing health initiatives with communities. To avoid duplicating efforts and resources around community engagement inside MSF, the appendices in this guideline largely refer to already existing MSF resources.09 This guideline is meant to be a living document that can evolve and be adapted given the experience of MSF staff and community members and diverse community contexts. This guide can be used by anyone in MSF who is interested in partnering with communities to improve the responsiveness and impact of their humanitarian programs. This is the first iteration of the document. Subsequent iterations will be published based on additional testing during future phases of the CommunityFirst TIC project.