DFID’s approach to value for money in programme and portfolio management - ICAI (Independent Commission for Aid Impact (ICAI), 2018)
ICAI published this review on DFID’s approach to value for money in February 2018, and as value for money is both a process and an outcome and cuts across all aspects of DFID’s operations, did not score this review. We made five recommendations and published a follow-up to this review in July 2019. All UK government departments are required to achieve value for money in their use of public funds. In recent years, DFID has been working to build value for money considerations further into its management processes and its relationships with implementers and multilateral partners, establishing itself as a global champion on value for money. Review This review was published in February. Though it was not rated, ICAI made five recommendations, and found that the Department for International Development’s approach to value for money was helping to make UK aid spending go further, but improvements are still needed. Findings This review found that DFID has strengthened its processes and systems for ensuring it gains maximum value for each pound spent, has taken swift remedial action to tackle under-performing programmes, and has become a strong global champion on value for money. The review also found that DFID has been diligent in its efforts to cut waste, detect fraud, and improve efficiency, and that this work is improving the return on the UK investment in aid. However, the review found that DFID’s approach was not adequately reporting and capturing results and value for money at the country portfolio level, or how programmes work together to deliver lasting impact, including reducing future dependency on aid. It also found that weakness in the annual review process could undermine DFID’s approach to value for money. It found that targets were frequently revised, and that there could be pressures for optimistic scoring of programmes. Recommendations Based on this review, we made the following five recommendations to help DFID improve its approach to value for money still further: DFID country offices should articulate cross-cutting value for money objectives at the country portfolio level, and should report periodically on progress at that level. Drawing on its experience with introducing adaptive programming, DFID should encourage programmes to experiment with different ways of delivering results more cost-effectively, particularly for more complex programming. DFID should ensure that principles of development effectiveness – such as ensuring partner country leadership, building national capacity and empowering beneficiaries – are more explicit in its value for money approach. Programmes should reflect these principles in their value for money frameworks, and where appropriate incorporate qualitative indicators of progress at that level. DFID should be more explicit about the assumptions underlying the economic case in its business cases, and ensure that these are taken into account in programme monitoring. Delivery plans should specify points in the programme cycle when the economic case should be fully reassessed. Senior responsible owners should also determine whether a reassessment is needed following material changes in the programme, results targets or context. Annual review scores should include an assessment of whether programmes are likely to achieve their intended outcomes in a cost-effective way. DFID should consider introducing further quality assurance into the setting and adjustment of logframe targets.

Fostering Learning in Large Programmes and Portfolios: Emerging Lessons from Climate Change and Sustainable Development - Harvey, Blane; Pasanen, Tiina; Pollard, Alison; Raybould, Julia (Sustainability, 2017)
In ﬁelds like climate and development, where the challenges being addressed can be described as “wicked”, learning is key to successful programming. Useful practical and theoretical work is being undertaken to better understand the role of reﬂexive learning in bringing together different knowledge to address complex problems like climate change. Through a review of practical cases and learning theories commonly used in the areas of resilience, climate change adaptation and environmental management, this article: (i) reviews the theories that have shaped approaches to reﬂexive learning in large, highly-distributed climate change and resilience-building programmes for development; and (ii) conducts a comparative learning review of key challenges and lessons emerging from early efforts to promote and integrate reﬂexive learning processes in programmes of this nature. Using a case study approach, the authors focus on early efforts made in four large, inter-related (or nested) programmes to establish, integrate and sustain learning processes and systems. Eight themes emerged from the review and are considered from the perspective of learning programmes as emergent communities of practice. By investigating how these themes play out in the nested programming, the paper contributes to the limited existing body of evidence on learning in large climate change programmes and identiﬁes areas where future efforts might focus.

Impact evaluation for portfolio programmes on policy influence: Reflections on the Indonesian Poverty Reduction Support Facility - Mackenzie, Jessica; Hearn, Simon (Methods Lab, 2016)
• Donors are increasingly using portfolio-based programmes that embrace ‘good failure’ and adaptive, political programming. • However, measuring the impact of these programmes is challenging, especially for those working on policy influence and building country systems; not only do you need to measure the positive and negative impact of the overall portfolio, but also the different pathways tested. • Programmes, therefore, need a light-touch monitoring and evaluation system that allows it to remain flexible. • Good practice examples of portfolio-based programmes present six strategies to evaluate impact: 1. Develop appropriate logic models 2. Collect observational data throughout implementation 3. Develop stories of change or case studies 4. Understand causal relationships without a counterfactual 5. Purposefully select which activities to study 6. Be explicit about how impact will be valued across the portfolio. • These strategies are only useful if monitoring and evaluation is placed at the centre of programme decision-making.

Integrated Governance: Achieving Governance Results and Contributing to Sector Outcomes - Frazer, Sarah; Granius, Mark; Brinkerhoff, Derick W.; McGregor, Lisa (RTI Press Publication, 2022)
Achieving broad-based socio-economic development requires interventions that bridge disciplines, strategies, and stakeholders. Effective sustained progress requires more than simply an accumulation of sector projects, and poverty reduction, individual wellbeing, community development, and societal advancement do not fall neatly into sectoral categories. However, researchers and practitioners recognize key operational challenges to achieving effective integration that stem from the structures and processes associated with the current practice of international development. Integration calls for the intentional linking of intervention designs, implementation, and evaluation across sectors and disciplines to achieve mutually reinforcing outcomes. In this report, we summarize the results of a study we conducted to explore the challenges facing governance programs that integrate with sector interventions to achieve governance outcomes and contribute to sector outcomes. Through a review of policy documents and project reports from recent integrated governance programs and interviews with donor and practitioner staff, we found three integrated governance programming variants, an emphasis on citizen and government collaboration to improve service delivery, interventions that serve as the glue between sectors, and a balancing act for indicators to measure contribution to sectoral outcomes. Our analysis identified four key success factors: contextual readiness, the application of learning and adapting approaches, donor support, and recognition of the limitations of integrated governance. We then discuss recommendations and implications and for answering the challenge of integrating governance and sector programming to achieve development outcomes.

Measuring Impact. Making Use of the Portfolio: Organizational Learning at USAID - Peabody, Shawn (Measuring Impact, 2015)
This technical analysis explores previous and ongoing social learning efforts, best practices, challenges, and lessons in USAID as a foundation for improving the implementation and design of the Agency’s forestry and biodiversity programs. This analysis is particularly relevant as the Bureau of Economic Growth, Education and the Environment’s Office of Forestry and Biodiversity (E3/FAB) begins to develop a Cross-Mission Learning Program under the Measuring Impact initiative, a five-year activity to promote the adoption of best practices in the USAID program cycle among Missions using biodiversity funds. The findings reported in this document can help inform the overall design and structure of the Learning Program and provide insight into possible challenges and best practices.

Portfolios of Agile Projects: A Complex Adaptive Systems’ Agent Perspective - Sweetman, Roger; Conboy, Kieran (Project Management Journal, 2018)
While agile approaches can be extremely effective at a project level, they can impose significant complexity and a need for adaptiveness at the project portfolio level. While this has proven to be highly problematic, there is little research on how to manage a set of agile projects at the project portfolio level. What limited research that does exist often assumes that portfolio-level agility can be achieved by simply scaling project level agile approaches such as Scrum. This study uses a complex adaptive systems lens, focusing specifically on the properties of projects as agents in a complex adaptive portfolio to critically appraise current thinking on portfolio management in an agile context. We then draw on a set of 30 expert interviews to develop 16 complex adaptive systems (CAS)-based propositions as to how portfolios of agile projects can be managed effectively. We also outline an agenda for future research and discuss the differences between a CAS-based approach to portfolio management and traditional approaches.

Program Cycle - How-To Note: Strategy-Level Portfolio Review - USAID (USAID PPL, 2017)

Seeing the Combined Effects of Aid Programmes - Burge, Richard; Nadelman, Rachel; McGee, Rosie; Fox, Jonathan; Anderson, Colin (IDS Policy Briefing, 2022)
Multiple aid agencies often try to support change in the same places, at the same time, and with similar actors. Surprisingly, their interactions and combined effects are rarely explored. This Policy Briefing describes findings from research conducted on recent aid programmes that overlapped in Mozambique, Nigeria, and Pakistan, and from a webinar with UK Foreign, Commonwealth & Development Office (FCDO) advisors and practitioners. The research found three distinct categories of ‘interaction effects’: synergy, parallel play, and disconnect. We explore how using an ‘interaction effects’ lens in practice could inform aid agency strategies and programming.

Summarising portfolio change: results frameworks at organisational level - Simister, Nigel (M&E Paper 10, 2016)

System Change: A Guidebook for Adopting Portfolio Approaches - Wellsch, Brent (UNDP, 2022)
This guidebook codifies the principles and methods of applying systems change and portfolio approaches to complex development challenges with practical tools and examples. It is based on the empirical learning generated from the collaborative initiatives in UNDP Country Offices in Bhutan, Pakistan, the Philippines, and Viet Nam with support from Regional Innovation Centre for Asia and the Pacific.