Webimar - Adaptive Management at the Strategy Level: Portfolio Reviews and Mid Course Stocktaking - USAID (USAID, 2017)
On Wednesday, November 15, 2017 at 8:00 a.m. EST, the Bureau for Policy, Planning and Learning (PPL) held a one-hour webinar exploring Strategy-level Portfolio Reviews and Mid Course Stocktaking.

Portfolios of Agile Projects: A Complex Adaptive Systems’ Agent Perspective - Sweetman, Roger; Conboy, Kieran (Project Management Journal, 2018)
While agile approaches can be extremely effective at a project level, they can impose significant complexity and a need for adaptiveness at the project portfolio level. While this has proven to be highly problematic, there is little research on how to manage a set of agile projects at the project portfolio level. What limited research that does exist often assumes that portfolio-level agility can be achieved by simply scaling project level agile approaches such as Scrum. This study uses a complex adaptive systems lens, focusing specifically on the properties of projects as agents in a complex adaptive portfolio to critically appraise current thinking on portfolio management in an agile context. We then draw on a set of 30 expert interviews to develop 16 complex adaptive systems (CAS)-based propositions as to how portfolios of agile projects can be managed effectively. We also outline an agenda for future research and discuss the differences between a CAS-based approach to portfolio management and traditional approaches.

Monitoring and learning for country-level portfolio decision-making and adaptation - Buffardi, Anne L; Mason, Paige; Hutchings, Claire; Sharp, Samuel (Briefing Note, 2019)
Most measurement and adaptive management approaches were developed for and from individual projects. This briefing aims to guide measurement and management of country-level portfolios of work. It identifies potential purposes portfolio-level analyses can fulfil, types of adaptation, and the relative role of monitoring, learning and evaluation (MEL). Drawing on reviews of practice from the United States Agency for International Development (USAID) and the United Kingdom Department for International Development (DFID), guidance notes, and experiences of members of the Global Learning for Adaptive Management (GLAM) initiative, it offers recommendations and considerations that are particularly relevant for this level of analysis and management. Key messages - Portfolio-level analyses can serve eight potential purposes, each of which answers different questions, involves adaptation at different times and levels, and requires different types of evidence. Identifying the purpose(s) and how the component parts relate to each other should guide the development of monitoring and learning systems. - Portfolios are oriented more towards breadth than depth, involve more people with different perspectives, and draw on multiple sources of evidence with potentially greater variation in quality. - In practice, four activities appear to be applied most frequently at a portfolio level: alignment of indicators and aggregation of monitoring data; synthesis of multiple sources and types of information to provide a summary of outputs, outcomes, common observations and trends; periodic review and reflection sessions; and strategic planning, design or refresh of the portfolio strategy. - The extent to which evidence-informed portfolio management is facilitating learning and adaptation has not been well documented to date, and we suggest potential indicators to do so.

DFID’s approach to value for money in programme and portfolio management - ICAI (Independent Commission for Aid Impact (ICAI), 2018)
ICAI published this review on DFID’s approach to value for money in February 2018, and as value for money is both a process and an outcome and cuts across all aspects of DFID’s operations, did not score this review. We made five recommendations and published a follow-up to this review in July 2019. All UK government departments are required to achieve value for money in their use of public funds. In recent years, DFID has been working to build value for money considerations further into its management processes and its relationships with implementers and multilateral partners, establishing itself as a global champion on value for money. Review This review was published in February. Though it was not rated, ICAI made five recommendations, and found that the Department for International Development’s approach to value for money was helping to make UK aid spending go further, but improvements are still needed. Findings This review found that DFID has strengthened its processes and systems for ensuring it gains maximum value for each pound spent, has taken swift remedial action to tackle under-performing programmes, and has become a strong global champion on value for money. The review also found that DFID has been diligent in its efforts to cut waste, detect fraud, and improve efficiency, and that this work is improving the return on the UK investment in aid. However, the review found that DFID’s approach was not adequately reporting and capturing results and value for money at the country portfolio level, or how programmes work together to deliver lasting impact, including reducing future dependency on aid. It also found that weakness in the annual review process could undermine DFID’s approach to value for money. It found that targets were frequently revised, and that there could be pressures for optimistic scoring of programmes. Recommendations Based on this review, we made the following five recommendations to help DFID improve its approach to value for money still further: DFID country offices should articulate cross-cutting value for money objectives at the country portfolio level, and should report periodically on progress at that level. Drawing on its experience with introducing adaptive programming, DFID should encourage programmes to experiment with different ways of delivering results more cost-effectively, particularly for more complex programming. DFID should ensure that principles of development effectiveness – such as ensuring partner country leadership, building national capacity and empowering beneficiaries – are more explicit in its value for money approach. Programmes should reflect these principles in their value for money frameworks, and where appropriate incorporate qualitative indicators of progress at that level. DFID should be more explicit about the assumptions underlying the economic case in its business cases, and ensure that these are taken into account in programme monitoring. Delivery plans should specify points in the programme cycle when the economic case should be fully reassessed. Senior responsible owners should also determine whether a reassessment is needed following material changes in the programme, results targets or context. Annual review scores should include an assessment of whether programmes are likely to achieve their intended outcomes in a cost-effective way. DFID should consider introducing further quality assurance into the setting and adjustment of logframe targets.

Fostering Learning in Large Programmes and Portfolios: Emerging Lessons from Climate Change and Sustainable Development - Harvey, Blane; Pasanen, Tiina; Pollard, Alison; Raybould, Julia (Sustainability, 2017)
In ﬁelds like climate and development, where the challenges being addressed can be described as “wicked”, learning is key to successful programming. Useful practical and theoretical work is being undertaken to better understand the role of reﬂexive learning in bringing together different knowledge to address complex problems like climate change. Through a review of practical cases and learning theories commonly used in the areas of resilience, climate change adaptation and environmental management, this article: (i) reviews the theories that have shaped approaches to reﬂexive learning in large, highly-distributed climate change and resilience-building programmes for development; and (ii) conducts a comparative learning review of key challenges and lessons emerging from early efforts to promote and integrate reﬂexive learning processes in programmes of this nature. Using a case study approach, the authors focus on early efforts made in four large, inter-related (or nested) programmes to establish, integrate and sustain learning processes and systems. Eight themes emerged from the review and are considered from the perspective of learning programmes as emergent communities of practice. By investigating how these themes play out in the nested programming, the paper contributes to the limited existing body of evidence on learning in large climate change programmes and identiﬁes areas where future efforts might focus.

Impact evaluation for portfolio programmes on policy influence: Reflections on the Indonesian Poverty Reduction Support Facility - Mackenzie, Jessica; Hearn, Simon (Methods Lab, 2016)
• Donors are increasingly using portfolio-based programmes that embrace ‘good failure’ and adaptive, political programming. • However, measuring the impact of these programmes is challenging, especially for those working on policy influence and building country systems; not only do you need to measure the positive and negative impact of the overall portfolio, but also the different pathways tested. • Programmes, therefore, need a light-touch monitoring and evaluation system that allows it to remain flexible. • Good practice examples of portfolio-based programmes present six strategies to evaluate impact: 1. Develop appropriate logic models 2. Collect observational data throughout implementation 3. Develop stories of change or case studies 4. Understand causal relationships without a counterfactual 5. Purposefully select which activities to study 6. Be explicit about how impact will be valued across the portfolio. • These strategies are only useful if monitoring and evaluation is placed at the centre of programme decision-making.

Summarising portfolio change: results frameworks at organisational level - Simister, Nigel (M&E Paper 10, 2016)

Program Cycle - How-To Note: Strategy-Level Portfolio Review - USAID (USAID PPL, 2017)

Measuring Impact. Making Use of the Portfolio: Organizational Learning at USAID - Peabody, Shawn (Measuring Impact, 2015)
This technical analysis explores previous and ongoing social learning efforts, best practices, challenges, and lessons in USAID as a foundation for improving the implementation and design of the Agency’s forestry and biodiversity programs. This analysis is particularly relevant as the Bureau of Economic Growth, Education and the Environment’s Office of Forestry and Biodiversity (E3/FAB) begins to develop a Cross-Mission Learning Program under the Measuring Impact initiative, a five-year activity to promote the adoption of best practices in the USAID program cycle among Missions using biodiversity funds. The findings reported in this document can help inform the overall design and structure of the Learning Program and provide insight into possible challenges and best practices.