A comprehensive framework for evaluating AI systems in development contexts. Features tools, case studies, and methodologies for responsible AI assessment. Comprehensive guidance, practical tools, and real-world case studies to help you implement effective AI evaluation in development contexts. Continuous evaluation is a critical tool for AI product developers. Generative AI (GenAI) is a relatively new technology, and as a result product development today is more of an art than a science. By rapidly iterating through different AI models, architectures, prompts, and knowledge bases, GenAI developers can steadily improve a product or workflow. So it is no surprise that AI evaluation tools have gained significant attention from software companies, investors, and academics alike. But what should evaluations look like in the social sector? One of the most compelling use cases for AI in the social sector is its potential to cost-effectively deliver personalized decision-making support for millions of people. Done right, this technology can help individuals exercise greater agency over their lives and improve outcomes in meaningful, measurable ways. To bring clarity and structure to the evaluation of AI services in the social sector – for both funders and service delivery organizations – we have introduced a four-level framework. We first shared the framework at a technical conference in Bangalore in March 2025. We later wrote about it in a blog post co-authored with the Center for Global Development (CGD) and J-PAL. This framework lays out four core questions that can guide AI evaluations in human development contexts: Introduction 1 Model evaluation: Does the AI model produce the desired responses? 2 Product evaluation: Does the product facilitate meaningful interactions? 3 User evaluation: Does the product positively support users’ thoughts, feelings, and actions? 4 Impact evaluation: Does access to the product improve human development outcomes?

Die folgenden Ressourcen beziehen sich durch Zitation auf diese Veröffentlichung.