Balancing Innovation and Rigor: Guidance for the Thoughtful Integratio n of Artificial Intelligence for Evaluation - Anuj, Harsh; Den Boer, Hannah; Raimondo, Estelle (IEG, The World Bank & IOE, IFAD, 2025)
One of the main questions evaluators have when using AI is: "How can I know if it performs well?" To answer this question, we have developed a guidance note aimed at capturing what we have learned so far about integrating AI into evaluation and to offer a framework for further ex ploration. Since we began our experiments with Large Language Models ( LLMs) in the Spring of 2023, we have made significant progress in usin g Generative Artificial Intelligence (GenAI) for processing and analyz ing large volumes of text data in our evaluations. Currently, we are r unning multiple experiments across IEG to test the various capabilitie s of LLMs in different evaluation use cases. As we continue to explore the strengths, limitations, and risks of this technology, we have par tnered with others in the evaluation community to learn from each othe r and identify promising practices. The guidance note is the result of our collaboration with the Independent Office of Evaluation of the In ternational Fund for Agricultural Development. Our guidance note estab lishes clear criteria to judge AI performance and good practices for d esigning meaningful experiments. We use these criteria to test AI on k ey tasks such as text classification, summarization, and synthesis. As sessing AI’s Performance LLMs do not always provide appropriate or ver acious responses, and their output must be thoroughly validated before use, especially in evaluation practice where analytical rigor is cruc ial. The guidance note proposes several performance dimensions dependi ng on the task at hand. For classification tasks, standard machine lea rning metrics like accuracy, precision, recall, balanced accuracy, and F1 scores are used (please see the guidance note for a description of metrics and scores). These metrics measure the overlap between machin e-annotated 'predicted' labels and human-annotated 'ground-truth' labe ls. Separate training, validation, testing, and prediction sets are ne cessary to compute useful performance metrics and reduce bias. For tex t summarization, text synthesis, and information extraction tasks, pro ducing human-annotated data as ground-truth is not always feasible. In such cases, we assess the quality of model responses using criteria f rom the field of Natural Language Generation, a subfield of AI: faithf ulness, relevance, and coherence. Faithfulness checks if the AI-genera ted information is factually consistent with the source, relevance ass esses if the content selected by AI is the most important, and coheren ce evaluates the overall quality of the generated sentences. To determ ine the minimum acceptable value for each metric, we take a context-sp ecific approach. For example, in a text classification task for a stru ctured literature review, recall and precision scores of 0.75 and 0.60 respectively may be sufficient for the initial literature identificat ion, especially with highly class-imbalanced data. This was the case i n our applied experiments within an ongoing IEG evaluation. However, f or tasks with higher evaluative stakes, such as judging the extent to which a portfolio of activities has reached key performance targets, h igher metric values are required. The guidance note provides a framewo rk of assessment criteria with clear definitions. Other Good Practices Identifying use cases where LLMs can add significant value compared t o traditional approaches is crucial. Not every task will have (net) be nefits from the use of LLMs, so aligning experiments with tasks that c an truly leverage their capabilities is essential. Planning workflows within use cases involves breaking down tasks into granular steps to u nderstand where and how LLMs can be effectively applied. Translating t ypical evaluation flows into AI-enabled workflows is a critical step. Incorporating a modular design for the AI-enabled workflows can furthe r enhance the process since it allows for the reuse of successful comp onents within and across use cases. Clear understanding and agreement on necessary resources and expected outcomes are vital within multidis ciplinary teams needed to carry out such work. This includes human res ources, technology, timeline, and defining success for each experiment . A robust sampling strategy is needed to parse datasets into training , validation, testing, and prediction sets, facilitating effective pro mpt development, model evaluation, and most importantly, model respons es that are helpful for the task. Iterative prompt development and val idation involve testing and refining prompts, including requests for j ustification to gauge the model's reasoning, which itself helps with p rompt refinement. Going Forward Experimenting with AI in evaluation pr actice involves thoughtful risk-taking, continuous learning, and adapt ation. It is a continuous process of questioning, testing, learning, a nd refining, analogous to how models like GPT themselves learn during their training phase: through constant trial and error. Our guidance n ote focuses on defining and adapting evaluation workflows to include L LMs where they fit best and building trust through thorough performanc e testing. Further research, experimentation, and collaboration are ne eded to standardize and expand frameworks for assessing LLM performanc e in evaluation. Sharing experiences and findings from experiments acr oss organizations and contexts is essential. Much has been written abo ut the potential and perils of leveraging LLMs in research and analyti cal tasks. However, it is through concrete, practical, context-specifi c experimentation that we can discover what works, what does not, and under what circumstances. We are committed to exploring and sharing ou r findings as widely as possible.