There has been a significant interest in AI and its application in the evaluation world and the IDEAS conference made this ever more evident. Already the first session of the conference started with AI. AI tools have the potential to transform the way evaluators and researchers collect, analyze, and interact with data.

However its surprising that in a context where AI rapidly advances and reshapes industries, there is surprisingly little awareness among evaluators about how to actually make it happen in practice. There is also a large fear of the unknown, limiting evaluators to use the full potential of AI.

While many evaluators are eager to harness AI’s power, the training and the conference in general showed that many struggle to use AI effectively. As the industry does not have access to tailored solutions that address the needs of evaluators, many turn to general AI tools—such as ChatGPT—as a starting point, exploring how these platforms could be applied to evaluation workflows.

In absence of ready-made solutions designed specifically for evaluators, also the workshop looked at more general AI platforms. The session introduced the participants to the basics of AI in the context of international development. The session provided more of the foundational knowledge for those hoping to integrate AI into their evaluation work and guided them on how to use general AI tools for evaluation.

This module therefore served as an introductory guide to leveraging AI for evaluation in international development. Participants were introduced to fundamental AI concepts and terminology, and guidance on when and how to use different AI tools. Fundamental AI concepts covered included the basics of machine learning and generative AI, with examples of their use in various contexts, including geospatial analysis and evidence synthesis. It also provided an overview of existing ML and generative AI-based internal assistants and chatbots.

While the session was highly relevant to introduce users to AI and available tools it also pointed to the need for tailored user friendly solution for the evaluators.

The training among others covered Prompting, stressing the importance of crafting clear, instructions (“prompts”) when interacting with generative AI applications. Yet, while prompting with general AI tools is valuable, it has clear limitations when it comes to addressing the specialized needs of evaluators. Recognizing this, we at Elevaid invested in training our own models specifically for evaluation purposes.

By designing solutions around the unique challenges evaluators face—such as handling large scale complex data and ensuring methodological rigor we aim to fill a critical gap that generic AI platforms could not fully address. Tailor-made tools will allow the sector to maximize potential while minimizing common pitfalls, ensuring that evaluators could benefit from technology truly designed to support their specific workflows.