Framework for analytical AI agent validation
Pitkäniemi, Joonas · LUTPub (LUT University) · 2025
Utilizing the Design Science framework for Information Systems, this Master’s Thesis creates a framework for the validation of analytical artificial intelligence agents. The main goal is to enable organizations using analytical agentic systems to verify the production readiness of their agents. The content within this thesis is structured according to the Design Science framework and the structure abides by the Design Science process. The work begins with a literature review, after which using the findings from the literature review it develops a designed artifact, the validity of which validity is demonstrated through a demonstration phase, where the framework is subjected to a simulated validation scenario. The created framework seeks to introduce measurable and repeatable steps for agent validation, which anyone adhering to the framework can perform regardless of the analytical purpose of the agent. The purpose of the framework is to ensure that the agent’s connection to data is sound, that the agent correctly interprets the data it is linked to, and that the agent returns only those values derived directly from the data, thereby avoiding hallucination.