[In]Credible Models – Verification, Validation & Accreditation of Agent-Based Models to Support Policy-Making
Michael Belfrage, Emil Johansson, Fabian Lorig, Paul Davidsson · Journal of Artificial Societies and Social Simulation · 2024
This paper explores the topic of model credibility of Agent-based Models and how they should be evaluated prior to application in policy-making.Specifically, this involves analyzing bordering literature from different fields to: (1) establish a definition of model credibilitya measure of confidence in the model's inferential capability -and to ( 2) assess how model credibility can be strengthened through Verification, Validation, and Accreditation (VV&A) prior to application, as well as through post-application evaluation.Several studies have highlighted severe shortcomings in how V&V of Agent-based Models is performed and documented, and few public administrations have an established process for model accreditation.To address the first issue, we examine the literature on model V&V and, based on this review, introduce and outline the usage of a V&V plan.To address the second issue, we take inspiration from a practical use case of model accreditation applied by a government institution to propose a framework for the accreditation of ABMs for policy-making.The paper concludes with a discussion of the risks associated with improper assessments of model credibility.