Documenting AI Systems under the EU AI Act: A UML Architectural Framework with Support for Post-Hoc XAI
José Uetanabara Júnior · Zenodo (CERN European Organization for Nuclear Research) · 2025
Artificial Intelligence (AI) has gained prominence in recent years, with widespread adoption raising challenges related to the auditability of AI-based systems. Explainable Artificial Intelligence (XAI) addresses this issue through post-hoc methods that provide interpretation. However, the integration of XAI methods into architectural representations and compliance-oriented documentation remains largely unstructured. At the same time, the European Union’s AI Act (Regulation 2024/1689) demands documentation requirements for high-risk AI systems without prescribing a standardized format. As a result, compliance material is often complex to produce and maintain, and may not accurately reflect the system implementation. To address this gap, this work proposes a UML architectural framework for AI systems incorporating post-hoc XAI, focusing on the structural representation of compliance-relevant items required by Annex IV. The framework defines a minimal set of Unified Modeling Language (UML) stereotypes, tagged values, and relationships, based on an architectural contract emerging from object-oriented (OO) Python implementations. As an additional contribution, this work introduces the UMLOOModeler, a tool that generates UML class diagrams from these implementations using a conservative extraction strategy, ensuring consistency between the implementation and architectural representations. The framework is illustrated through heterogeneous AI configurations and a partial example of technical documentation, supporting traceability, auditability, and documentation consistency.