A Digital Engineering Methodology for Design, Exploration and Validation of Safety‐Critical Software for Integrating AI‐based Algorithms
Gabriel Pedroza, Matthieu Paquet, Bernard Dion · INCOSE International Symposium · 2025
Abstract A strong shift towards the usage of AI/ML techniques is observed in a variety of applications and domains ranging from systems health monitoring, and fuel optimization up to novelties like Collision Avoidance and Unmanned Air Mobility. Developing such systems is challenging given the required levels of safety, autonomy, and the complexity of the environmental conditions. To provide guidance, standardization bodies like SAE, EUROCAE, and ISO work on guidelines for AI/MLs integration into safety‐critical systems (e.g., ED‐324/ARP6983 in aeronautics, ISO/PAS‐8800 in automotive). In this work, some AI/ML challenges are first surveyed including limited embedded HW resources and the AI/ML uncertainty. Then, a modular, iterative Digital Engineering Methodology based upon SysML® v2 is introduced to support AI/ML design and development. The methodology is aligned with the Department of Defense Digital Engineering Directive 5000.97 and the MIL‐STD‐881f standard for structuring work breakdown. To ensure datasets quality, the methodology includes validation of properties of the Operational Design Domain (ODD) like completeness, representativeness, and independence. To characterize the ML model, techniques are applied to assess stability, generalization, and robustness. Last, to validate system safety a probabilistic scenario‐based method is introduced to ensure the ML performance remains within safety thresholds. The approach is illustrated by a formation flying Use Case integrating a Reinforcement Learning model, and some MBSE techniques for engineering requirements, system, architecture, and ODD. Some perspectives towards the consolidation of the proposed methodology and to foster AI/ML maturity are finally given.