Explainable AI and Robustness-Based Test and Evaluation of Reinforcement Learning
Ali K. Raz, Kshitij Mall, Sean Matthew Nolan, Winston C. Levin, Linas Mockus, Kris Ezra, Ahmad Mia, Kyle Williams, Julie J. Parish · IEEE Transactions on Aerospace and Electronic Systems · 2024
Reinforcement Learning is a powerful and proven approach to generating near-optimal decision policies across domains, though characterizing performance boundaries, explaining decisions, and quantifying output uncertainties are major barriers to widespread adoption of Reinforcement Learning for real-time use. This is particularly true for high-risk and safety-critical aerospace systems where the cost of failure is high and performance envelopes for systems of interest may be small. To address these issues, this paper presents a three-part Test and Evaluation framework for Reinforcement Learning which is purpose-built from a systems engineering perspective on artificial intelligence. This framework employs Explainable AI techniques-namely Shapley Additive Explanations-to examine opaque decision-making, introduces robustness testing to characterize performance bounds and sensitivities, and incorporates output validation against accepted solutions. In this paper, we consider an example problem of a high-speed aerospace vehicle emergency descent problem where a Reinforcement Learning agent is trained to control vehicle angle of attack (AoA). Shapley Additive Explanations expose the most significant features that impact the selection of AoA command while robustness testing characterizes the acceptable range of disturbances to in flight parameters the trained vehicle can accommodate. Finally, the outputs from the Reinforcement Learning agent are compared to a baseline optimal trajectory as an acceptance criterion of RL solutions.