To Trust or Not: Towards Efficient Uncertainty Quantification for Stochastic Shapley Explanations
Joseph Cohen, Eunshin Byon, Xun Huan · PHM Society Asia-Pacific Conference · 2023
Recently, explainable AI (XAI) techniques have gained traction in the field of prognostics and health management (PHM) to enhance the credibility and trustworthiness of data-driven nonlinear models. Post-hoc model explanations have been popularized via algorithms such as SHapley Additive exPlanations (SHAP), but remain impractical for real-time prognostics applications due to the curse of dimensionality. As an alternative to deterministic approaches, stochastically sampled Shapley-based approximations have computational benefits for explaining model predictions. This paper will introduce and examine a new concept of explanation uncertainty through the lens of uncertainty quantification of stochastic Shapley attribution estimates. The proposed algorithm for estimating Shapley explanation uncertainty is efficiently applied for the 2021 PHM Data Challenge problem. The uncertainty in the derived explanation for a single prediction is also illustrated through personalized prediction recipe plots, improving post-hoc model visualization. Finally, important practical considerations for the implementation of Shapley-based XAI for industrial prognostics are provided.