Evaluation Criteria for Explainable AI in Intrusion Detection to Ensure the Creation of High-Quality Threat Intelligence
Noriyoshi Ozawa, Satoru Sunahara, Shigeki Hagihara · 2025
Threat intelligence is important for forming rapid, appropriate responses to increasing numbers of sophisticated cyberattacks.Highquality threat intelligence requires excellent threat information.Although intrusion detection systems that use deep learning are highly accurate, it is impossible to understand how they come to their decisions.Therefore, such systems cannot be used to created threat intelligence.Explainable artificial intelligence (XAI) is being intensively studied to clarify how deep learning decisions are made.However, standard XAI evaluation criteria are lacking, and comparative evaluations are impossible.This study aimed to establish XAI methods and evaluation criteria to determine whether it is possible to obtain high-quality threat intelligence.First, we carefully defined the role that a deep learning-based XAI intrusion detection system should play when creating high-quality threat intelligence.Then, we established XAI methods and evaluation criteria to examine the extent to which XAI fulfills this required role.The evaluation method was compared to current international standards.The results revealed that our criteria are generally applicable for assessing whether an XAI yields the information required to create high-quality threat intelligence.