EDGE: Evaluation Framework for Logical vs. Subgraph Explanations for Node Classifiers on Knowledge Graphs
Rupesh Sapkota, Dominik Köhler, Stefan Heindorf · 2024
As machine learning and deep learning become increasingly integrated into our daily lives, understanding how these technologies make decisions is crucial. To ensure transparency, accountability, and ethical adherence, these so-called "black-box" models should be accompanied by human-comprehensible explanations of their predictions. This clarity is essential for establishing trust in their real-world applications. Similarly, it is crucial to compare different types of explanations to evaluate and understand their effectiveness, interpretability, and generalization capabilities for informed selection in various applications. To this end, we propose a framework called EDGE to evaluate diverse knowledge graph explanations, assessing logical rule-based and subgraph-based explanations by various explainers in terms of prediction accuracy and fidelity to the Graph Neural Network (GNN) model. Our evaluations reveal that logical methods excel in explaining complex and structured data, while subgraph-based models exhibit higher fidelity to the GNN model, earning them the label "GNN Explainers". Although further diversified evaluations are necessary to determine the superiority of one explanation type over another, our study shows that each type has pros and cons.