The Interpretability of Quantum-Inspired Neural Network
Shikai Song, Yuexian Hou, Guangcheng Liu · 2021
Recently, the mathematical framework of quantum mechanics has boosted interpretability and performance in a variety of NLP tasks. Since different linguistic units are explicitly weighted by this framework in density matrix, it is also often believed that the models inspired by quantum mechanics afford transparency. However, the relationship between weights of linguistic units and model outputs is uncertain. In this work we test whether this assumption holds by performing extensive experiments in question answering (QA) model to manipulating semantic weights and assessing the resulting differences in its output. Although we observe in some ways that higher weights do correlate with greater effect on model, but most of the data does not support it. i.e., the weight is independent of the prediction, and different weight distributions can produce equivalent result. Our results illustrate that standard quantum-inspired model has interpretable structure but does not accurately model feature importance and requires improvement in semantic representation and model interpretability.