Interpretation of Deep Neural Networks Based on Decision Trees
Tsukasa Ueno, Qiangfu Zhao · 2018
Nowadays deep learning is becoming a core of machine learning. In many applications, a well-trained deep learner (DL) outperforms other models. However, we still face the black-box problem. That is, in most cases we use the DLs without knowing the reasons behind the decisions. We may try to interpret a DL using some existing methods developed for shallow learners (SLs), but this is usually not efficient nor effective because the scale of a DL can be very large. In our research, we try to interpret a DL layer by layer, using decision trees (DTs). The purpose is of two folds. First, we want to extract interpretable knowledge from the hidden layers. Second, we want to propose a criterion for determining the number of layers needed for solving a given problem. Experimental results show that accurate DTs can be extracted from the hidden layers, and the size of the DTs can be used as a criterion for determining the needed number of layers.