Opening the machine learning black box with Layer-wise Relevance Propagation
Sebastian Lapuschkin · DepositOnce · 2019
Machine learning techniques such as (Deep) Neural Networks are successfully solving a plethora of tasks, e.g. in image recognition and text analysis, and provide novel predictive models for complex physical, biological and chemical systems. However, due to the nested complex and non-linear structure of many machine learning models, this comes with the disadvantage of them acting as a black box, providing little or no information about the internal reasoning. This black box character hampers acceptance and application of non-linear methods in many application domains, where understanding individual model predictions and thus trust in the model’s decisions are critically important. In this thesis, we describe a novel method for explaining non-linear classifier decisions by decomposing the prediction function, called Layer-wise Relevance Propagation (LRP). We apply our method to Neural Networks, kernelized Support Vector Machines (with non-linear kernels) and Bag of Words feature extraction pipelines and evaluate LRP theoretically, qualitatively and quantitatively in comparison to other recent methods for interpreting model predictions. Using our method as a tool for comparative analyses between various pre-trained models we reveal different learned prediction strategies and flaws in datasets, predictors and the training thereof.