“Why Should I Trust You?”: Explaining the Predictions of Any Classifier
Marco Túlio Ribeiro, Sameer Kumar Singh, Carlos Guestrin · 2016
Despite widespread adoption in NLP, machine learning models remain mostly black boxes.Understanding the reasons behind predictions is, however, quite important in assessing trust in a model.Trust is fundamental if one plans to take action based on a prediction, or when choosing whether or not to deploy a new model.In this work, we describe LIME, a novel explanation technique that explains the predictions of any classifier in an interpretable and faithful manner.We further present a method to explain models by presenting representative individual predictions and their explanations in a non-redundant manner.We propose a demonstration of these ideas on different NLP tasks such as document classification, politeness detection, and sentiment analysis, with classifiers like neural networks and SVMs.The user interactions include explanations of free-form text, challenging users to identify the better classifier from a pair, and perform basic feature engineering to improve the classifiers.