A model explanation system
Ryan Turner · 2016
We propose a new methodology for explaining the predictions of black box classifiers. We use the motivating paradigm that predictive performance is of primary importance but human analysts (e.g., in fraud detection) desire a classifier's predictions to be augmented with useful explanations. To be truly general and principled, we derive a scoring system for finding explanations based on formal requirements. In this system, the explanations are assumed to take the form of simple logical statements. We derive an efficient Monte Carlo algorithm to find explanations for black box classifiers with finite sample guarantees. The methodology is then applied to interesting examples in facial recognition and credit data.