Algorithmic (in)explainability : causality and machine learning
You Jeen Ha · Smith ScholarWorks (Smith College) · 2020
Machine learning algorithms, which pervade much of society today, currently lack explainability and transparency, resulting in “The ‘Black Box’ Problem.” I will thereby explore this problem by investigating computer scientist Judea Pearl’s the- ory of causality, a theory grounded in causal models, as well as his “do-calculus” for determining the effects of interventions. The goal here is to understand the power, promise, and possible weaknesses in his theory, which has been accepted by many working on the problem of algorithmic explainability. Although he is not the only researcher of causality, I will concentrate on his theory, which is one of the most influential and the furthest developed. I will also briefly draw on philosophical investigations of causality, as it has been a long-studied topic in numerous traditions of thought. Following this contextual study, we will complete an experimental component with various methods of causal discovery and with PyCausalImpact, a Python library for causal inference. It is important to recognize that algorithmic explainability for ML with causal mechanisms is an active and un- resolved area of research. Nonetheless, based on my examination of Pearl’s theory in the context of current literature on ML and the philosophical analyses of causation, I conclude that current approaches to causal inference with respect to ML are very brittle and do not solve “The ’Black Box’ Problem” with great confidence.