Causal Inductive Biases for Cognitive Machine Learning
Saketh Vishnubhatla, Adrienne J. Raglin, Raha Moraffah, Huan Liu · 2024
One of the long-unsolved open problems in machine learning is imbuing machine learning algorithms with human-like cognitive reasoning capabilities. An essential aspect of cognitive reasoning is the causal reasoning capacity inherent in humans. While humans can easily find causal connections, many Google searches for "pandemic" do not cause a pandemic but the other way around; the algorithms relying only on observational data might learn such misleading patterns. Thus, an active area of research - causal machine learning - emerged, where we encode causal assumptions into the learning algorithms and constrain the hypotheses learned by the algorithm. Constraining the learning algorithm with prior knowledge is commonly referred to as "inductive bias." The core contribution of any machine learning algorithm is primarily to build the correct inductive biases for the problem at hand. Causal machine learning researchers regularly employ causal assumptions in their learning algorithms, yet many fail to recognize these assumptions as inductive biases. Thinking of causal assumptions in terms of inductive biases helps researchers design better learning algorithms for the relevant task. We present a case for how causal assumptions restrict the hypothesis space of the learning algorithm and discuss the standard ways of encoding these assumptions, which we refer to as causal inductive biases. Our position is that causal inductive biases are necessary, to build causally interpretable and generalizable models.