Understanding from Deep Learning Models in Context
Michael Tamir, Elay Shech · 2022
This chapter places into context how the term model in machine learning (ML) contrasts with traditional usages of scientific models for understanding and we show how the direct analysis of an estimator’s learned transformations (specifically, the hidden layers of a deep learning model) can improve understanding of the target phenomenon and reveal how the model organizes relevant information. Specifically, three modes of understanding will be identified: the difference between implementation irrelevance and functionally approximate irrelevance will be disambiguated, and how this distinction impacts potential understanding with these models will be explored. In addition, by distinguishing between empirical link failures and representational ones, an ambiguity in the concept of link uncertainty will be addressed, thus clarifying the role played by scientific background knowledge in enabling understanding with ML.