BINN-DT: Towards Better Interpretability of Multidimensional Decision Rules via Bivariate Nonlinear Node Decision Trees
Satoshi Arai, Shinichi Shirakawa, Tomoharu Nagao · 2024
In the practical application of machine learning, the opaqueness of models often poses significant challenges. While decision trees are known for balancing representability with interpretability, enabling humans to understand decision rules, their interpretability decreases as the complexity of the task increases and the tree size expands, making it difficult to trace and interpret the decision flow. In this paper, we introduce a new variant of decision tree called Bivariate Nonlinear Node Decision Tree (BINN-DT), designed to enhance the interpretability of decision trees. BINN-DT selects bivariate features at each node and utilizes nonlinear splitters to learn the data splitting rules. Additionally, each node visualizes the relationship between the data distribution and split boundaries through a two-dimensional map using the selected bivariate features. Our experiments compared the proposed BINN-DT method with traditional univariate decision trees. The results demonstrate that our approach not only maintains classification accuracy but also produces more compact models. BINN-DT clearly depicts the entire decision boundaries of a model as a tree-structured collection of two-dimensional maps with the bivariate feature axes selected from the entire features. Our method significantly improves the interpretability of models by producing more compact models than the traditional decision trees, without sacrifice of accuracy.