Optimal Decision Tree Pruning Revisited : Algorithms and Complexity
Juha Harviainen, Frank O. Sommer, Manuel Sorge, Stefan Szeider · arXiv (Cornell University) · 2025
We present a comprehensive classical and pa-rameterized complexity analysis of decision tree pruning operations, extending recent research on the complexity of learning small decision trees. Thereby, we offer new insights into the computa-tional challenges of decision tree simplification, a crucial aspect of developing interpretable and effi-cient machine learning models. We focus on fun-damental pruning operations of subtree replace-ment and raising, which are used in heuristics. Surprisingly, while optimal pruning can be per-formed in polynomial time for subtree replace-ment, the problem is NP-complete for subtree raising. Therefore, we identify parameters and combinations thereof that lead to fixed-parameter tractability or hardness, establishing a precise bor-derline between these complexity classes. For example, while subtree raising is hard for small domain size D or number d of features, it can be solved in (Formula Presented) time, where I is the input size. We complement our theoretical findings with preliminary experimental results, demonstrating the practical implications of our analysis.