Soft decision trees
Ozan İrsoy, Olcay Taner Yıldız, Alpaydın, Ahmet İbrahim Ethem · Akademik Açık Erişim (Işık Üniversitesi) · 2012
We discuss a novel decision tree architecture with soft decisions at the internal nodes where we choose both children with probabilities given by a sigmoid gating function. Our algorithm is incremental where new nodes are added when needed and parameters are learned using gradient-descent. We visualize the soft tree fit on a toy data set and then compare it with the canonical, hard decision tree over ten regression and classification data sets. Our proposed model has significantly higher accuracy using fewer nodes.