Improving Decision Tree Pruning through Automatic Programming
Stig-Erland Hansen, Roland Olsson · 2007
Automatic Design of Algorithms through Evolution (ADATE) is a machine learning system for program synthesis with automatic invention of recursive help functions. It is well suited for automatic improvement of other machine learning algorithms since it is difficult to design such algorithms based on theory alone which means that experimental tuning, optimization and even design of such algorithms is essential for the machine learning practitioner. To demonstrate the feasibility and usefulness of “learning how to learn” through program evolution, we used the ADATE system to automatically rewrite the code for the so-called error based pruning that is an important part of Quinlan’s C4.5 decision tree learning algorithm. We evaluated the resulting novel pruning algorithm on a variety of machine learning data sets from the UCI machine learning repository and found that it generates trees with seemingly better generalizing ability. The same meta-learning may be applied to most machine learning methods. keywords: Automatic programming, decision tree learning, meta learning 1