On the Greediness of Feature Selection Algorithms
Kan Deng · 1999
: Based on our analysis and experiments using real-world datasets, we find that the greediness of forward feature selection algorithms does not severely corrupt the accuracy of function approximation using the selected input features, but improves the efficiency significantly. Hence, we propose three greedier algorithms in order to further enhance the efficiency of the feature selection processing. We provide empirical results for linear regression, locally weighted regression and k-nearestneighbor models. We also propose to use these algorithms to develop an off-line Chinese and Japanese handwriting recognition system with automatically configured, local models. Keywords: feature selection, cross-validation, function approximation, handwriting recognition, memory-based learning, instance-based learning. 2 1. Introduction A fundamental problem of machine learning is to approximate the function relationship f() between an input and an output Y, based on a memory of data points, , ...