Exploring MARS: an alternative to neural networks
Will Dwinnell · PC AI archive · 2000
S ince their inception, neural networks have commanded a significant amount of attention in the world of machine learning – yet neural networks are not the only game in town. While they have been instrumental in solving many difficult real-world problems, keep in mind the wide array of alternatives. One such alternative is Multivariate Adaptive Regression Splines (MARS), developed by statistician Jerome Friedman. MARS segments the space of possible input cases into rectangular regions that are fit with linear or cubic splines — splines being moderately complex curves. The MARS approach has proven effective at a variety of learning problems and is competitive with neural networks and Non-parametric regressions, such as knearest neighbors. After its recent incorporation into the suite of products offered by Salford Systems (www.salfordsystems.com ), MARS has been further developed into a fine commercial tool (see Figures 1 and 2). MARS is an excellent example of the powerful modeling tools developed by statisticians and is explored in this article. A standard machine learning benchmarking data set, the Boston Housing data, is used to demonstrate MARS’s operation principles. This data, available on-line at the UCI Machine Learning Repository (www.ics.uci.edu/~mlearn/MLRepository.html ) contains information on 506 examples of housing data from the Boston area consisting of 13 numeric variables and 1 binary variable. One numeric variable, the learning algorithms may be placed on a spectrum from local to global. Local models utilize a large collection of small ‘micromodels’ that handle specific, small regions of the input space. Global models employ a single, monolithic model covering the entire input space. Very often, local models are fast to train, but slow to recall, whereas global models are slower to train, but quick to recall. More importantly, local models quickly learn to solve the specific types of problems that are presented during training, but are slow to generalize to other types of problems. In contrast, global models are Exploring MARS: An Alternative to Neural Networks