Fusing the Results of Diverse Algorithms
John F. Elder · 2001
Structurally adaptive methods -from decision trees and polynomial networks, to projection pursuit models, additive networks, and cascade correlation neural networks -iteratively add components in an attempt to construct models with complexity appropriate to the data. The diverse basis functions and search strategies employed usually lead to a distribution of results (with rankings hard to predict priori) yet, robustly combining the output estimates can achieve a consensus model with properties often superior to the best of the individual models. Additionally, other information learned by some of the modeling techniques can be shared to the benefit of the fused system, including identification of key variables to employ and outlying cases to ignore. This paper describes the fused model developed for a small but challenging classification dataset (where one infers the species of a bat from its chirps) and introduces a robust of combining the outputs of diverse models. Inductive Modeling Toolbox Though new inductive methods are introduced continually, they appear to be motivated by only a handful of key underlying ideas, and it is likely that a suite of methods can be identified which will essentially span method space. In a readable empirical survey, Michie, Spiegelhalter and Taylor (1994)t suggested such a sufficient toolbox would include linear discrimination, decision trees, and k-nearest neighbors, as well as the more recent methods of projection pursuit and radial basis functions. Elder and Pregibon (1996) further suggest adding polynomial networks and (perhaps) adaptive splines and neural networks to the set covering methods, due to some relatively unique properties of those algorithms. Four of these methods -decision trees, k-nearest neighbors, neural networks, and polynomial networks -were selected to be used, in a multi-stage procedure, to address a chaUenging classification problem. Reviewed by Elder (1996). Application Example: Identifying Bat Species Tracking populations of potentially endangered bats can be simplified if similar species can be distinguished using features of their sophisticated echolocation signals. Data recently collected and analyzed by Kaefer et al. (1996) demonstrated that the necessary information is present in the signals, but that a fused model (described here) seems required to extract it. The data was obtained by capturing and labeling bats, recording several of their in-flight biosonar calls, extracting Fourier and time-frequency features from such chirps, and iterating the signal feature extraction phase after analyzing early returns from individual induction algorithms charged with constructing discriminating classifiers. The data consists of 93 cases representing 18 different bats (with 3-8 signals per individual) from 5 species (some quite related physically) 2 all of which emit calls in the FM range (a spectrum area previously unexplored for the purposes of discrimination). Because there were multiple signals from each individual, a voting mechanism could be employed to arrive at a final classification for each set (bat). That is, bat was counted as correctly classified if a plurality of its acceptable signals were correctly identified. (The acceptance process is defined below.) For each algorithm, cross-validation accuracy was measured for both case-wise and bat-wise identification. That is, each trained a model using 17 of the bat signal sets, then tested it on the 18th set (with this repeated 18 times so each set could be the hold-out sample in turn) and the collection of test results accumulated. Thus, the results reflect the accuracy of each algorithm (represented by a bundle of models) when applied to new individuals, as is required to estimate potential field performance.