A Comparison of Projection Pursuit and Neural Network Regression Modeling
Jenq-Neng Huang, Hang Li, Martin Maechler, R. Douglas Martin, J. Schimert · Neural Information Processing Systems · 1991
Two projection based feedforward network learning methods for model-free regression problems are studied and compared in this paper: one is the popular back-propagation learning (BPL); the other is the projection pursuit learning (PPL). Unlike the totally parametric BPL method, the PPL non-parametrically estimates unknown nonlinear functions sequentially (neuron-by-neuron and layer-by-Iayer) at each iteration while jointly estimating the interconnection weights. In terms of learning efficiency, both methods have comparable training speed when based on a Gauss-Newton optimization algorithm while the PPL is more parsimonious. In terms of learning robustness toward noise outliers, the BPL is more sensitive to the outliers.