Control of the error signals in negative correlation learning
Yong Liu · 2016
Negative correlation learning has been proposed to create a set of negatively correlated artificial neural networks (ANNs) in a committee machine. In negative correlation learning, the error signals for each ANN on a given data are not only decided by the error differences between the output of ANN and the targets. Two terms are optimized at the same time. The first one is to minimize the error between the output of each ANN and the target output on the given data. The other one is to maximize the difference between the output of the ensemble and the output of each ANN on the given data. From the point of view on the bias-variance-covariance trade-off, the minimization of the first term would decrease the bias while the maximization of the second term would reduce the sum of bias and variance. In order to balance well among bias, variance and covariance, error signals in learning should be well adjusted. On one hand, when the learning would force itself to be closer to the ensemble, an individual ANN would choose to learn less so that the learning on that direction would be disencouraged. On the other hand, when the learning would help itself to be more different to the ensemble, an individual ANN would let itself to learn more so that the learning on that direction would be encouraged. A new version of negative correlation learning based on such error signal adjustment have been implemented in this paper. Experimental results were carried out to show how the error signal adjustment would help to achieve the better generalization.