Improved Particle Swarm Optimization Combined with Backpropagation for Feedforward Neural Networks
Fei Han, Jiansheng Zhu · International Journal of Intelligent Systems · 2012
Traditional particle swarm optimization (PSO) has good global search ability, but it easily loses its diversity and thus leads to premature convergence. Gradient descent methods such as backpropagation (BP) algorithm have good performance in searching local minima, whereas they are apt to converge to local minima. To improve search ability, two hybrid algorithms combining two improved PSOs individually with BP are proposed to train single-hidden-layer feedforward neural networks in this paper. In the two improved PSOs, other than the phases of repulsion and attraction, a new phase named as a mixed phase is introduced, in which the particles are attracting and repelling simultaneously to prevent premature convergence. Moreover, a modified mutation operation is performed to help particles jump out of local minima in the improved PSOs. The proposed hybrid methods achieve better convergence performance with faster convergence rate than some commonly used PSO–BP approaches and purely global or local search methods. The experiments results on function approximation and benchmark classification problems are given to verify the effectiveness and efficiency of the proposed hybrid algorithms.