Analysis of weight initialization routines for conjugate gradient training algorithm with Fletcher-Reeves updates
Sarfaraz Masood, M. N. Doja, Pravin Chandra · 2016
Among the various choices of improving the speed of convergence of sigmoidal feed forward neural network, the choice of the initial weights and the biases stands out as an important one. This paper presents an analysis of various weight initialization methods when the neural network was trained with the conjugate gradient training algorithm having Fletcher-Reeves updates. A set of experiments were performed over eight problems from the function approximation domain. The lesser value of mean test error obtained from the experiments, recommend that the weight initialization technique proposed by Nguyen and Widrow assist the neural network to converge faster and also to generalize better while the SCAWI technique is the most robust technique as it underperforms in the least number of cases.