A partially deterministic weight initialization method for SFFANNs
Sartaj Singh Sodhi, Pravin Chandra · 2014
A new method for the weight initialization of sigmoidal feedforward artificial neural networks (SFFANNs) is proposed. The proposed weight initialization routine initializes the input to the hidden layer weights on different regions of an interval corresponding to the [-1,1 + ε], where the value of ε depends on the number of nodes in the hidden layer. The thresholds of the hidden layers are initialized to one end of the sub-interval associated with the hidden node for input to hidden node initialization. The hidden nodes to the output node weights are initialized in a deterministic manner in an interval dependent on the number of hidden nodes in the network while the threshold of the output node is initialized to zero. The proposed weight initialization method is compared on a set of 8 function approximation tasks with four instances of the random weight initialization method. The results indicate that the networks initialized with the proposed methods, reach deeper minima of the error functional during training, generalize better and are faster in convergence.