Boundedness of a batch gradient method with penalty for feedforward neural networks
Huisheng Zhang, Wei Wu, Mingchen Yao · 2007
Abstract: This paper considers a batch gradient method with penalty for training feedforward neural networks. The role of the penalty term is to control the magnitude of the weights and to improve the generalization performance of the network. An usual penalty is considered, which is a term proportional to the norm of the weights. The boundedness of the weights of the network is proved. The boundedness is assumed as a precondition in an existing convergence result, and thus our result improves this convergence result. Key–Words: Batch gradient method; Feedforward neural network; Boundedness; Penalty 1