Fast and efficient and training of neural networks
Hao Yu, Wilamowski Auburn · 2010
In this paper, second order algorithms, such as Levenberg Marquardt algorithm, are recommended for neural network training. Being different from traditional computation in second order algorithms, the proposed method simplifies Hessian matrix computation, by removing Jacobian matrix computation and storage. Matrix multiplications are replaced by vector operations. The proposed computation not only makes the training process faster, but also reduces the memory cost significantly. Based upon the improvement, second order algorithms can be applied for application with unlimited number of patterns.