A memory optimal BFGS neural network training algorithm
Seán McLoone, Vijanth Sagayan Asirvadam, G.W. Irwin · 2003
This paper considers the implementation of a novel memory optimal neural network training algorithm which maximises performance in relation to available memory. Mathematically, it is similar to the full memory BFGS training when there are no constraints on memory and to the variable memory (VM) BFGS when memory is limited. However, it requires less computations per iteration than VM and uses a much better strategy for discarding old curvature information when memory is limited.