On complexity analysis of supervised MLP-learning for algorithmic comparisons
Eiji Mizutani, Stuart E. Dreyfus · 2002
This paper presents the complexity analysis of a standard supervised MLP-learning algorithm in conjunction with the well-known backpropagation, an efficient method for evaluation of derivatives, in either batch or incremental learning mode. In particular, we detail the cost per epoch (i.e., operations required for processing one sweep of all the training data) using "approximate" FLOPs (floating point operations) in a typical backpropagation for solving neural networks nonlinear least squares problems. Furthermore, we identify erroneous complexity analyses found in the past NN literature. Our operation-count formula would be very useful for a given MLP architecture to compare learning algorithms.