Parallel Implementation of Backpropagation Algorithm
Róbert Zsolt Szabó, M. Steinmetz, · Journal of Intelligent Systems · 1996
We present the results of computer simulations of several aspects of parallel programming in relation to massively parallel neural network algorithms.Backpropagation, as the most widely used massively parallel neural network learning algorithm, was used to investigate the different aspects of parallel programming with the intent of providing the fastest overall execution of the algorithm in both multiprocessor and multicomputer environments.Our paper discusses the results of an investigation into the suitability of the Backpropagation algorithm for concurrency, data parallelism and partitioning, synchronous iteration mechanisms, communication and synchronization delays, and performance on different parallel architectures.