Multiprocessor realization of neural networks
Robert W. Bennington, Nicholas DeClaris · 1990
This research provides a foundation for implementing neural networks on multiprocessor systems in order to increase simulation speeds and to accommodate more complex neural networks. The emphasis is on the use of affordable coarse grain multiprocessors to implement commerically available neural network simulators currently being run on single processor systems. A conceptual framework is presented based on the concepts of program decomposition, load balancing, communication overhead, and process synchronization. Four methodologies are then presented for optimizing execution times. A set of metrics are also introduced which make it possible to measure the performance enhancements over single processor systems, and analyze the effects of communication overhead, load balancing, and synchronization for various network decompositions. The application of these four methodologies to two neural network simulators on a multiprocessor computer system is discussed in detail. They are illustrated with practical implementations of networks ranging in size from six to twenty thousand connections. Two of the methodologies, the Pipeline and Hybrid approaches, exhibit speedups approaching the possible upper limits. The theoretical significance of this dissertation research is that it provides a basis for achieving efficient multiprocessor implementation of high complex and massive neural networks. Traditionally, neural network research and development requires a considerable amount of time be spent in repeatedly evaluating and modifying network architectures and algorithms. As such, the engineering value of this dissertation is that the time required to repeatedly execute networks in research and development can be significantly reduced.