A distributed discrete-time neural network architecture for pattern allocation and control
Anthony Theodore Chronopoulos, J. Sarangapani · 2002
The focus of this study is how we can efficiently im-plement a novel neural network algorithm on distributed systems for concurrent execution. We assume a dis-tributed system with heterogeneous computers and that the neural network is replicated on each computer. We propose an architecture model with efficient pattern al-location that takes into account the speed of processors and overlaps the communication with computation. The training pattern set is distributed among the heteroge-neous processors with the mapping being fixed during the learning process. We provide a heuristic pattern allocation algorithm minimizing the execution time of neural network learning. The computations are over-lapped with communications. Under the condition that each processor has to perform a task directly propor-tional to its speed, we show that the pattern allocation is a polynomial-time problem, solvable by dynamic pro-gramming. 1