On the implementation of backpropagation on the Alex AVX-2 parallel system
Hazem M. Abbas, Mohamed M. Bayoumi · Proceedings of International Conference on Neural Networks (ICNN'97) · 2002
Training backpropagation (BP) networks is a time-consuming process especially on sequential machines. This has motivated the use of parallel architectures to decrease the processing time required for training. In this paper the implementation of the BP algorithm on the Alex AVX-2 MIMD machine is investigated. Due to the high communication time caused by sending and receiving network information and due to the overhead of the message passing process, the conventional use of block-BP is not appropriate for this particular machine. Increasing the processing load of the workers with respect to the communication load will definitely increase the speedup factor. Here, we propose a block-update learning method for BP which reduces the communication time and produces results similar to those obtained with parallel block-BP.