Parallel computing for artificial neural network training
Osman Gürsoy, Md. Haidar Sharif · Periodicals of Engineering and Natural Sciences (PEN) · 2018
The big-data is an oil of this century. A high amount of computational power is required to get knowledge from data. Parallel and distributed computing is essential to processing a large amount of data. Artificial Neural Networks (ANNs) need as much as possible data to have high accuracy, whereas parallel processing can help us to save time in ANNs training. In this paper, we have implemented exemplary parallelization of neural network training by dint of Java and its native socket libraries. During the experiments, we have noticed that Java implementation tends to have memory issues when a large amount of training data sets are involved in training. We have remarked that exemplary parallelization of a neural network training will not outperform drastically when additional nodes are introduced into the system after a certain point. This is widely due to network communication complexity in the system.