GPU simulator of multilayer neural network based on multi-valued neurons
Christian Hacker, Igor N. Aizenberg, Jeff Gaines Wilson · 2016
In this paper, we consider principles of design and basic fundamentals of a GPU simulator of the multilayer neural network with multi-valued neurons (MLMVN). Slowing down a learning process due to a big learning dataset and/or a big neural network needed for solving a certain problem is a potential bottleneck preventing the use of neural networks for solving some challenging problems. The same is related to deep learning. MLMVN is a feedforward complex-valued neural network, which has a number of advantages when compared to real-valued neural networks. These advantages include derivative-free learning and significantly better generalization capability. To extend applicability of MLMVN, its GPU-based software implementation shall be considered. We present basic principles of the GPU simulator of MLMVN and how matrix algebra operations are specifically employed there. It is shown that the bigger the network is, the more beneficial is its GPU implementation. It is shown that up to 32× acceleration can be achieved for the MLMVN learning process. Some applications, which could not be even considered without a GPU simulator, are also presented.