Fast training algorithm for deep neural network using multiple GPUs
Li-Rong Dai · Journal of Tsinghua University(Science and Technology) · 2013
In recent years,deep neural networks(DNNs) have been successfully used for speech recognition as a popular recognition model with great potential.However,the computational complexity of the training algorithm means that the training time for this DNN model increases dramatically with larger amounts of training data and more neural network nodes.This paper describes a fast DNN training algorithm implemented on multiple graphic processing units(GPUs) to improve the training efficiency.Tests of phone speech recognition on the TIMIT corpus show that the training speed of the improved DNN training algorithm on 4 GPUs is 3.3 times faster than the general algorithm on a single GPU,while the recognition accuracy is almost the same.Tests indicate that the fast DNN training algorithm significantly improves the DNN training speed.