GMM-Free DNN Training
Andrew Senior, Georg Heigold, Michiel Bacchiani, Huan Liao · 2014
While deep neural networks (DNNs) have become the dom-inant acoustic model (AM) for speech recognition systems, they are still dependent on Gaussian mixture models (GMMs) for alignments both for supervised training and for context dependent (CD) tree building. Here we explore bootstrap-ping DNN AM training without GMM AMs and show that CD trees can be built with DNN alignments which are bet-ter matched to the DNN model and its features. We show that these trees and alignments result in better models than from the GMM alignments and trees. By removing the GMM acoustic model altogether we simplify the system required to train a DNN from scratch. Index Terms — Deep neural networks, hybrid neural net-work speech recognition, Voice Search, mobile speech recog-nition, flat start, Viterbi forced-alignment, context dependent tree-building. 1.