Improving Language Model Adaptation using Automatic Data Selection and Neural Network
Shahab Jalalvand · Recent Advances in Natural Language Processing · 2013
Since language model (LM) is very sensitive to domain mismatch between training and test data, using a group of techniques to adapt a big LM to specific domains is quite helpful. In this paper, we, benefit from salient performance of recurrent neural network to improve domain adapted LM. In this way, we first apply an automatic data selection procedure on a limited amount of in-domain data in order to enrich the training set. After that, we train a domain specific N-gram LM and improve it by using recurrent neural network language model trained on limited in-domain data. Experiments in the framework of EUBRIDGE 1 project on weather forecast dataset show that the automatic data selection procedure improves the word error rate around 2% and RNNLM makes additional improvement over 0.3%.