BUILDING A TOPIC-DEPENDENT MAXIMUM ENTROPY MODEL FOR
Very Large Corpora, Wu Jun, Sanjeev P. Khudanpur · 2002
Maximum ent ropy (ME) t echniques have been successfully used to combine different sources of li nguisticall y meani ng ful constraints in language models. However, most of the current ME models can only be used for small corpora , since the comput ational load in training ME models for large cor pora is unbearable. This problem is especially severe when non-local dependencies are considered. In this paper, we show how to train and use topic-dependent ME models effi ciently for a very large corpus, Broadcast News (BN). The training tim e is greatly reduced by hierarchical training and divide-an d-conquer approaches. The computation in using the model is also simplified by pre-normalizing the denomi nators of the ME model. We rep ort new speech rec ognition res ults showin g improvement with the topic model rel ati ve to the standard N-gram model for the Broad cast News task.