Structure and Performance of a Dependency Language Model
Ciprian I. Chelba, David M. Engle, Harry Printz, Frederick Jelinek, Eric Sven Ristad, Víctor Jiménez, Ronald Rosenfeld, Sanjeev P. Khudanpur, Andreas Stolcke, Lidia Mangue, Dekai Wu · 1997
We present a maximum entropy language model that incorporates both syntax and semantics via a dependency grammar. Such a grammar expresses the relations between words by a directed graph. Because the edges of this graph may connect words that are arbitrarily far apart in a sentence, this technique can incorporate the predictive power of words that lie outside of bigram or trigram range. We have built several simple dependency models, as we call them, and tested them in a speech recognition experiment. We report experimental results for these models here, including one that has a small but statistically significant advantage (p ! :02) over a bigram language model. 1. INTRODUCTION In this paper, we propose a new language model to remedy two important weaknesses of the well-known Ngram method. We begin by reviewing these problems. Let S be a sentence consisting of words w 0 : : : w n , each drawn from a fixed vocabulary of size V . By the laws of conditional probability, P (S) = P...