Grammatical Trigrams: A Probabilistic Model of Link Grammar
John Lafferty, Saniel Sleator, Davy Temperley · 1992
In this paper we present a new class of language models. This class derives from link grammar, a context-free formalism for the description of natural language. We describe an algorithm for determining maximum-likelihood estimates of the parameters of these models. The language models which we present differ from previous models based on stochastic context-free grammars in that they are highly lexical. In particular, they include the familiar n-gram models as a natural subclass. The motivation for considering this class is to estimate the contribution which grammar can make to reducing the relative entropy of natural language. Introduction Finite-state methods occupy a special position in the realm of probabilistic models of natural language. In particular, the simplicity, and simple-mindedness, of the trigram model renders it especially well-suited to parameter estimation over hundreds of millions of words of data, resulting in models whose predictive powers have yet to be seriously...