An Unsupervised Parameter Estimation Algorithm for a Generative Dependency N-gram Language Model
Chenchen Ding, Mikio Yamamoto · 2013
We design a language model based on a generative dependency structure for sen-tences. The parameter of the model is the probability of a dependency N-gram, which is composed of lexical words with four kinds of extra tags used to model the dependency relation and valence. We fur-ther propose an unsupervised expectation-maximization algorithm for parameter es-timation, in which all possible dependency structures of a sentence are considered. As the algorithm is language-independent, it can be used on a raw corpus from any lan-guage, without any part-of-speech annota-tion, tree-bank or trained parser. We con-ducted experiments using four languages: English, German, Spanish and Japanese. The results illustrate the applicability and the properties of the proposed approach. 1