Bayesian Modeling of Dependency Trees Using Hierarchical Pitman-Yor Priors
Hanna Wallach, Charles A. Sutton, Andrew McCallum · Scholarworks (University of Massachusetts Amherst) · 2008
In this paper, we introduce two hierarchical Bayesian dependency parsing models. First, we show that a classic dependency parser can be substantially improved by (a) using a hierarchical Pitman-Yor process prior over the distribution over dependents of a word, and (b) sampling the model hyperparameters. Second, we present a parsing model in which latent state variables mediate the relationships between words and their dependents. The model clusters dependencies into states using a similar approach to that used by Bayesian topic models when clustering words into topics. The inferred states have a syntactic character, and lead to modestly improved parse accuracy when substituted for part-of-speech tags in the parsing model.