Methods of Moments for Learning Stochastic Languages: Unified Presentation and Empirical Comparison
Borja Balle, William L. Hamilton, Joëlle Pineau · 2014
Probabilistic latent-variable models are a power-ful tool for modelling structured data. However, traditional expectation-maximization methods of learning such models are both computationally expensive and prone to local-minima. In contrast to these traditional methods, recently developed learning algorithms based upon the method of moments are both computationally efficient and provide strong statistical guarantees. In this work we provide a unified presentation and empiri-cal comparison of three general moment-based methods in the context of modelling stochastic languages. By rephrasing these methods upon a common theoretical ground, introducing novel theoretical results where necessary, we provide a clear comparison, making explicit the statisti-cal assumptions upon which each method relies. With this theoretical grounding, we then provide an in-depth empirical analysis of the methods on both real and synthetic data with the goal of elu-cidating performance trends and highlighting im-portant implementation details. 1.