A Convergence Analysis of Log-Linear Training
Simon Wiesler, Hermann Ney · RWTH Publications (RWTH Aachen) · 2011
Log-linear models are widely used probability models for statistical pattern recognition.Typically, log-linear models are trained according to a convex criterion.In recent years, the interest in log-linear models has greatly increased.The optimization of log-linear model parameters is costly and therefore an important topic, in particular for large-scale applications.Different optimization algorithms have been evaluated empirically in many papers.In this work, we analyze the optimization problem analytically and show that the training of log-linear models can be highly ill-conditioned.We verify our findings on two handwriting tasks.By making use of our convergence analysis, we obtain good results on a large-scale continuous handwriting recognition task with a simple and generic approach.