Weakly-Supervised Learning with Cost-Augmented Contrastive Estimation
Kevin Gimpel, Mohit Bansal · 2014
We generalize contrastive estimation in two ways that permit adding more knowledge to unsupervised learning.The first allows the modeler to specify not only the set of corrupted inputs for each observation, but also how bad each one is.The second allows specifying structural preferences on the latent variable used to explain the observations.They require setting additional hyperparameters, which can be problematic in unsupervised learning, so we investigate new methods for unsupervised model selection and system combination.We instantiate these ideas for part-of-speech induction without tag dictionaries, improving over contrastive estimation as well as strong benchmarks from the PASCAL 2012 shared task.