Disambiguated skip-gram model
Karol Grzegorczyk, Marcin Kurdziel · 2018
We present disambiguated skip-gram: a neural-probabilistic model for learning multisense distributed representations of words.Disambiguated skip-gram jointly estimates a skip-gram-like context word prediction model and a word sense disambiguation model.Unlike previous probabilistic models for learning multi-sense word embeddings, disambiguated skip-gram is end-to-end differentiable and can be interpreted as a simple feed-forward neural network.We also introduce an effective pruning strategy for the embeddings learned by disambiguated skip-gram.This allows us to control the granularity of representations learned by our model.In experimental evaluation disambiguated skip-gram improves stateof-the are results in several word sense induction benchmarks.