Sparsity Makes Sense: Word Sense Disambiguation Using Sparse Contextualized Word Representations
Gábor Berend · 2020
In this paper, we demonstrate that by utilizing sparse word representations, it becomes possible to surpass the results of more complex task-specific models on the task of finegrained all-words word sense disambiguation.Our proposed algorithm relies on an overcomplete set of semantic basis vectors that allows us to obtain sparse contextualized word representations.We introduce such an information theory-inspired synset representation based on the co-occurrence of word senses and nonzero coordinates for word forms which allows us to achieve an aggregated F-score of 78.8 over a combination of five standard word sense disambiguating benchmark datasets.We also demonstrate the general applicability of our proposed framework by evaluating it towards part-of-speech tagging on four different treebanks.Our results indicate a significant improvement over the application of the dense word representations.