Word Embedding Calculus in Meaningful Ultradense Subspaces
Sascha Rothe, Hinrich Schütze · 2016
We decompose a standard embedding space into interpretable orthogonal subspaces and a "remainder" subspace.We consider four interpretable subspaces in this paper: polarity, concreteness, frequency and part-of-speech (POS) subspaces.We introduce a new calculus for subspaces that supports operations like "-1 × hate = love" and "give me a neutral word for greasy" (i.e., oleaginous).This calculus extends analogy computations like "king-man+woman = queen".For the tasks of Antonym Classification and POS Tagging our method outperforms the state of the art.We create test sets for Morphological Analogies and for the new task of Polarity Spectrum Creation.