Modeling Word Meaning in Context with Substitute Vectors
Oren Melamud, Ido Dagan, Jacob H. Goldberger · 2015
Context representations are a key element in distributional models of word meaning. In contrast to typical representations based on neighboring words, a recently proposed ap-proach suggests to represent a context of a tar-get word by a substitute vector, comprising the potential fillers for the target word slot in that context. In this work we first propose a vari-ant of substitute vectors, which we find partic-ularly suitable for measuring context similar-ity. Then, we propose a novel model for rep-resenting word meaning in context based on this context representation. Our model outper-forms state-of-the-art results on lexical substi-tution tasks in an unsupervised setting. 1