Memory, Show the Way: Memory Based Few Shot Word Representation Learning
Jingyuan Sun, Shaonan Wang, Chengqing Zong · 2018
Distributional semantic models (DSMs) generally require sufficient examples for a word to learn a high quality representation.This is in stark contrast with human who can guess the meaning of a word from one or a few referents only.In this paper, we propose Mem2Vec, a memory based embedding learning method capable of acquiring high quality word representations from fairly limited context.Our method directly adapts the representations produced by a DSM with a longterm memory to guide its guess of a novel word.Based on a pre-trained embedding space, the proposed method delivers impressive performance on two challenging few-shot word similarity tasks.Embeddings learned with our method also lead to considerable improvements over strong baselines on NER and sentiment classification.