Word Embeddings with Limited Memory
Shaoshi Ling, Yangqiu Song, Dan Roth · 2016
This paper studies the effect of limited precision data representation and computation on word embeddings.We present a systematic evaluation of word embeddings with limited memory and discuss methods that directly train the limited precision representation with limited memory.Our results show that it is possible to use and train an 8-bit fixed-point value for word embedding without loss of performance in word/phrase similarity and dependency parsing tasks.