Distributed Representation Computation Using CBOW Model and Skip–gram Model
Takamune Onishi, Hiromitsu Shiina · 2020
Word2Vec is one of methods for generating a distribution representation. Two methods of calculation, CBOW model and skip-gram model, are proposed. However, the skip-gram model has high accuracy of the distribution representation of words, but it takes a long time to learn it. On the other hand, the CBOW model has been shown to be faster but less accurate than the skip-gram model in the distribution representation of words. In this study, we propose a method to combine the CBOW model and the skip-gram model in a way that shares the weight matrix of the CBOW model with the skip-gram model in order to achieve both learning speed and accuracy of the distribution representation of words.