GNEG: Graph-Based Negative Sampling for word2vec
Zheng Zhang, Pierre Zweigenbaum · 2018
Negative sampling is an important component in word2vec for distributed word representation learning.We hypothesize that taking into account global, corpuslevel information and generating a different noise distribution for each target word better satisfies the requirements of negative examples for each training word than the original frequency-based distribution.In this purpose we pre-compute word cooccurrence statistics from the corpus and apply to it network algorithms such as random walk.We test this hypothesis through a set of experiments whose results show that our approach boosts the word analogy task by about 5% and improves the performance on word similarity tasks by about 1% compared to the skip-gram negative sampling baseline.