Understanding the Source of Semantic Regularities in Word Embeddings
Hsiao-Yu Chiang, José Camacho-Collados, Zachary A. Pardos · 2020
Semantic relations are core to how humans understand and express concepts in the real world using language.Recently, there has been a thread of research aimed at modeling these relations by learning vector representations from text corpora.Most of these approaches focus strictly on leveraging the co-occurrences of relationship word pairs within sentences.In this paper, we investigate the hypothesis that examples of a lexical relation in a corpus are fundamental to a neural word embedding's ability to complete analogies involving the relation.Our experiments, in which we remove all known examples of a relation from training corpora, show only marginal degradation in analogy completion performance involving the removed relation.This finding enhances our understanding of neural word embeddings, showing that co-occurrence information of a particular semantic relation is the not the main source of their structural regularity.