Embedded Representation of Relation Words with Visual Supervision
Xue Wang, Youtian Du, Xuelian Li, Fuyuan Cao, Chang Su · 2019 Third IEEE International Conference on Robotic Computing (IRC) · 2019
Word representation learned from the analysis of natural language does not usually reflect the true semantics of words. The paper proposes a new method, named Visually Supervised Word2Vec (VS-Word2Vec) model to achieve the representation of the relation words that are important in knowledge related tasks. Our method first computes the visual feature vector of relation words based on deep networks, and then achieve the visual similarity matrix for all relation words, which we think reflects their true semantics. VS-Word2Vec model then combines the visual similarity and the CBOW and builds an optimization problem to jointly learn the word vector representation. Therefore, VS-Word2Vec fuses the visual modality and natural language together. Experiments implemented over the public datasets demonstrate that VS-Word2Vec model really changes the distribution of word representation and achieves more effective results in describing their true semantics than CBOW model.