Joint Representation Learning of Cross-lingual Words and Entities via Attentive Distant Supervision
Yixin Cao, Lei Jing Hou, Juanzi Li, Zhiyuan Liu, Chengjiang Li, Xu Chen, Tiansi Dong · 2018
Joint representation learning of words and entities benefits many NLP tasks, but has not been well explored in cross-lingual settings.In this paper, we propose a novel method for joint representation learning of cross-lingual words and entities.It captures mutually complementary knowledge, and enables cross-lingual inferences among knowledge bases and texts.Our method does not require parallel corpora, and automatically generates comparable data via distant supervision using multi-lingual knowledge bases.We utilize two types of regularizers to align cross-lingual words and entities, and design knowledge attention and crosslingual attention to further reduce noises.We conducted a series of experiments on three tasks: word translation, entity relatedness, and cross-lingual entity linking.The results, both qualitatively and quantitatively, demonstrate the significance of our method.1 https://en.wikipedia.org/wiki/Help: Interlanguage_links• We did qualitative analysis to have an intuitive impression of our embeddings, and quantitative analysis in three tasks: word translation, entity relatedness, and crosslingual entity linking.Experiment results show that our method demonstrates significant improvements in all three tasks.