Dependency-Based Relative Positional Encoding for Transformer NMT
Yutaro Omote, Akihiro Tamura, Takashi Ninomiya · 2019
In this paper, we propose a novel model for Transformer neural machine translation that incorporates syntactic distances between two source words into the relative position representations of a selfattention mechanism.In particular, the proposed model encodes pair-wise relative depths on a source dependency tree, which are the differences between the depths of two source words, in the encoder's selfattention.Experiments show that our proposed model achieved a 0.5 point gain in BLEU on the Asian Scientific Paper Excerpt Corpus Japanese-to-English translation task.