Domain Specific NMT based on Knowledge Graph Embedding and Attention
Hao Yang, Gengui Xie, Ying Hua Qin, Peng Song · 2019
In recent years, the encoder-decoder/attention-based Transformer architecture is superior to traditional methods and has been adopted as the core technology in Google services. But for specific areas, translation accuracy or BLEU scores are not good enough without domain knowledge. Typical problems include (1) domain entity such as subject/object translation error, and (2) relationship translation error, because lacking enough knowledge involved model and algorithms. This paper introduces the knowledge graph embedding and attention model based on the traditional encoder-decoder model. The KG embedded layer improve entities pair accuracy in diffident languages, by softly matching entities pair embedding in the encoder and decoder steps. The KG attention layer leverages entity relationship pair, by trained attention score from knowledge graph, just like business rules. The experiment proves Knowledge Graph Embedding and Attention (KGEA) can improve the ICT-NMT online production system, and also BLEU scores increase 6% with KG Embedding, 3% with KG Attention and total 7%, compared with general-purpose transformer, with semantic entities and relationship rules.