Enhancing Linguistic Structure Awareness in Machine Translation
Van-Dat Phan, Nhat-Anh Huynh, Ngoc-Dung Nguyen, Xuan-Dung Doan · 2024
Machine translation has seen many advancements with the advent of Transformer-based models. We investigate two prominent Transformer variants: the first variant restricts the attention scope by passing syntactic knowledge into the multihead self-attention phase forcing the model to focus more on the local region. The other approach incorporates syntactic structure information to enable the model to control the contents better. This paper exploits how to adapt both approaches to enhance the Transformer translation. This improvement is achieved by integrating additional grammatical information within the sentences. Specifically, information from the headword is incorporated to generate a weight matrix, thereby augmenting the attention of the dependent word to the headword. Consequently, our model effectively addresses the challenge of information loss in long-range dependency problems. The experimental results show that our method improves translation quality on largescale WMT English$\rightarrow$German dataset and low-resource VLSP English$\rightarrow$Vietnamese, Chinese$\rightarrow$Vietnamese datasets.