An Efficient Machine Reading Comprehension Method Based on Attention Mechanism
Wenzhen Jin, Guocai Yang, Hong Chun Zhu · 2019
Machine Reading Comprehension (MRC) answers the questions related to the specified passage to accurately locate the position of the answer. In recent years, many deep learning models based on attention mechanisms have evolved because they provide flexible and effective information interaction. Attention mechanism can further improved by introducing a multi-head self-attention mechanism to obtain the information in different subspaces. In this work, we propose a new model called MHT-Reader by improving multi-head self-attention mechanism with residual network, and convolutional neural network (CNN) combined with highway network to learn the vector representation of passage and question. The experimental results show that on DuReader Search, the BELU-4 is 24.8%, and the Rouge-L is 31.7%, they increased by 1.7% and 2.1% respectively compared with Bidirectional Attention Flow (BiDAF). On DuReader Zhidao, the BELU-4 is 42.1%, and the Rouge-L is 45.6%, they increased by 2.0% and 1.0% respectively compared with BiDAF. Experimental results show that our model has a better performance than BiDAF on the DuReader dataset.