Summarization Filter: Consider More About the Whole Query in Machine Comprehension
Kai Shuang, Yaqiu Liu, Wentao Zhang, Zhixuan Zhang · IEEE Access · 2018
Machine Comprehension (MC) is a challenging and valuable problem in natural language processing domain. The core method of the MC model is to get appropriate context representation to answer query. Query information such as what the query wants to know is injected into context representation by attention mechanism. The attention mechanism utilizes the relationship between query words and context words but ignores the whole semantics of the query, which is significant to understand what the query really wants to know. Inspired by the human’s reading process, we propose summarization filter mechanism in order to consider more about the whole query in MC. We obtain a summarization vector of the query to represent its whole meaning and fuse it into the gates of some layers. Our mechanism makes the model to better understand the query and give more precise answer. Our model promotes the baseline model’s performance by 0.7 F1 score and 1.1 EM score, which indicates that our mechanism contributes to a decent promotion for MC task.