Developing a How-to Tip Machine Comprehension Dataset and its Evaluation in Machine Comprehension by BERT
Tengyang Chen, Hongyu Li, Miho Kasamatsu, Takehito Utsuro, Yasuhide Kawada · 2020
In the field of factoid question answering (QA), it is known that the state-of-the-art technology has achieved an accuracy comparable to that of humans in a certain benchmark challenge.On the other hand, in the area of nonfactoid QA, there is still a limited number of datasets for training QA models, i.e., machine comprehension models.Considering such a situation within the field of the non-factoid QA, this paper aims to develop a dataset for training Japanese how-to tip QA models.This paper applies one of the state-of-the-art machine comprehension models to the Japanese how-to tip QA dataset.The trained how-to tip QA model is also compared with a factoid QA model trained with a Japanese factoid QA dataset.Evaluation results revealed that the how-to tip machine comprehension performance was almost comparative with that of the factoid machine comprehension even with the training data size reduced to around 4% of the factoid machine comprehension.Thus, the how-to tip machine comprehension task requires much less training data compared with the factoid machine comprehension task.