Semi-supervised Training Data Generation for Multilingual Question Answering
Kyungjae Lee, Kyoungho Yoon, Sunghyun Park, Seung Won Hwang · 2018
Recently, various datasets for question answering (QA) research have been released, such as SQuAD, Marco, WikiQA, MCTest, and SearchQA.However, such existing training resources for these task mostly support only English.In contrast, we study semi-automated creation of the Korean Question Answering Dataset (K-QuAD), by using automatically translated SQuAD and a QA system bootstrapped on a small QA pair set.As a naïve approach for other language, using only machine-translated SQuAD shows limited performance due to translation errors.We study why such approach fails and motivate needs to build seed resources to enable leveraging such resources.Specifically, we annotate seed QA pairs of small size (4K) for Korean language, and design how such seed can be combined with translated English resources.These approach, by combining two resources, leads to 71.50 F1 on Korean QA (comparable to 77.3 F1 on SQuAD).