More with Less: ZeroQA and Relevant Subset Selection for AI2 Reasoning Challenge
cu Cristian-Bogdan Pătras, George-Sebastian Pîrtoacă, Traian Eugen Rebedea, Ştefan Ruşeţi · Procedia Computer Science · 2022
Natural language processing has had a significant growth in recent years, two key factors being the increase of processing capabilities leading to very large models and the availability of larger standardized datasets. However, not every specific task has a fair amount of available data and using more data does not always lead to better results. Therefore, we also need to focus on obtaining good results even when small datasets are provided. For this reason, we propose a model for question answering, called ZeroQA, which ranked first place when submitted in two popular leaderboards for answering multiple-choice science questions: ARC Easy and ARC Challenge. Our ZeroQA model uses no transformers fine-tuned on the ARC training dataset, and relies mainly on transfer learning using a mixture of experts. We also propose methods of selecting relevant subsets from the ARC datasets for training and we use them to analyze how our model performs with less, but well chosen training datasets.