Towards a small language model powered chain‐of‐reasoning for open‐domain question answering
Jihyeon Roh, Minho Kim, Kyoungman Bae · ETRI Journal · 2024
Abstract We focus on open‐domain question‐answering tasks that involve a chain‐of‐reasoning, which are primarily implemented using large language models. With an emphasis on cost‐effectiveness, we designedEffiChainQA, an architecture centered on the use of small language models. We employed a retrieval‐based language model to address the limitations of large language models, such as the hallucination issue and the lack of updated knowledge. To enhance reasoning capabilities, we introduced a question decomposer that leverages a generative language model and serves as a key component in the chain‐of‐reasoning process. To generate training data for our question decomposer, we leveraged ChatGPT, which is known for its data augmentation ability. Comprehensive experiments were conducted using the HotpotQA dataset. Our method outperformed several established approaches, including theChain‐of‐Thoughtsapproach, which is based on large language models. Moreover, our results are on par with those of state‐of‐the‐artRetrieve‐then‐Readmethods that utilize large language models.