Self-Knowledge Guided Retrieval Augmentation for Large Language Models
Yile Wang, Peng Li, Maosong Sun, Yang Liu · 2023
Large language models (LLMs) have shown superior performance without task-specific finetuning.Despite the success, the knowledge stored in the parameters of LLMs could still be incomplete and difficult to update due to the computational costs.As complementary, retrieval-based methods can offer nonparametric world knowledge and improve the performance on tasks such as question answering.However, we find that the retrieved knowledge does not always help and even has a negative impact on original responses occasionally.To better make use of both internal knowledge and external world knowledge, we investigate eliciting the model's ability to recognize what they know and do not know (which is also called "self-knowledge") and propose Self-Knowledge guided Retrieval augmentation (SKR), a simple yet effective method which can let LLMs refer to the questions they have previously encountered and adaptively call for external resources when dealing with new questions.We evaluate SKR on multiple datasets and demonstrate that it outperforms chain-of-thought based and fully retrieval-based methods by using either In-structGPT or ChatGPT.