COMEM: In-Context Retrieval-Augmented Mass-Editing Memory in Large Language Models
Shanbao Qiao, Xuebing Liu, Seung‐Hoon Na · 2024
Noting that world knowledge continuously evolves over time, large language models (LLMs) need to be properly adjusted by performing the "knowledge editing", which involves updating outdated information or correcting false information.To achieve reliable and "massive" editing capabilities in terms of generalization and specificity, this paper proposes a unified knowledge editing method called in-COntext retrieval-augmented Mass-Editing Memory (COMEM), which combines two types of editing approaches: parameter updating and in-context knowledge editing (IKE).In particular, COMEM incorporates retrievalaugmented IKE, a novel extension of IKE designed for massive editing tasks, based on an updating-aware demonstration construction.Experimental results on the zsRE and Counter-Fact datasets demonstrate that COMEM outperforms all existing methods, achieving stateof-the-art performance.Our code is available at https://github.com/JoveReCode/ COMEM.git.