Multi-Head CNN-Attention Prompt Pools: Strengthening Prompt Combination and Contextual Insight for Domain Adaptation
Gyutae Oh, Jitae Shin · 2025
Modern AI models heavily depend on training data, so when they are presented with new inputs that were not encountered during the training process, their prediction performance can drop sharply. Furthermore, in real world environments, the data distribution constantly changes over time, which exacerbates these limitations. Therefore, continual learning techniques that allow models to continuously learn while effectively retaining existing knowledge even as new data is sequentially introduced are essential. However, simple retraining methods can lead to catastrophic forgetting, where new information overwrites previously learned knowledge, significantly degrading past performance. In this paper, we propose a novel approach to improve the prompt pool, an important component in prompt-based learning, in order to mitigate such forgetting. The proposed method achieves performance improvements ranging from approximately 2% to over 8% compared to conventional prompt-based learning techniques in various benchmark experiments.