Dynamic Routing and Distillation-Aware Fusion for LoRA-Based Continual Learning
Xiang Zheng, Hongquan Xu, Chao Fang · 2025
Continual Learning (CL) for large pre-trained models faces the challenge of knowledge forgetting when adapting to new data. While parameter-efficient fine-tuning techniques like LoRA show promise in mitigating catastrophic forgetting, existing methods either indiscriminately accumulate modules or fail to maintain knowledge consistency. We propose Dynamic Routing and Distillation-Aware Fusion for LoRA-based Continual Learning (DRDF-LoRA), a novel framework that introduces a dynamic LoRA selection mechanism to reuse historical knowledge modules in a task-boundary-free manner, and performs elastic LoRA fusion guided by behavioral distillation from earlier stages. We conduct experiments on challenging continual visual question answering (VQA-CL) benchmarks. The results demonstrate that DRDF-LoRA significantly outperforms baseline methods in terms of accuracy, knowledge retention, and parameter efficiency, showcasing its ability to achieve long-term knowledge preservation after fine-tuning large models on complex tasks.