Balancing Plasticity and Stability via Dual-Branch Learning in Online Continual Learning
Xu Han, Haodong Lu, Huiyi Wang, Dong Gong · 2025
Deep learning models struggle with catastrophic forgetting in continual learning (CL), specifically online continual learning, requiring a balance between plasticity (adapting to new tasks) and stability (retaining prior knowledge). Memory replay addresses this challenge by integrating new data with stored memories, but existing methods of replaying the samples often fail to achieve an effective balance, overemphasizing either old or new knowledge. Previous works address this issue by enhancing replay buffer quality through sample selection or distilled images but overlook the model's learning process itself. To tackle this, we propose a novel dual-branch framework designed to manage the plasticity-stability trade-off in CL. The fast-updated branch achieves plasticity by quickly adapting to novel tasks, while the slow-updated branch maintains stability by gradually consolidating knowledge across all seen tasks only through the fast-updated branch, instead of directly learning from samples. To further stabilize the learning process, we introduce a consistency loss to align the two branches and prevent divergence. This dual-branch structure integrates new information while maintaining robust retention of prior knowledge. Extensive experiments on online continual learning benchmarks demonstrate superior performance and a well-balanced plasticity-stability trade-off. Open source code is available at https://github.com/han030927/BPS.