Replay and regularization techniques to mitigate catastrophic forgetting in continual learning
Neeraj, Poonam Nandal · 2025
Artificial Intelligence adapts to learn continuously in a dynamic real-world environment. However, catastrophic (CF) forgetting is a significant challenge in continual learning, where neural networks degrade the performance of previously learned tasks while learning new tasks. To address catastrophic forgetting, this paper examines and compares three approaches: (1) Baseline (training without mitigation technique), (2) Regularization method, Elastic Weight Consolidation (EWC), and (3) Replay sampling (with buffer sizes 100 and 200). An experiment has been conducted on three datasets of binary classes. The performance was measured by forgetting level, average accuracy, memory required, and training time. The findings showed replay reduces significant forgetting and increases performance as buffer size increases from 100 to 200. EWC improves stability but may incur result trade-offs in plasticity. The strengths and limitations of replay and regularization are highlighted. The result emphasizes developing effective and scalable mitigation measures to balance the stability-plasticity and improve accuracy.