ALMM: Analytic Learning and Model Merging for Class Incremental Learning
He Han, Huiping Zhuang · 2025
In rapidly changing environments, traditional static deep learning models are unable to adapt to the evolving demands of new tasks. These models lack the adaptability required to address dynamic scenarios. While fine-tuning these models can be a solution to some extent, it frequently leads to a phenomenon known as catastrophic forgetting, which involves the loss of knowledge acquired from previous tasks. Existing class-incremental learning methods predominantly prioritize the mitigation of catastrophic forgetting, often at the cost of compromising plasticity and adaptability. To address these challenges, we draw inspiration from model merging techniques and propose the Analytic Learning and Model Merging (ALMM) method. ALMM integrates the strengths of model fine-tuning, analytic learning and model merging into a cohesive framework. This approach enables ALMM to achieve a balance between robustness and plasticity in class-incremental learning tasks, while also safeguarding data privacy—a critical consideration in real-world applications. Experimental results demonstrate that ALMM excels in class-incremental learning scenarios, effectively addressing the dual challenges of catastrophic forgetting and adaptability, and showcasing superior performance in balancing robustness and plasticity.