Feature Mixup with Normalized Logit Knowledge Distillation for Class Incremental Learning
Xijia Hong, Hongyu Zhao · 2025
Class-incremental learning aims to continually acquire new knowledge while retaining previously learned knowledge, mimicking the human learning process. To overcome catastrophic forgetting, various recent approaches based on dynamic-architectures achieve good performance. In this paper, we introduce a Feature Mixup technique when training the expanded model to preserve and strengthen knowledge from old tasks. However, dynamic-architectures bring the problem in memory budget. To address this, we propose the Normalized Logit Knowledge Distillation (NLKD) to control the model size while maintaining performance. We tested our proposed method on three public benchmark datasets, and experimental results show that our approach makes progress.