Multi-Granularity Knowledge Distillation Based Class Incremental Learning Method
Nan Lin, Lihao Wang, Cong Yang, Ying Han, Yongxia Chen · 2024
The issue of catastrophic forgetting is pervasive in class incremental learning scenarios, where the loss of old task knowledge significantly impacts model performance. Traditional knowledge distillation methods often struggle to effectively extract such knowledge. To tackle this challenge, this paper introduces a class incremental learning approach based on multi granularity knowledge distillation. This method regulates the training of the incremental model by extracting detailed and abstract features of the old model's knowledge. It encourages the incremental model to align with the outputs of the old model, thereby retaining more knowledge from previous tasks and preventing catastrophic forgetting. The proposed method can be integrated into various frameworks of class incremental learning. Comparative experiments are conducted using the CIFAR-100 and CIFAR-10 datasets. The results demonstrate that the proposed method effectively mitigates catastrophic forgetting and enhances the classification performance of incremental learning algorithms.