Maintaining Distinction and Fairness in Data-Free Class Incremental Learning
Feng Ying Yu, Pengcheng Zhao, Yadong Guo, Xue Zhao, Wangli Hao, Ran Zhao, Fuzhong Li · 2023
Catastrophic forgetting is the central challenge in class incremental learning (CIL), namely adapting a model to new data often results in severe performance degradation on previous tasks or classes. The most successful methods to alleviate this forgetting require extensive replay of previously seen data, which is problematic when memory constraints or data legality concerns exist. In non-exemplar-based CIL setting, an important factor causing catastrophic forgetting is the presence of severe bias between the new and previously learned classes in both the feature extractor and classifier. To address this problem, this paper proposes a simple and effective solution to calibrate the bias in non-exemplar-based CIL. Specifically, we propose to maintain the decision boundaries of the previously learned classes by preserving a collection of pseudo-samples in the latent space. By maintaining the distinction and balance between old and new classes mitigates representation bias. Then, to further maintain fairness, we propose classifier aligning that corrects the bias in the FC layer, which help the model output more discriminative results within older classes. Experimental results on benchmark datasets show that our method is significantly superior to non-exemplar-based methods, with a 10% improvement in average incremental accuracy, and achieves comparable performance compared to exemplar-based methods. The ablation study further confirms the effectiveness of the method.