Sampled Feature Replay and Multilayer Knowledge Distillation for Class-Incremental learning
Wen An, Yina Yang, Ruqiao Xu, Jingyi Zhu, Qiang Wang · 2023
Deep neural network is subject to catastrophic forgetting when learning new data incrementally. In this work we propose a class incremental learning algorithm based on feature replay and multi-layer spatial distillation to address the problem of data privacy and catastrophic forgetting. Firstly, different from general methods which save and replay images to retain old knowledge, this method saves prototype and generates feature for replay based on a new feature distribution assumption. Secondly, this method makes better use of the middle layer feature maps and performs multilayer spatial distillation, which alleviates catastrophic forgetting and improves the overall effect of class incremental learning. We implement multiple sets of experiments on public datasets and the experimental results show that the algorithm proposed in this paper can effectively improve the effect of class-incremental learning.