Robust Continual learning Model Driven by Diverse Samples
Xueyun Nie, Yifan Wei, Zhihong Liu · 2024
The replay method is one of the effective methods to alleviate the catastrophic forgetting of deep learning models when the data is updated incrementally, but the existing methods have some shortcomings. In order to solve the problems of overfitting and bias of new and old categories, the feature distribution of new and old categories is confused in the feature space, and the balance between stability and plasticity of the model is destroyed, a robust continual learning model driven by diverse samples is proposed in this paper. This method selects a small number of samples through uncertainty measurement, retains enough old information for replay training, and designs a task attention module, which effectively alleviates the recency bias problem caused by sample imbalance in new and old tasks. The algorithm maintains the stability and plasticity balance of the model from multiple perspectives to mitigate catastrophic forgetting in the replay continual learning method. Through the evaluation of CIFAR100 data set, the accuracy of the proposed method is improved by 0.31% to 1.48% compared with the most advanced method.