A Continual learning approach based on Gaussian process model
Zhaoyang Wen, Chaoyun Yao, Liuhao Sheng, Shiyu Chen, Qian Cao, Jiabao Cui · 2024
In light of the limitations associated with traditional continual learning methods, such as catastrophic forgetting and inadequate generalization capabilities, we propose a new continual learning model guided by Gaussian Processes. This method constructs a relationship between batch data and the global distribution using a Gaussian model. By integrating Gaussian Processes into the empirical replay mechanism, we enhance the global awareness of batch data. The learning process involves modeling the mapping relationship between the feature and label spaces. We utilize Gaussian process regression to compute the contextual prediction probability for each sample within the current batch. By exploring the joint optimization of contextual prediction probabilities, original labels, and network prediction probabilities, we improve the representation learning capability of the neural network model, thereby enhancing the accuracy of target recognition over time. Experimental results demonstrate that our proposed method significantly reduces training and validation losses across MNIST, CIFAR10, and CIFAR100 datasets. The integration of Gaussian Processes markedly improves both the convergence speed and final accuracy of the model. These findings underscore the potential of Gaussian Processes in the training of complex neural networks and offer a new avenue for future continual learning research.