A Learning Framework Combining Distillation-Generated Replay and Development Network in Continual Visual Scene Cognition for Autonomous Robot
Yuyang Zhang, Zhi Zheng · 2024
Continual learning for autonomous robots in complex environments is a challenging problem. Human beings have the lifelong ability to continuously acquire, adjust, and transfer knowledge. Although we tend to gradually forget previously learned knowledge throughout our lives, in very few cases does learning new knowledge catastrophically affect what we have already learned. Incremental learning aims to address a common flaw in model training: catastrophic forgetting. The primary drawback of most existing replay-based incremental learning is that they require a lot of additional computational resources and storage space to recall old knowledge. When the number of tasks keeps increasing, either the training cost becomes higher, or the representativeness of samples diminishes. In order to mitigate catastrophic forgetting and save storage space, we propose a new autonomous developmental neural network that combines distillation-generated replay(DGR-DN). Experimental results show that our approach not only has the better ability to integrate and refine new knowledge in new data, prevent significant interference from new input on existing knowledge, but also requires less storage space compared to existing generative replay networks. We verify the effectiveness of the method in the real environment scene, and also verify the generality using the MNISIT dataset. Through these experiments, it can be seen in the scene dataset that the recognition rate has increased by 300% compared to not using the generated network, and the storage space is reduced by more than 30% compared to the network that used generation. In the MNIST dataset, our approach significantly reduces storage space by over 50%, while preserving recognition rates at a level similar to the baseline.