Mitigating the catastrophic forgetting effect using generative models
Gianpaolo Bontempo · 2020
The catastrophic-forgetting effect is the inability of a neural network to remember objects already seen when it encounters new classes as in an incremental learning approach. There have been many attempts to mitigate it such as increasing the number of parameters of a model or keeping the most significant samples in memory for each class. However, it requires too much memory. For the following reason, the goal of this research was to address the problem without external memory support but rather through the use of generative models such as VAE and PGGAN. Then a model is proposed that can explore the limits of memory replay combined with knowledge distillation. It consists in transferring knowledge from a teacher model to a student when it comes to pseudo-images. The research was conducted on several datasets such as MNIST, CIFAR10 and a biological dataset of different types of colorectal cancer