Manifold learning to address catastrophic forgetting
Prathyusha Akundi, Jayanthi Sivaswamy · 2021
A major challenge that deep learning systems face is the Catastrophic Forgetting (CF) phenomenon that is observed when fine-tuning is used to try and adapt a system to a new task or a sequence of datasets with different distributions. CF refers to the significant degradation in performance on the old task/dataset. In this paper, a novel approach is proposed to address CF in computer aided diagnosis (CAD) system design in the medical domain. CAD systems often need to handle a sequence of datasets collected over time from different sites with different imaging parameters/populations. The solution we propose is to move samples from all the datasets closer to a common manifold via a reformer at the front end of a CAD system. The utility of this approach is demonstrated on two common tasks, namely segmentation and classification, using publicly available datasets. Results of extensive experiments show that manifold learning can yield about 74% improvement on an average in the reduction of CF over the baseline fine-tuning process and the state-of-the-art regularization based methods. The results also indicate that a Reformer when used in conjunction with the state-of-the-art regularization methods, has the potential to yield further improvement in CF reduction.