Learning Purified Feature Representations from Task-irrelevant Labels
Yinghui Li, Chen Wang, Yangning Li, Hai-Tao Zheng, Ying Shen · 2022 International Joint Conference on Neural Networks (IJCNN) · 2022
Learning an empirically effective model with generalization using limited data is a challenging task for deep neural networks. In this paper, we propose a novel learning framework called Purified Learning to exploit task-irrelevant features extracted from task-irrelevant labels when training models on small-scale datasets. Particularly, we purify feature representations by using the expression of task-irrelevant information, thus facilitating the learning process of classification. Our work is built on solid theoretical analysis and extensive experiments, which demonstrate the effectiveness of Purified Learning. According to the theory we proved, Purified Learning is model-agnostic and doesn't have any restrictions on the model needed, so it can be combined with any existing deep neural networks with ease to achieve better performance.