The Oil and Water Separation Phenomenon Inspired Loss for Feature Learning
Zijia Wang, Wenbin Yang, Zhi-Song Liu, Jiacheng Ni, Qiang Chen, Zhen Jia · 2023
Learning low-dimensional features that are both diverse and discriminative from high-dimensional data is a fundamental and critical task. This paper proposes a novel loss/principle for feature learning inspired by the "oil and water separation" phenomenon. The features learned with the proposed loss are shown to be more robust to label corruptions in empirical evaluations. Moreover, the proposed method achieves state-of-the-art performance in certain downstream tasks.