Reducing Overfitting In Deep Neural Networks By Intra-class Decorrelation
Tian Liu, Chenguang Zhang, Danmeng Li · 2023
Although deep learning models have achieved remarkable progress in various domains, the issue of overfitting remains a significant challenge. Our research proposes a novel regularization technique called ICD, that addresses this problem by enhancing generalization performance. We consider each hidden unit of a specific layer as a base-learner, and ICD regularization ensures the creation of diverse and non-redundant representations in deep learning models by minimizing intra-class cross-covariance among these base learners. Our experiments on diverse datasets and network architectures illustrate that ICD reduces overfitting and consistently enhances generalization performance.