Subclass-wise Logit Perturbation for Multi-label Learning
Zhu Yu, Ou Wu, Fengguang Su · ACM Transactions on Knowledge Discovery from Data · 2025
Logit perturbation refers to adding perturbation on logit, which has been shown to be capable of enhancing the robustness and generalization capabilities of deep neural networks in machine learning. However, studies on logit perturbation for multi-label learning are limited and they only consider the issue of class imbalance in the training data. Furthermore, the logit perturbation vectors in these methods are identical for negative classes containing different subclasses when multi-label learning is viewed as a multiple binary classification problem. This study investigates logit perturbation by exploring the characteristics of subclass-wise multi-label training data. First, the influence of the characteristics of multi-label training data on classification performance is analyzed in terms of the three data characteristics, namely, proportion, variance, and co-occurrence for each category (or subclass). Quantitative analyses reveal that variance differences among the subclasses in the negative class of a decomposed binary task also negatively impact the training performance, and if multiple characteristics affect simultaneously, the performance deterioration will be more severe. Second, theoretical analysis is performed for subclass-wise logit perturbation and a new subclass-wise logit perturbation method is proposed for multi-label learning. In our method, each class/subclass has a carefully designed perturbation implementation according to its proportion, variance, and co-occurrence. Finally, our proposed method is further explained through a regularization view. Extensive experiments demonstrate that our method consistently enhances the generalization performance of popular depth networks on multi-label benchmark datasets.