Reducing Uncertainty of Weak Supervised Signals via Mutual Information Techniques
Yichen Liu, Hanlin Feng, Xin Zhang · IEEE Access · 2025
Weakly supervised learning (WSL) refers to training models using imperfect or noisy labels, which can significantly reduce the costs associated with manual labeling. However, the model performance may be affected by the uncertainty of weakly supervised signals (WSS). To address this problem, we propose a novel Mutual Information-based Network (MINet), which reduces the reliance on weak labels by maximizing the mutual information (MI) and minimizing the conditional mutual information (CMI) between representations and labels. MINet differs from existing methods by using a dual-branch model structure that follows a mutual information-based objective function. The objective of one branch is to maximize MI, which helps to extract valuable information from weak labels. The second branch reduces the dependence of data representations on their corresponding weak labels, which lessens the negative effects of WSS. By applying variational approximation and reparameterization techniques, we optimize the proposed model and achieve significant improvements in robustness and generalization on four image classification datasets. Moreover, the experiments show that the model can effectively utilize the effective information from WSS and reduce their uncertainty.