Collaborative Global-Local Structure Network With Knowledge Distillation for Imbalanced Data Classification
Feiyan Wu, Zhunga Liu, Zuowei Zhang, Jiaxiang Liu, Longfei Wang · IEEE Transactions on Circuits and Systems for Video Technology · 2024
Multi-expert networks have shown great superiority for imbalanced data classification tasks due to their complementary and diverse. We have summarized two aspects for further explorations:(1)uncontrollable results, arising from the performance differences of individual experts and variations in sample difficulty;(2)insufficient exploration of the internal data structure. These factors result in inconsistent model performance across different data distributions, thereby impact the model’s generalization ability. To address the above issues, we propose a Collaborative Global-Local Structure Network (CGL-Net) with knowledge distillation for imbalanced data classification. Firstly, CGL-Net, as a new framework, decouples the representation learning of imbalanced data into global and local structure, enhancing the controllability of integration model in a hierarchical manner. Secondly, CGL-Net innovatively combines knowledge distillation, data augmentation, and multiple expert networks, efficiently extracting the internal structure of the data and improving robust recognition on imbalanced data. In particular, the global structure learning introduces an independent student network that integrates knowledge from diverse experts, enabling the model to achieve comprehensive and balanced performance across categories in imbalanced data. The local structure learning incorporates augmented data, allowing the model to focus on discriminative regional learning of individual objects, thereby enhances the robust representation for imbalanced data. After completing these two sequential learning stages, the model hierarchically integrates knowledge to achieve robust recognition performance on imbalanced data. Extensive experiments on six benchmark datasets demonstrate that the proposed CGL-Net significantly outperforms recent state-of-the-art methods.