Towards Unbiased Multi-Label Zero-Shot Learning With Pyramid and Semantic Attention
Ziming Liu, Song Guo, Jingcai Guo, Yuanyuan Xu, Fushuo Huo · IEEE Transactions on Multimedia · 2022
Multi-label zero-shot learning extends conventional single-label zero-shot learning to a more realistic scenario that aims at recognizing multiple unseen labels of classes for each input sample. Existing works usually exploit attention mechanism to generate the correlation among different labels. However, most of them are usually biased on severalmajor classeswhile neglect most of theminor classeswith the same importance in input samples, and may thus result in overly diffused attention maps that cannot sufficiently coverminor classes. We argue that disregarding the connection between major and minor classes, i.e., correspond to the global and local information, respectively, is the cause of the problem. In this paper, we propose a novel framework of unbiased multi-label zero-shot learning, by considering various class-specific regions to calibrate the training process of the classifier. Specifically,Pyramid Feature Attention(PFA) is proposed to build the correlation between global and local information of samples to balance the presence of each class. Meanwhile, for the generated semantic representations of input samples, we proposeSemantic Attention(SA) to strengthen the element-wise correlation among these vectors, which can encourage the coordinated representation of them. Extensive experiments on the large-scale multi-label benchmarksMS-COCO,NUS-WIDEandOpen-Imagesdemonstrate that the proposed method surpasses other representative methods by significant margins.