Cross Dense Feature Learning With Task Guidance for Few-Shot Classification
Qi Zhang, Long Chen, Wanfeng Shang · IEEE Transactions on Circuits and Systems for Video Technology · 2024
Few-shot classification aims to develop a classifier that adapts to new tasks using only a limited number of labeled images. To overcome the limitation of lacking training images in few-shot image classification, dense features have been extensively utilized to represent images by providing more subtle and discriminative clues. However, dense feature based methods are still facing challenges despite leveraging local details in images. Primarily, these methods deal with the support set images in each category independently, which ignores the information across different categories. Furthermore, dense features suffer from background noise, when performing similarity calculations based on a large number of dense feature pairs, these methods are susceptible to interference from task-irrelevant feature pairs. In this paper, we propose a cross dense feature learning with task guidance method to address the aforementioned issues. The key components of our method include two aspects. Firstly, a dense feature extraction approach based on transformer is proposed, aiming to better utilize inter-class information within the support set. We design two types of cross-attention mechanisms to get the across information among different categories for a better representation of dense features, named Support-Support Attention (SSA) and Support-Query Attention (SQA). Secondly, a task-relevant model is trained for dense feature pairs similarity calculating, aiming to filter out feature pairs that contribute more effectively to classification. Then we can get the final similarity to predict the label of query image through summarizing weighted local similarity. The experimental results prove that our method achieves a promising improvement for few-shot classification by taking information across different categories and task attention similarity into consideration.