Zero-Shot Image Classification Algorithm Based on SIF Fusion Semantic Tags
Rencun Dai · Automatic Control and Computer Sciences · 2022
Abstract Background: At present, two semantic vectors, word embedding vector and attribute vector, have been often used for category representation in the zero-shot image classification process. However, these two representation forms of the semantic vector suffer from two problems. The first problem is that words can be ambiguous, and another is that the word embedding vector is not necessarily related to the visual features of a category. In the zero-shot image recognition, the problem of the semantic interval will eventually arise due to the inconsistency in the visual information of an image and image semantic information understood by a human. Methods: Therefore, performing the zero-shot image classification using only the word embedding vector obtained from category words will introduce a large error to the experimental results. To reduce the experimental result error caused by the semantic interval problem, more information is needed to represent a category correctly, because the visual features of the category can be described using natural language. Results: If the semantic information describing the visual features of a category is infused into the word embedding vector of a category, the impact of the semantic interval problem can be effectively reduced theoretically. This paper proposes a semantic information fusion (SIF) algorithm. This algorithm can supplement the visual feature information of a category on the basis of the word embedding vector of the original category, thereby alleviating the impact of the semantic interval problem. Conclusion: The experimental results demonstrate that the proposed algorithm achieves satisfactory performance in image classification.