Semantic Similarity Score for Measuring Visual Similarity at Semantic Level
Senran Fan, Zhicheng Bao, Chen Dong, Haotai Liang, Xiaodong Xu, Ping Zhang · IEEE Internet of Things Journal · 2024
With the rapid development of Internet of Things (IoT) technology, more sensors are required to operate in complex channel scenarios and under limited communication resources. Semantic communication, as an emerging paradigm, extracts, transmits, and reconstructs information at the semantic level, offering advantages, such as high compression rates and strong noise resistance. These features are expected to find widespread application across various IoT scenarios. However, widely used image similarity evaluation metrics like peak signal-to-noise ratio and multiscale structural similarity index primarily focus on pixel or structural features, making it challenging to accurately measure the loss of semantic-level information during transmission. This limitation poses challenges for the performance evaluation of visual semantic communication systems and restricts the emergence of more novel and efficient systems. To address this issue, we propose a new semantic evaluation metric-semantic similarity score (SeSS). This metric is based on Scene Graph Generation and graph matching techniques, transforming image similarity scores into graph matching scores. By manually annotating thousands of image pairs, we fine-tuned the hyperparameters within SeSS to align it more closely with human semantic perception. The performance of SeSS has been tested across various image datasets and specific IoT visual tasks. Experimental results demonstrate the effectiveness of SeSS in measuring differences in semantic-level information between images, making it a valuable tool for evaluating visual semantic communication systems. This development is expected to encourage the emergence of more robust systems suited for diverse IoT scenarios. The code of SeSS is openly available onhttps://github.com/FSR3340/Semantic_Similarty_ScoreGitHub.