A Fuzzy Multigranularity Convolutional Neural Network With Double Attention Mechanisms for Measuring Semantic Textual Similarity
Butian Zhao, Runtong Zhang, Kaiyuan Bai · IEEE Transactions on Fuzzy Systems · 2024
Semantic textual similarity (STS) is a fundamental task in the field of natural language processing (NLP). Recent advances demonstrate that deep-learning-based approaches can achieve excitingly accurate STS measurement. However, existing studies cannot capture the spatial location of important information by attention mechanisms, fail to model sentences from the perspective of overall sentences, and neglect to deal with semantic fuzziness. In this article, we propose a novel double attentive fuzzy convolutional neural network (DAFCNN) to measure STS more accurately with the consideration of semantic fuzziness. This article first introduces the spatial attention module and combines it with the improved attentive convolutions to create a multigranularity convolutional neural network in DAFCNN, which not only extracts critical spatial location information but also models sentences from multiple perspectives at word and sentence levels. Second, DAFCNN pioneers a fuzzy learning module (FLM) to fulfill the extraction of fuzzy semantic features. By using the fuzzy membership function, fuzzy aggregation operator, and trainable parameters and weights, FLM can map sentence representations to fuzzy space to constitute representations with more accurate and rich semantics. Third, compared with various state-of-the-art STS models, DAFCNN decreases by 14.57% mean-square error, increases by 4.61% Pearson's γ and 8.57% Spearman's ρ on STS score datasets, and increases by 3.39% accuracy and 2.41%F1-score on semantic classification dataset. The ablation experiment demonstrates the effectiveness of each module of DAFCNN. Finally, the experimental results also indicate that FLM is a promising new attempt to incorporate fuzzy set theory in the NLP field.