Chinese Text Similarity Calculation Model Based on Multi-Attention Siamese Bi-LSTM
Zhongguo Wang, Zhang Bao · 2021
Measuring text similarity is a key research area in natural language processing technology. In this research, we proposed a multi-attention Siamese bi-directional long short-term memory (MAS-Bi-LSTM) to calculate the semantic similarity between two Chinese texts. The novel model used Bi-LSTM as the basic framework of the Siamese network, introduced a multi-head attention mechanism to capture the key features of the text, and used the Manhattan distance to calculate the similarity. Experiments were conducted on the large-scale Chinese question matching corpus dataset. Results showed that our model can achieve higher accuracy compared with other comparable models. The F1 value of our model reached 0.8070. The contribution of this research is to use the multi-head attention mechanism to re-weight the semantic features, and explore the influence of different pre-training corpus, distance formulas and heads of multi-attention on the model.