A Study of Sentence Similarity Based on the All-minilm-l6-v2 Model With “Same Semantics, Different Structure” After Fine Tuning
Yin Chen, Zixuan Zhang · Advances in computer science research · 2024
Traditional natural language processing models often find it difficult to distinguish between sentences with "similar structure and different semantics" and sentences with "different structure and similar semantics".Based on the all-MiniLM-L6-v2 and Bidirectional Encoder Representations from Transformers(BERT) model, this paper uses supervised learning and transfer learning methods to study the similarity of sentences with "similar structure, different semantics" and "different structure, similar semantics".New datasets in medical aspects with the same format as the hard datasets are artificially constructed and used as subdivided small-volume datasets to verify the model performance, thus simulating the needs of specific fields.On the basis of metalearning and small number of shots learning, different models are fine-tuned, and good verification results are obtained and compared.For the fine-tuned models, the performance has been improved, among which the most significant improvements are: BERT model: accuracy: 0.51 to 0.65, all-MiniLM-L6-v2 model: precision:0.74 to 0.91 and so on.In this paper, the supervised learning method is used to provide effective ideas and directions for sentence similarity division of "semantically similar, structurally different" and "semantically different, structurally similar".This optimization can be proved to be effective and necessary.