A Page-topic Relevance Algorithm Based on BM25 and Paragraph-Semantic Correlation
Shi Peng, Xiaodan Xie, Jia Zhai, Yusheng Jia, Yuxuan Gong · Journal of Physics Conference Series · 2021
Abstract It is difficult for the traditional topic relevance algorithm based on word frequency and probability statistics model to deal with the ambiguous topic of search keywords, which leads to the retrieval results containing much information that users are not interested in. To solve this problem, a page-topic relevance algorithm based on BM25 and paragraph-semantic correlation is proposed in this paper. The semantic correlation of one page is calculated by the paragraph-semantic classification using a pre-trained deep neural network model, and is weighted with the BM25 retrieval score. Experimental results show that compared with the traditional BM25 algorithm, this algorithm can effectively improve the retrieval accuracy.