Recognition of Junk Short Messages Based on Local Sensitive Hash KNN Algorithm
Jihui Fan, Fengshan Yuan · 2022
At present, the recognition and filtering of spam short messages mainly include the following technologies: black and white list recognition technology, transmission frequency limit recognition technology, keyword matching recognition technology, and machine learning based spam short message recognition technology. These commonly used spam short message recognition technologies have their own limitations. The K-Nearest Neighbor algorithm (KNN, K-Nearest Neighbor) based on locally sensitive hashing proposed in this paper can find the samples as close as possible when the eigenvectors are similar but not identical. First, define the features, and then use the local sensitive hash method(LSH). The task of calculating the similarity of the matrix is to find a measure, quantify the similarity of the matrix, compile and train the model, generate the matrix based on the text content of the short message, use TF-IDF algorithm, use Local Sensitive Hash to solve the most similar samples, record the category of the samples, import the training results to the KNN classifier, and obtain the best nearest neighbor through continuous experiments. The accuracy rate is evaluated on the test set or verification set. After KNN classifier, the classification accuracy rate reaches 98.7%.