A BERT-Based Spam SMS Filtering and Recognition Method with Adaptive Activation Functions
Weiwen Hao, Ruihong Yang, Yingbiao Ma, Jianglin Zhang · 2024
With the rapid development of mobile internet technology, smartphones have become an indispensable part of people's daily lives, greatly facilitating communication and living. However, the problem of spam SMS harassment has become increasingly prominent, not only affecting user experience but also potentially leading to fraud for users who lack sufficient awareness of network security, resulting in serious consequences such as information leakage and financial loss. Traditional methods for spam SMS recognition, such as keyword matching, machine learning, and deep learning, have achieved certain successes but still face significant challenges in understanding and processing the complexity of natural language. This paper innovatively proposes a spam SMS filtering and recognition method that combines BERT (Bidirectional Encoder Representations from Transformers) with an adaptive activation function layer. By introducing the adaptive activation function layer, this approach can dynamically adjust the non-linear transformation characteristics of the activation functions during fine-tuning based on the input data. This optimizes the structure of neurons in each layer of the neural network, enhancing the model's ability to learn different features. Specifically, this method uses a pre-trained BERT model as the base framework and adds an adaptive activation function layer for fine-tuning specific tasks, achieving efficient identification of spam SMS. Experimental results show that compared to traditional methods, the proposed BERT-based spam SMS filtering and recognition method with an adaptive activation function layer significantly improves recognition accuracy, demonstrating stronger stability and faster convergence speed. This not only provides new ideas and technical means for solving the problem of spam SMS but also demonstrates the great potential of large-scale parameter pre-trained models in specific text classification tasks. Additionally, this research has important theoretical value and practical application prospects, particularly in improving the efficiency and accuracy of text processing, thereby promoting the development of related fields.