Efficient fine-tuning of short text classification based on large language model
Likun Wang · 2024
With the booming development of social networks, a massive amount of short texts emerge every day, containing valuable information such as user interests and intentions. Therefore, the mining and classification of short text information is particularly important. However, the inherent characteristics of sparse features and high noise in short texts limit the performance of traditional machine learning methods in short text classification. Meanwhile, many neural network models often rely on a large amount of annotated data during the training process, but obtaining sufficient annotated data is a challenging task in practical situations. Taking inspiration from recent large-scale language models, this article proposes an efficient fine-tuning method for short text classification based on the LLaMA large-scale language model. Utilizing the powerful learning ability of large language models to expand text information, fine-tuning the freezing model and instruction learning through LoRA can more fully classify downstream specific tasks. From the experimental results obtained from real datasets, it can be seen that the method proposed in this paper has achieved an improvement in the accuracy of short text classification.