Optimizing News Topic Classification with Instructional Fine-Tuning of Chatglm3
Chunyang Ye, Xiaorui Shi · 2024
In the rapidly evolving landscape of natural language processing (NLP), the deployment of large language models has been at the forefront of innovation, particularly in the realm of text classification. This study probes into the utilization of advanced NLP techniques, emphasizing the instructional fine-tuning of the ChatGLM3-6B model —a variant of the General Language Model with 6.2 billion parameters—for the specific application of news topic classification. As the volume and velocity of news data burgeon, the imperative for models to not only accurately classify but also to efficiently process text becomes paramount. Our research undertakes a meticulous fine-tuning of ChatGLM3-6B using the QLora framework, which is renowned for its quantization efficiency and adaptability in processing, on the Twitter Financial News dataset. This dataset, characterized by its multi-label text classification challenge and inherent data imbalance, provides a rigorous testing ground for our model. We benchmarked ChatGLM3-6B against established models such as Roberta-Base, Roberta-Large, and the Deberta-V2 variants, measuring performance in terms of accuracy—a critical metric in the domain of news classification. The findings of our study are compelling, demonstrating that the instructionally fine-tuned ChatGLM3-6B model not only achieves but exceeds the classification accuracy of traditional models, registering an accuracy of 88.15%. This superior performance can be attributed to the strategic combination of the model's inherent capabilities and the fine-tuning process that was rigorously tailored to the complexities and nuances inherent in financial news texts. Moreover, we closely monitored the training stability and the model's ability to generalize, which is graphically evidenced by the consistent decline in training loss over approximately 1,500 iterations, without any indication of overfitting. This attests to the robustness of the chosen optimization strategies, including an empirically determined learning rate, batch size, and gradient accumulation strategy. Conclusively, this study not only elucidates the potential of fine-tuned large language models like ChatGLM3-6B in the efficient and accurate categorization of news topics but also propels the conversation forward regarding the scalability of these models. Our findings suggest significant implications for the automation of news categorization processes, paving the way for future research into the ethical and scalable applications of NLP in media and journalism.