News GPT: A Large Language Model for Reliable and Hallucination-Controlled News Generation
Shunyu Yao, Qingqing Ke, Kangtong Li, Qiwei Wang, Jie Hu · 2024
With the continuous development of natural language processing (NLP) technology, large models have become essential tools for handling natural language tasks. In the news domain, large models can be used for automating news generation, thereby enhancing the productivity and quality of news production. As a result, we introduce a new large model—News GPT—which utilizes an external knowledge retrieval module to inject real information, providing authentic and reliable news generation services. By combining Chinese and English news data with general domain data, we have constructed a high-quality, multi-domain news dataset consisting of 1.4B tokens. In the pre-training phase, we expand the Chinese vocabulary for the Llama2-70B model, and in the fine-tuning phase, we design expert prompts to help the model understand downstream tasks better. We have tested News GPT in various aspects, and the experimental results show that News GPT, as an intelligent news assistant, can effectively complete tasks using retrieval tools in different application scenarios. It possesses excellent natural language understanding, knowledge, and logical reasoning abilities, and effectively controls the hallucinations in the generation process. News GPT demonstrates high accuracy and reliability in news generation and can provide substantial support for the news industry.