An Exploration of Large Language Models for Verification of News Headlines
Yifan Li, ChengXiang Zhai · 2023
This study explores the capabilities of ChatGPT in news headline verification across different prompts and languages. We introduce an optimal prompt design and a novel difficulty ratio metric to analyze ChatGPT’s performance across various statement resources. Our findings highlight ChatGPT’s promising accuracy and cross-linguistic adaptability in fact-checking, while also identifying areas for further investigation, especially in expanding the analysis beyond headlines and exploring other Large Language Models (LLMs). Our findings suggest it is highly promising to leverage an LLM such as ChatGPT as a general tool in combating misinformation, enhancing the trustworthiness of digital information, and increasing trust-worithiness of text data mining algorithms by providing more reliable data sources for the algorithms.