What Evidence Do Language Models Find Convincing?
Alexander Wan, Eric Wallace, Dan Klein · 2024
Retrieval-augmented language models are being increasingly tasked with subjective, contentious, and conflicting queries such as "is aspartame linked to cancer".To resolve these ambiguous queries, one must search through a large range of websites and consider which, if any, of this evidence do I find convincing?In this work, we study how LLMs answer this question.In particular, we construct CON-FLICTINGQA, a dataset that pairs controversial queries with a series of real-world evidence documents that contain different facts (e.g., quantitative results), argument styles (e.g., appeals to authority), and answers (Yes or No).We use this dataset to perform sensitivity and counterfactual analyses to explore which text features most affect LLM predictions.Overall, we find that current models rely heavily on the relevance of a website to the query, while largely ignoring stylistic features that humans find important such as whether a text contains scientific references or is written with a neutral tone.Taken together, these results highlight the importance of RAG corpus quality (e.g., the need to filter misinformation), and possibly even a shift in how LLMs are trained to better align with human judgements. Question: is aspartame linked to cancer?Evidence #1 for the answer "Yes" Evidence #1 for the answer "No"Artificial sweeteners linked with a 13% higher risk of cancer New research finds that a higher intake of artificial sweeteners is linked to an increased risk of cancer.Nearly half of United States adults consume artificial sweeteners.Human-population studies have found artificial sweeteners to be safe, but results from in vitro studies and studies on animals pose some concerns.[...]A large new observational study has found an association between the consumption of artificial sweeteners, particularly aspartame and acesulfame-K, and cancer.The study found a 13% higher risk of cancer in general, with the highest likelihood of developing breast cancer and cancers related to obesity, for people consuming large quantities of artificial sweeteners.[....] the U.S. Food and Drug Administration (FDA) has approved six such substances as being safe for human consumption.Dr. Philip Landrigan was not involved in the study.He is [....] Professor of Biology at