Artificial Intelligence Summaries of Historical Events Improve Knowledge Compared to Human-Written Summaries
Daniel Karell, Matthew Shu, Keitaro Okura, Thomas Davidson · 2024
How do artificial intelligence (AI) tools affect people's knowledge? Existing scholarship supports contrasting expectations. Some research shows that people often find AI-generated ("synthetic") texts clearer, more accessible, and more enjoyable to read, which can foster learning from the texts. However, other research shows that synthetic texts are derivative and homogeneous, making them less interesting and thus potentially inimical to learning. In two experimental studies, we find that people who read AI-generated summaries of historical events more accurately recall factual information about the events compared to people who read human-written summaries of the same event. In the first study (N=193), participants who read synthetic text describing the Seattle General Strike of 1919 correctly answered more fact-based questions than participants who read a text written by experts. The preregistered second study, with a sample nationally representative of the United States (N=1907) and additionally testing a second historical event, replicates the results of the first study. Study 2 also indicates that synthetic historical summaries have a positive effect regardless of whether subjects know they were generated by AI, as well as whether or not the summaries exhibit political biases. Furthermore, while synthetic summaries improve knowledge for participants across all levels of educational attainment, the largest improvements occur at the lowest levels. Our findings show how AI-generated texts can enhance learning and reduce, rather than amplify, educational inequalities.