How ChatGPT’s Semantic Parrotting (Compared to Gemini’s) Impacts Text Summarization with Literary Text
Rodolfo Delmonte, Giulia Marchesini, Nicolò Busetto · Information · 2025
In this paper we explore ChatGPT’s ability to produce a summary, a precis, and/or an essay on the basis of excerpts from a novel—The Solid Mandala—by Nobel Prize Australian writer Patrick White. We use a number of prompts to test a number of functions related to narrative analysis from the point of view of the “sujet”, the “fable”, and the style. In the paper, we illustrate extensively a number of recurrent semantic mistakes that can badly harm the understanding of the contents of the novel. We made a list of 12 different types of semantic mistakes or parrotting we found GPT made, which can be regarded as typical for stochastic-based generation. We then tested Gemini for the same 12 mistakes and found a marked improvement in all critical key issues. The conclusion for ChatGPT is mostly negative. We formulate an underlying hypothesis for its worse performance, the influence of vocabulary size, which in Gemini is seven times higher than in GPT.