Are They Getting What They Expected? User Confirmation and Satisfaction with Generative AIs
Boryung Ju, Brenton J. Stewart · Proceedings of the Association for Information Science and Technology · 2025
ABSTRACT The purpose of this study is to explore users' expectations of LLMs, examine their confirmation of perceived system performance, and examine how these factors influence their overall satisfaction with the system. We analyzed data collected from LLM users through an online survey using Welch's ANOVA and regression analysis. The findings demonstrate that users' expectations and confirmation of LLMs are fluid across different socio‐cultural variables, spanning age, gender, and educational levels. Additionally, users' perceived system performance, of LLMS, significantly influences their confirmation of the system. Specifically, both perceived usefulness and perceived ease of use have a statistically significant effect on confirmation. Both of our sub‐models demonstrate that perceived system performance influences users' confirmation of a given system, and users' confirmation is a strong determinant of their satisfaction. Furthermore, our results indicate an uneven distribution and penetration of AI technologies with respect to age, gender and educational level.