Development and Reliability-Validity Testing of a Knowledge, Attitude, and Behavior (KAB)-Based Scale for University Students' Academic Use of AI Tools
Yangli Ou, Nancy A Aguila, Lulu Xiao, Zhiyuan Zhang, Fan Ouyang · International Journal of Public Health and Medical Research · 2025
To develop and validate a scale for assessing the knowledge and behaviors related to academic use norms of AI tools among university students, focusing on reliability and validity. Grounded in the Knowledge-Attitude-Behavior (KAB) theory, a pool of scale items was constructed through a review of the literature, semi-structured interviews, and Delphi expert consultations. A pilot test was conducted with 140 undergraduate nursing students from a certain university in China, employing item analysis and exploratory factor analysis to refine the scale items. A formal assessment was then conducted with 220 nursing undergraduates who had used generative AI tools for at least three months. Reliability analysis and confirmatory factor analysis were performed using SPSS and AMOS. The scale consists of 2 subscales, 3 dimensions, and 14 items. Exploratory factor analysis extracted three common factors, with a cumulative variance explanation of 70.878%. The Cronbach's α coefficients for the subscales were 0.925 and 0.855, respectively, while the split-half reliability coefficients were 0.891 and 0.860. The fit indices for the confirmatory factor model were as follows: χ2/df=2.978, GFI=0.859, PGFI=0.606, RMSEA=0.098, RMR=0.048, CFI=0.920, and PCFI=0.748. The scale demonstrates strong reliability and validity, making it suitable for assessing university students' knowledge and behaviors regarding the academic use of AI tools.