Research on Evaluating College Students' Usage Behaviors and Patterns of Generative AI Tools Using Natural Language Processing
Hanying Gan, Wei Ji · 2025
Generative AI tools have great potential for application in the field of education, but there are few studies on the behavioral patterns and influencing factors of students using these tools. This study explores the behavioral characteristics and text generation quality of college students when using generative AI tools, and proposes an integrated research framework that combines natural language processing (NLP), cluster analysis, and regression models. This study uses 200 students from a university in Guangdong as the research dataset, uses the BERT model to quantify the fluency, relevance, and creativity of generated texts, and uses the K-means algorithm to classify students' behaviors into three categories: high-frequency learners, low-frequency casual users, and mixed users. The experimental results show that the text quality under different task types is significantly different. For example, the scenarios of writing summary generation and creative writing are different. The summary generation task has the highest score in relevance (90 points), while the creative writing task performs well in creativity (85 points). Subsequent regression analysis found that professional background and technical attitude are key factors affecting behavioral patterns. Students of science and engineering tend to frequently use generative AI tools for complex tasks, while students of humanities pay more attention to language fluency and creativity.