Effectiveness of Automated Chinese Sentence Scoring with Latent Semantic Analysis
Chen-Huei Liao, Bor‐Chen Kuo, Kai-Chih Pai · The turkish online journal of educational technology · 2012
Automated scoring by means of Latent Semantic Analysis (LSA) has been introduced lately to improve the traditional human scoring system. The purposes of the present study were to develop a LSA-based assessment system to evaluate children’s Chinese sentence construction skills and to examine the effectiveness of LSA-based automated scoring function by comparing it with traditional human scoring. Twenty-seven fourth graders and thirty-one six graders were assessed on single-character sentence making test (subtest 1) and two-character words sentence making test (subtest 2). The outcomes of LSA-based automated scoring methods in three Chinese semantic spaces generated from three type weighting functions were compared to the traditional human scoring. The results showed that LSA-based automated scoring in three different Chinese semantic spaces and traditional human scoring were highly correlated in single-character sentence making test and moderately correlated in two-character words sentence making test. The Chinese semantic space generated from Log-IDF outperformed the other two types of weighting function in the present study.