Enhancing Rubric Development in Science Education through Topic Modeling Techniques
Minju Hong, Hye Sun You · The Journal of Experimental Education · 2024
Content experts conduct traditional approaches to developing rubrics for constructed response (CR) items, which is a labor-intensive process. We aim to illustrate the potential benefits of utilizing topic modeling techniques to improve the efficiency of the rubric workload. The results of this study reveal that over half of the keywords in the rubric corresponded with the top 20 words extracted from the supervised latent Dirichlet allocation (sLDA) model. The rubric-based scores assigned by human raters showed medium-to-high associations with the sLDA-predicted scores. The sLDA approach offers supporting evidence for refining the rubric concerning criteria for word usage and score assignments. It accomplishes this by concurrently incorporating the predetermined rubric of content experts and the actual responses provided by students.