SETSum: Summarization and Visualization of Student Evaluations of Teaching
Yinuo Hu, Shiyue Zhang, Viji Sathy, A. T. Panter, Mohit Bansal · arXiv (Cornell University) · 2022
Student Evaluations of Teaching (SETs) are used in many colleges and universities as a measurement of teaching performance and a reference for many high-stake decisions. A typical demonstration of SET results for instructors is a static report, including summary statistics for quantitative ratings and an unsorted list of student comments on open-ended questions. The long, unorganized raw comments create difficulties for instructors to efficiently obtain main takeaways from students’ feedback, make accurate and unbiased inferences, and effectively improve future instructional design. This work introduces a novel system, SETSUM, that leverages sentiment analysis, aspect extraction, summarization, and visualization techniques to provide instructors and other reviewers with more efficient and less biased interpretations of SET results. The work is designed for a real university setting and the first version of the website is evaluated by 10 university professors from diverse departments. The human evaluation result shows that all 10 instructors agree that SETSUM helps them interpret SET results more efficiently, and 6 out of 10 instructors prefer the SETSUM website over the usual PDF version report (while even the remaining 4 would like to have both). As SETSUM 1.0 version demonstrates that the work holds the potential of reforming the SET report convention, the updated version is under development and it has the potential to be deployed for real usage in the future.