A Multivariate Generalizability Analysis Based on the Table of Specifications Model
Kyong Hee Chon · 동중앙아시아연구(구 한몽경상연구) · 2011
The purpose of this study was to examine score dependability of a test from the perspective of multivariate generalizability theory based on the table of specificationsmodel. Using data sets collected over three consecutive years (2000 to 2002), multivariate generalizability analyses were conducted for raw scores for an English proficiency test resulted from two different scoring schemes. The results suggested that the test maintained an acceptable level of reliability ranging from .93 to .95 over the years 2000 to 2002 and it did not matter much whether equal or different weights were used for each item in terms of reliability and measurement precision. Regarding the total number of items, relatively small reductions in absolute standard error estimates were observed by increasing the total number of items. Finally, for the optimality issue, the numbers of items minimizing absolute error were not much different from the numbers used in the actual tests in 2000 and 2002.