An Evaluation of Deep Learning Approaches for Factor Analysis of Response and Response Time Data
Rudolf Debelak, Dylan Molenaar · 2025
An important aspect of psychometric model application is the choice of a suitable algorithm for parameter estimation. Particularly in the case of large datasets and high dimensional models, conventional algorithms may be inapplicable or relatively slow. Therefore, in a recent publication, Urban and Bauer (Psychometrika 86:1-29, 2021) proposed an estimation algorithm from the field of deep learning which is suitable for binary or ordered categorical data. In practice however, researchers may be interested in adding continuous response times to the model. As it has yet not been studied whether the resulting model for responses and response times is viable, in this article, we investigate the accuracy of this estimation method for a multidimensional variation of the joint model for responses and response times proposed by Van der Linden (Psychometrika 72: 287-308, 2007). Our evaluation focuses on comparing the effects of different sets of hyperparameters on the accuracy of the estimation method. Our results indicate that the deep learning algorithm can be used for accurate item parameter estimation of these models even in relatively small datasets. In addition it is computationally fast in large datasets, but it's accuracy depends on the number of importance-weighted samples and Monte Carlo samples. We applied the method to an empirical dataset from PISA 2018 related to mathematics and reading.