Challenges to the Use of Artificial Neural Networks for Diagnostic Classifications with Student Test Data
Derek C. Briggs, Ruhan Circi · International Journal of Testing · 2017
Artificial Neural Networks (ANNs) have been proposed as a promising approach for the classification of students into different levels of a psychological attribute hierarchy. Unfortunately, because such classifications typically rely upon internally produced item response patterns that have not been externally validated, the instability of ANN estimates of attribute probabilities may not be widely appreciated. The present study illustrates the problem with both empirical and simulated data. In particular, it is shown that when an ANN is “trained” multiple times using the same data, attribute probability estimates can vary, sometimes dramatically. Researchers hoping to apply ANNs in the context of diagnostic classification models with student test data need to be very deliberate in checking the sensitivity of their findings.