Beyond Human Boundaries: Exploring the Proficiency of AI Technology and its Potential in Psychometric Test Construction
Wolfgang Lenhard, Alexandra Lenhard · 2023
The substantial advancements in artificial intelligence, notably the Generative Pretrained Transformers exemplified by GPT4, have instigated comprehensive discussions on the potential and challenges of AI. Following on from research on automatic evaluation of open-ended responses using latent semantic analysis (LSA) undertaken in our team two decades ago, we take an updated look at the possibilities today. In this brief report, we re-evaluate our findings in assessing the linguistic capabilities of computational linguistics and review the performance of current approaches using language subtests of intelligence tests of the Wechsler series and tests of reading comprehension. The results show that current language models perform at the upper end of the spectrum of human abilities and that they therefore offer tremendous opportunities in test development, as well as in the automatic evaluation of open-ended responses.