OECD Education Spotlights
OECD education spotlights · 2022
Advancements in artificial intelligence (AI) are laying the groundwork for extensive and rapid transformations in society.Understanding the relationship between AI capabilities and human skills is essential to ensure policy responsiveness to ongoing and incoming changes.The OECD has tracked how well AI systems fare on tasks from the Programme for International Student Assessment (PISA), comparing AI performance to that of 15-year-old students in the test's core domains of reading, mathematics and science.Tests were conducted using the Generative Pre-Trained Transformer (GPT) family of large language models (LLMs), the AI behind ChatGPT, which took the world by storm after its public release in late 2022.Results show that both GPT versions outperform average student performance in reading and science.In addition, we observe rapid advances in mathematics where AI capabilities are quickly catching up with those of students.In November 2022, GPT-3.5 could answer 35% of a set of PISA mathematics tasks, a level of performance significantly below that of humans, who answer 51% of the tasks successfully on average.However, by March 2023, GPT-4 answered 40% of the tasks successfully.Policy implications of these results are discussed below. AI will revolutionise the world! … But how?The rapid development of AI technology and its potential impact on the economy and society have been an increasingly hot topic in recent years, both in research and policy spheres as well as in the media.Some observers have suggested that humans can and will continue to outperform AI in competences such as critical reasoning, creativity or social perceptiveness for the foreseeable future (Gil and Selman, 2019[1]).However, it is hard to maintain that argument confidently considering the progress made by AI applications over the past year.AI has advanced on tests designed to measure aspects of critical thinking and reasoning (OpenAI, 2023[2]; Bubeck et al., 2023[3]); the creation of ChatGPT and its visual counterpart DALL-E 2 show astounding results with respect to creativity (Roose, 2022[4]); and the empathy exhibited in chatbots through exchanges with people struggling with mental health or social issues suggests substantial progress in social perceptiveness as well (Martinengo, Lum and Car, 2022[5]).We have to be cautious about saying that AI "understands language," "reasons critically" or "is socially perceptive"phrases that suggest human levels of understanding.However, it is clear that AI can perform an increasing number of tasks where we use such phrases when humans carry them out.Putting AI to the test: How does the performance of GPT and 15-year-old students in PISA compare?