Using Large Language Models to Automatically Identify Programming Concepts in Code Snippets
Andrew Tran, Linxuan Li, Egi Rama, Kenneth Angelikas, Stephen MacNeil · 2023
Curating course material that aligns with students’ learning goals is a challenging and time-consuming task that instructors undergo when preparing their curricula. For instance, it is a challenge to find multiple-choice questions or example codes that demonstrate recursion in an unlabeled question bank or repository. Recently, Large Language Models (LLMs) have demonstrated the capability to generate high-quality learning materials at scale. In this poster, we use LLMs to identify programming concepts found within code snippets, allowing instructors to quickly curate their course materials. We compare programming concepts generated by LLMs with concepts generated by experts to see the extent to which they agree. The agreement was calculated using Cohen’s Kappa.