Automated Program Repair with the GPT Family, including GPT-2, GPT-3 and CodeX
Márk Lajkó, Viktor Csuvik, Tibor Gyimóthy, László Vidács · 2024
Automated Program Repair (APR) is a promising approach for addressing software defects and improving software reliability. There are various approaches to APR, including using Machine Learning (ML) techniques such as neural networks and evolutionary algorithms, as well as more traditional methods such as static analysis and symbolic execution. In recent years, there has been growing interest in using ML techniques for APR, including the use of large language models such as GPT-2 and GPT-3. These models have the ability to generate human-like text and code, making them well-suited for tasks such as generating repair patches for defective programs. In this paper, we explore the use of the GPT family (including GPT-2, GPT-J-6B, GPT-3 and Codex) for APR of JavaScript programs and evaluate their performance in terms of the number and quality of repair patches generated. Our results show that these state-of-the-art language models are able to generate repair patches that successfully fix the defects in the JavaScript programs, with Codex performing slightly better overall. To be precise, in our self-assembled dataset, Codex was able to generate 108 repair patches that are exactly the same as the developer fix for the first try. If we consider multiple patch generations, up to 201 buggy programs are being repaired automatically from the 1559 evaluation dataset (12.89%).