From Manual to Automatic: The Evolution of Test Case Generation Methods and the Role of GitHub Copilot
Sajid Mehmood, Uzair Iqbal Janjua, Adeel Ahmed · 2023
The emergence of AI tools, such as GitHub Copilot, introduces innovative approaches to software development tasks, particularly in the realm of test case generation. This research endeavors to evaluate the effectiveness of test cases generated by GitHub Copilot in comparison to manually crafted ones. Through the examination of four programming exercises, This study conducted a comparative analysis based on criteria including test passing rates, uniqueness, and equivalence. Preliminary findings suggest that the quality of test cases produced by Copilot is on par with manually created ones, indicating its potential to diversify the range of test cases when provided with precise prompts. While Copilot has the potential to enhance testing processes, it's important to consider certain limitations, such as constraints related to the range of test cases and potential prompt biases. Further research endeavors should explore a wider range of software development tasks and investigate how developer expertise influences the outcomes of test cases generated by AI tools.