RGD: Multi-LLM Based Agent Debugger via Refinement and Generation Guidance
Haolin Jin, Zechao Sun, Huaming Chen · 2024
Large Language Models (LLMs) have shown incredible potential in code generation tasks, and recent research in prompt engineering have enhanced LLMs’ understanding of textual information. However, ensuring the accuracy of generated code often requires extensive testing and validation by programmers. While LLMs can typically generate code based on task descriptions, their accuracy remains limited, especially for complex tasks that require a deeper understanding of both the problem statement and the code generation process. This limitation is primarily due to the LLMs’ need to simultaneously comprehend text and generate syntactically and semantically correct code, without having the capability to automatically refine the code. In real-world software development, programmers seldom produce flawless code on their first attempt. Instead, iterative feedback and debugging are heavily leveraged to refine the programs. Inspired by this process, we introduce a novel architecture of LLM-based agents for code generation and automatic debugging: Refinement and Guidance Debugging (RGD). The RGD framework is a multi-LLM-based agent debugger that leverages three distinct LLM agents-Guide Agent, Debug Agent, and Feedback Agent. RGD decomposes the code generation task into multiple steps, ensuring a clearer workflow and enabling iterative code refinement based on self-reflection and feedback. Experimental results demonstrate that RGD exhibits remarkable code generation capabilities, achieving state-of-the-art performance with a $9.8 \%$ improvement on the HumanEval dataset and a $\mathbf{1 6. 2 \%}$ improvement on the MBPP dataset compared to the state-of-theart approaches and traditional direct prompting approaches. We highlight the effectiveness of the RGD framework in enhancing LLMs’ ability to generate and refine code autonomously.