Hydra-Reviewer: A Holistic Multi-Agent System for Automatic Code Review Comment Generation
Xiaoxue Ren, Chaoqun Dai, Qiao Huang, Ye Wang, Chao Liu, Bo Jiang · IEEE Transactions on Software Engineering · 2025
Review comment generation is a crucial task in code review, and significant progress has been made in automating. Previous research has generated review comments by fine-tuning pre-trained models or Large Language Models (LLMs). However, these studies have overlooked the necessity of conducting code reviews from multiple perspectives, resulting in the omission of potential issues in code changes. Additionally, the complexity of review comments often hinders the accurate quantitative evaluation of automated tools’ effectiveness.In this paper, we first conduct an empirical study to propose a comprehensive taxonomy of code review dimensions. We also identify three major limitations of existing automated code review (ACR) methods: lack of comprehensiveness, incorrectness, and vagueness. Building on the insights from our empirical study, we introduce Hydra-Reviewer, a collaborative multi-agent framework powered by large language models, designed to automatically generate high-quality code reviews. We utilize the CodeReview and CodeReviewNew benchmark datasets, along with a newly constructed review comment generation dataset. We compare Hydra-Reviewerwith several baselines, including CodeReviewer, LLaMA-Reviewer, ChatGPT, Comprehensive-ChatGPT, and DeepSeek-V3.The experimental results show that Hydra-Reviewerachieves a BLEU score of 8.20, outperforming the state-of-the-art baseline, DeepSeek-V3, which scores 7.85. In qualitative evaluation, Hydra-Reviewer’s generated comments span an average of 7.8 review dimensions, addressing the limitations of existing ACR methods effectively. Additionally, Hydra-Reviewerdemonstrates strong generalization capabilities on unseen dataset. We further validate the contributions of each component of Hydra-Reviewerthrough an ablation study and confirm the helpfulness and readability of the generated comments via a User Study. Finally, a cost analysis reveals that Hydra-Reviewergenerates review comments at an average cost of 0.018 dollars and 62.63 seconds per code change.