DocChecker: Bootstrapping Code Large Language Model for Detecting and Resolving Code-Comment Inconsistencies
Anh T. V. Dau, Jin Guo, Nghi Bui · 2024
Comments in source code are crucial for developers to understand the purpose of the code and to use it correctly.However, keeping comments aligned with the evolving codebase poses a significant challenge.With increasing interest in automated solutions to identify and rectify discrepancies between code and its associated comments, most existing methods rely heavily on heuristic rules.This paper introduces DocChecker, a language model-based framework adept at detecting inconsistencies between code and comments and capable of generating synthetic comments.This functionality allows DocChecker to identify and rectify cases where comments do not accurately represent the code they describe.The efficacy of DocChecker is demonstrated using the Just-In-Time and CodeXGlue datasets in various scenarios.Notably, DocChecker sets a new benchmark in the Inconsistency Code-Comment Detection (ICCD) task, achieving 72.3% accuracy, and scoring 33.64 in BLEU-4 on the code summarization task.These results surpass other Large Language Models (LLMs), including GPT 3.5 and CodeLlama.DocChecker is available for use and evaluation.It is available on https://github.com/ FSoft-AI4Code/DocCheckerGitHub and as an Online Tool.A demonstration video of its functionality can be found on YouTube.