A Study on C Code Defect Detection with Fine-Tuned Large Language Models
Yue Wang, Xu Wang, Hongwei Yu, Fei Gao, Xueshi Liu, Xiaoling Wang · 2024
Large Language Models(LLMs) have demonstrated excellent capabilities in many areas of software engineering(SE), including code completion, code generation, code understanding, code repair, etc., and the most prominent performer in this regard is ChatGPT. However, its cost of use makes the integration of ChatGPT into code defect detection techniques costly. In this paper, we focus on low-cost-of-use, fine-tunable, open-source large language models with less than 10B parameters, and study their capabilities of C code defect detection when fine-tuned with real-world data and improved with prompt engineering. We studied LLaMa3-8B, DeepSeek-Coder-7b and Qwen2-7B, as they are the typical models with prompt capabilities, whose performance in SE is close to ChatGPT, and they are open-source models. Experimental results show that our method can significantly improve the performance of LLMs within 10B parameters on code defect detection, and the output of the models can be applied to several downstream tasks, such as improving the report quality of static analysis tools.