Code Defect Detection Method Based on BERT and Ensemble
Ye JianJie, Wei Le · 2023
A code defect detection method based on Bidirectional Encoder Representations from Transformers (BERT) and ensemble learning was proposed to solve the low accuracy and reliability of the existing neural networks in the task of code defect detection due to insufficient feature extraction and weak generalization ability. This method involves extracting the Abstract Syntax Tree (AST) and the Token depth feature information vector of the source code using the BERT model. Then, the different feature information expressions of text vectors were learned by establishing the heterogeneous multi-base classifiers of such neural network models as Graph Convolutional Networks (GCN), Convolutional Neural Networks (CNN), and Bidirectional Long Short-term Memory (BiLISM). Subsequently, the different feature information of text vectors was fused via Stacking ensemble learning for training and prediction. This fusion aimed to bolster the model feature extraction and generalization abilities. As demonstrated by the experimental findings, the proposed method exhibits an enhanced accuracy, showcasing improvements of 7.08% and 11.76% when juxtaposed against VulPecker and SySeVr, respectively.