CodeBERT-SENet: Adaptive Syntax-Semantic Fusion via Gated Attention for Python Bug Detection and Localization
Rozali Ilham, Bahtiar Imran, Hasan Basri, Erfan Wahyudi · DOAJ (DOAJ: Directory of Open Access Journals) · 2026
Syntax error detection is a critical step in software development to ensure code quality and reliability. The research proposes the first end-to-end multitask architecture that jointly detects Python syntax errors and localizes their exact line position by adaptively fusing CodeBERT’s semantic embeddings with handcrafted syntactic features via a lightweight gating mechanism, without relying on Abstract Syntax Trees (AST). The model is trained and evaluated on a structured Python code dataset (testing set: 342 files, 30% of total data) using consistent data splits and standardized evaluation metrics. The results demonstrate that CodeBERT-SENet achieves state-of-the-art performance with 99.71% accuracy, an F1-score of 0.9969, and a Mean Absolute Error (MAE) of 0.1157 in line-level error prediction, outperforming all baselines, including Vanilla CodeBERT, GraphCodeBERT, RoBERTa, Random Forest, and the rule-based approach. The confusion matrix confirms zero false negatives (all 160 buggy files detected) and only one false positive (181 non-bugged files, 180 correctly identified). Training converges stably within five epochs without signs of overfitting, underscoring the effectiveness of the architectural design and optimization strategy. However, limitations remain, including high inference latency (six seconds per file), lack of deep structural code representation (e.g., AST), regression-based line prediction instead of classification, and unverified generalization on real-world production code. Nevertheless, CodeBERTSENet conclusively demonstrates that adaptive fusion of semantic and syntactic features significantly enhances code diagnostic capability. Future research will focus on optimizing inference speed, integrating AST-based representations, and transitioning to per-line classification, transforming CodeBERT-SENet from a mere bug detector into a next-generation, context-aware code diagnostic assistant.