Malware-BERT: Enhancing Evasive Malware Detection with Multi-Head BERT Attention with Hybrid BERT models
Neha Nandinee, Deepak Singh Tomar, Yogesh Kumar Sharma, Vasudev Dehalwar · 2025
The increasing prevalence of malicious software poses a significant threat to computer systems and the Internet, with some malware employing sophisticated techniques to evade detection. Traditional detection methods, such as signature-based and heuristic-based approaches, often fail to identify unknown malware, underscoring the need for advanced solutions. This research introduces the Malware-BERT model for detecting evasive malware in PDF documents. Leveraging a novel dataset, Evasive-PDFMal2022, which comprises 10,025 samples (5,557 malicious and 4,468 benign), the model utilizes BERT embeddings and multi-head attention mechanisms. A comparative analysis of hybrid BERT models reveals significant improvements in detection performance, with the RoBERTa model achieving an accuracy of 96%. This study demonstrates the potential of advanced machine learning techniques in fortifying cybersecurity measures against sophisticated PDF-based threats.