Zero-Day Malware Detection Using Autoencoder and Hybrid Deep Learning Model
Chintha Reddy Sai Kiran, S. Hariharasitaraman, Sajjad Ahmed, Ajay Kumar Phulre · 2025
Malware continues to pose a serious threat to cybersecurity, especially with the rise of unknown or zero day attacks that bypass the traditional antivirus tools. This study proposes a hybrid detection framework that combines machine learning and deep learning for more effective threat identification. Known malware is classified using an RNN-LSTM model, while an Autoencoder detects unfamiliar threats by learning typical benign behavior and flagging anomalies. To improve accuracy, XGBoost is used to select the most relevant features for analysis. Experimental results show that the model achieves 99% accuracy in detecting known malware and reliably identifies previously unseen attacks. This approach demonstrates strong potential for enhancing proactive and scalable cybersecurity defenses.