A Detection Method for Malware Communication Traffic via Encrypted Traffic Analysis
Linfeng Wei, Yu Wang, Xueming Li, Jian Li, Yuqin Huang, Zhiquan Liu · IEEE Internet of Things Journal · 2025
As the proportion of encrypted traffic in network communications continues to increase, encryption technologies are widely used to protect user privacy and data security. Meanwhile, this also makes it more covert for hackers to spread malware, steal sensitive information, or conduct other harmful behaviors in the network. How to effectively detect encrypted malicious traffic in communications while protecting user privacy has become a key task to be addressed in the field of network security. To address this challenge, this paper proposes a detection method for malware communication traffic via encrypted traffic analysis. It extracts contextual correlations and temporal features from raw data traffic without decrypting the encrypted data using Session-Transformer. The method uses Deep Neural Networks as the classifier to detect and classify encrypted malicious traffic. The experimental results show that our method has the best performance in accuracy, precision, recall, and F1-score on the DataCon2020 encrypted malicious traffic dataset and the CIC-AndMal-2017 dataset. In particular, the recall on the DataCon2020 dataset reaches 98.34%, and the precision on the CIC-AndMal-2017 dataset achieves 93.54%.