An Encrypted Malicious Traffic Detection System Based on Neural Network
Tangda Yu, Futai Zou, Linsen Li, Ping Yi · 2019
In recent years, with the widespread use of encrypted traffic communication technology, network traffic encryption has been gradually becoming a standard of communication. This phenomenon has a great impact on traditional traffic detection methods, especially on anomaly detection methods, which are highly dependent on the type of network protocol types and traffic data. By surveying the existing encrypted traffic classification and analyzing these methods, we found that there are two main methods for detection: payload-based detection and feature-based detection. On this basis, this paper puts forward an Encrypted Malicious Traffic Detection System which is based on multi-AEs(Autoencoder). We use malicious sandbox for traffic data collection, mark malicious flow and normal flow with labels. After that, we use multilayer networks of AEs for feature extraction and training classifier model. Our system analyzes the feature of cryptographic protocol from handshake phase to Authentication phase on the basis of existing research, and we extract the traffic feature for a better classification by further expanding flow feature vector to the higher dimensions. In addition, we compared the performance of our model with the traditional learning model under the same environment. The experimental results showed that our system had higher detection accuracy and lower loss rate. We also studied the influence of dataset imbalance on results in the detection of encrypted traffic by several experiments. According to the experimental evaluation, dataset imbalance will lead to the decrease of positive rate.