Encrypt DNS Traffic: Automated Feature Learning Method for Detecting DNS Tunnels
Shuai Ding, Daoqing Zhang, Jingguo Ge, Xiaowei Yuan, Xinhui Du · 2021
In recent years, attacks on the DNS continue to proliferate due to the lack of security mechanisms. DNS over HTTPS (DoH) is a standard developed for encrypting plaintext DNS to protect user privacy, but attackers may use this protocol to bypass enterprise firewalls and exfiltrate private data through the establishment of DNS tunnels. Traditional DNS tunnel detection methods based on packet inspection are no longer applicable because of DNS encryption. Although popular machine learning methods are widely used, it require expert knowledge to extract statistical feature sets which are complicated. In order to solve the problem of detecting encrypted DNS tunnels, we propose an end-to-end anomaly detection model based on a variational autoencoder which incorporates the attention mechanism. By modeling the raw flow sequence data at the flow-level, we use bidirectional GRU-based network to automatically learn the feature representations and detect anomalies via reconstruction error. We conducted experiments on the public dataset, which contains raw normal DoH traffic and abnormal DNS tunnel traffic. The results show that our model achieves excellent performance, and has the ability to identify unknown DNS tunnels.