IAP-Based Self-Learning Real-Time Application Layer DDoS Detection Method on Storm Platform
Bin Zhang, Zihao Liu, Shuqin Dong · 2019
With increasing amount of information handled in application layer DDoS (App-DDoS) detection, the cluster-based App-DDoS detection methods not only face the tedious training process and complex training results updating, but also face the new problem of inability to detect in real time. To solve these problems, we propose an IAP-based self-learning real-time application layer DDoS detection method on Storm platform. The IAP algorithm embeds the operations of pre-grouping, dimension reduction and merger into affinity propagation (AP) algorithm to cluster large training data sets efficiently and accurately, and we further promote the training efficiency by means of Storm platform. Then we design the real time feature extraction method based on the slot-based statistical mechanism. In detection phase, the k-d tree algorithm is selected to reduce the detection delay and voting principle is chosen to determine the result for the same user in different slots. Next, we design the self-learning mechanism according to silhouette index, to prevent hackers from launching the attacks by analyzing the structures of clusters. Finally, we experiment with real dataset and heavy traffic generator Spirent TestCenter C100. The simulation results show that our method can improve the real-time performance and shorten the training time with the guarantee of detection performance compared with AP and KMPCA. The method can also reduce the false positive rate and improve the detection rate by constantly self-learning.