Classifying Attack Tactics in Zeek Conn Logs Using the UWF-ZeekDataFall22 Dataset and Spark ML within a Big Data Environment
C Rajeswari, Ishan Desai, C. P. Sanjay, Ashok Chanabasangouda Patil · 2025
The rapid proliferation of cybersecurity threats has heightened the necessity for real-time detection and classification of malicious activities within network environments. Zeek conn logs, widely utilized for network traffic monitoring, contain extensive data, which presents both opportunities and challenges for attack detection. This paper explores an innovative approach for classifying attack tactics using the UWF-ZeekDataFall22 dataset, processed within a big data environment leveraging Apache Spark ML. The proposed methodology includes comprehensive data preprocessing to clean and transform raw Zeek logs, followed by feature extraction to enhance the interpretability of network behavior. Advanced machine learning algorithms such as Decision Trees, Random Forest, and Gradient-Boosted Trees are trained on this dataset, achieving robust classification performance. Experimental evaluation highlights the scalability of the system, effectively handling large datasets without compromising accuracy. The Gradient-Boosted Trees model emerged as the most effective, achieving superior results across key performance metrics including precision, recall, and F1-score. This study demonstrates the potential of integrating big data technologies with machine learning to address complex cybersecurity challenges. The findings underscore the significance of a distributed computing framework for enabling real-time, large-scale network security analysis, paving the way for future innovations in this domain.