Cyber Threat Analysis Using Pearson and Spearman Correlation Via Exploratory Data Analysis
Samara Pires, Cynthia Mascarenhas · 2023
Cyber threats have been increasing in frequency and severity in recent years, and this trend shows no signs of slowing down. With the growing dependency on technology and the internet, steadily more individuals, organizations, and governments are becoming vulnerable to cyber-attacks. The proliferation of internet-connected devices enlarges the attack vector for cybercriminals, affording them a multitude of gateways to manipulate. This results in high volumes of cyber threat-related data being generated on a daily basis. Data analysis is the most promising way to understand the possible patterns in the cyber threats faced by organizations. This paper investigates the use of Pearson and Spearman correlation in cyber threat analysis by performing exploratory data analysis on the NUSW NB 15 dataset. Relationships between various cyber threats and their patterns with respect to duration, IP address, target systems/ports, etc. have also been studied. This paper performs hypothesis testing using the T-Test and chi-square test to validate the presence of noticeable patterns observed in the dataset during analysis. Ultimately, this study contributes to the field of cybersecurity by providing valuable insights on the use of correlation techniques for cyber threat analysis.