Data-driven Approach to Information Sharing using Data Fusion and Machine Learning for Intrusion Detection
Lars Christian Andersen, Katrin Franke, Andrii Shalaginov · 2016
Intrusion Detection System (IDS) sensors are employed in various locations in computer and communication networks to identify possible malicious activities. One of the main challenges with IDS is the high false positives rate, which creates a high unnecessary workload for human analysts at Security Operation Centre (SOC). Similarly, the exponential growth of captured sensor raw data combined with the application of Threat Intelligence (TI) creates a complex data flow. Considering mentioned challenges, this paper presents a model of heterogeneous sensor and TI data fusion and reduction in intrusion detection. We summarize found literature and qualitative research interviews with security experts from law enforcement and public and private organizations. Building on our qualitative research we identied feature subsets for corresponding data fusion that produce accurate classication model in Machine Learning (ML)-aided analysis. Proposed data fusion process model was successfully evaluated on a real-world dataset from a SOC. This work contributes to development of data-driven approach for automated classication of IDS events using reduction of raw log data.