Data Stream Classification for Anomaly Detection Using Ensemble of Classifiers
Bhakti Ghuse, Snehlata Dongre · 2023
A tough challenge in different fields, such as anomaly detection, fraud, and intrusion detection, is finding novel classes in data streaming. Classification techniques that are traditionally used have trouble detecting instances that correspond to undiscovered or unique classes that are absent from the dataset of training. This paper provides an ensemble model for enhancing the identification of anomalies in streaming data. Our method makes use of a group of classifiers, to make use of the combined wisdom and decision-making methods of various models. A portion of the data is used to train each classifier in the ensemble, which enhances generalization and accuracy. To enhance the detection of anomalies, we incorporate this ensemble model, which is trained on normal instances to capture the underlying distribution of known classes. It then detects instances that deviate significantly from the learned normal behavior, indicating the presence of potential novel classes. Concept drift detection is integrated into our methodology to monitor changes in the data distribution over time. Evaluation and experimentation on a real-world dataset, the KDD dataset, demonstrate the effectiveness of our ensemble-based approach for anomaly detection. The results show significant improvements in accuracy, precision, and recall compared to individual classifiers. Furthermore, the ensemble demonstrates robustness in handling concept drift and effectively identifies novel class instances in a streaming data environment.