Detection of anomaly over streams using big data technologies
Chellammal Surianarayanan, Saranya Kunasekaran · Institution of Engineering and Technology eBooks · 2022
Anomaly detection serves as a method for identifying and recognizing abnormal events that may occur over data in various application domains. Anomaly detection is very useful as it provides valuable and actionable information such as detection of fraud in financial domain, detection of intrusion in networking etc. Detecting anomaly over streaming data requires efficient tools and techniques as streaming data is continuously flowing one with no start or end. As streaming data is associated with speed, big data-based platforms provide the fundamental base over which machine learning algorithms can be employed so that the detection of anomaly over streaming data can be performed efficiently. This chapter describes Apache Kafka-based architecture in which detection of anomaly over streams is being done using machine learning algorithm.