SDADS: Stream Data Anomaly Detection System
Yusheng Wang, Anyi Zhang · 2023
In a cloud-native architecture, the operational data of various system components experiences a significant increase. From the distributed complex system, obtaining the operation status data and realizing real-time monitoring and abnormal alarm play an important role in guaranteeing the smooth production. However, handling a large volume of stream data in real-time poses challenges such as high computational demands, low latency, and high concurrency. Therefore, this study presents the design and implementation of a system for anomaly detection in stream data called Stream Data Anomaly Detection System (SDADS). SDADS leverages message queues for processing business data and further processes the data using a distributed computing framework. It also provides functions for persistent data storage and anomaly detection alerts. The use of SDADS enables the management of complex business processes such as server provisioning and application deployment, simplifies business development logic, reduces operational time cost, and allows users to focus solely on the anomaly detection algorithms themselves.