Anomaly Detection of Argo Data using Variational Autoencoder and K-means Clustering
Yongguo Jiang, Tingting Huang, Jiaxing Wang, Ce Kang · 2022 IEEE 5th Advanced Information Management, Communicates, Electronic and Automation Control Conference (IMCEC) · 2022
Aiming at the problem of abnormal data mining in marine Argo buoy monitoring data, due to the poor performance of traditional machine learning algorithms in anomaly detection, a two-stage anomaly detection method based on variational autoencoder and K-means clustering is proposed. The anomaly detection of Argo data is realized by combining deep learning and machine learning. The proposed method is divided into two parts, data preprocessing and anomaly detection. Specifically, standardization and normalization techniques are used to preprocess the input data. Then, the data are input into the training process of encoding and decoding in the variational autoencoder. Finally, the reconstruction error between the original data and the reconstructed data is calculated, and the upper and lower limits of the appropriate threshold are set. If the threshold exceeds the upper limit, it is directly determined as the abnormal data, and if the threshold is lower than the lower limit, it is determined as the normal data. It is called the boundary sample between the thresholds, and the extra clustering enhancement is used to determine the anomaly. The experimental verification is carried out on the simulation data set and the real data set, respectively. The experimental results show that compared with other methods, this method significantly improves the accuracy of anomaly detection.