Streaming Sensor Data Validation in Networked Infrastructure Systems through Synergic Auto and Cross Similarity Discovery and Analysis
Lu Xing, Amin Rasekh, M. Ehsan Shafiee, Lina Sela, Ami Preis · 2019
Streaming data can provide a timely understanding of the state of infrastructure networks to enable real-time monitoring and control. However, erroneous data is also inevitable and, if not identified and isolated effectively, may results in erroneous decisions and adverse consequences. This study leverages intra- and inter-similarity structures from monitoring stations for data validation in infrastructure networks. First, validation rules are developed to estimate similarity of new data from an individual sensing station to its routine patterns and flag suspicious data streams. Second, an unsupervised learning model is applied to identify clusters of stations that exhibit similar streaming data and test for cross-similarity to estimate the likelihood of the flagged data stream being invalid. The proposed model is demonstrated using steaming data collected from a real water distribution system. Preliminary results have revealed how a sensor’s readings diverge from its historical patterns when a sensor fault occurs, and have discovered the presence of clusters of sensor stations with significant cross-similarity for boosting data validation performance.