Outliers Detection in One Dimensional Meteorological Data Stream
Joel Paulin Tanzouak Vaumi, Blaise Omer Yenké, Ndiouma Bame, Idrissa Sarr · 2018
Sensors networks are some of technologies mostly used to gather informations from the environment. Indeed they collect a lot of data and send them to based stations for treatment. For meteorological monitoring, used sensors usually provide one dimensional data like temperatures, precipitations, humidity, etc. These data are speedily generated in the way that they form what we frequently call stream data. Among these data, there are usually bad values called outliers that need to be removed from the data stream. Many algorithms used to detect these outliers are usually designed for data with a static distribution and they do not consider the dynamic aspect of this distribution. However, one of the main characteristic of meteorological data is the dynamic behavior of the data distribution. Moreover, the speed of the data stream imposes to the outliers detection algorithms to be very fast in the data processing, otherwise a significant number of data could be lost. Regarding most of algorithms studied in the literature, it could be argued that these algorithms are not suitable for outliers detection in meteorological data stream. One of the reason is because of their time complexity and also their weak ability to easily detect contextual outliers. This work proposes a new outliers detection algorithm for one dimensional numeric data stream based on two filters that offer a complexity of O(n) and detect contextual outliers well, with a good precision.