Temporal Neighborhood Discovery Using Markov Models
Sandipan Dey, Vandana Pursnani Janeja, Aryya Gangopadhyay · 2009
Temporal data, which is a sequence of data tuples measured at successive time instances, is typically very large. Hence instead of mining the entire data, we are interested in dividing the huge data into several smaller intervals of interest which we call temporal neighborhoods. In this paper we propose an approach to generate temporal neighborhoods through unequal depth discretization. We describe two novel algorithms (a) similarity based merging (SMerg) and, (b) stationary distribution based merging (StMerg). These algorithms are based on the robust framework of Markov models and the Markov stationary distribution respectively. We identify temporal neighborhoods with distinct demarcations based on unequal depth discretization of the data. We discuss detailed experimental results in both synthetic and real world data. Specifically we show (i) the efficacy of our approach through precision and recall of labeled bins, (ii) the ground truth validation in real world datasets and, (iii) knowledge discovery in the temporal neighborhoods such as global anomalies. Our results indicate that we are able to identify valuable knowledge based on our ground truth validation from real world traffic data.