An Efficient Algorithm for Quantile Computation over Streaming Data
Bei Yang, Houkuan Huang, Zhihai Wang · 2007
Data stream is a new data model that has recently attracted attentions in numerous applications. Considering the continuity, limitlessness of streaming data, this paper proposes a novel method to build a synopsis data structure, Nord_Histogram, for storing streaming data summary and a one-pass approximate algorithm, NHQC, for quantile computation. The algorithm implements quantile queries over data stream with the time and space requirements being linear with the number of buckets, beating several previous synopsis structure and algorithms in terms of time and space costs which grow at least logarithmically with the length of data stream. The correlation between computation error and main memory requirement is also analyzed. Experiment results show the good performance of NHQC with desirable accuracy and efficient time and space requirements.