Finite Storage Discriminators for Ergodic Processes
Aaron D. Wyner, J. Ziv · 2005
We are looking for an "essential statistic" of a finite-alphabet ergodic source, that, under a given storage (memory) constraint, will allow discrimination between the given source and any other finite-alphabet source. In our model an encoder is given the n-th order statistics of a stationary process, and the encoder output is a binary N-vector. A discriminator, observes the n-th order statistics of a second source that is either identical to the first source or differs from it by a specified Kullback- Leibler divergence. In a sense made precise in the paper, we show that when n is large, this can be done if and only if N > exp(nH), where H is the entropy of the first source.