Entropy estimators and serial tests for ergodic chains
Stefan Wegenkittl · IEEE Transactions on Information Theory · 2001
Maurer (1992, A universal statistical test for random bit generators) discussed a statistic whose value is closely related to the per-bit-entropy of an ergodic stationary source. Here we derive an entropy estimate from a class of generalized serial tests and discuss its relationship to return-time-based entropy estimators and frequency-based goodness-of-fit tests. Our setup extends Kullback's I-divergence approach for independent stationary sequences to the class of ergodic Markov chains. The effects caused by the order of the source are examined theoretically and by an empirical study.