Innovation and pattern entropy of stationary processes
Alon Orlitsky, Narayana Santhanam, Krishna Viswanathan, J. Zhang · 2005
We obtain bounds on the probability that the n'th variable in a stationary random process differs from all previous ones, and use it to show that the pattern entropy rate of any finite-entropy stationary process equals the process entropy rate. In the particular case of i.i.d. processes we also bound the speed at which the per-symbol pattern entropy converges to the sequence entropy