Prefixes and the entropy rate for long-range sources
Ioannis Kontoyiannis, Yuri Suhov · 2002
The asymptotic a.s.-relation H=lim/sub n/spl rarr//spl infin//[(nlogn)/(/spl Sigma//sub i=1//sup n/L/sub i//sup n/(X))] is derived for any finite-valued stationary ergodic process X=(X/sub n/, n/spl isin/Z) that satisfies a Doeblin-type condition: there exists r/spl ges/1 such that ess/sub x/inf P(X/sub n+1/|x/sub /spl rarr//spl infin/,n/)/spl ges//spl alpha/>0. Here, H is the entropy rate of the process X, and L/sub i//sup n/(X) is the length of a shortest prefix in X which is initiated at time i and is not repeated among the prefixes initiated at times j, 1/spl les/i/spl ne/J/spl les/n. The validity of this limiting result was established by Shields in 1989 for i.i.d. processes and also for irreducible aperiodic Markov chains. Under our new condition, we prove that this holds for a wider class of processes, that may have infinite memory.>