From Finite-System Entropy to Entropy Rate for a Hidden Markov Process
Or Zuk, Eytan Domany, Ido Kanter, Michael Aizenman · IEEE Signal Processing Letters · 2006
A recent result presented the expansion for the entropy rate of a hidden Markov process (HMP) as a power series in the noise variable epsi. The coefficients of the expansion around the noiseless (epsi=0) limit were calculated up to 11th order, using a conjecture that relates the entropy rate of an HMP to the entropy of a process of finite length (which is calculated analytically). In this letter, we generalize and prove the conjecture and discuss its theoretical and practical consequences