Relative Entropy Rate between a Markov Chain and Its Corresponding Hidden Markov Chain
Hossein Yari, Zohre Nikooravesh · Journal of Statistical Research of Iran · 2011
In this paper we study the relative entropy rate between a homogeneous Markov chain and a hidden Markov chain defined by observing the output of a discrete stochastic channel whose input is the finite state space homogeneous stationary Markov chain. For this purpose, we obtain the relative entropy between two finite subsequences of above mentioned chains with the help of the definition of relative entropy between two random variables then we define the relative entropy rate between these stochastic processes and study the convergence of it.