Identification of a transition matrix of a Markov chain from noisy measurements of state
R.L. Kashyap · IEEE Transactions on Information Theory · 1970
We consider the problem of recursively estimating the transition matrix of a regular Markov chain with a finite number of states using only noisy measurements of the state. The measurement noise sequence is assumed to be independent with known mean and unknown variance. We also discuss the convergence rates and computational aspects of the algorithms and the methods of accelerating them.