Discrete Time Markov Chains and Extensions
David Rı́os Insua, Fabrizio Ruggeri, Michael Peter Wiper · Wiley series in probability and statistics · 2012
This chapter presents the Bayesian analysis of discrete time Markov chains, focusing on homogeneous chains with a finite state space. It analyzes many important subclasses and extensions of this basic model such as reversible chains, branching processes, higher order Markov chains, and discrete time Markov processes with continuous state spaces. The chapter outlines the properties of the basic Markov chain model and the variants from a probabilistic viewpoint. It considers the inference for time homogeneous, discrete state space and first-order chains. The chapter provides inference for various extensions and particular classes of chains, and presents a case study on the analysis of wind directions and the Markov decision processes. Controlled Vocabulary Terms Bayes estimator; Bayesian inference; Markov chain