Asymptotic Behavior of Discrete‐Time M arkov Chains
Eylem Tekin · Wiley Encyclopedia of Operations Research and Management Science · 2011
Abstract This article summarizes the theoretical foundations for analyzing the asymptotic behavior of discrete‐time Markov chains (DTMC). Consider a system that evolves randomly over time. Let X n denote the state of the system at time n = 0, 1, … . If, for all n ≥ 0, the future probability distribution of the system depends only on the current state and is independent of the past, the stochastic process { X n , n ≥ 0} is called a DTMC . In this article, we study the distribution of X n as n → ∞. We first introduce several concepts for classifying the states of a DTMC. These concepts include irreducibility, periodicity, recurrence, and transience. Using these concepts, we next describe the necessary and sufficient conditions for the existence and uniqueness of a limiting distribution, and explain how to compute a limiting distribution, if it exists.