8. Applications of the Drazin Inverse to the Theory of Finite Markov Chains
Society for Industrial and Applied Mathematics eBooks · 2009
1. Introduction and terminology Let be an indexed set of random variables. If P is a probability measure such that P ( Xt ≤b| X ti = xi ,for i=1,… ,n)=P ( Xt ≤b| X tn = xn ) whenever , then the set is called a Markov process. In other words, a Markov process is such that when the present state of the process is known, the probability of any future behaviour of the process is not changed by knowledge of its past behaviour. If the index set F is countable and if the range of each is the same finite set (the elements of are referred to as the states of the process), then the process is called a finite Markov chain. It is convenient to think of as being the outcome of the process on the kth step or trial. The probability of being in state given that was in state is . These numbers are called the one-step transition probabilities. If each of the one-step transition probabilities is independent of time, i.e. for , then the chain is said to be homogeneous. In this chapter, we will confine our attention to finite homogeneous Markov chains and will use ‘Markov chain’ or ‘chain’ to denote a finite homogeneous Markov chain. For an m-state chain, the matrix T whose (i, j)th entry is the one-step transition probability is called the one-step transition matrix, or simply the transition matrix of the chain. An ergodic set (or class) is a set of states in which every state of the set is accessible from every other state of the set and no state outside the set is accessible from any state in the set. A transient set is a set of states in which every state of the set is accessible from every other state of the set, but some state outside the state is accessible from each state in the set. An ergodic state is a member of an ergodic set and a transient state is a member of a transient set.