8. Finite Markov Chains
Society for Industrial and Applied Mathematics eBooks · 1994
1 INTRODUCTIONIn the present chapter we consider n-state homogeneous Markov chains. The study of such chains provides some of the most beautiful and elegant applications of the theory of nonnegative matrices.A Markov chain is a special type of stochastic process and a stochastic process is concerned with events that change in a random way with time. A typical stochastic process might predict the motion of an object that is constrained to be at any one time in exactly one of a number of possible states. It would then be a scheme for determining the probability of the object's being in a specific state at a specific time. The probability would generally depend upon a large number of factors, e.g., (1) the state, (2) the time, (3) some or all of the previous states the object has been in, and (4) the states other objects are in or have been in. For example, the object might be the reader, and the states might be the countries of the world. Clearly, aspects of all four factors just mentioned might influence the probability that the reader will be in a specific country at a specific time.