Markov Chains: First Steps
Robert P. Dobrow · 2016
This chapter provides the wide range of applications of Markov chains. A powerful feature of Markov chains is the ability to use matrix algebra for computing probabilities. To use matrix methods, the chapter considers probability distributions as vectors. The Markov property says that past and future are independent given the present. It is also true that past and future are independent, given the most recent past. Simulation is a powerful tool for studying Markov chains. For many chains that arise in applications, state spaces are huge and matrix methods may not be practical, or even possible, to implement. A Markov chain can be simulated from an initial distribution and transition matrix. To simulate a Markov sequence, simulate each random variable sequentially conditional on the outcome of the previous variable. Mathematical induction is a technique for proving theorems, or properties, which hold for the natural numbers.