Markov Process
Ken Chen · 2015
This chapter studies the Markov process, which is an important class among the stochastic systems. The principal characteristic of the Markov processes is the conditional independence between the future and the past if the present is known. Modern computer systems are complex systems which evolve dynamically in time. The stochastic processes and the associated mathematical tools offer a means for the description and study of these kinds of dynamic systems. The chapter considers only Markov chains, with both discrete time and continuous time. The Markov chains constitute a very popular modeling tool. The chapter considers only the irreducible, aperiodic and positive recurrent Markov chains, i.e. those Markov chains that have a stationary distribution.