Continuous Time Markov Chains
Richard T Durrett · Springer texts in statistics · 2016
In Chap. 1 we considered Markov chains X n with a discrete time index n = 0, 1, 2, … In this chapter we will extend the notion to a continuous time parameter t ≥ 0, a setting that is more convenient for some applications. In discrete time we formulated the Markov property as: for any possible values of j, i, i n−1, … i 0 $$\displaystyle{P(X_{n+1} = j\vert X_{n} = i,X_{n-1} = i_{n-1},\ldots,X_{0} = i_{0}) = P(X_{n+1} = j\vert X_{n} = i)}$$ In continuous time, it is technically difficult to define the conditional probability given all of the X r for r ≤ s, so we instead say that X t , t ≥ 0 is a Markov chain if for any 0 ≤ s 0 < s 1⋯ < s n < s and possible states i 0, …, i n , i, j we have $$\displaystyle{P(X_{t+s} = j\vert X_{s} = i,X_{s_{n}} = i_{n},\ldots,X_{s_{0}} = i_{0}) = P(X_{t} = j\vert X_{0} = i)}$$ In words, given the present state, the rest of the past is irrelevant for predicting the future. Note that built into the definition is the fact that the probability of going from i at time s to j at time s + t only depends on t the difference in the times.