Discrete-Time Markov Chains: On the transition from countable to continuous state spaces
Lorenz Mihatsch · Open access LMU (Ludwid Maxmilian's Universitat Munchen) · 2020
In this thesis we will discuss basic concepts of homogenous discrete time Markov chains.We will start by introducing stochastic processes as a series of random variables {X n } n∈N 0 taking values in the same measurable space (E, E), which are called states.The basic idea of Markov chains is that the probability of {X n } n∈N 0 to adopt some state only depends on the previous state, but not on the ones before.This property is called Markov property.In case of discrete state spaces we describe the transition from one state to another by stochastic matrices, which we will then generalize to continous state spaces by introducing Markov kernels.Throughout this thesis we will analyze serval different examples of Markov chains in discrete as well as continous state spaces.