Definition and Examples of DTMCs
Natarajan Gautam · Wiley Encyclopedia of Operations Research and Management Science · 2011
Abstract The objective of this article is to describe a special type of stochastic process called discrete‐time Markov chain (DTMC) . We first provide definitions for DTMCs using simple language as well as mathematics. Then, we describe a systematic technique to model a system as a DTMC. Finally, we provide examples to both illustrate the modeling technique and motivate the breadth of real‐world applications where DTMCs can be used.