Extension of First Passage Probability
Yiping Zhang · Scholarship at UWindsor (University of Windsor) · 2020
In this paper, we consider the extension of first passage probability. First, we present the first, second, third, and generally k-th passage probability of a Markov Chain moving from one state to another state through step-by-step calculation and two other matrix-version methods. Similarly, we compute the first passage probability of a Markov Chain moving from one state to multiple states. In all discussions, we take into account the situations that one state moves to a different state and returns to itself. Also, we find the mean number of steps needed from one state to another state in a Markov Chain for the first, second, third, and generally k-th passage. Besides, we find the probability generating function for the number of steps. This makes the calculation of passage probabilities, mean and variance of passage steps, easier. Additionally, if we extend a discrete Markov Chain to its corresponding continuous Markov Process with the same transition probabilities and transition time in the form of an exponential distribution with parameter 1 between two states, we can obtain the mean time needed from one state to another state by Laplace Transforms, which is the same as with the discrete situation. Subsequently, we can calculate the variance of the time needed from one state to another state in the same way.