On Markov chains generated by markovian controlled Markov systems: Structural stability
Yasuo Komota, Masayuki Kimura · International Journal of Systems Science · 1981
From the viewpoint of applying the framework of structural stability theory for dynamical systems defined on manifolds to the analysis of structural stability problems for discrete stochastic systems, this paper deals with the structural stability problem that arises when one considers the effects of small perturbations of transition matrices of an underlying Markov system M upon the ergodic properties and the other behaviour of a non-homogeneous Markov chain generated by a markovian controlled Markov system MB (proposed by Komota el al. previously) and of a Markov system M, respectively. The concepts of three kinds of stability, namely, ergodic stability, ultimate stability and strong stability, are introduced and the necessary and sufficient conditions for a given Markov chain and for a given Markov system to be stable in the sense of these three kinds of stability are presented. As a consequence of these results, it is made clear that the three kinds of stability are equivalent in both cases. Finally, it is proved that the class of structurally stable Markov systems is open and dense in the metric space of all Markov systems defined on fixed state set and input set. This open—dense theorem is analogous in some sense to Peixoto's open-dense theorem for the class of structurally stable dynamical systems defined on a compact differentiate manifold.