Numerical Methods in Markov Chain Modeling
Bernard Philippe, Yousef El-Mabruk Saad, William J. Stewart · Operations Research · 1992
This paper describes and compares several methods for computing stationary probability distributions of Markov chains. The main linear algebra problem consists of computing an eigenvector of a sparse, nonsymmetric matrix associated with a known eigenvalue. It can also be cast as a problem of solving a homogeneous, singular linear system. We present several methods based on combinations of Krylov subspace techniques, single vector power iteration/relaxation procedures and acceleration techniques. We compare the performance of these methods on some realistic problems.