Evaluating the Steady State Distribution of Cumulative Reward in Hidden Markov Models
Ana Paula, Couto da Silva, R osa M. M. Le, Edmundo de Souza · 2007
Abstract. Hidden Markov Models (HMMs) have been widely used in the literature for modeling computer systems, for describing and predicting the loss and delay packet processes over the Internet and for solving network planning/dimensioning problems. This paper proposes a system of differential equations for calculating the distribution of the cumulative reward, when HMMs are used. Furthermore, we propose an iterative algorithm for obtaining an approximated solution for the differential equations. The overall technique for calculating the measure of interest is numerically robust and has a significant smaller computational cost when compared with approaches found in literature. 1.