A software system for multi-level hierarchic Markov processes
Anders Ringgaard Kristensen · 2007
A general software system for construction, editing and optimization of Markov decision processes ranging from traditional models and hierarchic Markov processes to multi-level hierarchic processes is presented. A short description of the system is given and the facilities are illustrated through a sow model defined as a 3-level hierarchic Markov process combining decisions on two time scales. Emphasis is put on modeling tricks that may simplify the structure or speed up the calculations which in real world models are of vast dimensions. 1 Introduction Herd management science has a long and strong tradition for application of Markov decision processes for decision support. For a survey of studies, reference is made to [6]. Recent examples include [1, 9, 10, 11, 12, 13, 15]. From a theoretical point of view, the development of hierarchic Markov processes [5] and multi-level hierarchic Markov processes [8] has widened the applicational scope of the techniques. Even though the methods a...