Multi-partition time aggregation for Markov Chains
Edilson F. Arruda, Marcelo Dutra Fragoso, Fabricio de Oliveira Ourique · 2017
Motivated by Markov decision processes, this paper introduces a form of embedding for Markov chains which is based on the partition of the state space into a manageable number of subsets, with the aim of enabling a decomposition algorithm for calculating long-term costs and probabilities. The decomposition enables the decision maker to derive the long term distribution by making use of evaluations in the domain of the partitions, which presents reduced cardinality with respect to the original state space and hence yields reduced computational effort.