RAAM: The Benefits of Robustness in Approximating Aggregated MDPs in Reinforcement Learning
Marek Petrik, Dharmashankar Subramanian · 2014
We describe how to use robust Markov decision processes for value function ap-proximation with state aggregation. The robustness serves to reduce the sensitiv-ity to the approximation error of sub-optimal policies in comparison to classical methods such as fitted value iteration. This results in reducing the bounds on the γ-discounted infinite horizon performance loss by a factor of 1/(1 − γ) while preserving polynomial-time computational complexity. Our experimental results show that using the robust representation can significantly improve the solution quality with minimal additional computational cost. 1