Robust Reinforcement Learning for Quadcopter Control
Lukas Bjarre · KTH Publication Database DiVA (KTH Royal Institute of Technology) · 2019
Sim-to-Reality transfer in Reinforcement Learning is a promising approach ofsolving costly exploration in real systems, but it comes with the generalizationproblem of transferring policies from simulators to real systems. This thesislooks at ideas presented by Robust Markov Decision Processes, which combinesideas from Reinforcement Learning and Robust Control to create agentswith embedded uncertainty about the simulated environment, opting for pessimisticoptimization in order to handle potential gaps between simulators andreality. These ideas are adapted in order to apply it to a state-of-the-art DeepReinforcement Learning algorithm.The adaptations were tested on the task of positional control of a quadcopter,where agents were trained in a simple simulator and tested on versionsof the simulator with different environment parameters. Agents with higherlevel of robustness outperformed the standard agents in these environments,suggesting that the added robustness increases generality and can help whentransferring policies from simulators to reality.