A novel approach with safety metrics for real-time exploration of uncertain environments
Tommaso Mannucci, Erik-Jan Van Kampen, Coen C. de Visser, Q. P. Chu · 2016
Various research has been done on the application of Reinforcement Learning for adaptive controllers for aerospace, due to its core simplicity of design and its model-free capabilities resulting in a great flexibility of application.During real-life exploration of the environment, such a controller will employ various algorithms to accelerate the collection of significant data and therefore the convergence of the value function.If the environment presents any form of danger for the agent, these algorithms need to cope with the additional requirement of avoiding actions leading to such dangers, even when a definite model of the agent in the environment is not available.In this paper, computing a safety-weighted graph based on a tiling of the state space, and with the introduction of two different metrics for action selection is shown as a promising method for avoiding dangers during exploration.As proof of concept, the method is applied on two simulated tasks: a high-level navigation task for an autonomous UAV, and a classical, low-level task of controlling the elevator deflection of an aircraft.