Detecting and Exploiting Symmetry to Accelerate Reinforcement Learning
Ashish Pujari, Haoze Lin, William Liles Neal, Tony Dear, Scott David Kelly · Society for Industrial and Applied Mathematics eBooks · 2023
Reinforcement learning can be used to develop a feedback control policy for an autonomous system when the system's dynamics aren't known explicitly but can be explored through simulations or physical experiments. When a symmetry is present, this fact can be exploited to accelerate exploration and learning. It's possible to know in advance that a particular symmetry will be present in a particular problem — for instance, when a mobile robot with unknown internal dynamics moves through an environment with some known uniformity — but it's also possible to detect unknown symmetries during the learning process. The detection of a symmetry while learning can be used to trigger the exploitation of that symmetry, accelerating learning thereafter. We illustrate these ideas using computational models for two systems: a wheeled mobile robot learning to navigate in a particular spatial direction and an array of vibrating cylinders driving a planar viscous streaming flow to advect an inertial solid particle to a target location. The former example is contrived, in that an analytical model is available for model-based control design, but only a computational model exists for the latter system, and our control results for this system are novel.