Symmetry-Informed Reinforcement Learning and its Application to Low-Level Attitude Control of Quadrotors
Junchang Huang, Weifeng Zeng, Hao Xiong, Bernd R. Noack, Gang Hu, Shugao Liu, Yuchen Xu, Huanhui Cao · IEEE Transactions on Artificial Intelligence · 2023
Symmetry is ubiquitous in nature, physics, and mathematics. However, a classical symmetry-agnostic Reinforcement Learning (RL) approach cannot guarantee to respect symmetry. Researchers have shown that if the symmetry of a system cannot be respected, the performance of a symmetry-agnostic RL approach can be inhibited. To this end, this paper develops a generally applicable Neural Network (NN) module with symmetry that can enforce the symmetry of a system to be respected. Based on the NN module with symmetry, this paper proposes a symmetry-informed Model-Based RL (MBRL) approach that respects symmetry and improves data efficiency. The symmetry-informed MBRL approach is applied to the attitude control of a quadrotor in simulation to evaluate the effectiveness of the approach. The simulation results show that the data efficiency of the symmetry-informed MBRL approach is much superior to that of a symmetry-agnostic MBRL approach. An NN module with symmetry can respect the symmetry of a quadrotor while a naive NN cannot enforce the symmetry of a quadrotor to be respected.