Robust Robot Control Policy Learning Based on Gaussian Mixture Variational Autoencoders
Qingwei Dong, Peng Fei Zeng, Chuanzhi Zang, Guangxi Wan, Xiaoting Dong, Shijie Cui · 2024
Reinforcement learning methods have demonstrated advanced capabilities in training neural network controllers for specific tasks. In the industrial domain, where model discrepancies and state disturbances occur, robot control strategies must exhibit adaptability. Existing studies, such as Generative Motor Reflexes (GMR), employ Variational Autoencoders (VAE) to approximate the latent variable space distribution to a standard normal distribution. However, this method, while straightforward, offers limited capability in representing data. The Gaussian Mixture Variational Autoencoder (GMVAE) algorithm leverages a Gaussian Mixture Model (GMM) as the prior distribution for data generation, enhancing the representation of the original state distribution. To increase the robustness of the trained neural network controllers against unknown states, we utilize GMVAE to represent the original state space, subsequently learning the mapping from latent variables to actions. We further accelerate the policy search process by integrating state representation learning with Guided Policy Search (GPS) methods. Our method was evaluated on robotic reaching tasks. Experimental results validate the robustness of our method against state perturbations.