Asymmetric Actor Critic for Image-Based Robot Learning
Lerrel Pinto, Marcin Andrychowicz, Peter Welinder, Wojciech Zaremba, Pieter Abbeel · 2018
Deep reinforcement learning (RL) has proven a powerful technique in many sequential decision making domains.However, robotics poses many challenges for RL, most notably training on a physical system can be expensive and dangerous, which has sparked significant interest in learning control policies using a physics simulator.While several recent works have shown promising results in transferring policies trained in simulation to the real world, they often do not fully utilize the advantage of working with a simulator.In this work, we propose the Asymmetric Actor Critic, which learns a vision-based control policy while taking advantage of access to the underlying state to significantly speed up training.Concretely, our algorithm employs an actor-critic training algorithm in which the critic is trained on full states while the actor (or policy) is trained on images.We show that using these asymmetric inputs improves performance on a range of simulated tasks.Finally, we combine this method with domain randomization and show real robot experiments for several tasks like picking, pushing, and moving a block.We achieve this simulation to real-world transfer without training on any real-world data.Videos of these experiments can be found in www.goo.gl/b57WTs.